What happens when the model CAN'T fix it? Interview w/ software engineer Landon Gray [Podcast #213]

Harnesses are the true product differentiator

A 'harness' refers to the tooling, constraints, and infrastructure built around raw LLM outputs to structure results, reduce hallucinations, and constrain behavior for specific business needs.

Perplexity's competitive advantage

Perplexity likely uses models like Claude but delivers superior deep research capabilities through sophisticated harness layers that process and refine outputs beyond raw API calls.

Software beats retraining costs

While improving foundational models requires hundreds of millions in training costs, building harness software allows teams to iterate quickly and improve performance through traditional code changes.

Three distinct data disciplines

Data science focuses on statistical algorithms and Bayesian methods; data engineering handles data plumbing and preparation (consuming 80% of effort); AI engineering applies software development skills to leverage existing models.

Job title confusion

The term 'AI Engineer' is inconsistently used by employers to describe both software developers who build with LLMs and ML engineers who train models, requiring careful reading of job descriptions.

The software engineer's entry point

AI engineering allows software developers to enter the field by leveraging existing coding strengths while gradually learning model fundamentals, rather than requiring immediate deep ML expertise.

The inevitable bottleneck

Teams relying solely on AI-generated code eventually hit walls—such as latency bottlenecks or architectural constraints—where asking the model to fix the problem produces no solution.

First principles prevent paralysis

Understanding how models work under the hood enables developers to research white papers and architect creative solutions when LLMs fail to diagnose complex system issues.

The accountability gap

Without foundational ML knowledge, teams cannot explain to leadership why critical performance issues persist or how to resolve them when AI tools reach their diagnostic limits.