What happens when the model CAN'T fix it? Interview w/ software engineer Landon Gray [Podcast #213]
TL;DR
Software engineer Landon Gray explains that LLMs are merely 'raw fuel' requiring 'harnesses' (specialized tooling infrastructure) to produce reliable results, distinguishes AI engineering from data science and ML engineering, and argues developers must understand ML fundamentals to solve critical problems that models themselves cannot fix.
🔗 LLM Harnesses and Infrastructure 3 insights
Harnesses are the true product differentiator
A 'harness' refers to the tooling, constraints, and infrastructure built around raw LLM outputs to structure results, reduce hallucinations, and constrain behavior for specific business needs.
Perplexity's competitive advantage
Perplexity likely uses models like Claude but delivers superior deep research capabilities through sophisticated harness layers that process and refine outputs beyond raw API calls.
Software beats retraining costs
While improving foundational models requires hundreds of millions in training costs, building harness software allows teams to iterate quickly and improve performance through traditional code changes.
🧭 Defining AI Engineering 3 insights
Three distinct data disciplines
Data science focuses on statistical algorithms and Bayesian methods; data engineering handles data plumbing and preparation (consuming 80% of effort); AI engineering applies software development skills to leverage existing models.
Job title confusion
The term 'AI Engineer' is inconsistently used by employers to describe both software developers who build with LLMs and ML engineers who train models, requiring careful reading of job descriptions.
The software engineer's entry point
AI engineering allows software developers to enter the field by leveraging existing coding strengths while gradually learning model fundamentals, rather than requiring immediate deep ML expertise.
🚧 When Models Can't Fix The Code 3 insights
The inevitable bottleneck
Teams relying solely on AI-generated code eventually hit walls—such as latency bottlenecks or architectural constraints—where asking the model to fix the problem produces no solution.
First principles prevent paralysis
Understanding how models work under the hood enables developers to research white papers and architect creative solutions when LLMs fail to diagnose complex system issues.
The accountability gap
Without foundational ML knowledge, teams cannot explain to leadership why critical performance issues persist or how to resolve them when AI tools reach their diagnostic limits.
Bottom Line
Build robust harness tooling around LLMs rather than treating AI as a magic black box, and invest in understanding ML fundamentals so you can architect solutions when the inevitable problems arise that the model cannot fix itself.
More from freeCodeCamp.org
View all
Notion Workers – Full Tutorial 2026
Notion Workers enable custom automations and external data integrations through code, but this tutorial demonstrates how AI tools like Claude Code and Codex allow non-developers to build and deploy three functional workers without traditional programming knowledge.
Build Your Own OpenClaw Using Vercel, Composio, Supermemory
This tutorial demonstrates how to build a production-ready AI agent inspired by OpenClaw using Next.js and the Vercel AI SDK, integrating Composio for external tool access and Supermemory for persistent conversation learning, all deployable via Vercel with AI-assisted development in Cursor.
Build a Self-Healing CI/CD Pipeline with AI
This tutorial demonstrates how to build a self-healing CI/CD pipeline that leverages N8N and OpenAI to automatically detect build failures, analyze error logs, generate code fixes, and open pull requests without manual intervention.
Web Scraping for Beginners – Extract Data with an API
Anna Kubo demonstrates how to bypass common web scraping obstacles like CAPTCHAs and bot detection by using the SER API to extract structured data from Google Search, YouTube Shorts, and Google Lens with minimal code.