Fast Models Need Slow Developers — Sarah Chieng, Cerebras

Hardware breaks the memory wall

New architectures like Cerebras' wafer-scale engine use on-chip SRAM instead of off-chip HBM to eliminate memory bandwidth bottlenecks, while disaggregated inference separates compute-bound prefill from memory-bound decode onto specialized hardware.

Stack-wide optimizations compound

Efficiency gains come simultaneously from model architectures like Mixture of Experts (MoE) and pruning, plus inference techniques like KV cache reuse that minimize redundant computations.

The 20x danger multiplier

Without changing habits developed for 50 token/sec models, developers will generate massive amounts of unverified technical debt 20 times faster, turning agent swarms into instant spaghetti code.

Strategic model pairing

Use larger, more intelligent models for complex planning and long-horizon workflows, then deploy fast models like Codex Spark as pure executors for sub-tasks to maximize both quality and speed.

Codify success into skills

Capture successful AI trajectories as reusable "skills" using slower planning models, then have fast agents execute these verified patterns autonomously in the background.

Cherry-picking induces taste

Leverage extreme speed to generate 15 to 75 variations of UI or design elements simultaneously, then manually select the best to artificially inject "taste" that models lack without exhaustive prompt engineering.

Shift from batch to interactive

Treat fast models as real-time pair programmers; sit with the code, ask questions, and actively steer implementation rather than spawning agents and walking away.

Validation becomes free

At 1,200 tokens/sec, exhaustive validation—test suites, linting, diff reviews, and browser QA—should be baked into every step instead of deferred to pre-commit.

Externalize memory immediately

With context windows filling 20x faster (compaction in 30 seconds vs. 10 minutes), break tasks into bounded goals and use persistent files (agents.md, plan.md, progress.md, verify.md) to maintain state across sessions.