Solving the Wrong Problem Works Better - Robert Lange
TL;DR
Robert Lange from Sakana AI explains how evolutionary systems like Shinka Evolve demonstrate that scientific breakthroughs require co-evolving problems and solutions through diverse stepping stones, while current LLMs remain constrained by human-defined objectives and fail to generate autonomous novelty.
🧬 Evolutionary Discovery Principles 2 insights
Stepping stones precede breakthroughs
Innovation follows an evolutionary tree where diverse intermediate discoveries must be collected before converging on solutions, rather than through direct optimization toward fixed goals.
Problem invention enables solutions
True creativity often requires inventing new problems recursively before solving them, a capability current AI systems lack when restricted to static, human-defined evaluation functions.
⚙️ Shinka Evolve Architecture 3 insights
Adaptive multi-model ensembling
The system dynamically selects between frontier LLMs including GPT and Gemini based on which model best suits specific program parents, significantly improving sample efficiency over single-model approaches.
Island populations with knowledge diffusion
Programs evolve in parallel islands with an archive database that diffuses successful mutations across populations, using LLMs to edit or cross-over code based on real-time evaluator feedback.
Self-adaptive evolution
The evolutionary algorithm co-evolves its own parameters during runtime, continuously adjusting model prioritization strategies as the optimization progresses.
⚠️ Autonomous AI Limitations 3 insights
Novelty stagnation in autonomous mode
LLMs running autonomously quickly plateau without generating interesting discoveries, remaining parasitic on their starting conditions and unable to escape local optima without human intervention.
Starting point sensitivity
Systems beginning with highly optimized solutions get trapped in local optima, while impoverished starting points enable greater diversity but require significantly longer optimization horizons.
Verification bottlenecks
Generating candidate solutions is computationally cheaper than hard-verifying correctness, creating fundamental bottlenecks for autonomous scientific discovery systems.
Bottom Line
Future AI systems must co-evolve problems and solutions from diverse, unconstrained starting points rather than merely optimizing fixed objectives, embracing open-endedness to achieve true autonomous scientific discovery.
More from Machine Learning Street Talk
View all
He won a Nobel here for AlphaFold. Then he left. - John Jumper
Nobel laureate John Jumper explains how AlphaFold solved the 50-year protein structure prediction problem by collapsing years of experimental work into minutes, while emphasizing its narrow scope as a starting point for biological research rather than a universal model of life.
The Ex-Congressman Who Says AI Isn't Unstoppable — Brad Carson
Former Congressman and Pentagon official Brad Carson argues that AI development is not inevitable and can be controlled through strategic regulation, particularly by treating AI as products subject to liability laws rather than granting them human rights, while leveraging chip controls and mandatory testing to shape the technology's future.
Inference, not prediction — Prof. Michael I. Jordan on what modern AI is still missing
Professor Michael I. Jordan critiques the hype around AGI and prediction-based LLMs, arguing that modern AI lacks economic and social thinking; he advocates for 'inference' systems grounded in game theory and market dynamics that respect human agency and create collective value.
Inference, not prediction — Prof. Michael I. Jordan on what modern AI is still missing
Professor Michael I. Jordan critiques AGI as harmful PR that demoralizes young engineers, arguing that true intelligence requires economic and social systems thinking—treating billions of humans as agents in value-creating ecosystems—rather than isolated prediction engines built without intellectual foundations.