5 Papers That Show Where AI Research Is Heading Right Now
TL;DR
Researchers argue that achieving AGI requires moving beyond human-generated training data toward AlphaZero-style self-play methods, while highlighting critical unsolved challenges in learning efficiency per sample and per watt. A detailed presentation demonstrates that protein biology models now follow the same predictable scaling laws as language models, with the ESMC model showing continuous improvement when trained on 2.8 billion sequences compared to previous plateaus at 50 million.
🎯 Beyond Human Data: The AlphaZero Path 2 insights
Human data constrains discoverable solution spaces
Training on human-generated solutions limits models to subspace H, making it improbable to discover the full solution space F minus H regardless of test-time compute or recursive self-improvement efforts.
AlphaZero self-play enables unbiased AGI development
Unbiased self-play without human data represents a more viable path to advanced intelligence than AlphaGo-style training, avoiding the limitations of human "meandering" exploration patterns.
⚡ Learning Efficiency: Sample and Watt Constraints 2 insights
Intelligence per sample optimization remains critical challenge
Current methods like in-context learning fail to improve monotonically with more samples and hit context-length cliffs, unlike human learning which consistently improves with experience using the same algorithm.
Biologically inspired alternatives to backpropagation urgently needed
The brain likely does not use backpropagation, suggesting undiscovered learning procedures like SPSA could dramatically improve intelligence per watt and enable true continuous learning.
🧬 Biological Scaling: The Bitter Lesson Holds 2 insights
Protein models confirm scaling laws transfer to biology
The ESMC protein language model demonstrates clean log-linear scaling laws identical to LLMs, where contact prediction performance improves predictably with increasing compute and parameters from 300M to 6B.
Billion-scale metagenomic data eliminates performance plateaus
Unlike the previous ESM2 generation which plateaued at 50 million sequences, ESMC trained on 2.8 billion metagenomic sequences from uncultured organisms shows no diminishing returns, confirming data scaling drives biological AI capabilities.
Bottom Line
AI research must prioritize AlphaZero-style self-play exploration and massive cross-domain data scaling while urgently developing biologically plausible alternatives to backpropagation to overcome fundamental limits in sample efficiency and energy consumption.
More from Y Combinator
View all
How A Prototype Built During A Missed Flight Became A New Gusto Product
Gusto co-founder Eddie Kim explains how a prototype built during a 5-hour airport delay evolved into Gusto Co-founder, an AI agent that automates repetitive small business tasks by leveraging existing customer data and simple chat interfaces rather than requiring technical expertise.
India Can Create The Largest AI Companies
India is positioned to create the world's largest AI companies because the technology rewards deep technical expertise over local market knowledge, leveling the global playing field and allowing Indian founders to win enterprise customers through cold outreach and superior product merit rather than geographic proximity or networks.
Zynga Founder: Consumer Is Not Investible Right Now - Thats Why You Should Build It
Zynga founder Mark Pincus argues that while consumer startups are currently out of favor with investors, AI agents create unprecedented opportunities to reinvent everyday services. He shares his "Proven Better New" product framework and explains why founders must kill their ego to survive the inevitable failure of novel features.
Why Domain Experts Are Winning Right Now
Bryant Chou, co-founder of Webflow, demonstrates how his new startup Ploy enables domain experts to autonomously execute world-class marketing and web design, arguing that deep industry experience is becoming the ultimate competitive advantage for leveraging AI effectively.