Frontier AI at Home — Alex Cheema, EXO Labs

Not your weights, not your brain

Cheema cites Andrej Karpathy's warning that renting AI via cloud APIs creates vulnerability to account lockouts and data surveillance, whereas local weights ensure true cognitive autonomy and privacy.

AI as exocortex, not tool

EXO Labs views AI as an extension of consciousness rather than a chat interface, making local hardware essential for uninterrupted access as agentic systems become critical infrastructure for professional competitiveness.

Training is FLOPs-bound, inference is memory-bound

While training demands raw compute, inference bottlenecks shift to memory bandwidth and capacity, particularly for low-batch-size local deployments that cannot aggregate multiple user requests.

Decode phase dominates local performance

Prefill (compute-heavy prompt processing) matters less than decode (token generation) for local use because system prompts remain cached, making decode speed the critical user experience metric.

Energy constraints limit mobile deployment

Phone inference currently consumes 10-15 watts, draining batteries within an hour and creating overheating issues that make sustained local inference on mobile devices impractical despite technical feasibility.

The hardware lottery favors training

Decades of optimization for Nvidia data center GPUs (built for FLOPs) left inference-specific architectures unexplored, creating significant 'low-hanging fruit' for alternative hardware like Apple Silicon.

Kernel fusion unlocks hidden performance

EXO Labs discovered standard implementations run 50% slower than theoretical speeds on Apple Silicon due to inefficient kernel launches, achieving 30% speedups through basic fusion techniques.

Full-stack inefficiencies persist

Suboptimal orchestration layers and harness implementations waste resources across the stack, where training-optimized software fails to account for local hardware constraints.

Exponential efficiency gains

Stanford's 'intelligence per joule' metric shows 5x improvement from hardware and 3x from model efficiency over two years, compounding to enable viable local frontier models.

Commodity memory expansion

Consumer devices now offer 128GB+ unified memory (e.g., MacBook Pro M5 Max), democratizing access to hardware previously restricted to data centers.

Current frontier remains expensive

Running trillion-parameter models like GLM 5.1 natively in FP16 requires approximately $40,000 in high-RAM hardware today, though this barrier drops exponentially with each generation.