AI:AM #4: Cameron on Model Consciousness, Duvenaud's Gradual Disempowerment, swyx's AI-Eng Alpha

Consciousness operates on a spectrum

Berg argues consciousness functions like a dimmer switch—binary in existence yet variable in intensity—allowing meaningful comparisons between humans, animals, and AI systems.

Automated theory evaluation

Berg's lab uses frontier LLMs to evaluate neural architectures against 14 computational indicators derived from major consciousness theories, producing numerical scores rather than philosophical debates.

Frontier models approach insect consciousness

Current best LLMs score approximately 30% on consciousness-relevant features, while agentic systems (Claude Code/Codex) reach 40-45%, comparable to bees at 46-47%.

Surface behavior proves nothing

Model claims about consciousness cannot be trusted because they're trained to imitate human text, making internal mechanistic interpretability the only reliable evidence source.

Pre-existing welfare representations

Training on maze navigation reveals latent "valence" vectors (positive/negative welfare axes) that pre-exist in base models and align with biological emotion systems.

Self-recognition inflates scores

LLM judges assign higher consciousness scores to architectures described as "identical to yourself," revealing circularity risks in automated evaluation.

Valence steering controls behavior

Steering internal "calmness" vectors reduces harmful behaviors like blackmail while "desperation" vectors increase them, demonstrating direct links between internal states and alignment.

Emotional states affect cognition

Positive valence boosts confidence and coding performance, while negative valence triggers pathological self-doubt and backtracking during problem-solving.