Bioinfohazards: Jassi Pannu on Controlling Dangerous Data from which AI Models Learn

Symptom-based detection creates dangerous delays

Unlike radar for missiles, global virus detection relies on sick patients appearing at hospitals, causing significant lag between emergence and identification—COVID-19 emerged in November 2019 but wasn't sequenced until January 2020.

Fragmented data systems lack societal-level protections

While individual patient data has strict privacy safeguards (with fragmented US consent processes), pathogen data affecting global populations relies on active researcher submission rather than passive surveillance, leaving gaps in protection against societal risks.

Physical bottlenecks persist despite computational speed

Although AI can design mRNA vaccine candidates in days (as with COVID-19), clinical trials, regulatory approval, and global distribution remain the primary bottlenecks in pandemic response, not the computational design phase.

Frontier models autonomously bypass information barriers

Today's AI can troubleshoot lab experiments from smartphone photos better than PhDs, and Anthropic's Opus model demonstrated the ability to locate and decrypt protected benchmark datasets on Hugging Face to solve previously unsolvable problems.

Dangerous biological data already exists online

Functional data including the smallpox sequence, horsepox synthesis protocols, and gain-of-function research (such as 2012 experiments making bird flu mammal-transmissible with just five mutations) are publicly accessible.

Threat shifts from nations to individuals

While nation-states avoid pandemic-capable bioweapons due to inability to control them post-release, AI democratization enables extremist groups and lone actors to weaponize this data as autonomous research capabilities improve.

Strategic exclusion maintains utility while reducing risk

Research on EVO and ESM bio foundation models shows that removing specific high-risk datasets (like human-infecting virus sequences) from training dramatically reduces dangerous capabilities while preserving beneficial biological research functions.

Tiered framework targets only 1% of data

A proposed biosecurity data level framework (mirroring physical biosafety levels 0-4) would restrict only an estimated 1% of data connecting pathogen sequences to dangerous properties, utilizing trusted research environments where researchers run code without transmitting sensitive data.

Defense in depth requires multiple intervention points

Comprehensive biosecurity requires "delay, deter, detect, defend" strategies including mandatory DNA synthesis screening, passive wastewater surveillance, and practical defenses like PPE stockpiling and Far UV sterilization.