A thought experiment by James Kelly — mapped

What AI governance needs to survive the intelligence explosion

A single proposal combines three existing mechanisms — compute governance, a hybrid public/private certification market, and US domestic regulation — into one ecosystem. This map traces how the three pieces connect, where the combination quietly breaks, and what it takes to hold under a real emergency.

Compute governance Certification market Domestic regulation

Each piece is already in motion somewhere. The proposal's bet is that layered together, their gaps don't overlap — each pillar carries a matched cost.

01

Compute governance

Track and report frontier training runs at the hardware level.

Strengths

  • Easy to monitor — a handful of chip fabs and data centers, not millions of users
  • Already happening: export controls, the CHIPS Act

Weaknesses

  • Can't reach a model once it's released or leaked
  • Compute is not a perfect stand-in for capability

02

Public + private certification

A market for certification providers; courts and certifiers handle deployment harms.

Strengths

  • Uses courts that already exist
  • Creates a paper trail and a marketplace for certification

Weaknesses

  • Risks the conflict of interest that broke credit ratings in 2008
  • No company can pay enough to cover a true catastrophe — liability fails for the worst risks

03

US domestic regulation

Agencies oversee; power stays spread across companies; export controls hold the China lead.

Strengths

  • Not nationalized, but builds on what's already in motion
  • Includes public participation

Weaknesses

  • Agencies lack technical staff to actually check anyone's work
  • Without a crisis, enforcement stays weak

All three pillars lean on the same shortfall: government doesn't have the technical staff to verify anything itself. Follow what happens when that job gets handed off.

Step 1

The verification job gets handed to a certifier — chosen and paid by the company being checked.

Step 2

Right as a company nears a dangerous capability threshold, its own hand-picked certifier signs off.

Step 3

Nobody independent is watching, and liability can't backstop it — no company can pay for a true catastrophe.

Net result: paperwork that looks like oversight, but isn't.


Each fix targets a specific point in the failure chain above — replacing trust in paperwork with structural checks.

1

Check the hardware, not the paperwork

Build reporting into the chips themselves, so oversight doesn't depend on hiring more government auditors.

Short term · fixes Step 1
2

Take away the conflict of interest

Companies contribute talent to design certifications across every frontier lab, double-blind. They pay into a shared fund; certifiers are assigned by rotation, not chosen by the company they're checking.

Fixes Step 2
3

Split the job by severity

Courts and certifiers handle everyday harms. A small, focused agency — like an AISI — gets the sole power to halt a training run, but only for defined catastrophic thresholds, reauthorized by Congress on a set schedule.

Fixes Step 3

The plan concentrates the reporting system and the halt authority in one place. A fully distributed system would resist abuse but move too slowly for a real emergency — so different risks get different structures.

Distributed & slow

Courts & certifiers

Handle everyday harms — the rotation-assigned, double-blind certification market.

Centralized & time-limited

AISI-style agency

Sole halt authority. Power kept narrow; members are non-political joint appointees approved by both labs and Congress.

Centralized & fast

Catastrophic thresholds

Triggered automatically if a certifier is later caught missing something.

Courts and Congress back up the halt authority — knowingly slower than the agency itself, slow enough to act as a commercial deterrent to release. The highest cost: responding fast to a catastrophe means trusting one body's judgment before anyone can double-check it.


"Different risks need different structures: slow and spread out for everyday problems, fast and centralized for the worst-case ones."
As always, James is human and can make mistakes.