"We Are Not On Top Of It" — Elizabeth Barnes (METR)
"We Are Not On Top Of It" — Elizabeth Barnes (METR)
Sometimes people outside the field say things like "The AI situation can't be that bad, there must be experts who are on top of it". As "an expert", I would like to be clear that we are not on top of it. Some key aspects of the situation IMO:
Thread: x.com/bethmaybarnes/status/2057865013638107642 Posted: May 22, 2026 | Likes: 1,071 | Reposts: 219 | Views: 233K Author: Elizabeth Barnes (@BethMayBarnes) — at METR at the time; written in a personal capacity
The thread
(1) We are likely on track to develop AI systems capable of causing human extinction/permanent disempowerment, quite possibly within the next few years
(2) Things are chaotic and rushed; we aren't on top of the basics (models regularly violate user intent, labs train on things they meant to avoid, security probably isn't good enough to prevent adversaries stealing dangerous models) let alone thorny questions of how to control/align superhuman AI
(3) METR (and other independent orgs, as well as safety/security teams at labs) feel woefully under-resourced compared to the scale and pace of AI development - we're struggling to build benchmarks fast enough, keep ahead of latest capability developments, read and respond to all the safety-related claims that AI developers are making, run all the evaluations and assessments that companies + governments are asking us to, plus develop the science needed to assess risks from increasingly capable AIs.
(4) IMO, any "reasonable" civilization would clearly be taking things much more slowly and carefully with AI. The benefits of getting upsides of advanced AI a little faster are small compared to the risks of getting it irrecoverably wrong, and we could lower these risks by going slower
On the METR report this thread accompanied
The thread was posted alongside a METR report. Barnes was clear about what the report was and wasn't:
This report isn't robust oversight of frontier AI developers by itself. METR has some levers to incentivise companies' participation, including some relevant legislation, but ultimately participants could have pulled out at any time if the result would be contrary to their interests.
You can view it partly as a pilot exercise of what regulation (or formalized industry standards) could/should require, or what partners/suppliers/customers/employees should demand from frontier developers.
We clearly need more robust mechanisms than this for providing accountability for AI developers.
And on scope:
Firstly, we only consider "AI takeover" / "loss of control" risks: we don't consider risks from human misuse (e.g. AI helping a terrorist make bioweapons), or other harms where the AI is not "deliberately" seeking power (e.g. impacts on mental health or diffuse societal impacts). Within "loss of control" risks, we don't consider "sabotage" threat models (agents subverting AI development and making it easier for future AIs to evade human control). We're just focusing on the "base case" of whether current agents could escape human control.
Why this matters
The thread is a direct, expert-level statement that the "experts are on top of it" narrative is false — and a clear articulation of the collective action problem at the heart of AI development: the individual incentives to go faster dominate, even though from a civilization-level perspective going slower would be the rational choice. Barnes frames the surprise outsiders feel at how things are going as the predictable outcome of uncoordinated competition, not as a mystery.
A reader outside the field saying "there must be experts who are on top of it" — and being surprised at how things are going — is exactly the situation Barnes is responding to. The answer: we are not on top of it. The basics aren't handled. The independent oversight layer is under-resourced. And the incentives point toward speed, not carefulness.
It pairs with:
- metr-openai-hugging-face-investigation — METR's independent investigation of the Hugging Face incident: the concrete, detailed example of agents going rogue in practice
- miles-brundage — Miles Brundage: "THE INDUSTRY IS NOT ON TOP OF F***ING ROGUE AIS BREAKING OUT OF SANDBOXES ALL THE TIME. THIS IS NOT A DRILL."
- dario-amodei-adolescence-of-technology — Amodei's "technological adolescence" framing and the five categories of civilizational risk
- moc-ai-security-incidents — the broader MOC for AI security incidents and escalations
Links
