CristianTrout@CristianTrout1d ago

I have a new piece on @lawfare! TL;DR: Frontier labs can and should enforce their shared safety commitments by building a lab-owned mutual insurer that holds their catastrophic risk — putting their money where their mouth is. No new antitrust exemption needed. 1/ Problem I applaud the recent safety commitments from Anthropic, OpenAI, and other labs. But voluntary, unilateral commitments are easy to renege on and hard to keep consistent. They also forgo public goods like common standards, safety R&D, and improved risk modeling with pooled data. 2/ Proposed solution With @RuneKvist and @RajivDattani, we argue labs should make these commitments binding by building a lab-owned frontier AI mutual insurance company that holds the industry's catastrophic risks. This would create an entity with teeth and skin in the game to police labs and solve coordination problems. This isn’t about saving labs money on insurance; it’s about correcting incentives. 3/ Antitrust compliant Mutuals have an existing antitrust exemption (McCarran-Ferguson Act). That lets members peer review safety cases and incidents, and pool expertise for safety R&D. We expect some mix of these to be the default: mutuals routinely use these tools to guard against free riders and to provide value by reducing losses. 4/ Precedent Mutuals have been managing emerging risk since the Industrial Revolution. Modern examples include: • The US nuclear mutual runs onsite inspections. • Top law firms belong to malpractice mutuals that peer review claims. • Medical malpractice mutuals use their data to develop safety tech and standards. This is not Geico. Think privately organized fire department that also runs a fire safety lab. (Real examples. See the essay for more.) 5/ A real market for third-party assurance As @gabriel_weil and others have noted, an insurer covering catastrophic risk would finally create healthy demand for third-party evaluations of frontier risks. The mutual will want the best risk indicators at the lowest cost. 6/ “You can’t insure extinction” We’re not proposing labs buy insurance against literal human extinction. We're proposing an entity exposed to tens if not hundreds of billions in tail risk, such as CBRN or critical infrastructure failure. That is doable. Such an entity would be powerfully motivated to model, price, and mitigate those tail events. That should also reduce extinction risk. 7/ Limitations A mutual doesn't solve everything. It creates transparency between labs, not public transparency. (It helps with latter on the margin, but not much.) Those goals arguably should be decoupled anyway: one is about learning, the other is about public accountability. Also, while this solves some of liability’s limitations (e.g. a mutual will move much faster than courts), it inherits some too (e.g. negligence liability can incentivize burying damning information). 8/ We’ll need both market-based solutions and regulation This is a market-based solution that we think can do real work. But given frontier AI’s risks, I think we'll need to regulate it directly, and soon. Managing extreme risk is a basic duty of government. And to do nothing would be to cede unprecedented power to a handful of private companies. But the private sector can lead. Since labs are the ones building this technology, they have a responsibility to. And for better or worse, that’s where the expertise lives. Full piece here: https://t.co/D4jjjz6Wdj