I just finished hearing the CEO of Anthropic say the following on one of today’s “talking heads” programs:
The biggest dilemma is how we get China to agree to join the agreement on regulating the pace of AI development.
but I think he’s wrong. The dilemma that’s bigger than that is how we prevent a country we support that’s engaged in an existential conflict against a larger foe from developing their own LLM AI that breaks all the rules when it means their survival as a country.
That’s also the problem with Megan McArdle’s more modest and seemingly more doable agreement which she describes in her most recent Washington Post column:
Sure, I can imagine China and the United States coming to a rapid-fire agreement for an AI pause. But then, I can also imagine getting a MacArthur “genius grant,” which seems about as likely.
So back to my question about the murderbots. Could we at least not build the literal, physical murderbots, such as fully autonomous drones that can operate without humans in the kill chain? Controlling their development would be a very hard problem, as all arms control is. But it seems more manageable than completely stopping AI advancement.
Consider her proposal in the context of Ukraine. Should we actively discourage even prevent Ukraine from developing their own “murderbots” if they make the difference between their victory or defeat against the Russians?
The bigger dilemma may be how you construct an AI arms-control regime that remains rational to obey when one of its members believes violating the rules is necessary for national survival.
There’s an even more elementary problem. Suppose Anthropic, OpenAI, xAI, and Google agree on rules limiting what their LLM AIs may do. What prevents a well-financed startup from developing an LLM AI that ignores those rules?
If the prohibited capabilities are useful, that company’s product may be more competitive precisely because it breaks the rules. The agreement would then have accomplished something rather peculiar: it would have imposed competitive disadvantages on the companies that agreed to it while creating an opportunity for companies that did not.
Preventing that requires something considerably more ambitious than an agreement among AI companies—or even an agreement between the United States and China. It requires preventing *anyone capable of developing the technology* from defecting when there is an economic or strategic advantage in doing so.
Now consider Ukraine. If we cannot be confident that a startup will refrain from breaking the rules for money, why should we expect a country to refrain from breaking them when its leaders believe national survival is at stake?
