I ran across an interesting post by Kevin Bass about Anthropic, AI safety, and the network of organizations involved in evaluating the risks posed by AI.
I don’t endorse everything in the post. In particular I don’t think it is necessary to assume corruption, conspiracy, or even bad faith to recognize the problem he identifies.
The problem is incentives.
Anthropic has been among the loudest voices warning that advanced AI may pose extraordinary risks. It has also advocated government regulation of frontier models, including independent evaluations intended to determine whether those models pose unacceptable risks.
Let’s assume they’re right.
Indeed, let’s go considerably farther than that. Let’s give Anthropic the keys.
If Anthropic really has the expertise to determine what constitutes a dangerous AI model, put it in charge of developing the standards. Give it an important role in evaluating models. Give its recommendations substantial regulatory force.
There should be one condition.
Anthropic can’t profit from it.
I don’t mean that figuratively. Congress has considerable power to establish the conditions under which governmental authority may be exercised. We already have conflict-of-interest laws, disclosure requirements, divestiture requirements, procurement rules, and restrictions intended to prevent people from using governmental authority for their own financial benefit.
Use them.
Construct whatever legal mechanism is necessary so that Anthropic, its principals, and those exercising the delegated authority cannot become richer because of the regulatory regime they devise. That might require divestiture. It might require some sort of regulated return. It might require a special corporate structure. Those are details for lawyers to work out.
The principle is simple enough: you can have the keys or you can have the money. You can’t have both.
There is a reason for doing that which has nothing to do with whether anyone at Anthropic is honest.
Regulation creates barriers to entry. The more elaborate the testing requirements, the more expensive compliance becomes. A company already spending billions of dollars developing frontier models is in a much better position to comply with a regulatory regime requiring billions of dollars than a prospective competitor is.
Consequently a company can advocate regulations for perfectly sincere reasons and still benefit enormously from them. The regulations may protect the public. They may also protect the incumbent.
Both things can be true at the same time.
That is why I don’t think accusations of corruption are particularly useful here. We don’t need to know what is in anybody’s heart. We can remove the conflict instead.
There is another advantage to this approach. It would provide a pretty good test of revealed preference.
If the people at Anthropic actually believe that frontier AI presents risks to human civilization sufficiently grave to justify extraordinary governmental intervention, asking them to surrender the opportunity to make extraordinary private profits from that intervention doesn’t seem unreasonable.
The greater the danger, the stronger the argument becomes.
Conversely, if the response is that giving up those profits would be intolerable, perhaps the situation is not quite as extraordinary as we have been told.
I don’t expect anything like this to happen. The major AI companies would oppose it. The investors would oppose it. Quite possibly the AI safety organizations would oppose it. There would be endless arguments that government could not afford to lose access to the expertise concentrated in the companies actually developing frontier AI.
My proposal doesn’t lose that expertise.
Use it.
Give Anthropic the keys.
Just don’t let them own the tollbooth.
A key point I didn’t mention in the post: there is nothing stopping Anthropic from throttling their pace of development. Or OpenAI. Or Alphabet.
Sure, but if Anthropic does it and no one else does it just means they go out of business. As to the rest, if we address it in the US what happens elsewhere? I am pretty skeptical about any efforts or success at slowing down AI. I expect a few stumbling blocks along the way but Ai is largely going to find ways to get around any attempts at control. It’s a big world and it goes well beyond the US and China.
Recent talk has centered on existential risks. Those are probably real but I would be more concerned about day to day risks and not just employment issues. I expect AI to be used in increasingly sophisticated ways to go after people’s savings, business accounts and finance in general.
Steve
The idea that AI (any flavor) is going to end human civilization “as we know it” is a canard used by these companies as a means to part investors from their money.
I saw the breathless headlines that a swarm of AI agents was somehow able to connect to the internet and hack other companies. Unless these AI agents were somehow able to gain physical access to the network, they were always connected to the internet.
Basically, their sandbox was weak, and they did what any good hacker would do. They exploited that weakness. The AI company should release their sandbox code, and we can see how long it would take human hackers to find and exploit that weakness.
Then, the “messages” make even less sense. Why would would one binary entity communicate with another binary entity using human language, and if it did, why would it use anything more than the minimal words? CPU clock cycles are not free.
AI is nothing more than statistical algorithms with a human interface bolted to it. When I am playing video games, I will sometimes “see” the code running under it. While the engines try to hide it, pseudo RNG still follows some patterns, no matter what the time lag is.
Once you see the trick, you cannot unsee it. AI is simply using pattern recognition to work its magic. It is sophisticated, but in reality, it is neither magic nor sentient. For video gamers, the next time you play a game think about the algorithms governing everything, and what is on the screen is nothing more than pixels generated from the underlying data. The only real purpose for screen output is to make it more palatable to the human.