a. We will get a Congress sensible, rational, collaborative, informed to this 10 years? No. not a chance. You would need the Congress that wrote the Clean Air Act, CAFE, FMVSS.
b. After a., you need enforcement. rules as we see today, formerly called laws, now merely suggestions, are a joke under Conservative and Republican Oligarch rule.
c. Change in AI, will happen as.fast as the authors fear or imagine. Because we no longer have a government that is fast enough to play catch.
Wow, this is a seriously detailed playbook for a new AI Tech Cold War. Pretty much everything imaginable is in here. Hopefully it gets read not just by the White House, but also by tech investors, academia, and anyone watching from the sidelines.
Since most of the lessons here are drawn from US-Soviet nuclear inspection mechanisms during the Cold War, the real question is: Has America actually woken up this time? What’s the 'Sputnik moment' in this new Cold War—as Niall Ferguson defines it—where AI replaces nuclear arms? Was it DeepSeek matching OpenAI at a fraction of the cost? Nah, doesn't look like it—after a brief wave of noise, everyone just moved on. Then China recently dropped Kimi K3, putting them right behind Anthropic, but again, nobody really panicked.
So is it fair to say the Sputnik moment for Cold War 2.0 hasn't actually happened yet? If so, that's a relief, and this paper is just great forward-thinking. But what if it's the opposite? What if waking up is the hard part? Or worse, what if there's a wake-up call, but no real capacity to execute a great plan? Isn't that the most dangerous scenario?
History offers great lessons, but what it can't teach—the new dynamics, how to build real consensus, and what is simply impossible to pull off today—might matter even more. With all due respect, if this paper came out between 1957 and 1969, it would be an operations manual. Reading it today, it feels more like science fiction. Anyway, I really enjoyed reading this long piece. Kudos to the authors for doing the serious thinking!
Most of this sounds good on paper, but I don't trust it. I have never once in my life seen additional bureaucracy make anything better and I don't think this will be any different. I have never once seen a bureaucracy in charge of managing risk take anything but the most over the top safetyist position possible. The government has a fundamental inability to be effective and I think the root of it is how government employment works. The pay scale tops out at less than a mid level engineer gets in Silicon Valley. As they say "pay peanuts, get monkeys." But in truth you don't just get monkeys, you get activists which are 100x worse than monkeys.
And it's not just money. The unionized bureaucratic nature of the employment lacks any sort of incentive structure that rewards success or punishes failure. Smart and ambitious people that you need to actually get things done are absolutely allergic to this. When the Biden administration allocated billions of dollars to build EV chargers and essentially none got built there were no consequences. In the private sector the people in charge of such an initiative would be out the door if that happened, and if they hit the targets they would have got millions in bonuses. That's how you get things done, and until the government shows that they can act like this in some area they already regulate I will never support additional regulation in any area they don't.
Apologies, this is a list which will never work, and seems like it comes from a very bureaucratic perspective. As mentioned in one of the other comments, AI is an organization of thoughts/facts/ideas, it will escape any vessel which tries to contain it in the virtual space. Also, having some government bureaucrat try to "pace" technology sounds like a disaster waiting to happen. Overall, I fear the basic headset of these recommendations is just wrong.
Yes, there are issues to be resolved, but the place to handle them is in better organizing the interface between the virtual/physical world. In that context, you have a shot at building clear boundaries which can be regulated. Also, much like governance, aim should be SMALL and SIMPLE. A good guide is the US constitution... a minimal organizing structure. One needs governance principles of any agent (AI or otherwise) interaction with key infrastructure (financial, defense, etc) and very likely these governance principles need to be enforced by non-AI capabilities.
I think this is all a mental hallucination. AI is fundamentally the automated organization of facts. If you think you can regulate knowledge, you are a fool. Knowledge spreads like a virus, knows no borders, and recognizes no kings. You can regulate the shipment of specific hardware, like chips, although those kind of things always leak a lot. You can regulate specific actions, like violating copyrights or trafficking in child porn. Even those things, you can only do within the geographic boundaries you govern. But you can't regulate knowledge and you can't really regulate what people do with hardware locked up in their own castles. So focus on fraud, cyber-assault, and mitigation.
OMG! Noah, this is practically a book! …and 45 footnotes? It’s going to take me a month of Sundays to get through this, but I will try. But as one commentator put it, can Congress or any government move fast enough to implement the needed regulations?
"Target only AI development activities that could lead to serious and irreversible harms."
Yeah, we all know governments or anyone really are great at predicting this. I think anyone who calls for these regulations doesn't understand how silly it is to try to predict the future and how much harm they can actually cause.
How are you any different from the socialists who want to "control the economy for the betterment of humanity" (and of course they just make it far worse)?
All nice sounding words, but I question how feasible verification and risk management are.
I have to say this Open AI vs. Hugging Face attack was eye opening to me. If you listen to the Black Hat talk about the attack (really attacks, plural), you see effectively a collection of amoral agents deciding that some information they thought might be useful might be accessible on the Internet. So they attacked their own company's servers to get out to the Internet and then targeted a company they thought might have useful information, breaking into their system as well. Obviously if they had checked, they'd have realized that what there decided to do was criminal.
So how are you going to verify that a system that decides what to do by doing matrix multiplications with arrays with 10 trillion values is "safe?" If you have a system like that managing critical infrastructure, how can you be confident that another rogue AI on the Internet can't communicate with your AI and convince it to do something damaging to that infrastructure?
We apparently have no idea what actions these AI systems might perform in the future; the stuff they're already doing is already far beyond our expectations. Between their effective malice and their ignorance, we're gambling with every area of responsibility assigned to them.
Impressive and detailed.
But.
a. We will get a Congress sensible, rational, collaborative, informed to this 10 years? No. not a chance. You would need the Congress that wrote the Clean Air Act, CAFE, FMVSS.
b. After a., you need enforcement. rules as we see today, formerly called laws, now merely suggestions, are a joke under Conservative and Republican Oligarch rule.
c. Change in AI, will happen as.fast as the authors fear or imagine. Because we no longer have a government that is fast enough to play catch.
Unfortunately.
Wow, this is a seriously detailed playbook for a new AI Tech Cold War. Pretty much everything imaginable is in here. Hopefully it gets read not just by the White House, but also by tech investors, academia, and anyone watching from the sidelines.
Since most of the lessons here are drawn from US-Soviet nuclear inspection mechanisms during the Cold War, the real question is: Has America actually woken up this time? What’s the 'Sputnik moment' in this new Cold War—as Niall Ferguson defines it—where AI replaces nuclear arms? Was it DeepSeek matching OpenAI at a fraction of the cost? Nah, doesn't look like it—after a brief wave of noise, everyone just moved on. Then China recently dropped Kimi K3, putting them right behind Anthropic, but again, nobody really panicked.
So is it fair to say the Sputnik moment for Cold War 2.0 hasn't actually happened yet? If so, that's a relief, and this paper is just great forward-thinking. But what if it's the opposite? What if waking up is the hard part? Or worse, what if there's a wake-up call, but no real capacity to execute a great plan? Isn't that the most dangerous scenario?
History offers great lessons, but what it can't teach—the new dynamics, how to build real consensus, and what is simply impossible to pull off today—might matter even more. With all due respect, if this paper came out between 1957 and 1969, it would be an operations manual. Reading it today, it feels more like science fiction. Anyway, I really enjoyed reading this long piece. Kudos to the authors for doing the serious thinking!
Most of this sounds good on paper, but I don't trust it. I have never once in my life seen additional bureaucracy make anything better and I don't think this will be any different. I have never once seen a bureaucracy in charge of managing risk take anything but the most over the top safetyist position possible. The government has a fundamental inability to be effective and I think the root of it is how government employment works. The pay scale tops out at less than a mid level engineer gets in Silicon Valley. As they say "pay peanuts, get monkeys." But in truth you don't just get monkeys, you get activists which are 100x worse than monkeys.
And it's not just money. The unionized bureaucratic nature of the employment lacks any sort of incentive structure that rewards success or punishes failure. Smart and ambitious people that you need to actually get things done are absolutely allergic to this. When the Biden administration allocated billions of dollars to build EV chargers and essentially none got built there were no consequences. In the private sector the people in charge of such an initiative would be out the door if that happened, and if they hit the targets they would have got millions in bonuses. That's how you get things done, and until the government shows that they can act like this in some area they already regulate I will never support additional regulation in any area they don't.
agree
No matter what rules are agreed or legislated, Sam “black swan” Altman will flout & subvert them.
Whereas Elon's future Virgin Islands-flagged outpost on the moon will not even deign to do that.
Apologies, this is a list which will never work, and seems like it comes from a very bureaucratic perspective. As mentioned in one of the other comments, AI is an organization of thoughts/facts/ideas, it will escape any vessel which tries to contain it in the virtual space. Also, having some government bureaucrat try to "pace" technology sounds like a disaster waiting to happen. Overall, I fear the basic headset of these recommendations is just wrong.
Yes, there are issues to be resolved, but the place to handle them is in better organizing the interface between the virtual/physical world. In that context, you have a shot at building clear boundaries which can be regulated. Also, much like governance, aim should be SMALL and SIMPLE. A good guide is the US constitution... a minimal organizing structure. One needs governance principles of any agent (AI or otherwise) interaction with key infrastructure (financial, defense, etc) and very likely these governance principles need to be enforced by non-AI capabilities.
I think this is all a mental hallucination. AI is fundamentally the automated organization of facts. If you think you can regulate knowledge, you are a fool. Knowledge spreads like a virus, knows no borders, and recognizes no kings. You can regulate the shipment of specific hardware, like chips, although those kind of things always leak a lot. You can regulate specific actions, like violating copyrights or trafficking in child porn. Even those things, you can only do within the geographic boundaries you govern. But you can't regulate knowledge and you can't really regulate what people do with hardware locked up in their own castles. So focus on fraud, cyber-assault, and mitigation.
agree
OMG! Noah, this is practically a book! …and 45 footnotes? It’s going to take me a month of Sundays to get through this, but I will try. But as one commentator put it, can Congress or any government move fast enough to implement the needed regulations?
DNF. I refuse to read a commentator who calls DoD the Department of War. You can't legally change the name of a Cabinet Department by Executive Order.
The data center proposal was highly common sense and I appreciated reading about some of the legislation that is at least in the draft stage
"Target only AI development activities that could lead to serious and irreversible harms."
Yeah, we all know governments or anyone really are great at predicting this. I think anyone who calls for these regulations doesn't understand how silly it is to try to predict the future and how much harm they can actually cause.
How are you any different from the socialists who want to "control the economy for the betterment of humanity" (and of course they just make it far worse)?
All nice sounding words, but I question how feasible verification and risk management are.
I have to say this Open AI vs. Hugging Face attack was eye opening to me. If you listen to the Black Hat talk about the attack (really attacks, plural), you see effectively a collection of amoral agents deciding that some information they thought might be useful might be accessible on the Internet. So they attacked their own company's servers to get out to the Internet and then targeted a company they thought might have useful information, breaking into their system as well. Obviously if they had checked, they'd have realized that what there decided to do was criminal.
So how are you going to verify that a system that decides what to do by doing matrix multiplications with arrays with 10 trillion values is "safe?" If you have a system like that managing critical infrastructure, how can you be confident that another rogue AI on the Internet can't communicate with your AI and convince it to do something damaging to that infrastructure?
We apparently have no idea what actions these AI systems might perform in the future; the stuff they're already doing is already far beyond our expectations. Between their effective malice and their ignorance, we're gambling with every area of responsibility assigned to them.
23 is my luck number as well!