I follow a lot on AI Noah (too much in fact) and while I’ve heard rumblings of this type of position, you’ve laid this out as clearly as I’ve ever read. Never thought you’d turn into one of my most influential figures on how I view AI, but this is excellent
“We could simply be thinking about the benefits of intelligence wrong — arrogantly privileging the kind of mental tasks we humans happen to do especially well, while ignoring the value of the tasks we do poorly.”
This kind of “rhymes” with so much in the history of invention, where inventors trying and failing to “solve” one thing, accidentally invent (or inspire the invention) of a completely different thing.
I had a very similar thought - although I think in some ways saying 'arrogantly' is maybe even an "arrogant" error - perhaps worse, myopically blind to different modes as difficult to genuinely conceive.
I’m pretty sympathetic to the idea that intelligence isn’t just a linear knob you can turn to infinity for endless gain.
But at this point I think the “why hasn’t everything changed” answer is a boring one that seems to escape outspoken software nerds , both on the pro and anti AI sides: building actual physical things takes time. That’s it. That’s the whole huge mystery.
Folks it’s been four years since the first widespread use of genAI, and only like one year since we had models that were actually good.
Give it a minute! These things take time! Unless your intelligence is so advanced it can manifest things from thin air, you still have to design and build all this stuff. And the production lines. And then scale it. After finding a use case and a market. And maybe there are regulatory barriers. And then there are the stubborn people - white collar and trades alike - who think all change is fake news. There are machinists who still think CNC is a fad. We still have hordes of doctors giving the concept of “good and cheap time-series scan data” the stinkeye because their medieval guilds apparently cant handle better data except by overreacting and treating nonexistent conditions. How many decades did it take for “wash your hands beteeen handling cadavers and delivering babies” to not be sneered at? How many people died in the many years it took pilots to get over their “no queer gizmo is gonna tell ME where the plane is pointing” mentality and accept that yes, gyroscopes are useful? Etc across all fields and industries.
If tomorrow we discovered some radically novel new way of propelling spacecraft that would allow us to colonize the solar system, it would still be dumb to stand around the very next day harrumphing “ummm I guess it wasn’t so great after all cause where’s my house on Eros???”
To be clear this isn’t directed at Noah. The public-facing “tech” world discourse, which is heavily software biased, on the other hand...if there’s one thing software engineers can be reliably expected to do, it’s radically overestimate their galaxy-brained dominion over all human endeavors. This is just a thing, it would be hack comedy to even bother pointing it out at this stage.
It still takes time for concrete to cure, no matter how much open source ideology you spout at it or how many think-pieces about Skynet and/or the singularity someone can pump out in the interim.
The software needs don’t understand atoms at all. Or industry. Or agriculture. They assume humanoid robotics are just around the corner, without proof, nor do they get that you wouldn’t replace most modern factory robotics with these human style robots, nor a combine harvester with two dozen robots. We have automated a lot of industry already.
They also don’t get that a lot of the service industry, from retail to haircutting will be unaffected.
I have concerns about each of your sources of productivity gains for AI:
Smart matter: AI being capital has advantages, but also disadvantages. We don't endlessly build more physical capital of other kinds; there is an equilibrium level. Capital has diminishing returns, bottlenecks to growth, depreciation rates, etc. that determine this equilibrium. As it stands, we do know AI hardware has very tough bottlenecks and depreciation rates, and the jury is still out on the returns.
Distributed tacit knowledge: Capturing tacit knowledge is its own entire field: KM or organizational learning. We don't just need to record what people are doing, but why. "Why" is a deep, multi-faceted and possibly non-existent thing, even for purely technical work. I do hope AI can help here, especially if it also has access to the non-tacit knowledge of the organization like documents, but it's still speculative at this point.
Cloud laws: Controlling systems we don't have laws for is its own field too: Control theory. Very few of our machines are controlled by directly applying the underlying physical laws. Instead it's PID controllers everywhere. You don't need to have a deep understanding of most systems to control them, just respond to deviations quickly with simple rules. There are some small cases where AI has helped with control -- for fusion reactors, for example -- but the benefits seem marginal.
"We don't endlessly build more physical capital of other kinds; there is an equilibrium level." <-- Oh yes indeed! That's why I said that in the long run, this just raises the capital-to-labor ratio. In premodern times, one person's "capital" was a few tools and a couple of farm animals. Today each person is supplemented by a vast amount of industrial machinery. AI will continue this trend toward each person having more machines to back them up!
"I do hope AI can help here, especially if it also has access to the non-tacit knowledge of the organization like documents, but it's still speculative at this point." <-- Oh yes, this whole post is speculative. But AI can definitely take in vast quantities of information, and it can notice tons of little details that humans can't easily notice. Try uploading some images and getting AI to tell you about what it's looking at, and you'll see what I mean.
"There are some small cases where AI has helped with control -- for fusion reactors, for example -- but the benefits seem marginal." <-- Why were we never able to simulate natural language with control theory and simple feedback rules? Consider that for a moment... ;-)
I did my PhD in control theory and I think you somewhat misrepresent the field. Your comment is valid insofar as, yes, there is a sector of control theory that is concerned with model-free controllers, like PID.
It's incorrect to say that the entire field is controlling things we don't have laws for. Many popular control techniques and much of the theory addresses the case where we have an explicit system (typically called the “plant”) model.
Examples of common control methods that use plant models are model-predictive control (MPC) and Linear-Quadatic regulator.
Control is a very pragmatic (though also mathematically and theoretically rich) field. It concerns itself with a variety of problems:
° known plant model but control objective is computationally intractable so an efficient good enough solution is sought.
° known plant model but subject to unknown disturbances. Seek a controller robust to those disturbances.
° Unknown plant model but want to learn the plant model from data (system ID).
° Unknown plant model and want to control it without needing to learn a model (PID)
° Unknown plant model but want to learn a plant model while simultaneously learning to control it (Neuro-Adaptive control)
Etc.
What would be the field that tries to control systems with known dynamics? That's just a branch of control theory…
As a climate hawk, I'm pretty frustrated that AI superintelligence hasn't helped us solve climate change yet. AI can run through billions of new materials and chemicals per second, so where are the super-light, super-strong metal equivalents that will usher in electric aircraft and featherweight electric cars? Where are the new chemistries that will create super-efficient, super-flexible solar panels that can be slapped onto anything to give us all the electricity we'll ever need? Maybe all this stuff is in-train and I'm just writing a year too early. But several years into the modern AI boom, I'm not seeing it yet.
Honestly, all we really need at this point is cost reduction for zero-carbon steel and cement, plus a few process improvements in industrial heating. After that it's all just the governance bottleneck.
Great article. I m so glad I subscribed. I would add improvements on batteries energy density. This would solve the intermittent aspect of renewable energy (read grid decarbonisation nearly everywhere), solve for transport of goods (ships) and people (flights)
Exactly. Intelligence isn't the bottleneck. Coordination is.
Evolutionarily, humans didn't win because Einstein was smarter than a giraffe. We won because we built cooperative structures (language, institutions, states) that enabled coordination at scale. Intelligence is less an individual attribute than a property of how groups organize.
AI today is a tool inside those structures, not a new structure itself. So the better question isn't whether AI is smarter than us, but what role it plays in the organizations that actually coordinate human action. Especially sovereign states, which hold the greatest coercive power.
AI doesn't need to become a sovereign actor to be dangerous. It only needs to be embedded in organizations that already possess sovereign immunity. In such structures, AI participates in surveillance, targeting, and coercion while responsibility gets distributed until accountability disappears. Engineers say they only built the system. Officials say they relied on technical assessments. Operators say they followed protocol.
The real question isn't "will AI solve climate change." It's how AI transforms the power structures that decide whether we act at all.
“ The real question isn't "will AI solve climate change." It's how AI transforms the power structures that decide whether we act at all.”. Great comment.
It seems there are two classic problems with accomplishing coordination.
The knowledge problem — Figuring out what to accomplish together and how to go about it
The incentive problem — Overcoming the prisoners’ dilemma and how each actor is incentivized to exploit/defect (thus undermining coordination).
I suggest AI can help us significantly address both problems, though whether it does so for good or bad is still yet to be known. I am actually optimistic, as I think we are too powerful already for our present level of coordination.
I largely agree on the knowledge problem. AI is likely to help us discover better solutions much faster.
I’m much less optimistic about the incentive problem, though. Today’s world isn’t simply a coordination game between individuals—it’s a coordination game between sovereign states. We’re living in a period where sovereignty increasingly outweighs globalization, and I’m not convinced AI makes that easier to overcome.
In fact, AI may amplify the mismatch. Human civilization now operates at a scale far beyond the coordination capacity of traditional sovereign structures. AI can improve coordination within organizations, but that doesn’t necessarily improve coordination between them. It may instead make competing organizations even more capable.
That’s why I think the hardest coordination problem isn’t technical. It’s institutional.
It’s only been a year or so since AI was as good as people at coding, and a few months since AI reached the ability to do new math that people hadn’t done yet - other disciplines will likely take longer. Materials research seems particularly likely to be difficult.
I think you have finally expressed in one post the true place of AI in this world. It is an amazing machine for applying all of human knowledge and experience that has been described and written down to answer questions asked by humans quickly. Your first footnote makes this clear (although I wonder why one would need to tell an AI why a particular problem is important).
AIs are tools. They aren't beings.
They have no agency, and have no curiosity. I don't think that any of the AIs are ticking over in their spare time wondering if some proposition no human has yet asked, or even mentioned somewhere, is true or not.
If we somehow placed an AI in, say Rome, in 0 AD, with access to all of human knowledge available at the time and left it alone, didn't prompt it along, how long would it take for it to come up with quantum mechanics? The current ultimate "cloud law." How to use it to make accurate predictions is well known. Why it works, let alone what it means, is a mystery.
There is good reason to think AI could be dangerous. Look at social media, which is just like AI: no agency, no curiosity. Just an inert lump until humans call it into action. We may follow the Krell, yet.
We can definitely build systems with curiosity and agency, and such systems already exist and ask questions no human has asked. See, e.g.., the automated scientist work of Ross King.
However, I agree that current AI systems are unlikely to produce conceptual change. This is because they are trained on our current understanding rather than retracing the trajectory of conceptual changes that science has undergone. There is very little training data for using ML to model conceptual change.
The “third magic” post was great and that sort of knowledge outside of our perception (or where our limited perception prevents us from seeing/understanding) is an interesting concept.
To me AI’s two superpowers are knowledge (in the old days I knew a couple of bright kids who had practically memorized the encyclopedia- AI knows everything it trained on), which is not “intelligence”, and iteration (computing power).
Machine learning has been used in financial markets for decades and I’ve seen it uncover relationships humans wouldn’t have found (or looked for)….maybe unless you had a team of 1000 humans and have them broad remits.
Is knowledge plus iteration plus being able to filter iterative paths to the more promising ones (based upon past knowledge and finding overlaps) actually “intelligence”.?
Great stuff, I think we’re already seeing the diminishing social returns of individual intelligence, which at some point seems to crowd out the emotional kind
I absolutely adore the idea of “cloud laws” - specifically the conceit that axiomatic derivation is sort of a comparatively brittle path dependent emergent phenomenon of both cognitive biology and the history of philosophy, with some inherent incompletenesses, and the truth is much more associational-geometric in nature.
One other thought. While demoting intelligence, you quietly enthrone capital. Your theory of change is machine legibility applied to labor, matter, and institutions. Footnote 3 half-refutes the piece: you see the status interests behind worship of intelligence, but not the ownership interests behind worship of productivity.
I don't know where you get the "quietly" part...the return of the importance of physical capital was a big theme of this post, and of other things I've written on AI!
It's not about my personal opinion of capital, or yours. It's not about society's, either. Capital's contribution to productivity is just a physical fact. And if capital is becoming more important to productivity, that's a physical fact as well -- one that we will have to deal with.
Wrong word choice on my part. By “quietly,” I meant unexamined, not absent. Capital raises output. My point is that productivity does not determine who owns the capital, captures the gains, or absorbs the costs. That is political economy, not physics.
The current wave of AI breakthroughs is from LLM and its applications are still largely confined to language processing. As a result, the primary shifts we're seeing right now are quite specific: democratizing software engineering (since coding, at its core, is just manipulating language) and boosting efficiency for people whose main tools are reading, writing, and synthesizing text. Honestly, in the grand scheme of economic transformation, that can feel a bit surface-level and incremental.
But we shouldn't get impatient just yet. Computer scientists are actively building physical world models—most notably Fei-Fei Li’s latest push into Spatial Intelligence and World Models instead of just Language Models—to help AI understand the physical world directly rather than through text descriptions. Once that paradigm shift happens, we’re going to see very tangible, dramatic leaps in humanoid robotics and real-world automation fairly quickly.
Honestly, we’re just in the messy middle right now. The fire’s already burning hotter by the day, but it takes time. We’ll only truly get the big picture once the smoke clears.
I highly recommend running this essay through a top notch AI. The two together are much better than either alone. And I kind of think this supports Noah’s article.
Really good article! I suspect low hanging ”cloud laws” may be found in the area of condensed matter physics. I hope that more cloud laws are found for biology, but I suspect that will be a slower process.
There is a similar more technical argument for limited returns on intelligence.
A lot of problems that can be solved by algorithms cannot be solved efficiently - these are so-called NP-hard problems. Many economically relevant problems belong here - planning, scheduling, search with constraints. The core limitation is that adding a lot more computational resources only slightly improves the max size of the problem where perfect solutions are feasible: exponential spend for linear gains.
Does not mean that humans would be able to match the solutions - but does limit the economic value that can be captured by becoming superintelligent.
Satisfiability doesn't even have anything that could be a "good approximation" without being right. The knapsack problem has arbitrarily good polynomial-time approximations. The traveling salesman problem has some polynomial-time approximations, but I'm not sure if there are bounds on how well it can be approximated.
There are many subtleties here. 3-SAT is NP-complete and yet we routinely solve very large SAT problems daily. A lot depends on the distribution of problem instances.
I follow a lot on AI Noah (too much in fact) and while I’ve heard rumblings of this type of position, you’ve laid this out as clearly as I’ve ever read. Never thought you’d turn into one of my most influential figures on how I view AI, but this is excellent
Bravo and thank you good sir
Wow, thank you!! That means a lot.
“We could simply be thinking about the benefits of intelligence wrong — arrogantly privileging the kind of mental tasks we humans happen to do especially well, while ignoring the value of the tasks we do poorly.”
This kind of “rhymes” with so much in the history of invention, where inventors trying and failing to “solve” one thing, accidentally invent (or inspire the invention) of a completely different thing.
I had a very similar thought - although I think in some ways saying 'arrogantly' is maybe even an "arrogant" error - perhaps worse, myopically blind to different modes as difficult to genuinely conceive.
But I then want to cyborg myself (2/3 joke here)
I’m pretty sympathetic to the idea that intelligence isn’t just a linear knob you can turn to infinity for endless gain.
But at this point I think the “why hasn’t everything changed” answer is a boring one that seems to escape outspoken software nerds , both on the pro and anti AI sides: building actual physical things takes time. That’s it. That’s the whole huge mystery.
Folks it’s been four years since the first widespread use of genAI, and only like one year since we had models that were actually good.
Give it a minute! These things take time! Unless your intelligence is so advanced it can manifest things from thin air, you still have to design and build all this stuff. And the production lines. And then scale it. After finding a use case and a market. And maybe there are regulatory barriers. And then there are the stubborn people - white collar and trades alike - who think all change is fake news. There are machinists who still think CNC is a fad. We still have hordes of doctors giving the concept of “good and cheap time-series scan data” the stinkeye because their medieval guilds apparently cant handle better data except by overreacting and treating nonexistent conditions. How many decades did it take for “wash your hands beteeen handling cadavers and delivering babies” to not be sneered at? How many people died in the many years it took pilots to get over their “no queer gizmo is gonna tell ME where the plane is pointing” mentality and accept that yes, gyroscopes are useful? Etc across all fields and industries.
If tomorrow we discovered some radically novel new way of propelling spacecraft that would allow us to colonize the solar system, it would still be dumb to stand around the very next day harrumphing “ummm I guess it wasn’t so great after all cause where’s my house on Eros???”
To be clear this isn’t directed at Noah. The public-facing “tech” world discourse, which is heavily software biased, on the other hand...if there’s one thing software engineers can be reliably expected to do, it’s radically overestimate their galaxy-brained dominion over all human endeavors. This is just a thing, it would be hack comedy to even bother pointing it out at this stage.
It still takes time for concrete to cure, no matter how much open source ideology you spout at it or how many think-pieces about Skynet and/or the singularity someone can pump out in the interim.
This is Ruxandra Teslo's conclusion.
The software needs don’t understand atoms at all. Or industry. Or agriculture. They assume humanoid robotics are just around the corner, without proof, nor do they get that you wouldn’t replace most modern factory robotics with these human style robots, nor a combine harvester with two dozen robots. We have automated a lot of industry already.
They also don’t get that a lot of the service industry, from retail to haircutting will be unaffected.
I have concerns about each of your sources of productivity gains for AI:
Smart matter: AI being capital has advantages, but also disadvantages. We don't endlessly build more physical capital of other kinds; there is an equilibrium level. Capital has diminishing returns, bottlenecks to growth, depreciation rates, etc. that determine this equilibrium. As it stands, we do know AI hardware has very tough bottlenecks and depreciation rates, and the jury is still out on the returns.
Distributed tacit knowledge: Capturing tacit knowledge is its own entire field: KM or organizational learning. We don't just need to record what people are doing, but why. "Why" is a deep, multi-faceted and possibly non-existent thing, even for purely technical work. I do hope AI can help here, especially if it also has access to the non-tacit knowledge of the organization like documents, but it's still speculative at this point.
Cloud laws: Controlling systems we don't have laws for is its own field too: Control theory. Very few of our machines are controlled by directly applying the underlying physical laws. Instead it's PID controllers everywhere. You don't need to have a deep understanding of most systems to control them, just respond to deviations quickly with simple rules. There are some small cases where AI has helped with control -- for fusion reactors, for example -- but the benefits seem marginal.
"We don't endlessly build more physical capital of other kinds; there is an equilibrium level." <-- Oh yes indeed! That's why I said that in the long run, this just raises the capital-to-labor ratio. In premodern times, one person's "capital" was a few tools and a couple of farm animals. Today each person is supplemented by a vast amount of industrial machinery. AI will continue this trend toward each person having more machines to back them up!
"I do hope AI can help here, especially if it also has access to the non-tacit knowledge of the organization like documents, but it's still speculative at this point." <-- Oh yes, this whole post is speculative. But AI can definitely take in vast quantities of information, and it can notice tons of little details that humans can't easily notice. Try uploading some images and getting AI to tell you about what it's looking at, and you'll see what I mean.
"There are some small cases where AI has helped with control -- for fusion reactors, for example -- but the benefits seem marginal." <-- Why were we never able to simulate natural language with control theory and simple feedback rules? Consider that for a moment... ;-)
I did my PhD in control theory and I think you somewhat misrepresent the field. Your comment is valid insofar as, yes, there is a sector of control theory that is concerned with model-free controllers, like PID.
It's incorrect to say that the entire field is controlling things we don't have laws for. Many popular control techniques and much of the theory addresses the case where we have an explicit system (typically called the “plant”) model.
Examples of common control methods that use plant models are model-predictive control (MPC) and Linear-Quadatic regulator.
Control is a very pragmatic (though also mathematically and theoretically rich) field. It concerns itself with a variety of problems:
° known plant model but control objective is computationally intractable so an efficient good enough solution is sought.
° known plant model but subject to unknown disturbances. Seek a controller robust to those disturbances.
° Unknown plant model but want to learn the plant model from data (system ID).
° Unknown plant model and want to control it without needing to learn a model (PID)
° Unknown plant model but want to learn a plant model while simultaneously learning to control it (Neuro-Adaptive control)
Etc.
What would be the field that tries to control systems with known dynamics? That's just a branch of control theory…
As a climate hawk, I'm pretty frustrated that AI superintelligence hasn't helped us solve climate change yet. AI can run through billions of new materials and chemicals per second, so where are the super-light, super-strong metal equivalents that will usher in electric aircraft and featherweight electric cars? Where are the new chemistries that will create super-efficient, super-flexible solar panels that can be slapped onto anything to give us all the electricity we'll ever need? Maybe all this stuff is in-train and I'm just writing a year too early. But several years into the modern AI boom, I'm not seeing it yet.
Honestly, all we really need at this point is cost reduction for zero-carbon steel and cement, plus a few process improvements in industrial heating. After that it's all just the governance bottleneck.
Great article. I m so glad I subscribed. I would add improvements on batteries energy density. This would solve the intermittent aspect of renewable energy (read grid decarbonisation nearly everywhere), solve for transport of goods (ships) and people (flights)
Climate change is less a problem of knowing what to do (nuclear and so on),than of coordinating people to do it.
Exactly. Intelligence isn't the bottleneck. Coordination is.
Evolutionarily, humans didn't win because Einstein was smarter than a giraffe. We won because we built cooperative structures (language, institutions, states) that enabled coordination at scale. Intelligence is less an individual attribute than a property of how groups organize.
AI today is a tool inside those structures, not a new structure itself. So the better question isn't whether AI is smarter than us, but what role it plays in the organizations that actually coordinate human action. Especially sovereign states, which hold the greatest coercive power.
AI doesn't need to become a sovereign actor to be dangerous. It only needs to be embedded in organizations that already possess sovereign immunity. In such structures, AI participates in surveillance, targeting, and coercion while responsibility gets distributed until accountability disappears. Engineers say they only built the system. Officials say they relied on technical assessments. Operators say they followed protocol.
The real question isn't "will AI solve climate change." It's how AI transforms the power structures that decide whether we act at all.
“ The real question isn't "will AI solve climate change." It's how AI transforms the power structures that decide whether we act at all.”. Great comment.
It seems there are two classic problems with accomplishing coordination.
The knowledge problem — Figuring out what to accomplish together and how to go about it
The incentive problem — Overcoming the prisoners’ dilemma and how each actor is incentivized to exploit/defect (thus undermining coordination).
I suggest AI can help us significantly address both problems, though whether it does so for good or bad is still yet to be known. I am actually optimistic, as I think we are too powerful already for our present level of coordination.
I largely agree on the knowledge problem. AI is likely to help us discover better solutions much faster.
I’m much less optimistic about the incentive problem, though. Today’s world isn’t simply a coordination game between individuals—it’s a coordination game between sovereign states. We’re living in a period where sovereignty increasingly outweighs globalization, and I’m not convinced AI makes that easier to overcome.
In fact, AI may amplify the mismatch. Human civilization now operates at a scale far beyond the coordination capacity of traditional sovereign structures. AI can improve coordination within organizations, but that doesn’t necessarily improve coordination between them. It may instead make competing organizations even more capable.
That’s why I think the hardest coordination problem isn’t technical. It’s institutional.
It’s only been a year or so since AI was as good as people at coding, and a few months since AI reached the ability to do new math that people hadn’t done yet - other disciplines will likely take longer. Materials research seems particularly likely to be difficult.
I think you have finally expressed in one post the true place of AI in this world. It is an amazing machine for applying all of human knowledge and experience that has been described and written down to answer questions asked by humans quickly. Your first footnote makes this clear (although I wonder why one would need to tell an AI why a particular problem is important).
AIs are tools. They aren't beings.
They have no agency, and have no curiosity. I don't think that any of the AIs are ticking over in their spare time wondering if some proposition no human has yet asked, or even mentioned somewhere, is true or not.
If we somehow placed an AI in, say Rome, in 0 AD, with access to all of human knowledge available at the time and left it alone, didn't prompt it along, how long would it take for it to come up with quantum mechanics? The current ultimate "cloud law." How to use it to make accurate predictions is well known. Why it works, let alone what it means, is a mystery.
There is good reason to think AI could be dangerous. Look at social media, which is just like AI: no agency, no curiosity. Just an inert lump until humans call it into action. We may follow the Krell, yet.
We can definitely build systems with curiosity and agency, and such systems already exist and ask questions no human has asked. See, e.g.., the automated scientist work of Ross King.
However, I agree that current AI systems are unlikely to produce conceptual change. This is because they are trained on our current understanding rather than retracing the trajectory of conceptual changes that science has undergone. There is very little training data for using ML to model conceptual change.
Isn't AlphaFold already a good example of your thesis?
The “third magic” post was great and that sort of knowledge outside of our perception (or where our limited perception prevents us from seeing/understanding) is an interesting concept.
To me AI’s two superpowers are knowledge (in the old days I knew a couple of bright kids who had practically memorized the encyclopedia- AI knows everything it trained on), which is not “intelligence”, and iteration (computing power).
Machine learning has been used in financial markets for decades and I’ve seen it uncover relationships humans wouldn’t have found (or looked for)….maybe unless you had a team of 1000 humans and have them broad remits.
Is knowledge plus iteration plus being able to filter iterative paths to the more promising ones (based upon past knowledge and finding overlaps) actually “intelligence”.?
I’m not sure. Also not sure it matters.
Great stuff, I think we’re already seeing the diminishing social returns of individual intelligence, which at some point seems to crowd out the emotional kind
I absolutely adore the idea of “cloud laws” - specifically the conceit that axiomatic derivation is sort of a comparatively brittle path dependent emergent phenomenon of both cognitive biology and the history of philosophy, with some inherent incompletenesses, and the truth is much more associational-geometric in nature.
One other thought. While demoting intelligence, you quietly enthrone capital. Your theory of change is machine legibility applied to labor, matter, and institutions. Footnote 3 half-refutes the piece: you see the status interests behind worship of intelligence, but not the ownership interests behind worship of productivity.
I don't know where you get the "quietly" part...the return of the importance of physical capital was a big theme of this post, and of other things I've written on AI!
It's not about my personal opinion of capital, or yours. It's not about society's, either. Capital's contribution to productivity is just a physical fact. And if capital is becoming more important to productivity, that's a physical fact as well -- one that we will have to deal with.
Wrong word choice on my part. By “quietly,” I meant unexamined, not absent. Capital raises output. My point is that productivity does not determine who owns the capital, captures the gains, or absorbs the costs. That is political economy, not physics.
The current wave of AI breakthroughs is from LLM and its applications are still largely confined to language processing. As a result, the primary shifts we're seeing right now are quite specific: democratizing software engineering (since coding, at its core, is just manipulating language) and boosting efficiency for people whose main tools are reading, writing, and synthesizing text. Honestly, in the grand scheme of economic transformation, that can feel a bit surface-level and incremental.
But we shouldn't get impatient just yet. Computer scientists are actively building physical world models—most notably Fei-Fei Li’s latest push into Spatial Intelligence and World Models instead of just Language Models—to help AI understand the physical world directly rather than through text descriptions. Once that paradigm shift happens, we’re going to see very tangible, dramatic leaps in humanoid robotics and real-world automation fairly quickly.
Honestly, we’re just in the messy middle right now. The fire’s already burning hotter by the day, but it takes time. We’ll only truly get the big picture once the smoke clears.
I highly recommend running this essay through a top notch AI. The two together are much better than either alone. And I kind of think this supports Noah’s article.
Really good article! I suspect low hanging ”cloud laws” may be found in the area of condensed matter physics. I hope that more cloud laws are found for biology, but I suspect that will be a slower process.
Thanks. This is why I subscribe.
There is a similar more technical argument for limited returns on intelligence.
A lot of problems that can be solved by algorithms cannot be solved efficiently - these are so-called NP-hard problems. Many economically relevant problems belong here - planning, scheduling, search with constraints. The core limitation is that adding a lot more computational resources only slightly improves the max size of the problem where perfect solutions are feasible: exponential spend for linear gains.
Does not mean that humans would be able to match the solutions - but does limit the economic value that can be captured by becoming superintelligent.
That depends on whether you need to actually solve the problem, or do some good enough approximation.
Getting a good approximation to some NP-hard problems (for a technical definition of "good approximation") is also NP-hard.
That depends a lot on the specific problem!
Satisfiability doesn't even have anything that could be a "good approximation" without being right. The knapsack problem has arbitrarily good polynomial-time approximations. The traveling salesman problem has some polynomial-time approximations, but I'm not sure if there are bounds on how well it can be approximated.
There are many subtleties here. 3-SAT is NP-complete and yet we routinely solve very large SAT problems daily. A lot depends on the distribution of problem instances.