Utilitarianism has received a great deal of brilliant 'buttressing' since Bentham's first draft, not least by his follower and friend J S Mill, who provided excellent analytic refinements to deal with the often rather crude, reductio arguments levelled against it. Most of Mill's refinements are different ways of pointing out that in its purest, simplest form, utilitarianism, like any moral system, presents an ideal which needs caveating and adapting to specific situations and the wider human condition.
Nozick's utility monster is frankly daft. Nozick was much given to inventing things that definitely didn't, almost certainly wouldn't and probably couldn't exist, as the premise of thought experiments. All good fun, but hardly persuasive.
One real strength of Utilitarianism is that it is a (probably 'the') consequentialist philosophy. That is, it judges actions based on their consequences rather than on their conformity to a specific rule. It is extremely difficult to argue for any moral position without referring to consequences and Utilitarianism is by far the best thought-through and intuitive moral philosophy focused on the consequences of actions.
The “utility monster” and Omelas “problems” are actually very easy to answer: the utility monster isn’t actually utility-maximizing because wealth has diminishing utility returns in reality, and the Omelas thought experiment is completely unrealistic.
Utilitarianism is a moral philosophy designed for use in the real world, intended to maximize human welfare in the real world. It has no reason to care about unrealistic scenarios and cannot be disproven by nonsense like this. If you’re supposing something that seems to satisfy the formal precepts of utilitarianism but feels unfair, it is invariably because what you’re suggesting wouldn’t actually be utility-maximizing if attempted in reality.
Not only that, many of these objections hinge on the assumption that what feels wrong IS wrong. That’s not necessarily correct! “What feels wrong is wrong” is a distinct moral philosophy which is not utilitarian as such.
The insect thing is also silly, because bugs have very short lives and are constantly being eaten by larger and longer-lived animals. Not only that, we don’t know how they experience the world or how much utility they enjoy even when things go well for them. As such, we can’t realistically utility-maximize for them.
I’ve found that utilitarianism is never wrong when applied realistically, with everyone’s utility given equal weight and all relevant considerations included in the analysis. It is the correct moral philosophy.
"I’ve found that utilitarianism is never wrong when applied realistically, with everyone’s utility given equal weight and all relevant considerations included in the analysis. It is the correct moral philosophy."
No True Utilitarian?
Utilitarianism cannot fail, it can only be failed?
Actually? I believe the answer is yes. Utilitarianism gets the right answer every time, if you keep it within the parameters of things that can actually happen in the real world and if you have enough self-discipline to not selfishly over-value yourself and your own utility.
I’ll grant that including all relevant considerations in the analysis is not always easy. But when trying to bat down an attempted objection to utilitarianism coming from someone in the philosophy department, finding the flaw isn’t usually hard. It’s basically always either “that is an unrealistic scenario” or “utility doesn’t work that way in real life”.
In all honesty, I suspect the reason why utilitarianism takes such a beating in philosophy courses even though it’s clearly correct is because serious utilitarians who are smart and capable don’t become academic philosophers. They become economists, politicians, technologists, business leaders, and doctors, because they can see that a highly capable person’s lifetime spent in one of those fields does a lot more to raise humanity’s aggregate utility than one spent in academic philosophy. Philosophy does have value and it is entirely defensible, even on fully utilitarian logic, to give their departments some funding. But the limits ought to be appreciated here: eventually you need to move away from the theoretical utility function on the chalkboard and look at how people are actually doing in the real world.
What is more important among the following, and how do you measure?
* Suffering avoided
* Happiness obtained (and what do you mean by happiness)
* Fulfilment of meaning / purpose
And add whatever other values of "good" you might have to that pot.
Then answer the question: how do you convert between them? That is rigor demanded by utilitarianism. If you want to do that "locally" and use whichever measures seem to fit the moment, I'm more or less good with that, but that's not utilitarianism.
Utilitarianism tells you that if you are walking to work, and in your pocket you have a blockchain key on a USB drive that contains the money you just collected to save 100 starving children in Africa, and you pass a drowning child, and diving and to save them will certainly save them while certainly destroying your money, and failing to do so will certainly let them die, you should let that drowning child die. That is utilitarianism of the most common type practiced in EA circles. There are plenty who will say letting that child drown is the right decision! I'm far more conflicted than that.
As to how we balance different forms of utility: we largely do that through market price signals and the democratic governing process. These don’t work perfectly but they work quite well. It’s not realistic to do it perfectly, so the best we can do is all utilitarianism really demands of us.
I completely reject the idea that moral philosophy ought to be a problem set of unrealistic scenarios divorced from reality. The real world is the thing that actually matters and we should think about *it*, not dwell on idle what-ifs.
That's fine. But that's not utilitarianism -- at least not as practiced by the most famous EA folks like Will MacAskill. I inverted exactly an argument he makes in favor of EA: that if you would save a drowning child in front of you, which we all agree we have a moral obligation to do, then we have an equal moral obligation irrespective of distance, familiarity, or other circumstances, and thus you have the same moral obligation to children dying of starvation that you have to a child drowning in front of you.
There is a valid utilitarian argument in favor of prioritizing those nearer to you: you know more about their needs and are better positioned to help them, especially if you have limited resources. You know less about people farther away and have less ability to reach them, so it usually makes sense to entrust their welfare to those who are nearer to them. With that said, we are better able to tell what is going on across the world than we were in the past, so the argument that we should do more for the world’s poorest is absolutely valid. Bill Gates clearly made the correct moral choice by devoting his fortune to public health in poor countries.
But even with that said, saving a drowning child in front of you and donating to help starving children on the other side of the world are never realistically going to conflict. Starvation is a slow killer and in real life you’ll have plenty of time to send your donation after you’ve saved the drowning child.
I strongly believe that a unit of avoided suffering should be weighted much more than an additional unit of happiness and that you cannot offset one person’s suffering with another person’s additional happiness. The neutral point of personal suffering/happiness is a profound transition in quality of life. Under-appreciated perhaps.
Under that logic, levying taxes on childless people to pay for schools would be morally indefensible. For that reason I can’t agree with you on that score. Sometimes it is totally valid to impose some suffering for the greater good, albeit only if the good being done clearly outweighs the suffering. When “the ends justify the means” fails morally, it’s because the damage from the means was in fact greater than the good of the ends. I’ll grant that it’s important for people to be careful about this, because it’s hard to predict in advance whether the desired ends will pan out as hoped. But some forms of “impose a lesser suffering for a greater happiness” are well established and proven, like levying taxes to pay for public goods.
To what extent is the Omelas "completely unrealistic"? It can be readily argued that the current world is running on quite similar a setup, where the suffering of the labors in the developing countries on the supply-chains of many essential technologies relied on by the OECD citizens are indispensable.
But ultimately what has evolved is the point of moral philosophy itself. Since Rawls, it is no longer about "what I think is correct" but "what can be accepted by all as reasonable". Because in the end moral philosophy needs to be a basis people can live upon, and in real world people live together and negotiate together what that basis should be, and no one has the time and wisdom to apply utilitarianism correctly all the time.
We source raw materials from the US, Canada and Australia just fine without abusing anybody: natural resource extraction in wealthy countries tends to be done by a small number of workers operating heavy machinery to raise their productivity sky-high, allowing them high wages, and they also benefit from workplace safety rules.
In poor countries natural resource extraction is typically done by large numbers of low-paid workers using small tools without workplace safety rules, but it does not have to be that way and as these countries escape their premodern economic setup and grow their economies, their natural resource extraction operations will look more like those in wealthy countries. There is no supply chain that *needs* to be exploitative.
A utopian city whose supply chains in total do not need to be exploitative I feel is as fictional as Omela. In reality, the supply chains are exploitative because they need to be, to stay in market competition, and to keep inflation under control.
Supply chain does not mean you can pick and choose freely which parts come from where. Barely any electronics supply chain at scale can bypass China. Many manufacturers of electronic assembly in China are giving hourly rate below 20 RMB (US$3) to hordes of laborers, with factory-provided living conditions similar to company towns in the US history. There are so many laborers though, if they do not keep wages down they may lose orders to the factory next door which does. Are they applying utilitarianism right?
For all the tariffs and intentions to source from elsewhere, you may end up being scammed by a drop shipper and get made-in-China stuff anyway, see SmarterEveryday's attempt at making a grill scratcher entirely in the US. He ended up with a very good but very expensive grill scratcher. For cheaper mass marketed goods, you don't have alternatives to Chinese factories. Are US businesses relying on Chinese factories applying utilitarianism right?
By deontologist logic, labor laws are good and China should enforce them. But many Chinese economists claim that by consequentialist logic the workers are better off with a job than without, and China *needs* to keep their wages down to grow even more.
Many factories are built on land confiscated from farmers (who then died of execution or starvation) over the years, in the name of Revolution for a fairer world. Now they also become part of cheap China manufacturing.
The Chinese government then justifies all that by claiming that, they will in the end achieve dominance of world market and Chinese people can share in the glory while world population can share in the cheap products. They in fact succeeds to an extent in converting national glory to people's utility by investing in nationalist education.
Is the Chinese government applying utilitarianism right?
I have no confidence I can tell if someone has applied utilitarianism right or wrong in any realistic scenario.
It is perfectly possible to build a factory on land that was bought from a farmer who freely sold it; that’s how stuff in the US normally gets built.
Manufacturing jobs tend to shift out of high-wage countries into low-wage countries that have managed to get enough security and good policy to be worth investing in. Manufacturing jobs at wages that we consider shockingly low tend to be better paid than what people in premodern economies get from subsistence agriculture, so people flock to these jobs without any coercion. Nobody in the US is a poor subsistence farmer anymore and we, as a people, have forgotten how awful that lifestyle actually is.
China is at the point in the growth cycle where they ought to be shifting out of manufacturing into more services, but Xi is against this; it would be much more utilitarian to let the Chinese labor force shift into more services at this point but the CCP government has never been utilitarian, they have always prioritized the utility of their highest elites over that of everyone else. They are exploitative because they don’t think that the people being exploited matter. A good utilitarian would say that they matter. You keep asking if they’re applying utilitarianism right, but they’re not applying it at all.
I forget who pointed out that obviously, by any objective standard, Omelas is a vastly superior society to ours, or to any society that has ever existed or is ever likely to exist.
You're telling me Omelas has 9,999 profoundly happy people and 1 utterly miserable person? Do you think that any society in all of history has ever been 99.99% profoundly happy, or even 90% profoundly happy? Do you think your preferred set of policies could deliver a society in which fewer than 1 out of every 10,000 people were miserable, and all the rest lived in near-constant euphoria? Is your own life 99.99% profoundly happy? Would you want your own life to be 99.99% profoundly happy, and if your answer is "no", are you stupid?
The story is remembered for its usefulness to those looking for reasons to reject utilitarian logic, because it frames stunning social success as if it were somehow sad and monstrous.
Our society is superior to Omelas by one perfectly objective standard: it is not intentionally structured around torturing children in order to make the rest of the population flourish. The starting point of the Omelas story is the judgement that it is wrong to torture children, even if it increases net happiness. If you don’t share that judgement, of course Omelas won’t move you.
Hmm, did even one child suffer, perhaps via illness, in the process of mining, manufacturing, and/or polluting activities that created and powered the device you used to write this comment?
So, umm....yeah, there is like a tiny difference between unsafe working conditions and/or untaxed externalities and forced permanent imprisonment with infinite torture.
Joss Weedon's meta-horror film The Cabin In the Woods explores this idea with a world in which a small number of children/ teenagers have to be sacrificed periodically to appease ancient gods and stop them from destroying the World. The perspective is split but pitched asymmetrically in favour of the sacrificial youths over against the jaded, results-focused bureaucracy engaged in serving up the sacrifice according to the rules of the deal made with the ancients. I ended up thinking that even the survival of everything may not be worth the price: of course it's easy to say that when you're sat at home on your sofa watching a film; not so easy when huge, vengeful divinities are smashing the world to pieces!
Utilitarianism is basically Newtonian physics. It works pretty well on a small, well defined problem ("should we let someone build an apartment here that will make 50 new residents very happy but five neighbors very sad?"), but quickly falls apart when you try to make it a full theory of the universe.
It's a philosophical "one weird trick" that appeals to a certain set that think they can solve the entire problem of ethics via some single-variable optimization, who are very pleased with themselves that they can answer the "is it worth flipping a 49/51 coin to make everyone twice as happy/destroy humanity" by doing math.
"GDP, at its most simple, is a measure of the degree to which people in our society get what they want."
Another example of the tendency of economists to view reality on the macro rather than the micro level. As you point out later in your essay, 90% of the members of a high GDP society could be impoverished, miserable, enslaved, and barely clinging to life.
I gave an extreme example to gesture at the current American situation. American GDP continues to grow at a healthy manner while wages for the bottom 50% of American workers have essentially flatlined in the last 40 years. There's a big GDP but the way it is distributed is increasingly unequal.
No, you did not give an example at all. You just gave hysterical hyperbole.
If you seriously think that the American people in 2026 are "impoverished, miserable, enslaved, and barely clinging to life," then you are completely detached from reality.
"Flatlining wages" is not even close to "impoverished, miserable, enslaved, and barely clinging to life" and you know it. Nor is it true that "the bottom 50% of American workers have essentially flatlined in the last 40 years."
Inequality is a totally different issue from being "impoverished, miserable, enslaved, and barely clinging to life." There are dozens of highly unequal societies where the vast majority of people are living well. In fact, it is the norm in Western societies and increasingly East Asia.
For me, utilitarianism always fails on two simple grounds:
1) It pretends that whatever you might think "the good" is (QALY, hedonism, etc.), that all things can be converted into an objective measure of that as if they were as objective as length, and it's simply measuring in inches or meters or kilometers.
2) That there is some objective "the good." If there is no objective "the good" then what exactly are we measuring again? A moral realist generally doesn't have a problem with this point; they'd simply argue over what the objective good is. I'm not a moral realist.
Quite opposite to the end of utilitarianism, I believe those questions are exactly where the fun with utilitarianism begins. That is the marginalist revolution of utilitarianism. It is exactly when the different subjective "good"-s interact that we observe compromises, and to the spirit of utilitarianism, tools are developed to work out how and whether the compromise is guided by some kind of common "good". Arguably that goes into liberalism, but utilitarianism is guiding too. There are many papers along that line.
It seems to me that you could reject *any* moral philosophy or standard on nearly identical logic, by saying that it’s too subjective to be knowable at all what right and wrong are.
I find that profoundly unsatisfying, and also likely to get bad results.
It seems to me that you can in fact get a usable approximation of objective population-wide utility values via market price signals and the results of free and fair elections.
cp6 asks the right question — if there's no objective "the good," what's the standard? — and I don't think the answer has to be "there isn't one."
Drop the assumption that a standard must be a quantity to maximize, and a different kind is available: a direction. Do a person's or a society's values become more coherent while taking more into account, or do they hold together by leaving things out? That's checkable without a common currency. It doesn't need to convert suffering avoided into meaning gained; it asks whether the process that weighs them is widening or narrowing.
It also explains the track record LV points to. Utilitarians were first on women's rights, animal welfare, and the rest — but what carried them there was the widening, not the sum. The sum is what produces the utility monster and insects-over-humans. Keep the direction, drop the arithmetic, and you keep the part that had the good track record.
And it answers cp6's own mechanism. "Market signals and democratic process" aren't a calculation of utility. They're a process by which many valuers revise what they want under contact with consequences — which is the standard, not a stand-in for it.
wrt "[putting] a lot of weight on the well-being of our descendants in the far future" I agree with what Keynes wrote in his 1906 unpublished manuscript on Burke:
it can seldom be right… to sacrifice a present benefit for a doubtful advantage in the future…. It is not wise to look too far ahead; our powers of prediction are slight, our command over results
infinitesimal. It is therefore the happiness of our own contemporaries that is our main concern; we should be very chary of sacrificing large numbers of people for the sake of a contingent
end, however advantageous that may appear…. We can never know enough to make the chance worth taking. There is this further consideration that is often in need of emphasis: it is not sufficient that the state of affairs which we seek to promote should be better than the state of affairs which preceded it; itmust be sufficiently better to make up for the evils of the transition…"
The data center controversy is a simple case of different people disagreeing about the utility we can expect from AI.
Some people think that AI will do great things for human utility that will massively outweigh the disadvantages, and some people think AI will either give us a dystopia or wipe us out. Some are pointing out the possibility that AI will only have modest impacts on human utility. As of the current time, we don’t know who’s right.
The problem here isn’t with utilitarian moral philosophy, it’s figuring out what the utility tradeoffs of AI and data centers actually are.
I think a lot of what this diagnosis identifies is less a problem with utilitarianism and more a problem with the utility function *as specifically defined by various optimization vectors at the aggregate population level*.
These tend to share the common error that they don't 'bake in' multidirectional shapes as the object being optimized. The less abstract way of saying that is that — given most observed utility functions involve multiple competing axes — we should be extremely suspicious of any construction of utilitarianism that selects an attribute or state or quantity and parameterizes the optimization function along that particular axis at the system level, devoid of any internal equilibrators.
I'm not clear on whether Noah's argument here is that utilitarianism is worse than some existing known alternative, or just that it's not yet a fully complete solution to moral philosophy and there are more things we still need to figure out.
If the former, I think Richard Chappell put up a good counterargument: all known competing theories are also vulnerable to many of the problems Noah brings up here. To the extent this isn't obvious, it's because other theories are judged by a much lower standard and aren't asked to consistently provide rigorously defensible correct answers. https://www.goodthoughts.blog/p/puzzles-for-everyone
If the latter, then every moral philosopher already agrees with that, but then the framing of the article is kind of odd and gives the impression of wanting to selectively distance from a thing that's currently getting bad press.
The issue does not really have to be put into fancy philosophical terms.
Is there a distinction between what people want and what will make them happy? And, if so, who ought to be given the authority to make that call?
I am going to guess that there is a strong correlation between social class and responses to that. People surrounded by people who have gotten massive amounts of what they want, yet are not happy, are far more open to the idea that there is a crucial distinction. People who have largely seen a life of not getting what they want, not so much. It's the money and the stuff I want and lack that ought to be blamed for unhappiness.
So... arguing the point in the language of the well-educated is likely to skew the argument by excluding others. And elites are going to think that some authority other than people would do better at making people (especially other people) happy.
My skepticism is rooted in my experience of being a liberal in a red county. When I argue politics, I tend to take the nuanced view that I disagree with my county, but I respect their views. Invariably, my fellow educated blues tell me that my neighbors are voting against their own interests.
Think about that claim for a moment. The arrogance of being certain that one knows what is in the interest of others. "I know what ought to matter to these unmet people and I know far better than they know themselves. No need to walk in their shoes, I know what they are and what they should be."
I have no doubt that all of us make errors in our pursuit of happiness, but I have extreme doubts that distant authority would make fewer errors. Or be motivated, for long, to even try.
“We generally assume that this is causing less total suffering than torturing similar numbers of humans would.”
Although this is far from proven and the ratio may in fact be 1:1 or higher for non-humans who don’t have the same contextual understanding of their predicament. Assuming a lower ratio may be a holdover from not that long ago when we denied non-human animals having an inner life at all.
Wonderful piece, Noah. Very high density of relevance and comprehensiveness.
We also can’t maximize the utility of aphids without wrecking the utility of ladybugs, because ladybugs eat aphids.
Once you consider the ways that living things wreck each other’s utility for the sake of their own *all the time*, often in ways essential to their continued existence and the functioning of the ecosystem as a whole, it becomes obvious that we can’t realistically maximize utility for all living things and have to limit ourselves to maximizing utility for humans and maybe also our pets.
What affirmative reason is there to believe that an LLM that states that it is happy is experiencing something akin to an emotion rather than mimicking language patterns? What physical mechanism even exists that would enable that to be happening?
I think you're setting the bar for evidence quite high given that we don't even have a great understanding of the physical mechanisms that enable our own emotions.
Good for you, Noah, for addressing this complex set of questions.
My utopia (or contentment) may not be your, or someone else's, utopia (or contentment)...though likely there are some points of overlap. And the way people experience and label suffering also differs, though there are some general agreements. Lessons learned by a hospital chaplain!
If AI could write better novels than really good writers, I suspect many would-be-writers for whom this particular creative expression is a vocation would end up miserable, unfulfilled, and feeling useless in the world. Creating art is healing, builds contentment for recipients and creators, forges human connections and community, and helps us be more self-reflective and other-curious and empathic.
Almost all of your identified limitations with utilitarianism or EA have almost nothing to do with AI though. Having said that, AI can (theoretically) help each society get close to its desired mix of utilitarianism or any other philosophy.
Our version of utilitarianism follows the Sanskrit saying of 'Vasudaiva Kutumbakam' which means 'the world is one family' - and that includes all living things. We try to live this in practice by being more sensitive to higher sentient beings (we may step on a piece of grass or eat a plant, but not a mammal).
Utilitarianism has received a great deal of brilliant 'buttressing' since Bentham's first draft, not least by his follower and friend J S Mill, who provided excellent analytic refinements to deal with the often rather crude, reductio arguments levelled against it. Most of Mill's refinements are different ways of pointing out that in its purest, simplest form, utilitarianism, like any moral system, presents an ideal which needs caveating and adapting to specific situations and the wider human condition.
Nozick's utility monster is frankly daft. Nozick was much given to inventing things that definitely didn't, almost certainly wouldn't and probably couldn't exist, as the premise of thought experiments. All good fun, but hardly persuasive.
One real strength of Utilitarianism is that it is a (probably 'the') consequentialist philosophy. That is, it judges actions based on their consequences rather than on their conformity to a specific rule. It is extremely difficult to argue for any moral position without referring to consequences and Utilitarianism is by far the best thought-through and intuitive moral philosophy focused on the consequences of actions.
The “utility monster” and Omelas “problems” are actually very easy to answer: the utility monster isn’t actually utility-maximizing because wealth has diminishing utility returns in reality, and the Omelas thought experiment is completely unrealistic.
Utilitarianism is a moral philosophy designed for use in the real world, intended to maximize human welfare in the real world. It has no reason to care about unrealistic scenarios and cannot be disproven by nonsense like this. If you’re supposing something that seems to satisfy the formal precepts of utilitarianism but feels unfair, it is invariably because what you’re suggesting wouldn’t actually be utility-maximizing if attempted in reality.
Not only that, many of these objections hinge on the assumption that what feels wrong IS wrong. That’s not necessarily correct! “What feels wrong is wrong” is a distinct moral philosophy which is not utilitarian as such.
The insect thing is also silly, because bugs have very short lives and are constantly being eaten by larger and longer-lived animals. Not only that, we don’t know how they experience the world or how much utility they enjoy even when things go well for them. As such, we can’t realistically utility-maximize for them.
I’ve found that utilitarianism is never wrong when applied realistically, with everyone’s utility given equal weight and all relevant considerations included in the analysis. It is the correct moral philosophy.
"I’ve found that utilitarianism is never wrong when applied realistically, with everyone’s utility given equal weight and all relevant considerations included in the analysis. It is the correct moral philosophy."
No True Utilitarian?
Utilitarianism cannot fail, it can only be failed?
Actually? I believe the answer is yes. Utilitarianism gets the right answer every time, if you keep it within the parameters of things that can actually happen in the real world and if you have enough self-discipline to not selfishly over-value yourself and your own utility.
I’ll grant that including all relevant considerations in the analysis is not always easy. But when trying to bat down an attempted objection to utilitarianism coming from someone in the philosophy department, finding the flaw isn’t usually hard. It’s basically always either “that is an unrealistic scenario” or “utility doesn’t work that way in real life”.
In all honesty, I suspect the reason why utilitarianism takes such a beating in philosophy courses even though it’s clearly correct is because serious utilitarians who are smart and capable don’t become academic philosophers. They become economists, politicians, technologists, business leaders, and doctors, because they can see that a highly capable person’s lifetime spent in one of those fields does a lot more to raise humanity’s aggregate utility than one spent in academic philosophy. Philosophy does have value and it is entirely defensible, even on fully utilitarian logic, to give their departments some funding. But the limits ought to be appreciated here: eventually you need to move away from the theoretical utility function on the chalkboard and look at how people are actually doing in the real world.
What is more important among the following, and how do you measure?
* Suffering avoided
* Happiness obtained (and what do you mean by happiness)
* Fulfilment of meaning / purpose
And add whatever other values of "good" you might have to that pot.
Then answer the question: how do you convert between them? That is rigor demanded by utilitarianism. If you want to do that "locally" and use whichever measures seem to fit the moment, I'm more or less good with that, but that's not utilitarianism.
Utilitarianism tells you that if you are walking to work, and in your pocket you have a blockchain key on a USB drive that contains the money you just collected to save 100 starving children in Africa, and you pass a drowning child, and diving and to save them will certainly save them while certainly destroying your money, and failing to do so will certainly let them die, you should let that drowning child die. That is utilitarianism of the most common type practiced in EA circles. There are plenty who will say letting that child drown is the right decision! I'm far more conflicted than that.
Your last paragraph is an unrealistic scenario.
As to how we balance different forms of utility: we largely do that through market price signals and the democratic governing process. These don’t work perfectly but they work quite well. It’s not realistic to do it perfectly, so the best we can do is all utilitarianism really demands of us.
I completely reject the idea that moral philosophy ought to be a problem set of unrealistic scenarios divorced from reality. The real world is the thing that actually matters and we should think about *it*, not dwell on idle what-ifs.
That's fine. But that's not utilitarianism -- at least not as practiced by the most famous EA folks like Will MacAskill. I inverted exactly an argument he makes in favor of EA: that if you would save a drowning child in front of you, which we all agree we have a moral obligation to do, then we have an equal moral obligation irrespective of distance, familiarity, or other circumstances, and thus you have the same moral obligation to children dying of starvation that you have to a child drowning in front of you.
There is a valid utilitarian argument in favor of prioritizing those nearer to you: you know more about their needs and are better positioned to help them, especially if you have limited resources. You know less about people farther away and have less ability to reach them, so it usually makes sense to entrust their welfare to those who are nearer to them. With that said, we are better able to tell what is going on across the world than we were in the past, so the argument that we should do more for the world’s poorest is absolutely valid. Bill Gates clearly made the correct moral choice by devoting his fortune to public health in poor countries.
But even with that said, saving a drowning child in front of you and donating to help starving children on the other side of the world are never realistically going to conflict. Starvation is a slow killer and in real life you’ll have plenty of time to send your donation after you’ve saved the drowning child.
One correction: I think the drowning child was actually created by Peter Singer, but MacAskill also talks about it.
I strongly believe that a unit of avoided suffering should be weighted much more than an additional unit of happiness and that you cannot offset one person’s suffering with another person’s additional happiness. The neutral point of personal suffering/happiness is a profound transition in quality of life. Under-appreciated perhaps.
Under that logic, levying taxes on childless people to pay for schools would be morally indefensible. For that reason I can’t agree with you on that score. Sometimes it is totally valid to impose some suffering for the greater good, albeit only if the good being done clearly outweighs the suffering. When “the ends justify the means” fails morally, it’s because the damage from the means was in fact greater than the good of the ends. I’ll grant that it’s important for people to be careful about this, because it’s hard to predict in advance whether the desired ends will pan out as hoped. But some forms of “impose a lesser suffering for a greater happiness” are well established and proven, like levying taxes to pay for public goods.
Seems reasonable to me in most circumstances.
To what extent is the Omelas "completely unrealistic"? It can be readily argued that the current world is running on quite similar a setup, where the suffering of the labors in the developing countries on the supply-chains of many essential technologies relied on by the OECD citizens are indispensable.
But ultimately what has evolved is the point of moral philosophy itself. Since Rawls, it is no longer about "what I think is correct" but "what can be accepted by all as reasonable". Because in the end moral philosophy needs to be a basis people can live upon, and in real world people live together and negotiate together what that basis should be, and no one has the time and wisdom to apply utilitarianism correctly all the time.
We source raw materials from the US, Canada and Australia just fine without abusing anybody: natural resource extraction in wealthy countries tends to be done by a small number of workers operating heavy machinery to raise their productivity sky-high, allowing them high wages, and they also benefit from workplace safety rules.
In poor countries natural resource extraction is typically done by large numbers of low-paid workers using small tools without workplace safety rules, but it does not have to be that way and as these countries escape their premodern economic setup and grow their economies, their natural resource extraction operations will look more like those in wealthy countries. There is no supply chain that *needs* to be exploitative.
A utopian city whose supply chains in total do not need to be exploitative I feel is as fictional as Omela. In reality, the supply chains are exploitative because they need to be, to stay in market competition, and to keep inflation under control.
Supply chain does not mean you can pick and choose freely which parts come from where. Barely any electronics supply chain at scale can bypass China. Many manufacturers of electronic assembly in China are giving hourly rate below 20 RMB (US$3) to hordes of laborers, with factory-provided living conditions similar to company towns in the US history. There are so many laborers though, if they do not keep wages down they may lose orders to the factory next door which does. Are they applying utilitarianism right?
For all the tariffs and intentions to source from elsewhere, you may end up being scammed by a drop shipper and get made-in-China stuff anyway, see SmarterEveryday's attempt at making a grill scratcher entirely in the US. He ended up with a very good but very expensive grill scratcher. For cheaper mass marketed goods, you don't have alternatives to Chinese factories. Are US businesses relying on Chinese factories applying utilitarianism right?
By deontologist logic, labor laws are good and China should enforce them. But many Chinese economists claim that by consequentialist logic the workers are better off with a job than without, and China *needs* to keep their wages down to grow even more.
Many factories are built on land confiscated from farmers (who then died of execution or starvation) over the years, in the name of Revolution for a fairer world. Now they also become part of cheap China manufacturing.
The Chinese government then justifies all that by claiming that, they will in the end achieve dominance of world market and Chinese people can share in the glory while world population can share in the cheap products. They in fact succeeds to an extent in converting national glory to people's utility by investing in nationalist education.
Is the Chinese government applying utilitarianism right?
I have no confidence I can tell if someone has applied utilitarianism right or wrong in any realistic scenario.
It is perfectly possible to build a factory on land that was bought from a farmer who freely sold it; that’s how stuff in the US normally gets built.
Manufacturing jobs tend to shift out of high-wage countries into low-wage countries that have managed to get enough security and good policy to be worth investing in. Manufacturing jobs at wages that we consider shockingly low tend to be better paid than what people in premodern economies get from subsistence agriculture, so people flock to these jobs without any coercion. Nobody in the US is a poor subsistence farmer anymore and we, as a people, have forgotten how awful that lifestyle actually is.
China is at the point in the growth cycle where they ought to be shifting out of manufacturing into more services, but Xi is against this; it would be much more utilitarian to let the Chinese labor force shift into more services at this point but the CCP government has never been utilitarian, they have always prioritized the utility of their highest elites over that of everyone else. They are exploitative because they don’t think that the people being exploited matter. A good utilitarian would say that they matter. You keep asking if they’re applying utilitarianism right, but they’re not applying it at all.
I forget who pointed out that obviously, by any objective standard, Omelas is a vastly superior society to ours, or to any society that has ever existed or is ever likely to exist.
You're telling me Omelas has 9,999 profoundly happy people and 1 utterly miserable person? Do you think that any society in all of history has ever been 99.99% profoundly happy, or even 90% profoundly happy? Do you think your preferred set of policies could deliver a society in which fewer than 1 out of every 10,000 people were miserable, and all the rest lived in near-constant euphoria? Is your own life 99.99% profoundly happy? Would you want your own life to be 99.99% profoundly happy, and if your answer is "no", are you stupid?
The story is remembered for its usefulness to those looking for reasons to reject utilitarian logic, because it frames stunning social success as if it were somehow sad and monstrous.
Our society is superior to Omelas by one perfectly objective standard: it is not intentionally structured around torturing children in order to make the rest of the population flourish. The starting point of the Omelas story is the judgement that it is wrong to torture children, even if it increases net happiness. If you don’t share that judgement, of course Omelas won’t move you.
Hmm, did even one child suffer, perhaps via illness, in the process of mining, manufacturing, and/or polluting activities that created and powered the device you used to write this comment?
So, umm....yeah, there is like a tiny difference between unsafe working conditions and/or untaxed externalities and forced permanent imprisonment with infinite torture.
Joss Weedon's meta-horror film The Cabin In the Woods explores this idea with a world in which a small number of children/ teenagers have to be sacrificed periodically to appease ancient gods and stop them from destroying the World. The perspective is split but pitched asymmetrically in favour of the sacrificial youths over against the jaded, results-focused bureaucracy engaged in serving up the sacrifice according to the rules of the deal made with the ancients. I ended up thinking that even the survival of everything may not be worth the price: of course it's easy to say that when you're sat at home on your sofa watching a film; not so easy when huge, vengeful divinities are smashing the world to pieces!
Utilitarianism is basically Newtonian physics. It works pretty well on a small, well defined problem ("should we let someone build an apartment here that will make 50 new residents very happy but five neighbors very sad?"), but quickly falls apart when you try to make it a full theory of the universe.
It's a philosophical "one weird trick" that appeals to a certain set that think they can solve the entire problem of ethics via some single-variable optimization, who are very pleased with themselves that they can answer the "is it worth flipping a 49/51 coin to make everyone twice as happy/destroy humanity" by doing math.
"GDP, at its most simple, is a measure of the degree to which people in our society get what they want."
Another example of the tendency of economists to view reality on the macro rather than the micro level. As you point out later in your essay, 90% of the members of a high GDP society could be impoverished, miserable, enslaved, and barely clinging to life.
Yes, we have to know the GDP plus the society's Gini coefficient.
Can you give an actual example of “a high GDP society” that is “ impoverished, miserable, enslaved, and barely clinging to life?”
And to be clear, I mean real per capita GDP, not nominal GDP.
My guess is that you will find when you look at actual nations, all those problems are strongly inversely correlated with real per capita GDP.
I gave an extreme example to gesture at the current American situation. American GDP continues to grow at a healthy manner while wages for the bottom 50% of American workers have essentially flatlined in the last 40 years. There's a big GDP but the way it is distributed is increasingly unequal.
Do you have a source for flatlining wages?
Everything I have been able to find shows something like this, where real wages have grown in that time frame for all income groups.
https://www.visualcapitalist.com/growth-in-real-wages-over-time-by-income-group-usa-1979-2023/
No, you did not give an example at all. You just gave hysterical hyperbole.
If you seriously think that the American people in 2026 are "impoverished, miserable, enslaved, and barely clinging to life," then you are completely detached from reality.
"Flatlining wages" is not even close to "impoverished, miserable, enslaved, and barely clinging to life" and you know it. Nor is it true that "the bottom 50% of American workers have essentially flatlined in the last 40 years."
Inequality is a totally different issue from being "impoverished, miserable, enslaved, and barely clinging to life." There are dozens of highly unequal societies where the vast majority of people are living well. In fact, it is the norm in Western societies and increasingly East Asia.
For me, utilitarianism always fails on two simple grounds:
1) It pretends that whatever you might think "the good" is (QALY, hedonism, etc.), that all things can be converted into an objective measure of that as if they were as objective as length, and it's simply measuring in inches or meters or kilometers.
2) That there is some objective "the good." If there is no objective "the good" then what exactly are we measuring again? A moral realist generally doesn't have a problem with this point; they'd simply argue over what the objective good is. I'm not a moral realist.
Quite opposite to the end of utilitarianism, I believe those questions are exactly where the fun with utilitarianism begins. That is the marginalist revolution of utilitarianism. It is exactly when the different subjective "good"-s interact that we observe compromises, and to the spirit of utilitarianism, tools are developed to work out how and whether the compromise is guided by some kind of common "good". Arguably that goes into liberalism, but utilitarianism is guiding too. There are many papers along that line.
So what is your preferred moral standard?
It seems to me that you could reject *any* moral philosophy or standard on nearly identical logic, by saying that it’s too subjective to be knowable at all what right and wrong are.
I find that profoundly unsatisfying, and also likely to get bad results.
It seems to me that you can in fact get a usable approximation of objective population-wide utility values via market price signals and the results of free and fair elections.
cp6 asks the right question — if there's no objective "the good," what's the standard? — and I don't think the answer has to be "there isn't one."
Drop the assumption that a standard must be a quantity to maximize, and a different kind is available: a direction. Do a person's or a society's values become more coherent while taking more into account, or do they hold together by leaving things out? That's checkable without a common currency. It doesn't need to convert suffering avoided into meaning gained; it asks whether the process that weighs them is widening or narrowing.
It also explains the track record LV points to. Utilitarians were first on women's rights, animal welfare, and the rest — but what carried them there was the widening, not the sum. The sum is what produces the utility monster and insects-over-humans. Keep the direction, drop the arithmetic, and you keep the part that had the good track record.
And it answers cp6's own mechanism. "Market signals and democratic process" aren't a calculation of utility. They're a process by which many valuers revise what they want under contact with consequences — which is the standard, not a stand-in for it.
wrt "[putting] a lot of weight on the well-being of our descendants in the far future" I agree with what Keynes wrote in his 1906 unpublished manuscript on Burke:
it can seldom be right… to sacrifice a present benefit for a doubtful advantage in the future…. It is not wise to look too far ahead; our powers of prediction are slight, our command over results
infinitesimal. It is therefore the happiness of our own contemporaries that is our main concern; we should be very chary of sacrificing large numbers of people for the sake of a contingent
end, however advantageous that may appear…. We can never know enough to make the chance worth taking. There is this further consideration that is often in need of emphasis: it is not sufficient that the state of affairs which we seek to promote should be better than the state of affairs which preceded it; itmust be sufficiently better to make up for the evils of the transition…"
The data center controversy is a simple case of different people disagreeing about the utility we can expect from AI.
Some people think that AI will do great things for human utility that will massively outweigh the disadvantages, and some people think AI will either give us a dystopia or wipe us out. Some are pointing out the possibility that AI will only have modest impacts on human utility. As of the current time, we don’t know who’s right.
The problem here isn’t with utilitarian moral philosophy, it’s figuring out what the utility tradeoffs of AI and data centers actually are.
I think a lot of what this diagnosis identifies is less a problem with utilitarianism and more a problem with the utility function *as specifically defined by various optimization vectors at the aggregate population level*.
These tend to share the common error that they don't 'bake in' multidirectional shapes as the object being optimized. The less abstract way of saying that is that — given most observed utility functions involve multiple competing axes — we should be extremely suspicious of any construction of utilitarianism that selects an attribute or state or quantity and parameterizes the optimization function along that particular axis at the system level, devoid of any internal equilibrators.
I'm not clear on whether Noah's argument here is that utilitarianism is worse than some existing known alternative, or just that it's not yet a fully complete solution to moral philosophy and there are more things we still need to figure out.
If the former, I think Richard Chappell put up a good counterargument: all known competing theories are also vulnerable to many of the problems Noah brings up here. To the extent this isn't obvious, it's because other theories are judged by a much lower standard and aren't asked to consistently provide rigorously defensible correct answers. https://www.goodthoughts.blog/p/puzzles-for-everyone
If the latter, then every moral philosopher already agrees with that, but then the framing of the article is kind of odd and gives the impression of wanting to selectively distance from a thing that's currently getting bad press.
The issue does not really have to be put into fancy philosophical terms.
Is there a distinction between what people want and what will make them happy? And, if so, who ought to be given the authority to make that call?
I am going to guess that there is a strong correlation between social class and responses to that. People surrounded by people who have gotten massive amounts of what they want, yet are not happy, are far more open to the idea that there is a crucial distinction. People who have largely seen a life of not getting what they want, not so much. It's the money and the stuff I want and lack that ought to be blamed for unhappiness.
So... arguing the point in the language of the well-educated is likely to skew the argument by excluding others. And elites are going to think that some authority other than people would do better at making people (especially other people) happy.
My skepticism is rooted in my experience of being a liberal in a red county. When I argue politics, I tend to take the nuanced view that I disagree with my county, but I respect their views. Invariably, my fellow educated blues tell me that my neighbors are voting against their own interests.
Think about that claim for a moment. The arrogance of being certain that one knows what is in the interest of others. "I know what ought to matter to these unmet people and I know far better than they know themselves. No need to walk in their shoes, I know what they are and what they should be."
I have no doubt that all of us make errors in our pursuit of happiness, but I have extreme doubts that distant authority would make fewer errors. Or be motivated, for long, to even try.
Awesome to see a moral philosophy post from Noah! Anti-utilitarian moral philosophy is my day job, and I thought you made great points.
On animals:
“We generally assume that this is causing less total suffering than torturing similar numbers of humans would.”
Although this is far from proven and the ratio may in fact be 1:1 or higher for non-humans who don’t have the same contextual understanding of their predicament. Assuming a lower ratio may be a holdover from not that long ago when we denied non-human animals having an inner life at all.
Wonderful piece, Noah. Very high density of relevance and comprehensiveness.
We also can’t maximize the utility of aphids without wrecking the utility of ladybugs, because ladybugs eat aphids.
Once you consider the ways that living things wreck each other’s utility for the sake of their own *all the time*, often in ways essential to their continued existence and the functioning of the ecosystem as a whole, it becomes obvious that we can’t realistically maximize utility for all living things and have to limit ourselves to maximizing utility for humans and maybe also our pets.
What affirmative reason is there to believe that an LLM that states that it is happy is experiencing something akin to an emotion rather than mimicking language patterns? What physical mechanism even exists that would enable that to be happening?
I think you're setting the bar for evidence quite high given that we don't even have a great understanding of the physical mechanisms that enable our own emotions.
It may be that the best way to predict the proper emotional reactions in language is to simulate the experience of having those emotions
Good for you, Noah, for addressing this complex set of questions.
My utopia (or contentment) may not be your, or someone else's, utopia (or contentment)...though likely there are some points of overlap. And the way people experience and label suffering also differs, though there are some general agreements. Lessons learned by a hospital chaplain!
If AI could write better novels than really good writers, I suspect many would-be-writers for whom this particular creative expression is a vocation would end up miserable, unfulfilled, and feeling useless in the world. Creating art is healing, builds contentment for recipients and creators, forges human connections and community, and helps us be more self-reflective and other-curious and empathic.
'Do not do unto others as you expect they should do unto you. Their tastes may not be the same'. George Bernard Shaw
Excellent philosophical piece!
Almost all of your identified limitations with utilitarianism or EA have almost nothing to do with AI though. Having said that, AI can (theoretically) help each society get close to its desired mix of utilitarianism or any other philosophy.
Our version of utilitarianism follows the Sanskrit saying of 'Vasudaiva Kutumbakam' which means 'the world is one family' - and that includes all living things. We try to live this in practice by being more sensitive to higher sentient beings (we may step on a piece of grass or eat a plant, but not a mammal).
Whatever it is, AI can help us get closer there.