On the wisdom of the historians
Just as in economics, beware untested theories.
When I first started blogging, I mostly wrote about my frustrations with the field of macroeconomics. After having endured several years of education in that field, I was exasperated with the way unrealistic theories became conventional wisdom and even won Nobel prizes while refusing to submit themselves to rigorous empirical testing. The theory-first culture of macro was absolutely antithetical to the ethos I had learned as an undergrad physics major. I think one reason my blog caught on was that the financial meltdown and the Great Recession had made a lot of people angry about how poorly these celebrated models performed in the crisis. Suddenly, models that might have been passed off as abstract thought-experiments a few years earlier were being confronted with cold, stark reality — and were falling short.
Then I got into the world of Twitter and media and political commentary, and I started encountering historians. Though I never studied history, when I saw the way that some professional historians applied their academic knowledge to public commentary, I started to recognize some of the same problems I had encountered in macroeconomics.
A lot of people are talking about the history profession this week. There was a kerfuffle when James Sweet, the president of the American Historical Association, wrote a rambling and somewhat opaque post criticizing what he felt was his profession’s excessive focus on the politics of the present, and singling out the 1619 Project for criticism. A subset of historians predictably flew into a rage at this, and forced Sweet to issue a stumbling apology.
I’m not particularly interested in the “woke vs. anti-woke” politics of this dispute. But I think a big part of the reason people care so much about the goings-on in history academia is that in recent years, history professors have become some of the most important voices that we look to in order to understand our current political and social troubles. Jay Caspian Kang explained it well in a New York Times column today:
Over the past decade or so, history has become the lingua franca of online political conversation. This is a relatively new phenomenon…[T]he shift has something to do with the centrality of Twitter over the past decade (historical documents and photos make for great screenshots) and, more important, the changes in the country itself. Once Donald Trump became president, it was harder to write about “Breaking Bad” and Taylor Swift in such self-serious tones…
Twitter has also allowed historians to assume a place in the public discourse that would’ve only been available to a select few before the advent of social media…As a result, history does seem to have an unusual amount of weight in the public discourse.
In the wake of the Great Recession, we talked a lot about whether economists and their theories were afforded too much credence, but as far as I can tell there has been no similarly critical public discourse about academic history. But there ought to be. Just as economists became a sort of priestly order that we relied upon to tell us how to achieve prosperity and distribute resources in society, historians have become a sort of priestly order that we rely on to tell us about where our politics are headed and how we should think about our sense of nationhood.
This is not a blanket criticism of the history profession (although some people on Twitter are certain to interpret it as such, and react accordingly). I am not saying that history needs to stay out of politics and go back to the ivory tower. Nor am I saying that our current crop of historians have bad takes on modern politics. All I am saying is that we ought to think about historians’ theories with the same empirically grounded skepticism with which we ought to regard the mathematized models of macroeconomics.
Historical analogies are theories
One frustrating thing about demanding empirical confirmation for historians’ theories is that some (many?) historians will insist that they don’t make theories at all. For example, when I brought this issue up on Twitter, Bret Devereaux — a historian of the ancient Mediterranean who is currently a visiting professor at the University of North Carolina — asserted that historians don’t make predictive models:
But let’s take a look at the article Devereaux cites as an example:
No ancient Greek would have had any difficulty in understanding the scenes that Americans watched on January 6: the inciting speeches, the marching mob, the insurrectionist attack on the seat of government…
[T]he Greek experiment in self-government ran for centuries and was repeated hundreds of times in different ancient Greek states. That robust “data set” allowed more observant Greeks to notice recurrent themes both in how self-government functioned and how self-government failed…
Another key lesson from this history should be even more sobering: Would-be tyrants keep trying until they succeed.
This isn’t stated in mathematical equations, but it’s undeniably a predictive model of how politics work. Deveraux starts by asserting that ancient Greece is an appropriate analogy for our modern politics. He then cites the experience of ancient Greek city-states as a “laboratory” that conducted an “experiment” and generated a “data set”. He asserts that this “data set” allows us to predict how self-government fails. He then claims an explicit law of human society: “Would-be tyrants keep trying until they succeed.”
Now, I suppose it’s technically possible that Devereaux is making a purely definitional claim — he might be saying that if someone eventually gives up trying to seize power, they weren’t really a “would-be tyrant” after all. If this were all he was trying to say, then we could conclude that, for example, Richard Nixon — the man who famously said that “When the president does it, that means that it is not illegal” — must not really have been a would-be tyrant after all, because he eventually resigned.
But such a purely definitional, “no true Scotsman” argument would be utterly vapid and not worth writing a column about. It is overwhelmingly likely, therefore, that Devereaux is arguing that we can have get some indication of who is a “would-be tyrant” in advance, and predict that this would-be tyrant will not give up trying to seize power. That, folks, is what we call a predictive theory.
Devereaux argues that these theories are not actually predictive theories because they are qualified, contingent, and probabilistic:
But to argue that this isn’t what theorists in other social sciences do is to fundamentally misunderstand disciplines like sociology, economics, and political science. Theorists in those disciplines also present menus of possible outcomes, and allow for contingent elements. Essentially no one thinks that they have invented a social-science version of the Standard Model of Particle Physics that applies perfectly in every conceivable situation.
In short, Devereaux claims that he describes only “what was”, not “what will be”, but by using ancient Greece as a historical analogy to predict the behavior of Donald Trump, he is clearly making claims about what possible futures are more likely.
In fact, though I don’t want to pick on Devereaux too much, I should point out that he has done this many times. In a column on Russian war crimes in Ukraine, he makes a explicit historical analogies with various armies of the past, and uses these analogies to predict that Russia’s atrocities are likely to harden opposition against its invasion. In a column on the U.S. withdrawal from Afghanistan, he uses an analogy with the Roman army to predict that the U.S. military would be more effective if it reorganized itself in certain ways.
Nor is Devereaux a singular example here. You can find plenty of examples of historians using historical analogies to make predictions about the feasibility of various political strategies.
I am not claiming that these analogies are misplaced or that these predictions and recommendations are wrong. I am merely recognizing that historical analogies, when used as these historians do to make either predictions and/or recommendations for the present, are social-science theories.
And theories ought to be subject to empirical tests. To go back to Deveraux’s theory of would-be tyrants, what are we to make of Richard Nixon stepping down and leaving politics? Or the historical list of dictators who gave up power? Or other would-be leaders who had dictatorial aspirations but who gave them up and went into other lines of work? Before we conclude that “would-be tyrants keep trying until they succeed”, we should rigorously and systematically check the historical record to see if we could identify, ex ante, a set of characteristics that allowed us to predict who would keep trying to seize power and who would give up.
This is hard, often unrewarding work, yes. But hard, often unrewarding work is what social scientists do.
Chameleons in economics and history
Devereaux’s assertion that he only chronicles the past, rather than predicting the future, was echoed by others, for example Lora Burnett, formerly a professor at Collins College:
Don’t be afraid of the big words here — “nomothetic” just means something that tries to discover laws of the Universe, while “idiographic” means something that simply chronicles specific historical events without claiming any lessons for any other events. In fact, I’ve read quite a few history books that merely chronicle what happened in the past. But as I noted above, there are also plenty of examples of historians claiming in the media that examining the past can teach us lessons that we can apply to our present situation.
Applying the lessons of history is a fundamentally nomothetic exercise — not an attempt to create a Standard Model that applies in every conceivable situation, but nonetheless a very clear attempt to apply the lessons of history to more than just the specific events that generated those lessons.
I am reminded of economist Paul Pfleiderer’s recent paper complaining about so-called “chameleon” models in finance and economics. “Chameleons”, as Pfleiderer calls them, are theories that the creators claim have no real-world applicability and are just thought experiments, but which they and others later try to apply to the real world. He argues that before actually applying a model, we need to test it empirically in some way, in order to have some reason to think it’s a useful and applicable model.
Similarly, if historians generate historical “descriptions” that they claim are applicable only to the past, and then turn right around and write mainstream media articles deriving “lessons” for the present, they are creating chameleon models.
Note that I am not arguing that we shouldn’t try to draw lessons from the past! Instead I am arguing that when we do try to draw lessons from the past, we should have some sort of empirical procedure for testing whether and when those lessons are good or bad ones. Otherwise, historians are just free to use their personal judgment — or their personal politics — to pick and choose whichever historical examples they feel like analogizing with the present day.
And if we don’t have any such empirical testing procedure in place, then we should view the predictions and policy recommendations historians make in the media as punditry rather than as academic knowledge.
Correlation vs. causation
In fact, I’ve seen some examples of historians going beyond the analogy approach and explicitly creating causal theories of the type that political scientists, economists, or sociologists would create — but with much less empirical verification than scholars in those other social sciences would insist on.
The most notorious and important example of this is the so-called “New History of Capitalism”, in which historians such as Cornell’s Ed Baptist and Harvard’s Sven Beckert allege that American slavery was a crucial factor in igniting the Industrial Revolution.
The historical work in the New History of Capitalism (or “NHC”) contains its share of methodological weaknesses, which economic historians Alan Olmstead and Paul Rhode critique in a 2016 survey paper. But a deeper question is whether the causal claim of Baptist, Beckert et al. is true. Yes, America used slavery to produce a lot of cotton. Yes, that cotton was often used in the textiles industry during the early Industrial Revolution. But does that mean that without slavery, the Industrial Revolution wouldn’t have happened, or would have happened later, or would have been more difficult? This is the question that many economic historians have asked.
Ed Baptist and the other historians of the NHC school have vigorously refused to even contemplate this question. Things happened as they did, they argue, so why should we ask if things could have happened otherwise?
The answer is that correlation is not causation. If I eat a slice of cake every morning and I lose 20 pounds, does that mean that eating cake is a good weight loss technique? Certainly not. In fact, there is pretty good evidence that slavery slows down economic development, by saddling a country or region with bad institutions. It’s also the case that after slavery was abolished, the Industrial Revolution accelerated. So although the matter is far from settled, it’s highly likely that rather than being a pivotal factor in the creation of modern capitalism, American slavery actually held back the development of modern capitalism. (Update: Note that this is far from a settled debate! A paper was just released showing that British regions with more connections to the overseas slave economy industrialized slightly faster in the early 1800s, raising the UK’s total GDP by about 3.5%. So economic historians are still puzzling this one out.)
Now, that doesn’t mean that the work of Baptist et al. is useless. Indeed, Matthew Desmond’s chapter in the 1619 Project, which drew heavily on the NHC work, focuses on the ways that slavery left its malign imprint on U.S. economic institutions. And in fact, this is perfectly consistent with the empirical work on slavery and institutions, and it’s something we should definitely care about in the present day.
But at the same time, the NHC historians’ causal theory about the genesis of industrialization is something that should be subject to rigorous empirical testing, rather than simply assumed from a temporal correlation. To refuse to test this causal theory makes us vulnerable to catastrophic causal misinterpretations. We might conclude — incorrectly — that broad economic prosperity depends crucially on forcibly extracting labor from human beings, thus tricking us into thinking that society must choose between poverty and slavery. That mistake would be a recipe for big, big trouble.
In general, historians should avoid to make assertions about causal mechanisms based purely on temporal correlations. For example, Columbia historian Karl Jacoby recently noted that integration coincided with the underfunding of higher education, heavily implying that the former caused the latter:
Now, maybe Jacoby was merely making a statement about unfortunate timing. I don’t want to put words in his mouth here. But from the way this tweet is worded, it sure seems like he’s saying that state governments became less interested in paying for college kids’ education when those kids started being people of color.
And this is a proposition we could actually test! For example, we could probably look at the differences in college funding between more segregated and less segregated states before and after Brown v. Board of Education (this is known as a differences-in-differences approach). Or we could find other factors that integrated colleges in different states at different times and see how state funding decisions responded. Whatever the method, though, it’s important to test this sort of causal hypothesis before concluding that we believe it. This is what political scientists do, and if historians want to horn in on the territory of political scientists, they should be held to the same standards.
Social science being the flawed and inaccurate endeavor that it is, empirical tests will never give us perfectly definitive answers to questions of historical causality, the way they will in physics or chemistry. But we should still do them. To rely on the personal judgment and wisdom of sages — even brilliant sages who teach at Ivy League schools — is simply not enough when deciding what to believe about the forces that shape the destiny of our society and our nation. Empiricism — the forced comparison of theory and reality — engenders a kind of humility that no amount of circumspection can match.
Update: Bret Devereaux has a response post, which is a little nastier and more personal than I’d like, and which (perhaps predictably) I think doesn’t really manage to address my critique. I may revisit this in a future post.
Andrew Gelman has some thoughts as well, incisive as usual.
Historian Anton Howes has a post making somewhat related criticisms of his field:
And here is a follow-up post by Ian Leslie, discussing some truly shoddy history work that became conventional wisdom in the U.S. media":
If historians think their field is only about recording past events, there's no reason to listen to their opinions about current events. They're effectively saying they have no more expertise on the matter than my hair stylist.
This is such a good article, I really appreciate how you took the time to explain the steps in your thought processes clearly.