> And if you can't cite two papers that serve as paragons or exemplars of the vast literature, it means that the knowledge contained in that vast literature must be very diffuse and sparse. Which means it has a high likelihood of being a mud moat.
For the counterfactual idiocy in me, I can't help but think of the recent ATIS article on messing around with bad ideas or sparse literature (not as a good foundation but as greenfield research) https://atis.substack.com/p/embrace-interesting-ideas
If we were to try and "do our own (fun) research" are there similar rules that can be applied (other than avoiding comspiracies)?
"Voila - a peer-reviewed literature chock full of misinformation. In practice, I doubt anyone ever does this intentionally. It takes too much coordination and long-term planning."
And yet that is what the Kochs have funded over 50 years or so of libertarian economics propaganda. Pretty much all of Austrian economics, Public Choice economics and large amounts of Chicago economics have been funded that way. And there's a wealth of critiques of their basic reasoning that has been emerging recently, such as Donut Economics and Ha-Joon Chang's books. Most philosophy, libertarian and otherwise, works similarly.
I was going to write more or less this exact comment :-) Noah wrote a decent article about this once but from a semi-defensive stance of "not all economists". He concentrated on the ratio of right wing quacks to others among economists being low and the amount of right wing quackery posing as serious intellectual work having declined in recent decades.
These are true. But there's still the vast literature you're talking about that's plutocratic ideology with mathy dressing. The default econ 101 book that drives a lot of economic intuition among college graduates is still from Mankiw, who may do some respectable work but also is a total fraud ideologue as a public intellectual. And you can see in the public debate in the US that the overclass has wildly succeeded in leveraging their bought "research" literature to inculcate as public intuition the self-serving myth that encouraging plutocratic rent extraction and crushing everyone else is, ironically, the only path not towards serfdom.
I'm not sure that would work with every field. If suddenly string theory became a political battleground, how much would citing "String theory dynamics in various dimensions" and "The Large N limit of superconformal field theories and supergravity" actually help you to understand if string theory is a worthwhile research subject ?
String theory is obviously a pretty extreme example, but econ can get very math heavy, very fast and I personally would not be comfortable judging whole economic theories by reading two recommended papers.
This is my concern too. I really like this idea for cases where the literature in question is sufficiently close to your own comfort and expertise to be able to easily judge its quality. But it's likely to lead to an overly dismissive response to work that approaches problems very differently from work you're familiar with.
If you're trying to convince a non-economist that microeconomic theory is valuable, it would probably be hard to do so by sharing two excellent, insightful, theoretical papers. Any work from the past 50 years would, at a minimum, use utility functions without explaining what they were (and for more recent work, use properties of general utility functions), and pretty much any work I can think of would do some combination of assuming a continuum of agents, working with a representative agent, or having a very limited number of "types." It would be totally reasonable for someone who isn't immersed in this work to read a microeconomic theory paper, conclude that it's a bunch of obscurantist math based on absurd assumptions, and walk away.
The solution probably wouldn't be to hand someone early work developing utility theory or talking about marginal decisions, because that work is highly abstract, theoretical, and old. Even if it was understandable, it wouldn't be clear why it's useful.
Instead, what you'd want to do would be to tell someone to read a good introductory econ textbook, and then chase it down with a good intermediate econ textbook, and then read a few clear, simple theory papers by someone like George Akerlof or Kaushik Basu who knows how to write and does interesting, simple models. Which is to say--you'd tell someone to go to college and major in Economics.
I have a PhD in statistics and worked with economists in the past. I'd actually agree that economic theory is (predominantly? mostly?) "obscurantist math based on absurd assumptions".
Econ isn't a physical science. The fact is you can literally always take a theoretical economic model, change a few assumptions or parameter distributions in plausible ways, maybe add in one or two new totally plausible terms/inputs, and create an equally plausible theoretical model that "proves" exactly the opposite ideological conclusion as the original.
In recent years there's more expectation that a model author not totally ignore predictive power. But even there, you have freedom to invent arbitrary models. Even worse, you can publish respected papers where, for example, you simply rename your predictive error as technological development or productivity shifts. Back predictive accuracy is hardly a major check.
But the point isn’t prediction. People who only care about prediction (say, quant hedge funds) use deep learning and other ML. The point of models is inference and policy insight/recommendation.
But there is no reasonable insight if the model is empirically wrong. As you said, the space of models is too large, without some discipline, like prediction, they are just empty. There is zero point about inference if the model is misspecified, and it always is. It's crazy how people care about standard errors of fundamentally wrong models. Deep learning and other ML does not work that well when predicting returns, there is just too much variance without feeding it good features, and creating good features is just creating good priors for the functional form.
I think microeconomic theory is really useful, even though I agree with your specific criticisms.
Economic theory is about making assumptions about how the world works, expressing them mathematically, and then doing a proof/solving the model in order to draw conclusions. In my opinion the way to get value from this is to take the assumptions as seriously as the conclusions.
In other words, the idea isn't that you draw a conclusion and say "I know that this is true in the real world because the math backs it up." Instead, the idea is to tell a story about how the world might work, but to do so as precisely and rigorously as possible. From this perspective, changing some assumptions and getting a completely different answer is exactly the point: once you've done this exercise, you've potentially identified which assumptions about the world are most important for the question you're trying to understand.
So, for example, the first welfare theorem proves that when individuals make choices to maximize their utility, are small enough components of every market so that their decisions do not affect prices, produce and consume goods with no externalities, and hold no private information, market equilibria will be pareto optimal.
It would be absolutely wrong to read this as a theorem proving that markets are efficient. Anyone telling you that the first welfare theorem proves that markets are efficient would absolutely be engaging in "obscurantist math based on absurd assumptions" in order to push an ideological agenda. What the theorem actually does is points at the factors that are most likely to cause economic inefficiency, which has helped focus policy analysis across a broad range of fields, has helped to emphasize similarities among seemingly very different problems, and thus allowed us to apply lessons across multiple domains.
For what it's worth, my sense is that the field is in some ways moving away from demanding that models have predictive power. At least in labor economics, approaches that try to directly test theoretical models against data (structural estimation) have become much less popular in the last 20 or so years, in favor of methods that try to plausibly estimate directly policy-relevant parameters in particular contexts using natural experiments. Theory has been used much more to tell stories that help explain empirical findings, suggest limits to the applicability of findings, or understand why different contexts give contradictory findings. I think it's really useful for all of this stuff!
Fair call on the wording. In prediction I’m including looking at how wide your error distribution is when inferring parameters.
On the rest, IMO you're explaining the attitude among non fraudulent economists that leads them down the wrong path and creates an atmosphere where ideologues can gain respect with their mathy propaganda.
Like I said originally, economics is not physics. Making up a bunch of (potentially differential) equations and then giving the variables names and solving for various things doesn't prove anything or reveal insights. It's just a math and story time game.
To your example, FWT is a lot of games with basic convex optimization of inner products from arbitrarily defined vector spaces. Just to take two of many examples. The whole thing embeds Pareto optimality as the correct measure. Which is dubious. Then waves hands over the concept of utility. Any attempt to dive deeper there is, more or less, putting mathy obscurantist frosting on moral philosophy. There's a reason economics was a sidekick to moral philosophy for the first few centuries.
It's a good point and a useful tactic. One risk is that the person who is giving you the two articles may be part of a subgroup within the supporters of the ideas in question which is particularly bad.
A while back I agreed to debate an Austrian and asked him in advance for an account of the position he would be defending. He pointed me at a book by Rothbard. It was clear from that that the views in question were wrong [http://www.daviddfriedman.com/Ideas%20I/Economics/Critique%20of%20a%20Version%20of%20Austrian%20Economics.pdf], but I also concluded that Rothbardians were only one faction among Austrians and I am still not sure if the movement as a whole has serious problems.
One other approach to your problem that I have been thinking about is looking for work that you can audit. I recently did a back of the envelope calculation of the increase in the amount of useful land from climate change shifting temperature contours towards the poles. I then looked at a couple of published articles on the subject. They were much more sophisticated than mine, looking at things like soil quality and humidity as well as temperature, which was the only variable I considered. One got a result similar to mine, a huge increase, one concluded that the amount of arable land might go up or down a little.
Their procedure is much better than mine but mine had one large advantage — it was simple enough so a reader willing to spend a few hours could check it, see what I was doing and that I did it honestly and competently. There is no way anyone outside the field could do the equivalent for either of those papers, and it would take more than a few hours even for someone in the field. Given both the fact that the issue is politically loaded and that the two professional papers disagreed so much, the fact that I can't check them for myself is a strong argument in favor of looking at simpler models like mine instead of sophisticated ones.
I think the principle generalizes. Try to find work with an important result that you can check. If it turns out to be bogus — I'm thinking of one example in the climate field — and other people in the field haven't pointed it out, that tells you something useful about the quality of work in the field.
It works for people too. You wrote things about Adam Smith, trying to represent him as a modern progressive rather than the 18th c radical that he was, that could not have been written by an honest man who had read the texts he was quoting. [http://daviddfriedman.blogspot.com/2021/07/noah-smith-on-adam-smith.html] That sharply reduced my interest in reading other things you wrote on other subjects, although in the case of this post I thought you were making a good point.
"Suppose you and your friends wanted to push a weak argument for political purposes. You could all write a bunch of papers about it, with abstracts and numbered sections and bibliographies and everything. You could cite each other's papers. If you wanted to, you could even create a journal, and have a peer review system where you give positive reviews to each other's B.S. papers."
Climate change?
To be fair, it appears to me as an outsider to academe, that this is always a risk. Academics have always to try to build a reputation, which requires publishing. Academics will try to establish their own ideas (and hence their own reputations) as obviously dominant in the field, leading publications and reviewers to summarily reject competing ideas. If you add political implications and support for political agendas, you greatly magnify the effect.
A couple of non-political examples I can think of: Einstein and relativity were initially rejected, even though competing physicists were working on very similar theories and furiously working to flesh out the details in order to be the first to publish. Plate tectonics (continental drift) was initially rejected because it seemed so different from then-contemporary understandings of the nature of the earth. I suppose these are both examples of science working as it should: revolutionary theory should require huge amounts of support before it can displace current understanding.
But, in today's frenetic academy, there are more researchers than ever before, dividing knowledge into ever smaller sub-specialties, and lots of new dominant narratives to establish.
The best evidence I can offer, not that the conclusions of the climate change literature are wrong but that the field cannot be trusted, is something often quoted and provably false that doesn't get called by people in the field. Here is my example:
I'd say this is one example of many where people misuse "science" to support an agenda. In this case, it's more dishonest than you showed. Aside from the misuse of the data you explained, the characterizations of the papers were based on Cook's interpretation of the papers' abstracts, not on any material in the papers themselves. Cook et al made no attempt to contact any of the papers' authors to verify that they properly characterized the authors' positions (even as a quality check); some of the authors who were later contacted said that their positions were misrepresented in Cook's paper.
Another example I encountered was the "medical bankruptcy" studies conducted by Elizabeth Warren et al (2005 and 2007). Links to the two papers here:
These studies carefully refrained from identifying a cause, or a primary cause of bankruptcy. They listed several factors (medical bills not paid by insurance, loss of income due to illness, loss of income due to caring for a family member, any statement by a bankrupt person that there were medical issues), and added them all together to conclude that a large percentage (54.5% in 2005 and 62% in 2007) of personal bankruptcies were "medical" bankruptcies. In media interviews, the authors carefully avoided using the word "cause", preferring such terms as "in the wake of". People who conducted or heard these interviews reported that these studies had found that medical bills "caused" most American bankruptcies, and these studies were used prominently by Democrats and other supporters of some sort of universal medical system to justify Democrats' medical reforms, leading to Obamacare. They implied that universal medical insurance would eliminate most bankruptcies, even though the papers provided no support for this position. If you read the reports, "loss of income" is reported as a larger cause of bankruptcies than "medical bills".
All the examples of such misuse I can think of are by Democrats or their ideological allies. I don't think this is because Republicans and their allies are more honest - I think it's because Democrats and their supporters believe that scientific or academic findings are a persuasive justification for policy choices, and therefore are willing to misuse such findings, while Republicans and their supporters don't need academic support for their positions.
Fun post. I've recently been wading through the vast "brain games" literature (can playing games improve your brain), and this rule works. On the cognitive side (do they make you smarter), I could not find two articles worth recommending. On the emotional side (can they make you feel better" I could name two - including one where games outperformed anti-depressant drugs. Probably says more about drugs than brain games.
Can you site two successful used of this strategy. I like the concept and I want it to be true... Or a bit probable. What a good compendium that would be. "The 2 Paper Library of Usefulness". My approach ... Is more like "The 2 book rule". I would give a new science or school of thought 2 books to explain the ideas and usefulness. Short books I hope. Like Toynbee, if the ideas are solid, they can be condensed. This does remind me of the Google Page index approach. Ah, IF only citations did not suffer from manipulation and sycophants.
Ties back to your research subpoint on your recent post about China. There's good research being conducted there, no doubt, but also a fair amount of the low-quality, insular garbage you describe. (There's, of course, garbage publications from all countries as well, but the sense is that it's more systematic in China.)
This is a recent medical article about research publication in China that's made a bit of a splash due to a very alarming claim. I'm still not sure how much to attribute to poorly written or edited publications and how much to attribute to malicious practices or shoddy medical care (including incorrectly performed apnea tests), but none of these option is a good defense of the other. Either way, it does show the difficulty of assessing the quality of literature when there are so many unknown variables.
Your two paper rule makes a lot of sense. Back in the days of the Partisan Review there used to be a lot of respect for public intellectuals. If that respect is gone, it should return. If people cannot present a good case in layman's terms for a theory, there is good reason to suspect that they are hiding behind jargon and credentials.
As a defensive rule, I don't know a better one, but there's one weakness here. It's entirely possible that a credible and relevant field has two "foundational" papers that contain a lot of weak thought. They may have introduced new ideas, but done it poorly, but if a group of academics are unwilling to replace those papers despite their flaws. You'd then end up with a situation in which certain key ideas are introduced but poorly defended in the "foundational" papers, and the good defense is diffused throughout.
Of course, an inability to take one of the possible fixes to this situation indicates a weakness in the field. The original authors could replace their flawed papers with better ones. The field could recognize the need to fix this problem trumps the academic credit system. But despite that, I know this can persist a long time.
A real-life example is Philosophy. Socrates had a great mind and did say many interesting things, but there are many flaws as well. I've personally engaged with philosophical groups that refused to move past that and only wanted to engage with the "vast literature" of ancient Greek philosophers, to the exclusion of all else.
You've written a lot about American suburbs so this piece I discovered (love substack for that very reason), well let me quote part of it.
The mind loves a mystery. And there’s a mystery to be solved in California. Why does a state with purportedly progressive values have such regressive outcomes?
The national hypothesis is to view things through a partisan lens: for left-leaners, Dem = good, GOP = bad.
But in California, that tribal shortcut doesn’t get us any closer to the answer.
At Modern Power we’re early on the journey to solve this mystery. So far we’ve discovered three noteworthy barriers to progressive outcomes: Narrow Interests, Progressive Proceduralism, and Anti-Statism.
I think you're giving short shrift to a simpler explanation: "1-d political spectrums are poor tools for explaining complex political economy outcomes".
California doesn't necessarily have to be a uniformly "progressive" state just because it codes far to the left on a lot of indices. It's pretty clear that despite an aggregate progressive stance, the actual municipal politics have been downright regressive at times. Sure, there are plenty of hypocritical rabid lefties to find in Berkeley planning meetings, but it's not a stretch to speculate that most of the people throughout the last 70 years of California's history of not-dense-enough auto-centric development ranged from normies to outright conservatives, and that _they_ are the consensus that drove most of those decisions.
IOW, today's hypocrisy is only notable because the longstanding committment to bad development patterns is now directly in opposition to the state's aggregate orientation to national politics, which has been evolving the entire time. People didn't just wake up and say, "I'm going to be a woke NIMBY!". They were always NIMBYs, and their national politics just changed without much changing the NIMBY stuff, because national politics has largely become decoupled from the local since the advent of national media.
> And if you can't cite two papers that serve as paragons or exemplars of the vast literature, it means that the knowledge contained in that vast literature must be very diffuse and sparse. Which means it has a high likelihood of being a mud moat.
For the counterfactual idiocy in me, I can't help but think of the recent ATIS article on messing around with bad ideas or sparse literature (not as a good foundation but as greenfield research) https://atis.substack.com/p/embrace-interesting-ideas
If we were to try and "do our own (fun) research" are there similar rules that can be applied (other than avoiding comspiracies)?
"Voila - a peer-reviewed literature chock full of misinformation. In practice, I doubt anyone ever does this intentionally. It takes too much coordination and long-term planning."
And yet that is what the Kochs have funded over 50 years or so of libertarian economics propaganda. Pretty much all of Austrian economics, Public Choice economics and large amounts of Chicago economics have been funded that way. And there's a wealth of critiques of their basic reasoning that has been emerging recently, such as Donut Economics and Ha-Joon Chang's books. Most philosophy, libertarian and otherwise, works similarly.
I was going to write more or less this exact comment :-) Noah wrote a decent article about this once but from a semi-defensive stance of "not all economists". He concentrated on the ratio of right wing quacks to others among economists being low and the amount of right wing quackery posing as serious intellectual work having declined in recent decades.
These are true. But there's still the vast literature you're talking about that's plutocratic ideology with mathy dressing. The default econ 101 book that drives a lot of economic intuition among college graduates is still from Mankiw, who may do some respectable work but also is a total fraud ideologue as a public intellectual. And you can see in the public debate in the US that the overclass has wildly succeeded in leveraging their bought "research" literature to inculcate as public intuition the self-serving myth that encouraging plutocratic rent extraction and crushing everyone else is, ironically, the only path not towards serfdom.
I'm not sure that would work with every field. If suddenly string theory became a political battleground, how much would citing "String theory dynamics in various dimensions" and "The Large N limit of superconformal field theories and supergravity" actually help you to understand if string theory is a worthwhile research subject ?
String theory is obviously a pretty extreme example, but econ can get very math heavy, very fast and I personally would not be comfortable judging whole economic theories by reading two recommended papers.
This is my concern too. I really like this idea for cases where the literature in question is sufficiently close to your own comfort and expertise to be able to easily judge its quality. But it's likely to lead to an overly dismissive response to work that approaches problems very differently from work you're familiar with.
If you're trying to convince a non-economist that microeconomic theory is valuable, it would probably be hard to do so by sharing two excellent, insightful, theoretical papers. Any work from the past 50 years would, at a minimum, use utility functions without explaining what they were (and for more recent work, use properties of general utility functions), and pretty much any work I can think of would do some combination of assuming a continuum of agents, working with a representative agent, or having a very limited number of "types." It would be totally reasonable for someone who isn't immersed in this work to read a microeconomic theory paper, conclude that it's a bunch of obscurantist math based on absurd assumptions, and walk away.
The solution probably wouldn't be to hand someone early work developing utility theory or talking about marginal decisions, because that work is highly abstract, theoretical, and old. Even if it was understandable, it wouldn't be clear why it's useful.
Instead, what you'd want to do would be to tell someone to read a good introductory econ textbook, and then chase it down with a good intermediate econ textbook, and then read a few clear, simple theory papers by someone like George Akerlof or Kaushik Basu who knows how to write and does interesting, simple models. Which is to say--you'd tell someone to go to college and major in Economics.
I have a PhD in statistics and worked with economists in the past. I'd actually agree that economic theory is (predominantly? mostly?) "obscurantist math based on absurd assumptions".
Econ isn't a physical science. The fact is you can literally always take a theoretical economic model, change a few assumptions or parameter distributions in plausible ways, maybe add in one or two new totally plausible terms/inputs, and create an equally plausible theoretical model that "proves" exactly the opposite ideological conclusion as the original.
In recent years there's more expectation that a model author not totally ignore predictive power. But even there, you have freedom to invent arbitrary models. Even worse, you can publish respected papers where, for example, you simply rename your predictive error as technological development or productivity shifts. Back predictive accuracy is hardly a major check.
Think of economic models as priors, if they are good, it allows for better prediction with fewer data
But the point isn’t prediction. People who only care about prediction (say, quant hedge funds) use deep learning and other ML. The point of models is inference and policy insight/recommendation.
But there is no reasonable insight if the model is empirically wrong. As you said, the space of models is too large, without some discipline, like prediction, they are just empty. There is zero point about inference if the model is misspecified, and it always is. It's crazy how people care about standard errors of fundamentally wrong models. Deep learning and other ML does not work that well when predicting returns, there is just too much variance without feeding it good features, and creating good features is just creating good priors for the functional form.
I think microeconomic theory is really useful, even though I agree with your specific criticisms.
Economic theory is about making assumptions about how the world works, expressing them mathematically, and then doing a proof/solving the model in order to draw conclusions. In my opinion the way to get value from this is to take the assumptions as seriously as the conclusions.
In other words, the idea isn't that you draw a conclusion and say "I know that this is true in the real world because the math backs it up." Instead, the idea is to tell a story about how the world might work, but to do so as precisely and rigorously as possible. From this perspective, changing some assumptions and getting a completely different answer is exactly the point: once you've done this exercise, you've potentially identified which assumptions about the world are most important for the question you're trying to understand.
So, for example, the first welfare theorem proves that when individuals make choices to maximize their utility, are small enough components of every market so that their decisions do not affect prices, produce and consume goods with no externalities, and hold no private information, market equilibria will be pareto optimal.
It would be absolutely wrong to read this as a theorem proving that markets are efficient. Anyone telling you that the first welfare theorem proves that markets are efficient would absolutely be engaging in "obscurantist math based on absurd assumptions" in order to push an ideological agenda. What the theorem actually does is points at the factors that are most likely to cause economic inefficiency, which has helped focus policy analysis across a broad range of fields, has helped to emphasize similarities among seemingly very different problems, and thus allowed us to apply lessons across multiple domains.
For what it's worth, my sense is that the field is in some ways moving away from demanding that models have predictive power. At least in labor economics, approaches that try to directly test theoretical models against data (structural estimation) have become much less popular in the last 20 or so years, in favor of methods that try to plausibly estimate directly policy-relevant parameters in particular contexts using natural experiments. Theory has been used much more to tell stories that help explain empirical findings, suggest limits to the applicability of findings, or understand why different contexts give contradictory findings. I think it's really useful for all of this stuff!
Fair call on the wording. In prediction I’m including looking at how wide your error distribution is when inferring parameters.
On the rest, IMO you're explaining the attitude among non fraudulent economists that leads them down the wrong path and creates an atmosphere where ideologues can gain respect with their mathy propaganda.
Like I said originally, economics is not physics. Making up a bunch of (potentially differential) equations and then giving the variables names and solving for various things doesn't prove anything or reveal insights. It's just a math and story time game.
To your example, FWT is a lot of games with basic convex optimization of inner products from arbitrarily defined vector spaces. Just to take two of many examples. The whole thing embeds Pareto optimality as the correct measure. Which is dubious. Then waves hands over the concept of utility. Any attempt to dive deeper there is, more or less, putting mathy obscurantist frosting on moral philosophy. There's a reason economics was a sidekick to moral philosophy for the first few centuries.
Let me recommend, for the purpose, my _Hidden Order: The Economics of Everyday Life_.
Thanks David! Your book looks great and sounds like a fantastic recommendation for my students (and for me :))
It's a good point and a useful tactic. One risk is that the person who is giving you the two articles may be part of a subgroup within the supporters of the ideas in question which is particularly bad.
A while back I agreed to debate an Austrian and asked him in advance for an account of the position he would be defending. He pointed me at a book by Rothbard. It was clear from that that the views in question were wrong [http://www.daviddfriedman.com/Ideas%20I/Economics/Critique%20of%20a%20Version%20of%20Austrian%20Economics.pdf], but I also concluded that Rothbardians were only one faction among Austrians and I am still not sure if the movement as a whole has serious problems.
One other approach to your problem that I have been thinking about is looking for work that you can audit. I recently did a back of the envelope calculation of the increase in the amount of useful land from climate change shifting temperature contours towards the poles. I then looked at a couple of published articles on the subject. They were much more sophisticated than mine, looking at things like soil quality and humidity as well as temperature, which was the only variable I considered. One got a result similar to mine, a huge increase, one concluded that the amount of arable land might go up or down a little.
Their procedure is much better than mine but mine had one large advantage — it was simple enough so a reader willing to spend a few hours could check it, see what I was doing and that I did it honestly and competently. There is no way anyone outside the field could do the equivalent for either of those papers, and it would take more than a few hours even for someone in the field. Given both the fact that the issue is politically loaded and that the two professional papers disagreed so much, the fact that I can't check them for myself is a strong argument in favor of looking at simpler models like mine instead of sophisticated ones.
I think the principle generalizes. Try to find work with an important result that you can check. If it turns out to be bogus — I'm thinking of one example in the climate field — and other people in the field haven't pointed it out, that tells you something useful about the quality of work in the field.
It works for people too. You wrote things about Adam Smith, trying to represent him as a modern progressive rather than the 18th c radical that he was, that could not have been written by an honest man who had read the texts he was quoting. [http://daviddfriedman.blogspot.com/2021/07/noah-smith-on-adam-smith.html] That sharply reduced my interest in reading other things you wrote on other subjects, although in the case of this post I thought you were making a good point.
"Suppose you and your friends wanted to push a weak argument for political purposes. You could all write a bunch of papers about it, with abstracts and numbered sections and bibliographies and everything. You could cite each other's papers. If you wanted to, you could even create a journal, and have a peer review system where you give positive reviews to each other's B.S. papers."
Climate change?
To be fair, it appears to me as an outsider to academe, that this is always a risk. Academics have always to try to build a reputation, which requires publishing. Academics will try to establish their own ideas (and hence their own reputations) as obviously dominant in the field, leading publications and reviewers to summarily reject competing ideas. If you add political implications and support for political agendas, you greatly magnify the effect.
A couple of non-political examples I can think of: Einstein and relativity were initially rejected, even though competing physicists were working on very similar theories and furiously working to flesh out the details in order to be the first to publish. Plate tectonics (continental drift) was initially rejected because it seemed so different from then-contemporary understandings of the nature of the earth. I suppose these are both examples of science working as it should: revolutionary theory should require huge amounts of support before it can displace current understanding.
But, in today's frenetic academy, there are more researchers than ever before, dividing knowledge into ever smaller sub-specialties, and lots of new dominant narratives to establish.
The best evidence I can offer, not that the conclusions of the climate change literature are wrong but that the field cannot be trusted, is something often quoted and provably false that doesn't get called by people in the field. Here is my example:
http://daviddfriedman.blogspot.com/2014/02/a-climate-falsehood-you-can-check-for.html
I'd say this is one example of many where people misuse "science" to support an agenda. In this case, it's more dishonest than you showed. Aside from the misuse of the data you explained, the characterizations of the papers were based on Cook's interpretation of the papers' abstracts, not on any material in the papers themselves. Cook et al made no attempt to contact any of the papers' authors to verify that they properly characterized the authors' positions (even as a quality check); some of the authors who were later contacted said that their positions were misrepresented in Cook's paper.
Another example I encountered was the "medical bankruptcy" studies conducted by Elizabeth Warren et al (2005 and 2007). Links to the two papers here:
http://pnhp.org/PDF_files/MedicalBankruptcy.pdf
https://www.amjmed.com/article/S0002-9343(09)00404-5/fulltext
These studies carefully refrained from identifying a cause, or a primary cause of bankruptcy. They listed several factors (medical bills not paid by insurance, loss of income due to illness, loss of income due to caring for a family member, any statement by a bankrupt person that there were medical issues), and added them all together to conclude that a large percentage (54.5% in 2005 and 62% in 2007) of personal bankruptcies were "medical" bankruptcies. In media interviews, the authors carefully avoided using the word "cause", preferring such terms as "in the wake of". People who conducted or heard these interviews reported that these studies had found that medical bills "caused" most American bankruptcies, and these studies were used prominently by Democrats and other supporters of some sort of universal medical system to justify Democrats' medical reforms, leading to Obamacare. They implied that universal medical insurance would eliminate most bankruptcies, even though the papers provided no support for this position. If you read the reports, "loss of income" is reported as a larger cause of bankruptcies than "medical bills".
All the examples of such misuse I can think of are by Democrats or their ideological allies. I don't think this is because Republicans and their allies are more honest - I think it's because Democrats and their supporters believe that scientific or academic findings are a persuasive justification for policy choices, and therefore are willing to misuse such findings, while Republicans and their supporters don't need academic support for their positions.
What about the qualities of the 'you' in your article who is recommending the 2 papers, doesn't that also enter into the equation?
I remember reading this article when your first published it - it's a good heuristic!
Fun post. I've recently been wading through the vast "brain games" literature (can playing games improve your brain), and this rule works. On the cognitive side (do they make you smarter), I could not find two articles worth recommending. On the emotional side (can they make you feel better" I could name two - including one where games outperformed anti-depressant drugs. Probably says more about drugs than brain games.
Good shtuff! Thanks
Can you site two successful used of this strategy. I like the concept and I want it to be true... Or a bit probable. What a good compendium that would be. "The 2 Paper Library of Usefulness". My approach ... Is more like "The 2 book rule". I would give a new science or school of thought 2 books to explain the ideas and usefulness. Short books I hope. Like Toynbee, if the ideas are solid, they can be condensed. This does remind me of the Google Page index approach. Ah, IF only citations did not suffer from manipulation and sycophants.
Ties back to your research subpoint on your recent post about China. There's good research being conducted there, no doubt, but also a fair amount of the low-quality, insular garbage you describe. (There's, of course, garbage publications from all countries as well, but the sense is that it's more systematic in China.)
This is a recent medical article about research publication in China that's made a bit of a splash due to a very alarming claim. I'm still not sure how much to attribute to poorly written or edited publications and how much to attribute to malicious practices or shoddy medical care (including incorrectly performed apnea tests), but none of these option is a good defense of the other. Either way, it does show the difficulty of assessing the quality of literature when there are so many unknown variables.
https://onlinelibrary.wiley.com/doi/10.1111/ajt.16969
Reminds me of religion discourse.
"I think you should read the whole Bible before critiquing it"
Unfortunately in Islam, the moat isn't as wide because the book is much shorter. The good news is it only counts if you read it in Arabic.
Sorry for bringing the discussion back to the 00s.
this helps me get over my concern about being bored reading papers that repeat what others have said.
Your two paper rule makes a lot of sense. Back in the days of the Partisan Review there used to be a lot of respect for public intellectuals. If that respect is gone, it should return. If people cannot present a good case in layman's terms for a theory, there is good reason to suspect that they are hiding behind jargon and credentials.
As a defensive rule, I don't know a better one, but there's one weakness here. It's entirely possible that a credible and relevant field has two "foundational" papers that contain a lot of weak thought. They may have introduced new ideas, but done it poorly, but if a group of academics are unwilling to replace those papers despite their flaws. You'd then end up with a situation in which certain key ideas are introduced but poorly defended in the "foundational" papers, and the good defense is diffused throughout.
Of course, an inability to take one of the possible fixes to this situation indicates a weakness in the field. The original authors could replace their flawed papers with better ones. The field could recognize the need to fix this problem trumps the academic credit system. But despite that, I know this can persist a long time.
A real-life example is Philosophy. Socrates had a great mind and did say many interesting things, but there are many flaws as well. I've personally engaged with philosophical groups that refused to move past that and only wanted to engage with the "vast literature" of ancient Greek philosophers, to the exclusion of all else.
You've written a lot about American suburbs so this piece I discovered (love substack for that very reason), well let me quote part of it.
The mind loves a mystery. And there’s a mystery to be solved in California. Why does a state with purportedly progressive values have such regressive outcomes?
The national hypothesis is to view things through a partisan lens: for left-leaners, Dem = good, GOP = bad.
But in California, that tribal shortcut doesn’t get us any closer to the answer.
At Modern Power we’re early on the journey to solve this mystery. So far we’ve discovered three noteworthy barriers to progressive outcomes: Narrow Interests, Progressive Proceduralism, and Anti-Statism.
https://modernpower.substack.com/p/why-california-politics-is-interesting?r=8ahwm&s=r&utm_campaign=post&utm_medium=web
I think you're giving short shrift to a simpler explanation: "1-d political spectrums are poor tools for explaining complex political economy outcomes".
California doesn't necessarily have to be a uniformly "progressive" state just because it codes far to the left on a lot of indices. It's pretty clear that despite an aggregate progressive stance, the actual municipal politics have been downright regressive at times. Sure, there are plenty of hypocritical rabid lefties to find in Berkeley planning meetings, but it's not a stretch to speculate that most of the people throughout the last 70 years of California's history of not-dense-enough auto-centric development ranged from normies to outright conservatives, and that _they_ are the consensus that drove most of those decisions.
IOW, today's hypocrisy is only notable because the longstanding committment to bad development patterns is now directly in opposition to the state's aggregate orientation to national politics, which has been evolving the entire time. People didn't just wake up and say, "I'm going to be a woke NIMBY!". They were always NIMBYs, and their national politics just changed without much changing the NIMBY stuff, because national politics has largely become decoupled from the local since the advent of national media.
As a Californian, I wish you good luck. It's a mess, but a mess I wouldn't trade for anywhere else.