78 Comments
Aug 25, 2021Liked by Noah Smith

As a scientist I've spent my entire adult life campaigning for the importance of expertise. The importance of climate change, for one, motivated me, as well as the goofy letters from cranks we would get positing who new kids of physics.

Covid has knocked me back a solid 10% on this. The fields of public health and epidemiology seem to have put forward really poor efforts, from giving bad advice, to being unable to make even the very roughest projections or paint plausible pictures of how this would play out (remember test, trace, and extinguish the virus without vaccines? Then models that showed every human on the planet infected inside a month? Then flatten the curve? Then ethics experts telling us challenge trials, which would have saved hundreds of thousands of lives, were unethical? Then approving vaccines for 12 year olds but not for 11 year olds six months later? Then not allowing tests unless they were perfect, but slow? Then constantly deflating vaccines? Then fighting perpetual war on case loads post-vaccine?).

Over and over reasonably smart generalists with backgrounds in statistics routed them.

It's been a change in perspective for me for sure.

Expand full comment
author

What do you think the reason is that the public health people failed this hard?

Expand full comment

I think there are a few reasons for the failure.

The first is that I think public health is inherently a fusion of politics, sociology, economics, ethics, and virology. Nobody is actually an expert in all those fields, so at the end of the day even the "experts" are just firing off takes like everybody else.

Second, I think a bunch of petty bullshit got in the way. Experts oversteered on masks because they were afraid of losing access to PPE, and then took way too long to walk it back. The FDA continually chose rigid bureaucratic procedure over any sort of initiative or flexibility. A weird longstanding debate about droplets vs aerosol got in the way of taking common sense protective steps. There was also a lot of arrogance in western doctors who were uninterested in learning best practices from asian countries who were more experienced with respiratory epidemics.

Third, experts felt like they were under constant attack from the political system (specifically republican politicians). This made them very cliquey and defensive, eager to circle the wagons against any criticism at all, regardless of how well-intentioned. Good criticism was treated as a trojan horse for bad criticism rather than being evaluated on its own merits.

And on top of that, in the face of a pandemic you only have one chance to get it right. Scientists are wrong all the time. Usually that means you toss out the hypothesis and try the next one. But when public health experts were wrong, that wasn't just a failed experiment - it meant people died.

I think this reply might sound harsh, but at the end of the day, petty bullshit and defensiveness are endemic to every organization I've seen. The real surprise would be if public health communities were somehow able to rise above those very human flaws. At the end of the day though, expertise isn't magic, and an organization full of experts is still and organization, with all the messiness that implies.

Expand full comment

My "expertise" is public financial management, and I taught at a School of Public Health. I was welcomed by the public health community. They explained to me that public health is an interdisciplinary field and my knowledge/experience was important, valid, and needed by students. So i was shocked by the epistemic closure that I saw by some in the public health academics when they essentially told Tufecki and Tabarrok to stay in their lanes. It seemed counter to the ethos of the field.

Expand full comment
Aug 25, 2021Liked by Noah Smith

I think its very hard at this point to know how hard they failed. Predicting the future is hard! Estimating the parameters of a disease model requires fitting a nonlinear differential equation, often with very very poor data.

That being said, I think the difference between economics after 2008 and public health now is instructive. Economics seems, to an outsider, to have had a major empirical revolution as a result of its failures in the first financial crisis. I'm not sure public health is doing the same thing... it seems just looking that there's a lot of blaming the patient. While obviously there are a lot of stupid people out there, social factors do matter in a pandemic...

Expand full comment
Aug 25, 2021Liked by Noah Smith

This is going to sound really self indulgent and low-n, but there was a point, pre delta, in the spring when we had a little wave just as vaccines were coming online, and experts were issuing these predictions of just huge case loads when anyone who knew the vaccination rate and efficacy could do the math in their head that it would crest in a week or two, and it did.

Expand full comment
Aug 25, 2021Liked by Noah Smith

Edit: I mean in my state specifically.

Expand full comment

Yeah I agree with that statement as well.

Expand full comment
Aug 25, 2021Liked by Noah Smith

Good question. I don't know entirely, but think partly it was a combination of trying to out-flank people instead of just being honest, and not being able to give up certain battles and move on. And just a lack of numerical proficiency, like understanding why rapid, imperfect tests are better than perfect slow ones.

Expand full comment

Zeynep Tufekci (who is not a journalist, she's an academic sociologist with a background in computer science who has taught a course on the sociology of pandemics; formerly at UNC, now at Columbia) has also written extensively about this. Here are just a few from her Substack & NYT.

https://www.theinsight.org/p/the-few-sentences-that-explain-much

https://www.theinsight.org/p/pandemic-as-metaphor

https://www.theinsight.org/p/the-gaslighting-of-science

https://www.nytimes.com/2021/06/25/opinion/coronavirus-lab.html

Expand full comment

I followed the first of the recommended links to Zeynep Tufekci and found a long, but very well written, piece that covered a host of mistakes, many systematic that I will categorize as the “Good Ol’ Boy” network of closed minds.

Expand full comment

One huge reason is the inability to distinguish between two questions:

“Do masks work at preventing infection?”

AND

“Should we recommend people wear masks?”

Decision analysis and Bayesian reasoning are not something most public health experts realise they need to understand if they want to make reasonable recommendations.

Expand full comment

The scientific consensus as of late 2019 was that widespread masking did not work to prevent infection for surface-spreading viruses, especially when used by untrained individuals. There had been numerous studies done up to that point that looked at community face mask usage and the flu, most of which showed little to no benefit.

As we learned more details about COVID-19, it became clear that it did not spread via surfaces but instead via aerosols. But until that information became known, the answer to "Do masks work at preventing infection" was widely believed to be no, and that's what most scientific literature supported.

Expand full comment

The idea that COVID-19 was surface spreading rather than being a respiratory disease was always stupid.

Expand full comment

I think they realise that and that's why they didn't recommend them initially. Later they changed their recommendations because the public was scared and wanted anything that would make them to feel safe. The story that they wanted to keep them for healthcare workers therefore they lied is not true, but widely believed because it is such a catchy mime.

On the balance of evidence, masks don't seem to make much difference except in specific circumstances. But many countries pushed too far and even demanded them outdoors all the time which was pointless.

Expand full comment

To my admittedly non expert eyes - in field expertise prioritiEs the inside view. That can be fine if the problem is familiar / has known solutions. This was a qualitatively different issue that required more Bayesian reasoning and focus on the outside view, which humans are generally bad at, but especially narrow experts as their conceptual universe naturally makes the inside view more dominant. Which is why it seems that novel problems with great deal of uncertainty are best solved with the exact types of cross functional teams Noah is advocating.

Expand full comment

This may not have been a particularly original thought but nonetheless…

Expand full comment

It was due to political interference. In Sweden they didn't fail. They properly identified uncertainty we have with models and managed to retain sanity and introduce appropriate and balanced measures and give people realistic expectations. Their pharma company (Astra Zeneca) developed a vaccine and the rest is history.

They mostly failed in other countries because public got scared and demanded politicians that something has to be done. Politicians were looking for answers that people would accept and not the realistic ones, so they fired experts whose message they didn't like and promoted those that played the tune.

Public Health English very quickly downgraded the severity of this pandemic to one of very little importance. When non-experts read that they were terrified and thought that PHE is nuts. There is great disconnect between what public health experts historically consider to be a severe case and what the public thinks.

The public health experts had acted correctly as they had evaluated the burden of possible restriction measures and expected benefit. Masks had very little effect, so their use was questionable and even lockdowns have very questionable gains if expressed in terms of QALY.

Expand full comment

I was going to object that AstraZeneca was British rather than Swedish, but then I checked to make sure and learned it was formed from parent companies from both countries, hence the portmanteau name. You learn something new every day.

Expand full comment

Epi is inter disciplinary and the vast majority have at best a solid understanding of extremely basic statistical modelling. Biology, virology, and public health of course are fields where folks get less, mostly near zero, statistical training. So it shouldn't be surprising that high quality stats and decision theory weren't inputs into PH decisions or communications.

Expand full comment
Aug 25, 2021Liked by Noah Smith

One related thought: it's also just unclear why certain types of expertise are needed. I can't count how many times prognosticators would be told "you're not a virologist", when projecting the twists and turns of the pandemic was an entirely statistical question that only needed a few basic observational inputs.

Expand full comment

this sounds like something I might have written

Expand full comment

Epistemic Trespassing implies Epistemic Property implies Epistemic Rent-Seeking. Thank you for coming to my TED talk.

Expand full comment
Aug 24, 2021Liked by Noah Smith

Epistemic rent seeking is definitely a thing... I mean literally, being the expert at something gives you money when that thing becomes important!

Expand full comment

The entire concept of a patent on a method or mechanism represents the legal formalization of epistemic rent seeking, as way of encouraging people to try to discover useful ideas.

Expand full comment

Is it really? Or. Possibly a loop hole rather in the steps manufacturing and or repair or replacements. Product can't go without that critical step however sighting a dept into visuals found a niche and the rest is history. I think we all know the true term for this tactic. Still I wouldn't shame or slap the hand that forced a way, it is important that these holes are filled even by a mole bc what if a huge rat chewed the wire first then you would of found one hell of a mess .

Depends on the mole too. Probably a hard working smart ass, good parent and a hoover. On looker if you will. Basically the motivation in us all is different at intervals in our life times. I feel why Knott have more then one great mind on a job each head assigned to appropriate participation assuming they fill the spot with qualifications generate an outcome grater then the possibility of the outcome expected without.

Yet one man or woman could and have taken on expertise for millennia and never require a hitch. Give a point to the peanut gallery bc , the hammerhead banged a nail , a figure of speech , that's all. He's right mixing business with pleasure is not always the latter way to go especially when involving two twin soul mates. Two fanatic beautiful brain for one. Ouch. Plato and Pluto his sweet road God. Epistemic trespassing , lol, I myself stem from a rather hefty stem and he's brilliant true blue. I supposed trespassing depends on shoes that walk on what lays unearthed and the legs that belong to the body that carries it to weather or Knott epistemic trespassing is actual notable. Property is only as private and mandated by its proprietor. And if property has a sound mind and ticking heart of its own then your defining a prop in place of liberty ought Knott be assumed to own such.

I known a lot of things yet I never got twisted over a title like expert. That's just another word for experienced teacher or trainer. What is wrong with that title. Perhaps that's what's wrong is the expertise eating the expert. Grow up and assume a career degree if your concerned about what is before or after your introduction. You could always thumb through the dictionary and propose another ackcronam

Expand full comment
Aug 24, 2021Liked by Noah Smith

I've definitely seen the phenomenon recently of experts using their expertise to disguise as science what are actually ethical questions. For example, Matt Yglesias advocated human challenge trials for COVID vaccines, and epidemiologists told him to stay in his lane, that there was an expert consensus for how we do trials. But I don't think they even had a scientific disagreement. Everyone knew that human challenge trials would lead to faster approval, it's just that the medical establishment has an ethical norm against them. Fair enough as far as it goes, but that's not a scientific question, and the epidemiologists have no more expertise on the matter than anyone else.

Expand full comment

I think a lot of the discussion of challenge trials, from both sides, has avoided discussing the important question of what precisely you learn from challenge trials.

One of the important things you get from standard Phase III trials is that you get to observe how frequently a population of people living ordinary life get infected when they do or don't have the vaccine. That's where we got the initial 95% or 85% efficacy numbers.

With challenge trials, we could learn that people facing a particular challenge dose of virus get infected at particular rates when they do or don't have the vaccine - but it remains open that a vaccine that fails to protect against the challenge dose succeeds in protecting at environmentally real doses (or conversely, that a vaccine that does protect against the challenge dose fails to protect against environmentally real doses, though this seems less plausible at first).

That said, challenge trials could be done a lot faster and tell us something, even if not something as ecologically useful as standard Phase III trials. And outside the world of vaccines we could have learned a lot about real-world transmission by intentionally sending some number of infected shoppers into study concerts or grocery stores where everyone else is a study participant (some with masks and some without).

Expand full comment

That latter idea is interesting, but as with the kind of challenge trials Matt advocated, there's a large confounding factor -- the kinds of people willing to participate in the trial almost certainly don't represent the population at large. I still think challenge trials for the purpose of narrowing in rapidly on stuff that seems effective in at least some people may be a good idea. (I also am frustrated that we did not invest however much money it would take to train up an army of public health workers to do stuff like make repeated follow-up calls to _millions_ of vaccine subjects in order to keep consolidating data on the long-run outcomes, because right now it seems like our understanding of the rates of breakthrough infections among various demographics of vaccinated people, and the rates for breakthrough infections producing really bad outcomes, are all extremely vague. When you have a pandemic that's majorly disrupting the economy, it is worth almost any conceivable amount you could spend, to get good data.)

Aside from that big confounder, you'd have to be very thorough in making sure that none of your concertgoers actually are infected coming in -- so, probably keep them isolated for a few days in advance, because if they inhaled an infectious dose six hours before the test concert, that will screw up your numbers. And the number of test subjects who might have that issue is going to vary over time, as the number of cases circulating in the community changes.

Expand full comment

A German group did the concert thing. They tried it with no masks, reduced capacity and good spacing in a large venue. They found out that COVID spread all too well.

Expand full comment
Aug 24, 2021Liked by Noah Smith

I think one reason credentialed experts hate outsider input is because sometimes outsiders dispute the entire foundation of their field. Most obviously, Dawkins has written the whole discipline of theology is nonsense and doesn’t belong in the academy; and the field of critical theory frequently draws commentators from outside saying it's inherently nonsense. People generally don’t like being told their job is a waste of their time and their employers’ money, so there’s some sunk-cost fallacy happening. Indeed, I also think the anger at ‘epistemic trespassing’ is partly driven by overproduction of PhDs – if someone spent eight years studying something and then can’t get an academic job in the field, but some 60-year-old guy with an unrelated career and a house in Palo Alto decides he wants to comment on the concept and immediately gets published to wide press coverage, you can see how that would cause resentment.

But I think this post is really important and people should really resist the impulse to denounce outsider/generalist commentary. Some fields, like journalism and legislation, will always need people who can ably and quickly learn the basics of expert consensus in any given field. Plus, sometimes entire fields really do need questioning – YMMV re Dawkins but I'd say if the cost of admission into religious studies debates is averring that religion is a healthy part of human nature that’s never going away and must not be subjected to value judgments, that’s a problem. And just in general, part of the the ultimate progress of human knowledge is not just someone in an ivory tower making a discovery, but making that discovery public, which necessarily entails subjecting it to people’s questions and their skepticism, however dumb. We can’t have an educated populace without academic debates becoming open to laypeople at least at some point.

Expand full comment
Aug 24, 2021Liked by Noah Smith

John McWhorter talked about this idea in a recent podcast with Glenn Loury. He pointed out that it's only ever used to dismiss those with whom you disagree, or who those fromm the outgroup. Nobody minds epistemic trespassers that uphold the party line.

Expand full comment
Aug 25, 2021Liked by Noah Smith

Interesting to see a philosophy professor dedicate a 24-page essay to defense of a narrow form of ad hominem.

Expand full comment

I think you give too much credit to the CDC owning up to their mistakes. A few isolated individuals partially owned up, but the truth is the CDC and the Doctors who blindly followed their masks aren’t required message and the who perpetuated the flawed droplets theory are responsible for thousands and thousands of deaths.

Doctors and the CDC killed people. Their credibility is shot. Epidemiologists might as well be alternative medicine practitioners until they own up in mass unequivocally to their manslaughter of Americans.

I could be equally as harsh on the officers and strategists who over the last 20-years deployed over and over to Afghanistan and lied to their superiors, civilians leaders and the public for the sake of promotion. Any of my enlisted friends who deployed to Afghanistan could of told you the truth and inevitable result, simply because their promotions were not based on the same pressures. Every officer who deployed to Afghanistan has a promotion statement that states they left it better than when they arrived.

Expertise is overrated in any of the most concrete of fields like engineering or mathematics.

Expand full comment
Aug 24, 2021Liked by Noah Smith

one other comment... though Gosset worked for Guiness, he was an oxford trained mathematician and probably shouldn't be thought of as an epistemic trespasser in a field he was one of the major creators of!

Expand full comment
author

So he was an epistemic tresspasser in brewing? :D

Expand full comment

He worked as a statistician for Guiness. He wasn't a brewer.

Expand full comment

And who remembers if he was any good at that?

Expand full comment

great article. very insightful

Expand full comment
Aug 24, 2021Liked by Noah Smith

One thing I find amusing, coming from religious studies, is that in general, the old timey religious studies scholars generally find the "philosophers of religion" utterly vapid and complete epistemic trespassers (also, they don't actually know anything about anyone's religion!). Perhaps this has some bearing on the issue...

Expand full comment
author

Thanks! Others told me this as well. Added an update!

Expand full comment

This idea of epistemic trespassing is just 'argument from authority' in different clothing.

Obviously if you're going to wade into a discussion in which you are not an expert, humility is in order. But, if somebody is not staying within their lane and they offer bad arguments, then let's attack the arguments, not whether they're staying in their lane or not.

Expand full comment

The professional classes are very insecure, and thus become very defensive when "lay" persons question their pearls of wisdom. Expert fragility, is probably a thing :) Pink Floyd's "Money" comes to mind: "I'm all right, Jack, keep your hands off of my stack."

Expand full comment

This post brings to mind a book I enjoyed reading, "Range" by David Epstein. He sort of gives a bunch of examples (with statistics to back it up) of generalists "finding their path" late in life and then excelling. I kind of think there's a lot of overlap between "finding their path" and epistemic trespassing.

Expand full comment

I actually took a class with Nathan Ballantyne back when he was working on this and you might have more agreement with him than you think. I'm sure he'd be happy to have a discussion and address your questions if you reached out to him.

Expand full comment

Not to dismiss the meta-level point, but the string theory example is poorly thought out. There is no other theory of quantum gravity, by which I mean there is no other theoretical framework of quantum gravity. Everything else has been ruled out. That's not to say string theory has empirical predictions (nor am I saying that it doesn't), but word of its demise is extremely premature.

String theory has given us holography and AdS/CFT correspondence, for instance, which is used everywhere from condensed matter to nuclear physics. (I know that someone calculated the exact viscosity of quark-gluon plasmas using string theory.) String theory definitely has not been a waste of funding, which is often the implication of "string theory does not make predictions". (See also: https://4gravitons.com/2014/03/28/gravity-is-yang-mills-squared/)

Those you cite don't have the greatest track record either. Sabine Hossenfelder has been saying that quantum gravity should be focused on hashing out predictions, e.g. particle scattering, which is what a selection of string theorists are doing. Lee Smolin's brainchild, loop quantum gravity, can't even derive Hawking radiation, and is pretty much dead in the water after we've seen neutron star collisions. Sabine, at least, is usually excellent elsewhere (I'm not familiar enough with the other two), but they come off as having an axe to grind.

Expand full comment

While I'm against random people claiming to be experts on Afghanistan, it's not like the experts were any good at it either. They were not being frank on afghan security forces.

Technocrats can be good at creating policy, but they need to be beholden to public scrutiny. This does mean plenty of stupid questions, but I'd prefer that if we get one good one every so often.

Expand full comment