92 Comments
User's avatar
Alan Goldhammer's avatar

Excellent post. I wonder how many of the critics of public education actually have children either in or who have gone through public schools K-12? Our experience was uniformly positive and both went on to graduate college with honors and go on to post-graduate education. The key is adequate financing for resources, good teachers, and small classes. Unfortunately, lots of excellent teachers are leaving the profession and that is where the crisis is right now.

The big issue is with the Title One schools that serve low income students (usually but not exclusively, those of color). Trying to understand why these students do not achieve at the same success rates is multi-factorial but whether it is family environment or something else has been difficult to tease out.

Expand full comment
Belisarius's avatar

At least locally, the Title One schools receive quite a bit more funding per capita (20-30%, IIRC) for the last few generations, and there is relatively little to show for it.

I strongly suspect that it is primarily a function of culture and parental involvement, than the $ itself.

And it is worth noting that, while most of our Title One schools are majority-black, there are still several rural ones that are majority-white and in the same boat, so race doesn't seem to be the deciding factor.

Expand full comment
Jeremy's avatar

Should be a distinction between middle class suburban or higher public schools where we compete very strongly and our lowest performing schools which can really drag down averages. If my choice was the school in the inner city I'd be fighting like hell for chatters or vouchers or whatever. Fortunately I like basically everyone else reading this will never be in that boat with our kids.

Expand full comment
Jeremy's avatar

If I lived in the inner city where the only option for my son was a failing school of desperation, then I would be fighting like hell for chatters or vouchers or a thing to give my child a better shot at life. Thing is neither I nor probably anyone who reads this will ever be in that situation.

Expand full comment
David Roberts's avatar

Thanks for this post. Very helpful data.

I always thought the issue with our k-12 education was high dispersion of results. That our distribution curve would show a much larger left tail than other countries. If true, then our top 80% might actually be close to number one, while our bottom 20% would be far behind.

Do comparative statistics like that exist?

Expand full comment
Ben's avatar

https://www.oecd.org/pisa/PISA%202018%20Insights%20and%20Interpretations%20FINAL%20PDF.pdf

The authors break down reading scores by socioeconomic decile on p.9. Just at first glance it doesn't really seem like the US has a particularly wide distribution of scores.

Expand full comment
David Roberts's avatar

Thanks. My hypothesis was wrong based on this data.

Really appreciate your sharing it.

Expand full comment
Auros's avatar

America has no public schools. We just call the tuition "property taxes" and "mortgage payments".

Expand full comment
Noah Smith's avatar

That's not right, in recent years we've moved away from the local property tax funding model!

Expand full comment
Auros's avatar

Still seems pretty true about California, with a lot of funding coming in through external non-profit "education foundations". You can't possibly tell me that, for instance, you think the Ravenswood School District (East Palo Alto) has as much actual resources as Palo Alto, regardless of the headline state funding number.

Expand full comment
El Monstro's avatar

Finances at Palo Alto Unified School District:

Palo Alto Unified School District spends $25,690 per student each year. It has an annual revenue of $316,749,000. Overall, the district spends $15,598.1 million on instruction, $7,155.4 million on support services and $284.7 million on other expenses.

https://www.usnews.com/education/k12/california/districts/ravenswood-city-elementary-113363

Finances at Ravenswood City Elementary

Ravenswood City Elementary spends $25,723 per student each year. It has an annual revenue of $56,283,000. Overall, the district spends $10,854.9 million on instruction, $6,856.1 million on support services and $911.3 million on other expenses.

https://www.usnews.com/education/k12/california/districts/palo-alto-unified-111350

Government funding is about the same, but as you point out the the PTA gives much more in Palo Alto. It is hard to find these numbers but I will try.

More to the point, PA pays it's teachers much better:

https://transparentcalifornia.com/salaries/search/?a=school-districts%2Fsanta-clara%2Fpalo-alto-unified&q=teacher&y=2020&s=-gross

You can see that there are about 1/3 of the teachers there that make a salary of over $100k/yr. On top of that they get retirement benefits worth another $30-50k/yr.

https://transparentcalifornia.com/salaries/search/?a=school-districts%2Fsan-mateo%2Fravenswood-city-elementary&q=teacher&y=2020

You can see a much smaller percentage of Ravenswood teachers make over $100k and none make the large salaries that top Palo Alto teachers enjoy,

Expand full comment
El Monstro's avatar

I found this PA foundation (PA Partners in Education) which spends $6M/ yr which is $500/student by looking on Guidestart:

"PiE raises money from parents and the community to provide all Palo Alto Unified School District students an educational experience beyond what is possible with public funding."

https://www.guidestar.org/profile/77-0186364

I think there is some PTA groups I am not finding though.

Expand full comment
Alex Karjeker's avatar

Texas recently did an education financing reform (HB3) in 2019, but even then it just moved the state/local split back to 50-50 from something like 40-60.

Which states have moved away from the local property tax model?

Expand full comment
Benita Brown's avatar

Washington state caps local support for schools at no more than $2500 per student ($3000 in Seattle, the city with the highest costs in the state), while the state spent $11,500 per student in 2019. Those caps can be adjusted annually. Washington's constitution says the state's paramount duty is to make ample provision for the education of all children raising within its borders.

Expand full comment
John_E's avatar

Illinois is still heavily dependent on prop taxes for public schools.

Expand full comment
Ziggy's avatar

Kevin Drum keeps pounding away at this. Median American primary and secondary education are okay. But kids of color don't do very well. And there is no excuse for that.

Expand full comment
Frog H Emoth's avatar

I read somewhere a while back (possibly Matt Yglesias) that while Texas does worse than Massachusetts in various scholastic competencies overall, when you break down the data at a racial level, children of race <X> in TX score better than children of race <X> in MA; for white, Black, Asian and Hispanic.

I wonder if this is also true at a national level - I mean, we know that Asian children score incredibly well on tests in general, and Black and Hispanic children generally do not, and it seems fair that when we compare the exceptionally racially diverse US to the much less diverse countries of Asia and Europe, that we at least consider the impact that the racial disparities have on our relative ranking. I have a suspicion that if you break down the data by racial category, the children in the US are at, or near the top, for every race.

We should be working to identify ways to make education better for Black and Hispanic children in the US. We also should not be shy about pointing out our successes, either.

Expand full comment
tgof137's avatar

I graphed NAEP math scores against school funding for each state in this article, and also broke down the numbers by race:

https://medium.com/@tgof137/increased-school-funding-doesnt-improve-test-scores-here-s-what-might-3d46ebf5c1d

The short summary is that there's not much correlation between school funding and test scores to begin with, and the correlation goes away almost entirely when you adjust for race.

Whatever it is that drives the gaps in test scores, it seems to be mostly a factor of the kids or their homes, not of their schools.

One rebuttal to Noah's article would be to ask why we can't have all states fund education at the same level as, say, Idaho and Utah, which spend much less than states like New York but get similar test scores.

Expand full comment
El Monstro's avatar

No, there is a pretty huge body of research, including quasi-experiments (before and after changes in funding) that challenge these claims. More funding, especially in resource constrained district, leads to better outcomes.

Here's a metanalysis:

https://www.nber.org/system/files/working_papers/w25368/w25368.pdf

A summary by an institute that measure educational outcomes:

https://www.shankerinstitute.org/resource/does-money-matter-second-edition

I agree that segregation is a big driver of educational gaps and that desegregating schools would lead to better overall outcomes with improvement for Black students at no cost to Whites and Asian students.

Expand full comment
tgof137's avatar

I made a brief effort to look through the meta-analysis. It looks like there are 2 clear claims made:

after court orders regarding school funding, school funding goes up.

if funding goes up, graduation rates increase.

So those are entirely different metrics than what I was talking about. It's quite possible that increased funding increases graduation rates without improving test scores.

I have not run a national regression to see if graduation rates correlate with funding by state (either over-all or adjusted by race). Do you predict that there will be a correlation? How would it influence your views if there is no relationship?

Do you not find it the least bit odd that per pupil spending across US states varies by a factor of 3 without any clear impact on test scores? (AFAIK, the funding differences remain after adjusting for cost of living)

It looks like there's a third claim made in the meta-analysis that test scores are somehow correlated to school reliance on state revenues, but I didn't read the supporting papers yet to understand the proposed relationship.

Expand full comment
El Monstro's avatar

If you are serious about doing this analysis you should consider test scores other than just math scores. I don't believe you have proven your claim that per pupil spending differs by a factor of 3 without any impact on overall test scores.

This is an interesting line of research though. If more spending does not increase graduation rates I would be surprised.

" However, once we adjust for labor market factors, we estimate that raising teacher wages by 10% reduces high school dropout rates by 3% to 4%"

https://cepa.stanford.edu/sites/default/files/loebpage.pdf

Expand full comment
tgof137's avatar

"If you are serious about doing this analysis"...

LOL. Dude sends me a meta-analysis to read, I make a good faith effort to read it and one of the supporting papers, next response says I'm not serious in my desire to understand this subject...

It's been a few years since I've worked through the NAEP data but I still have some spreadsheets sitting around or I can recreate them. That is, if you're serious about your request for more graphs.

A similar analysis averaging scores across multiple subjects was done here:

https://reason.com/2018/10/07/everything-you-know-about-stat/

IIRC, their methodology was to divide scores by subject (math, reading, science), by grade (4th and 8th), and by 4 racial groups. They gave equal weight to each of those 24 possible combinations and got an average score by state.

That average score shows zero relationship to state funding.

I saw that analysis before I tried doing my own project, I assumed it must be misleading, they must be hiding something with the equal weight aggregation. But, nope, it holds up. It's true for 4th grade and 8th grade, for math or for reading. There's little correlation between test scores and funding, and it goes away almost entirely once scores are broken down by race.

I did some basic sanity checks, like I correlated funding against class sizes to see if funding had any impact there. Sure enough, more funding does produce smaller class sizes.

But class sizes don't correlate with test scores, either, so that doesn't seem to help.

Expand full comment
El Monstro's avatar

This is all I can find. It's an old Vox article and doesn't quite support your claims though it comes pretty close. If you can find this essay, please share it:

https://www.vox.com/2015/10/26/9617514/test-scores-naep-2013

Expand full comment
Kaleberg's avatar

You are probably thinking of Mike the Mad Biologist who did a series of posts about this back in 2012. I mentioned this elsewhere in the comments for this post and got a few things wrong. Having checked the blog: Black students in Massachusetts do as well as white students in Alabama on the TIMSS (not PISA). Mike, in another post, sarcastically suggests that this is for genetic reasons.

One post to start on is: https://mikethemadbiologist.com/2012/12/12/instead-of-racing-to-tops-or-not-leaving-children-behind-why-dont-we-just-clone-what-massachusetts-has-done/ (I'd include more links, but don't want this comment rejected as spam.)

Expand full comment
Richard Nerland's avatar

I am surprised you didn't frame it: Massachusetts is already a top performing educational system, let's impose reforms to get export Massachusetts education to all US states.

https://www.nber.org/papers/w21770

Second, the US system educates a lot of children of immigrants that "lower" scores through composition effects. Yet immigrant education is a dramatic improvement for the world and a sign of educational success. Note, when decomposed the US does quite well again, especially Massachusetts!

Expand full comment
Kaleberg's avatar

Thanks for the citation.

Expand full comment
Zack's avatar

I think we could do better on the parts of a school that aren’t the curriculum. There’s no reason for any school in the country to have mold on the walls, or to offer non-nutritious food. Adding air purifiers to classrooms would probably be good too.

The IRA bill does take a step forward here with funding for EV school buses, which is nice to see.

Expand full comment
Nicholas Weininger's avatar

Two important arguments this doesn't address:

1. the argument that US education spending has increased drastically over time and in the aggregate we haven't gotten better outcomes for it, e.g. at the beginning of this classic Scott Alexander post: https://slatestarcodex.com/2017/02/09/considerations-on-cost-disease/. This is compatible with a few well- and hopefully-studied interventions succeeding in improving outcomes, much as it can simultaneously be true that the Perry and Abecedarian preschools worked great but most state preschool subsidies don't do much good. And it should lead us to be skeptical by default of the modal proposed spending increase.

2. the Bryan Caplan argument that even for K-12 the social return to education spending is low in terms of real productivity impact, even if it raises test scores.

Both of these are totally consistent with the US doing reasonably well compared to other rich nations and not spending an outlier amount: the claim (which I think Caplan at least would heartily endorse) would be that all the other rich nations are wasting their money too and suffer from the same cost disease. It's worth looking at whether the trajectory of education spending over time is similar in other rich countries and how that correlates, or doesn't, to whatever we can measure about their outcomes.

Expand full comment
Kaleberg's avatar

A lot of the new spending has been for special education. Children with a variety of emotional and developmental problems used to be thrown by the wayside or warehoused. Now they are educated to some extent. It tends to be expensive and low yield, but it is very popular with parents, particularly wealthier parents who know how to game the system. If you get the right educational consultant, the public system will pick up a good chunk of your child's private school tuition.

There's also the issue of benefits. The rising cost of medical insurance has hit every employer in the US. The ongoing baby boom retirement wave isn't helping either.

Expand full comment
Alan Goldhammer's avatar

"A lot of the new spending has been for special education." This certainly not the case with my daughter who is a special ed instructor at a Title 1 school. Maybe it works at a more affluent school with a different demographic breakdown.

Expand full comment
Kaleberg's avatar

You are probably right. It probably is different for most people. I've just known too many people who were good at gaming the system. (My mother was a junior high school teacher, so I learned something about the business from her.)

Expand full comment
John C's avatar

"We spend about $13,000 per student (at purchasing power parity), while the average is around $10,000. Not a huge difference."

Isn't 30% more a big difference, especially for average-to-below-average results? That's a meaningful chunk of GDP, wasted!

Expand full comment
Richard Gadsden's avatar

A significant chunk of that is just Baumol's Cost Disease.

If you want teachers of a given quality, then you have to pay more in the US than in other countries because the US is a richer country and the competing careers pay better in the US than in other countries.

Education is an area where a large fraction of costs are skilled labour, the classic case for Baumol, so you'd expect costs for a given level of quality to be closer to a constant fraction of GDP than a constant amount at PPP.

Expand full comment
John C's avatar

No doubt that's part of it, but I still would not predict a constant ratio of education labour costs to GDP relative to other countries (all else being equal), especially given:

1) US GDP growth has been concentrated at the high end in recent decades (even more so than peer countries)

2) Inflation-adjusted teacher salaries have dropped over time in the US (https://www.nea.org/advocating-for-change/new-from-nea/average-teacher-salary-lower-today-ten-years-ago-nea-report-finds)

3) US teachers are not particularly highly paid in comparative terms, so a big chunk of cost must come from elsewhere (see https://www.businessinsider.com/teacher-salaries-by-country-2017-5 or https://www.chalk.com/resources/most-least-paid-teachers-in-world/)

I'd be interested to see more depth on this from Noah! This post also ignores the distribution of outcomes to the low end, in which I assume the US performs quite poorly.

Expand full comment
El Monstro's avatar

I would like to see median teacher salary compared to median overall salary and media salary for degreed professionals before I could make an informed comment as to whether US teachers are paid more comparatively. What little I have seen tends to indicate that they are not.

Expand full comment
Noah Smith's avatar

Except we get above average results as I show.

Expand full comment
John C's avatar

I think that’s debatable. Such a reading requires heavily discounting the PISA math scores, where the US is not just bad, but the worst country presented by a margin!

Expand full comment
john jacobs's avatar

Any studies showing what we spend on sports - playgrounds, stadiums and coaches etc in k-12 vs other countries?

Expand full comment
Erich DeLang's avatar

This is the sort of thing that Noah scratches the surface of in the last few paragraphs of the post but doesn’t quite flesh out. Could we, if we diverted playground, stadium, and coaching money from wealthy districts to poorer ones, actually get MORE bang for our buck in the form of average higher scores without expending an additional dime, better greater utility for each dollar?

Expand full comment
john jacobs's avatar

Would be nice to see it fleshed out or be pointed to where the answer might be. I truly am not even sure that other country's K-12 even have athletic programs and the buildings/stadiums etc that we have vs. sports being outside their k-12 programs in sport clubs.

Expand full comment
Kaleberg's avatar

Massachusetts has better educational results than most states, particularly among its poorer students. If I remember correctly, Black Massachusetts students outperformed white Alabama students on the PISA tests. This may be because Massachusetts has what they call the "cherry sheets" which are state level educational payments designed to balance disparities between wealthier and poorer school districts.

Mike the Mad Biologist used to post on this fairly often. He argued that instead of looking at Finland or China, we should be looking at Massachusetts for an educational model. This obviously won't fly since this would lead to the state taking over educational spending and cutting the advantage of moving to a more highly taxed school district to get one's child a better education.

Expand full comment
Belisarius's avatar

Here in Republican land, the opposition to increased school spending comes down to three major factors:

1) reflexive opposition to increased taxes and/or more govt.

2) cultural concerns about godless heathen liberals trying to indoctrinate our kids.

3) racial dynamics where we are basically forced to pour much/most of the additional spending into majority-black schools.

1 is boring and there probably isn't much we can do about it. Getting the message out that each $1 of educational spending results in >$1 of benefit long-term may win over a tiny % of these people, but that's about it.

2 is more interesting, I think, but I don't see how to overcome it except maybe allowing school choice vouchers or whatever. Which Dems would never support.

3 is a local issue due to consent decrees from the 60s and 70s, so it's not really important from a national perspective.

I still like seeing the data, and maybe if word got out widely, it would have an effect in the purple-ish suburban areas. But that's probably the extent of it, w.r.t. persuading people that greater education spending is worthwhile.

Expand full comment
Irene C's avatar

This is one of those broad issues with a *lot* of different moving parts, a lot of different metrics/measurements you could use, and the vast majority of people both within and outside the country having very strong opinions, because most adults in developed countries received some non-home-based education.

I have to say that, given my own definitions on this issue, my own observations of what is important in a good education, and my observations of what people outside the US think ... the US does actually have a very crappy education system. Why? First, I can't speak to math, so I won't. I have my own weird reason for being bad at it, so I know I can't generalize.

But I can talk about reading and knowledge acquisition.

The vast majority of US school districts use a balanced literacy approach to (not do) reading instruction. Balanced literacy is rebranded/slightly retooled whole language, an approach that is not supported by cognitive science research. Additionally, there is pervasive ideological opposition to teaching facts to schoolchildren under the age of 10, because these facts are deemed "developmentally inappropriate" and "oppressive". The dominance of this ideology means that ordinary classroom teachers will say things like "I have my own science" to support their classroom practices. As a result, by the US's *own* nationally standardized reading comprehension tests, no more than 40% of students read at grade level. And those scores have been flat for literal *decades*, ever since the widespread adoption of whole language in the 1950s.

Since the US is so dominated by the anti-knowledge-teaching ideology, we don't even have exams that directly test for general knowledge acquisition in our students. But everyone knows about those civics and geography tests that are done on teens every so often that show that they don't know the three branches of government or that New Mexico is part of the US?

Like... somehow white liberals never connect the dots and realize that those results mean the US educational system is bad. That is what a failing education system looks like. Children are supposed to leave the system with more knowledge than they had when they entered, and that is not happening.

This is so obvious a point that it's an article of faith among people in other developed countries. YouTube is full of mocking, gotcha videos where Brits and Australians go up to American adults on the streets of US cities and ask them basic general knowledge questions. The results are embarrassingly bad. Like, you wonder how these adults function, bad. If some ranking system says the US has a slightly better system than the UK using whatever metric it uses, but these videos are still being made? I'm skeptical the US actually has a good system. And it stands to reason that if kids aren't taught knowledge, they will age into being these adults.

Connect the dots: a good education system would not produce adults who lack basic knowledge.

Expand full comment
Bob Eno's avatar

Irene, What do you find when Americans go up to Brits and Australians on their streets, ask them basic knowledge questions, and post the gotcha video clips they choose on YouTube?

Not to say we should be satisfied with US public education, but I think we shouldn't be careless thinking about it.

I suspect part of the reason US kids don't do as well on some test measures is, indeed, because rote learning plays an important role in them, and is not valued here as highly as it is in other countries (particularly in Asia). Perhaps that's reflected in the view of teaching facts as "oppressive" that you see as pervasive. I think that since the 1930s US education has gradually moved towards a theory of optimal learning as primarily building creativity and problem-solving skills, rather than quickly retrievable information. My own view is that we may have generally gone too far in this direction, but that it's very clear to anyone who has taught high-performing students from Asia that the countries outperforming us on standardized tests haven't gone far enough.

Expand full comment
Irene C's avatar

There's no indication that US people are playing video taped gotcha on the streets of the UK or Australia. How does that refute the point, though?

It would be easier to not be careless if the US measured how much general knowledge young people enter and leave the US public school system with. As for what is valued in the US, there's a pattern that is so predictable that it's a broken record: some instance of an American being stupid about a piece of general knowledge makes the news, and there is a *chorus* of voices on social media lamenting that that general knowledge should be taught in schools.

And just .... do we all not realize that as a society we get to control what and how our society's kids are taught?

Expand full comment
Bob Eno's avatar

Irene, My point about gotcha YouTubes was that there is no inference to be drawn from the fact that it's possible to interview people on the street, find some who are ignorant of things we might label "general knowledge," and select those people to show on YouTube. It doesn't show that "American people are ignorant" or that our education system is bad. All it shows is that people in other English-speaking countries think it's fun to ridicule Americans. (Has any one of those YouTubes ever mentioned how many adults were interviewed to yield the few that have been selected?) It has no diagnostic value whatever, and what seems more concerning to me is the chorus of people who think it does. Poor reasoning skills is a deeper problem than lack of general knowledge.

I mentioned similar YouTubes in other countries precisely because you were making a comparative point vis a vis the UK and Australia, without any sign of observing that you had no basis for the comparison. Do you really think that if a video team spent a day in London or Sydney they couldn't come up with a series of clips of space cadets like those in the YouTubes you saw? It doesn't mean that adults in the UK and Australia are the same as, better than, or worse than in the US--this sort of comedic data is simply worthless, except as entertainment.

I don't really understand your complaint about testing. If you're saying that we should test students, then the response would be that we already do: up to their ears. Regular standardized testing at prescribed intervals is required by law in order to qualify for federal funding. All that data is analyzed at the state level to monitor educational quality. We spend many billions of dollars compiling these data, which form the basis for comparison with other countries. But your conclusion is that you doubt the data's value because of gotcha YouTubes.

If you're complaining that we should test "general knowledge" specifically, you're using a vague and subjective term that would have broadly different interpretations in different regions, for different economic classes, and so forth. It is not an idea that could guide either test designers or educators designing a curriculum.

Expand full comment
Griff's avatar

So your anecdotes trump the data? In what world does that tell us about education?

Expand full comment
Irene C's avatar

No, my point is that there are other aspects that people value in an education system, such as how much general knowledge it teaches, and there can be (and is) a sense that the US education system isn't effective based on those other values. It's also pernicious, because if the US did have a standard measure of its high school kids' general factual knowledge, we could just talk about that.

But that measurement does not currently exist. So yes, anecdotal evidence that we're producing people without basic general knowledge (like that New Mexico is part of the US) is just going to have to do.

Expand full comment
Troy Flint's avatar

How did you develop the impression that white liberals have a high opinion of the U.S. educational system? In major cities, the rat race among white liberal parents to get their children into one of the three local public schools that meet their standards makes the exit from Afghanistan look like an orderly procession. (This phenomenon is not limited to white liberals, but since you set those parameters, I’ll go with your language).

As to the larger question of educational quality, I think two things can be true: the U.S. educational system is better, according to the standard data-based metrics, than frequently given credit for, while at the same time being deficient in important ways like you reference in your anecdotes. This is not just an idle consideration because the public perception of school quality drives, to an extent, public willingness to allocate additional funds (more taxes) toward public education, which in turn, has an ill-defined but non-zero impact on educational outcomes.

One important issue raised by the social media videos of Americans flunking elementary knowledge questions is the distribution of performance across the overall student population.

The post doesn’t explore this at length but the group difference in proficiency levels between higher and lower income students is quite large. The gap is even more dramatic when comparing white students and students of east Asian descent to their black and Latino peers – and often overlooked -- to Native American students, Pacific Islander students, and students of many Asian ethnicities that are subsumed into the larger Asian category.

I’m sure there are papers out there that can rule on this more decisively, but I think it’s likely that the bottom quartile of U.S. students compares especially poorly to the bottom quartile of other OECD countries. Some of these people are probably even the subjects of the YouTube videos you mention! That doesn’t necessarily indicate that the average student is receiving a poor education.

Now, if you’re arguing that any school system in which certain groups of students consistently lag country miles behind the average is unacceptable, I agree wholeheartedly. Or, if you’re arguing that any school system in a rich country where the bottom quarter of the student population brings up the rear internationally is unacceptable, I would also strongly agree. That’s a bit different though than the original premise of Noah’s position, which is that the U.S. educational system is better than perceived and more or less average. Regardless of how you slice it though, we have a lot of work to do.

Expand full comment
darawk's avatar

There's another fact that makes your point here substantially stronger, which is that our scores improve dramatically if you disaggregate them by race:

https://nces.ed.gov/surveys/pisa/pisa2018/index.asp#/math/achievement

What you will find is that the US is only a tiny bit worse at educating asians than the best asian countries, better at educating "whites" than the best "white" countries, and of course much better at educating blacks and hispanics than their respective countries. So, when properly demographically adjusted, it's obvious that the US isn't just middle of the pack, but top tier in its educational performance.

Expand full comment
Freddie deBoer's avatar

Dozens of studies have shown that randomly placing students into schools of very differently perceived quality has no impact whatsoever on different performance

https://freddiedeboer.substack.com/p/education-doesnt-work-20

All educational effects are downstream of intrinsic ability differences and perceived effects are ~100% the result of hidden selection effects

Expand full comment
Bob Eno's avatar

The Substack discussion you link to is interesting, surprising, and depressing. I'd like to pursue a couple of lines that stem from it, to clarify its implications.

My teaching experience is at the higher ed level, so it can only be used for analogic thinking. But having seen class performance in a language course sequence vary wildly according to the perceived (and observed) quality of the instructors at each level for different student cohorts, it would be hard to convince me that teaching quality is irrelevant. It may well be that the A and C students in Years X and Y remained in place vis a vis their cohorts, but the B (and even the C) students in cohorts that had talented instructors in first-year classes outpaced the A students in cohorts that had poor instructors. That experience makes me naturally skeptical of data that show that only intrinsic ability matters. (I taught a third-year class, and the gap was so profound I was unable to bridge it with a remedial approach, and virtually all students who got bad breaks at the start failed to move on to the final, fourth year of the sequence--analogous to high school drop outs.)

If you look at US education by state and measure average SAT scores, there is pretty wide variance. The highest scores (Minnesota) outpace the lowest (Delaware) by 30%, on a fairly smooth continuum. But participation rates vary wildly, and in general much of the difference correlates high scores with low participation (better students being more likely to participate). But if you look in a band of eight states with participation rates of 41-48%, you get some substantial differences. The extremes are neighboring states: Virginia (1151) and West Virginia (1007). That means Virginia students score 751 points above SAT-"Zero", which is 24% higher than WV students' 607. Maryland, another Virginia neighbor, is in the same participation group, has an average score of 1073 (673), pretty much splitting the difference. The results are a little more complicated if you note that WV has about a 10% higher participation rate compared to Virginia, but Maryland's rate is 4% higher than WV. (My extensive data research can be found at https://blog.prepscholar.com/average-sat-scores-by-state-most-recent, which required an exhausting forty seconds to discover.)

If intrinsic ability accounts for 100% of the differences--let's call that "IQ" to be tendentious--then if we allow that Maryland SAT participants score 100, similar Virginia kids score 112 and WV kids 90. How likely is this to be accurate?

Going back to my own experience, which involved teaching student groups with disproportionately East Asian background (all very successful high school or college graduates), it is very hard for me to doubt that different educational systems can tend to produce very different learning styles and skill strengths, and also different standardized test results (and, actually, different non-standardized test results).

My only-semi-relevant experience and five minutes of online research don't amount for much compared to the many sources you cite in your Substack post. But they are reasons, along with stubbornness and generic liberal optimism, why I'm impressed by your arguments, but not persuaded that they will hold up over time, as future data accumulate and novel school approaches and research methodologies are tested.

Expand full comment
Alex S's avatar

What if you randomly assigned the students to a school where they had so many schoolyard fights they all got brain injuries? That’d affect their academic performance.

nb you mention the ol “genetic influence on academic performance”, very popular with skull measurers and people who comment at Slate Star Codex, but there is of course no reason to believe this exists and no way to generate high quality evidence for it; people who believe in it are mostly demonstrating they don't know the difference between “heritable” and “genetic”. For instance, observational twin studies don’t prove genetic effects.

Expand full comment
richnugen's avatar

Parents know how much extra $ goes into fundraising, school supplies and other out of pocket costs like pre-K, after-school, tutoring, academic enrichment that probably aren't counted. Some teachers probably incur meaningful spend out of their pocket as well. Looking only at official spend figures is like judging a company's financial performance with only EBITDA or its many derivatives.

Expand full comment
myst_05's avatar

You’re assuming that this is not the case in European countries?

Expand full comment
richnugen's avatar

Nothing about my comment specifies any country. If there is an assumption being made anywhere, it is being made by you.

Expand full comment
Alan Goldhammer's avatar

"Some teachers probably incur meaningful spend out of their pocket as well."

My daughter teaches at a Title 1 school in the Baltimore/Washington area and she spends about $200 for supplies each year. We have friends who teach in the DC school system and it's the same thing there. They all have Amazon wish lists which we contribute to each year. DC has one of the higher spends per pupil of any school system and it's still a wreck.

Expand full comment
Terry P's avatar

I am not sure of the comparison, but vs. most developed countries, I’d suspect the US education system has a much wider range of spending per student due to our local funding model. Also hard to look at education in isolation of the differences in the social safety net. We ask schools to do a lot more than educate. Most of the data I have looked at suggests academic achievement is highly correlated to family income.

Expand full comment
User's avatar
Comment deleted
Jul 31, 2022
Comment deleted
Expand full comment
Terry P's avatar

Looking at the data from a few years ago shows ~30% of states that skew support to low income districts, leaving 70% or so who don't. NC (which I know best) is "neutral", leaving over a billion dollar discrepancy between low and high income districts on a per pupil basis. I have no belief funding will "fix" educational attainment due to the second half of my comment above, but I suspect it would help.

Expand full comment
John E's avatar

While states are mediocre at best in balancing funding, federal money is heavily tilted towards low income districts.

Expand full comment
Terry P's avatar

Thanks. Fair point. The comparison I looked at was for state and local funding sources. It doesn’t look like federal funds (8% of total spend) would close the gap, but directionally helpful.

Expand full comment
Greg Costigan's avatar

Yeah I don’t know Noah I’m with you on most things but this one feels like a stretch.

- US is mid table on most indexes

- Should we not be aspiring for East Asia levels?

- If not, why not?

My read is the performance is mediocre. Sure it’s not terrible. But we should not accept mediocrity.

Expand full comment
Kenny Easwaran's avatar

This isn't about "accepting" mediocrity - it's about showing that anyone who says "every developed country has better education than us" is just wrong. It's not a reason not to improve, just a reason to recognize that we don't have any obvious easy gains to make by reverting to the mean.

Expand full comment
Kaleberg's avatar

Be careful of comparisons with China. We tend to see the more developed nation, but there's still a lot of China well behind. They don't show up in most international comparisons. The US could score a lot better if we just dropped maybe a dozen states from our national ranking.

Expand full comment