i worked with detailed goverment survey data on gdp, retail & consumption, household & employer jobs for 30 years.
noah gets it absolutely right: initial prints are noisy, updates are systematically produced using more info, and surveys rebenchmarked to periodic censuses.
so it's messy but honest, & probably as good as it gets. thx noah!
As a fellow tech guy, my hunch is Balaji and Sacks are operating from the tech perspective that nearly everything you care about in your app/service/etc can be measured consistently and in real time. And the need to go back and revise isn’t there. I can see how this could lead them to think there’s something wrong with government stats on something as noisy as employment.
A slightly above average tech person would have first hand experience with data updates. There certainly are arenas that are as you describe - exact and real time - but just as many that aren’t. For example, compensating transactions, paxos algorithms, the CAP theorem. Not to mention a common marketing analytics approach is a using a cohort view to understand performance over time where it takes time for data to “mature”.
I have some really odd/newbie questions. This seems like a redundant inaccurate way to collect data that everyone seems to drool over. When any business issues a paycheck, if everything were automated and made anonymous, wouldn't the IRS (had it been updated and fully funded of course) have real time data and more accurate models? Especially if there interoperability, data sharing and eliminating redundancy?
I'm sure lots of people have already thought of these issues but I'm curious since I was surprised by how the data is actually collected.
Barring under the table jobs of course.
Also I'm assuming it makes more sense to look at 6 months, Year over year data?
But in critical moments, in order for fiscal/monetary policy work in tandem you'd want month to month trends? Not sure if I articulated that well....
Every two weeks when I get my paycheck I see some money gets taken out for payroll taxes. Is it possible to use some sort of tax data as a way to measure employment on a faster interval?
I swim in the world of normie tech people, but they are tech people who are rational. There are many tech people who have their heads screwed on straight. In fact most are in that category, I'd guess.
In Econ 101 we're taught about 'rational economic man' who would buy the Ford Pinto as a rational, economical family car. Across campus in business Marketing 101 images of a Ford Mustang Convertible with some babe sitting high on the seat back demonstrate how to sell to that 'rational man.'
Good reminder of difference between the two surveys, thanks. Divergence between the two has been pretty high since Covid. The way the government does seasonals (and each survey has its own) doesn’t really work well after precipitous events like Covid or the Great Recession. Eventually the two surveys come back into line (or in line with explanations, eg changes in self employment) so it is good to look at both surveys and draw one’s own conclusions.
Divergent numbers do not necessarily mean that the government is faking anything- why not fake both surveys, then? However, one would be a fool to place great faith in government. A good example is the 2020 census data, which was likely faked to benefit blue states in the electoral college (probability of several different states all being slanted in the same, wrong direction by accident is very low) , and the government was happy to correct the stats fairly quickly afterwards…maybe because there won’t be another adjustment for 10 years.
While in general government statisticians are very professional, and FBI agents are very professional, we know the deep state bias and most professional employees keep their mouths shut if they know what is good for them, meaning government workers cannot be trusted. I assume things are usually on the up and up in most cases, but I also have to leave a little room for doubt.
I think anything David Sacks writes needs to be seen in the context that he’s the finance chair for Ron DeSantis and hosted that disasterousTwitter launch for DeSantis’ campaign.
I’m not even trying to say this in a cynical or harsh way (trust me, I have many many harsh things to say about Sacks) but just as a framing that we should treat his utterances the same way we would if the DNC chair or RNC chair were making a public statement. It’s politics 101 for Sacks to highlight anything that may put President Biden in a bad light and would maybe help strengthen the case that Ron DeSantis should be President. In this case, that means trying to twist employment data to make it seem like the economy is worse than it really is and to make it seem like job losses are worse than they really are.
So, I think it’s completely fair to say that Sacks is being willfully dishonest as it serves the narrative interest he’s invested in; Biden is bad for the economy and this is why we should elect Ron DeSantis.
Your comment about forecasts being lazy to update their model make me remember about Reserve Bank of Australia, its forecast for wage growth was consistently wrong from 2013 to 2021, and that's why its governor (Philip Lowe) got a lot of criticism.
Noah, your explanations are always excellent. If people are interested in detail review of these type government report they should listen to the podcast "Inside Economics" from Moody Analytics and Mark Zandi, it is very good. Odd Lots is also an excellent podcast on economics.
Lots of people don't understand the differences between the two surveys, or how the numbers are collected, or what statistical assumptions or involved, or what a revision is, so...these are important things to explain, I think!
Not my place to tell you how to pick your friends but I can't say I agree that either he operates in good faith or that he deserves any kind of response. Fundamentally a bad actor in every domain.
Nah, he's just a bit out there. I've known him for 20 years and he has always been like this. I personally like weird people and original thinkers even if they're wrong 9 out of 10 times. As far as I can tell he's not hurting anyone.
Respectfully, I think you’re too close to the situation to have s truly unbiased opinion. This is someone who has openly advocated for Silicon Valley seceding from the US and who holds some pretty deranged views that are incompatible with liberal democracy.
They may or may not be cranks -- I don't know either of them or their work -- but this kind of shallow, confident, incorrect analysis is somewhat pervasive with engineers outside of their chosen field, as anyone who has ever worked with any will know. Just spend time on HackerNews, where they pat themselves on the back about high quality they think their discussions are relative to the rest of the internet and see how low-quality the conversations are when the subject is anything but computers.
Well, OK, but I think lots of people from lots of fields do this...look at humanities people telling economists that their whole discipline is bunk... ;-)
I'm sure Sacks and Balaji did some research and asked some economists about the discrepancies they noticed before posting to Twitter. Right? *Right*??!?
i worked with detailed goverment survey data on gdp, retail & consumption, household & employer jobs for 30 years.
noah gets it absolutely right: initial prints are noisy, updates are systematically produced using more info, and surveys rebenchmarked to periodic censuses.
so it's messy but honest, & probably as good as it gets. thx noah!
Thank you!!
Thanks, this was helpful.
As a fellow tech guy, my hunch is Balaji and Sacks are operating from the tech perspective that nearly everything you care about in your app/service/etc can be measured consistently and in real time. And the need to go back and revise isn’t there. I can see how this could lead them to think there’s something wrong with government stats on something as noisy as employment.
A slightly above average tech person would have first hand experience with data updates. There certainly are arenas that are as you describe - exact and real time - but just as many that aren’t. For example, compensating transactions, paxos algorithms, the CAP theorem. Not to mention a common marketing analytics approach is a using a cohort view to understand performance over time where it takes time for data to “mature”.
Most insightful and thoughtful. Thank you.
Thanks!!
I have some really odd/newbie questions. This seems like a redundant inaccurate way to collect data that everyone seems to drool over. When any business issues a paycheck, if everything were automated and made anonymous, wouldn't the IRS (had it been updated and fully funded of course) have real time data and more accurate models? Especially if there interoperability, data sharing and eliminating redundancy?
I'm sure lots of people have already thought of these issues but I'm curious since I was surprised by how the data is actually collected.
Barring under the table jobs of course.
Also I'm assuming it makes more sense to look at 6 months, Year over year data?
But in critical moments, in order for fiscal/monetary policy work in tandem you'd want month to month trends? Not sure if I articulated that well....
The government can't just use it without people's consent, but there is a private data set that's like this.
https://adpemploymentreport.com/
How is it a privacy violation to publish just the count of how many people are paying FICA, not the amounts being paid or the payee or payer?
https://adpemploymentreport.com/
Every two weeks when I get my paycheck I see some money gets taken out for payroll taxes. Is it possible to use some sort of tax data as a way to measure employment on a faster interval?
Perhaps!
I swim in the world of normie tech people, but they are tech people who are rational. There are many tech people who have their heads screwed on straight. In fact most are in that category, I'd guess.
Love this analysis.
If macroeconomics is based on the rational behavior of buyers and sellers, why are there more than few irrational economists?
Because if people are rational, they are rational only in aggregate, not individually!
So much for me remembering my prof Samuelson a half century ago!!
🐾🐾🐾🐾🐾🐾🐰
Power to rabbits!
In Econ 101 we're taught about 'rational economic man' who would buy the Ford Pinto as a rational, economical family car. Across campus in business Marketing 101 images of a Ford Mustang Convertible with some babe sitting high on the seat back demonstrate how to sell to that 'rational man.'
The entire business department is low-key warring with an econ dept that expects competition and rational behavior to drive profits to zero...
The key insight for me: “we shouldn’t put too much stock in any one month or even one quarter of data “
Good reminder of difference between the two surveys, thanks. Divergence between the two has been pretty high since Covid. The way the government does seasonals (and each survey has its own) doesn’t really work well after precipitous events like Covid or the Great Recession. Eventually the two surveys come back into line (or in line with explanations, eg changes in self employment) so it is good to look at both surveys and draw one’s own conclusions.
Divergent numbers do not necessarily mean that the government is faking anything- why not fake both surveys, then? However, one would be a fool to place great faith in government. A good example is the 2020 census data, which was likely faked to benefit blue states in the electoral college (probability of several different states all being slanted in the same, wrong direction by accident is very low) , and the government was happy to correct the stats fairly quickly afterwards…maybe because there won’t be another adjustment for 10 years.
While in general government statisticians are very professional, and FBI agents are very professional, we know the deep state bias and most professional employees keep their mouths shut if they know what is good for them, meaning government workers cannot be trusted. I assume things are usually on the up and up in most cases, but I also have to leave a little room for doubt.
Good article. You have a hanging “The” at the end of one of the paragraphs.
Thanks, fixed!
I think anything David Sacks writes needs to be seen in the context that he’s the finance chair for Ron DeSantis and hosted that disasterousTwitter launch for DeSantis’ campaign.
I’m not even trying to say this in a cynical or harsh way (trust me, I have many many harsh things to say about Sacks) but just as a framing that we should treat his utterances the same way we would if the DNC chair or RNC chair were making a public statement. It’s politics 101 for Sacks to highlight anything that may put President Biden in a bad light and would maybe help strengthen the case that Ron DeSantis should be President. In this case, that means trying to twist employment data to make it seem like the economy is worse than it really is and to make it seem like job losses are worse than they really are.
So, I think it’s completely fair to say that Sacks is being willfully dishonest as it serves the narrative interest he’s invested in; Biden is bad for the economy and this is why we should elect Ron DeSantis.
Sacks is absolutely operating in bad faith almost 100% of the time, whereas Balaji is just a crank who clearly hates the US.
Thank you for this great thread Noah!
Your comment about forecasts being lazy to update their model make me remember about Reserve Bank of Australia, its forecast for wage growth was consistently wrong from 2013 to 2021, and that's why its governor (Philip Lowe) got a lot of criticism.
Binder prompted me to look up the ADP numbers:
https://adpemploymentreport.com/
I wonder how those two would respond?
Noah, your explanations are always excellent. If people are interested in detail review of these type government report they should listen to the podcast "Inside Economics" from Moody Analytics and Mark Zandi, it is very good. Odd Lots is also an excellent podcast on economics.
They’re a pair of cranks, nothing more to it than that
Lots of people don't understand the differences between the two surveys, or how the numbers are collected, or what statistical assumptions or involved, or what a revision is, so...these are important things to explain, I think!
Also Balaji is a friend of mine, and while I think his ideas are often pretty wacky, he's operating in good faith and deserves a good-faith response!
Not my place to tell you how to pick your friends but I can't say I agree that either he operates in good faith or that he deserves any kind of response. Fundamentally a bad actor in every domain.
Nah, he's just a bit out there. I've known him for 20 years and he has always been like this. I personally like weird people and original thinkers even if they're wrong 9 out of 10 times. As far as I can tell he's not hurting anyone.
Indeed! Reading your Twitter is a lot of fun!
Respectfully, I think you’re too close to the situation to have s truly unbiased opinion. This is someone who has openly advocated for Silicon Valley seceding from the US and who holds some pretty deranged views that are incompatible with liberal democracy.
Sorry I didn't mean to come off as critical. Absolutely agree that this is super valuable stuff to explain, thanks for doing it :)
Thanks, man!
They may or may not be cranks -- I don't know either of them or their work -- but this kind of shallow, confident, incorrect analysis is somewhat pervasive with engineers outside of their chosen field, as anyone who has ever worked with any will know. Just spend time on HackerNews, where they pat themselves on the back about high quality they think their discussions are relative to the rest of the internet and see how low-quality the conversations are when the subject is anything but computers.
Well, OK, but I think lots of people from lots of fields do this...look at humanities people telling economists that their whole discipline is bunk... ;-)
Nasty. Why do that! Who does it convince?
I'm sure Sacks and Balaji did some research and asked some economists about the discrepancies they noticed before posting to Twitter. Right? *Right*??!?