The Urban Exodus Narratives Are Wrong

It is possible that New York City lost 468,000 residents between 2020 and 2022, as the Census Bureau estimates, which would be a serious alarm bell for the city’s political and corporate leadership. This data serves as the basis of an urban exodus narrative that has gripped the country’s major media outlets and intellectual commentariat and buttresses all kinds of claims about how cities are dead or crime has made America’s urban areas unlivable.

But it is also possible, using that exact same data, to come to a very different conclusion. It is possible New York City lost less than 100,000 people, perhaps as few as 90,000, over the same two-year period that included a global pandemic that killed 45,000 more people than typically die in the city. After accounting for the pandemic-related excess deaths, you’d be talking about half a percent of a population change over a two-year period, a time in which basically everything that made the city worth living in shut down for a prolonged period. If anything, this would be a story of urban resilience, not decay, during some of the most disruptive years in the city’s history.

Only one of these stories about New York City is accurate. But we don’t know which one. And neither does the Census Bureau. That’s because, as impressive as the annual Census data estimates are, they are just that, estimates. And estimates come with margins of error, something the Census Bureau doesn’t clearly state in its press releases but exist nonetheless in the fine print of the methodology page. And the average margin of error for the Census estimates across all counties is 2.9 percent. 

“I would definitely use a lot of caution interpreting some of these population changes, especially in cities that have a lot of immigrants,” said Adrian Pietrzak, a PhD student at Princeton who uses Census data to study urban land use and housing politics. “That seems to be the biggest place these estimates might go wrong.”

This is important not just for the narratives that appear in papers. These population estimates dictate how more than a trillion dollars in federal funding is allocated every year. In 2015, the most recent year specific data is available, almost $700 billion was allocated through programs where Census population estimates dictate who gets how much, but more recent estimates suggest more than $1.5 trillion is allocated this way.

On the surface, the annual population estimates are calculated using a simple formula. The Census takes the numbers from the last official count that occurs every 10 years, known as the “population base,” subtracts deaths, and adds births and migration to come up with a new estimate. 

“But that’s a lot harder than it sounds,” Pietrzak said. For both cross-border migrants and people moving within the U.S., it is actually quite hard to track where people go to within a reasonable degree of certainty. Many researchers use USPS change of address data to track internal migrations, but lots of people don’t use that tool. The Census Bureau looks at IRS data and Medicare enrollment, but, similarly, lots of people don’t file taxes—estimates vary from around seven to 10 million people a year—and not everyone is eligible for Medicare. 

For this decade, the Census Bureau has an even bigger challenge considering the “population base” is an unstable one. We know, due to the challenges of conducting a census during the height of a global pandemic, that the 2020 count was deeply flawed. We already know it undercounted Hispanic, Black and Native American residents more than in past counts. And we know these errors disproportionately impact cities and urban areas, suggesting that the margin of error is likely even higher for urban areas than the national average.

These errors mimic some of the same distortions we saw in the way the pandemic was covered more generally, with a focus on what it meant, as Pietrzak put it, for “affluent white Americans leaving for the Hamptons or something like that. We were coloring too much of the experience based on that story, and I think this might be extending the issue because of the difficulties estimating immigrant groups, racial minorities, and undocumented people.”

At the very least, an easy way to alleviate some confusion is for the Census to prominently mention the margin of error in its press releases and other publications relating to annual estimates, both for the country as a whole and for the specific counties it chooses to highlight. 

Instead, it does the opposite. Not only does it not mention that it even has a margin of error, but it publishes precise round numbers that give the illusion of accuracy. For example, the press release claims 9,421 people moved out of San Francisco County, California in 2022. This gives the impression the Census has identified 9,421 discrete individuals it has actually counted, especially since most people associate the Census Bureau with the decennial Census in which it does indeed try to count every single individual. But it has done nothing of the sort. That 9,421 figure is a very educated guess, but a guess nonetheless.

Not only that, but even if one is aware of the margin of error concept, it is difficult to find it. One has to go to a link in the footnote of the press release to get the estimation data by county and then calculate the margin of error yourself. The only margin of error estimate the Census directly states is an average across all counties on its methodology page—which they don’t even call a margin of error but “mean absolute percent error” or MAPE—but that margin could vary tremendously based on the demographics of the county.

The danger here is not only that $1.5 trillion in annual funding gets misallocated based on bad guesses, but that politicians, business leaders, major corporations, and other influential figures form narratives about our cities that are fundamentally incorrect. “Harder to count, harder to estimate groups might be underrepresented,” said Pietrzak, “and it might be affecting how we think about our cities evolving in a pretty major way.”