What Are They Measuring?

I became aware of this working paper by Phil Kerpen, Stephen Moore and Casey B. Mulligan at the National Bureau of Economic Research on the “final report card” on the various states’ response to COVID-19 based on mortality, impact on the states’ economies, and impact on education within the states via the Wall Street Journal. Here’s a summary of their methodology:

Our measures fall into three categories: the economy, education, and mortality. For economic performance we used two measures: unemployment and GDP by state. For education we used a single metric: the Burbio cumulative in-person instruction percentage for the complete 2020-2021 school year, with hybrid instruction weighted half. For ID-associated deaths reported to the CDC and all-cause excess mortality. Of course, even without a pandemic, states populations are heterogeneous and their economies emphasize different industries. And because the pandemic had a much more negative effect on economic output in some industries (such as entertainment, energy production, mining, hotels and food), we adjust unemployment and GDP changes for industry composition. We adjust COVID mortality (through March 5, 2022) for age and “metabolic health,” by which we mean the pre-pandemic prevalence of obesity and diabetes – as these are highly correlated with higher death rates from the virus.

Economy and schooling are positively correlated (correlation coefficient = 0.43), which suggests a relationship between the willingness of the population (or its politicians) to resume normal activity in business and school. MT, SD, NE, and UT are the states highest on the economy score and also among only seven states to exceed 85 percent open schools. The correlation between health and economy scores is essentially zero, which suggests that states that withdrew the most from economic activity did not significantly improve health by doing so.

and here’s a quick take on their findings:

The approach I generally use in assessing such studies is to come up with the simplest explanation I can that might explain the results. In this case what leapt out at me was two things. First, the states that were in the top 10 included many of the states with the lowest level of income inequality while the states in the bottom 10 were among those with the highest levels of income inequality. The second thing was that quite a few although not all of the states in the top 10 were among the whitest states in the Union, e.g. Utah, Nebraska, Vermont. Those two observations may be related.

I also briefly considered whether billionaires per capita were not sufficient to explain high levels of income inequality. It isn’t although, oddly, absolute number of billionaires seems to be.

All of which leads me to suspect that the authors may not be measuring what they think they’re measuring but rather they may be measuring social cohesion.

My observation throughout the COVID-19 pandemic is that despite all of the huffing and puffing policy seems to be a lot less important than other factors like demographics, population density, and lifestyle. I just don’t see the findings of this study as being as important as the authors seem to.