Modeling COVID-19

I found this article at STAT by Sharon Begley interesting:

A widely followed model for projecting Covid-19 deaths in the U.S. is producing results that have been bouncing up and down like an unpredictable fever, and now epidemiologists are criticizing it as flawed and misleading for both the public and policy makers. In particular, they warn against relying on it as the basis for government decision-making, including on “re-opening America.”

“It’s not a model that most of us in the infectious disease epidemiology field think is well suited” to projecting Covid-19 deaths, epidemiologist Marc Lipsitch of the Harvard T.H. Chan School of Public Health told reporters this week, referring to projections by the Institute for Health Metrics and Evaluation at the University of Washington.

and

The chief reason the IHME projections worry some experts, Etzioni said, is that “the fact that they overshot” — initially projecting up to 240,000 U.S. deaths, compared with fewer than 70,000 now — “will be used to suggest that the government response prevented an even greater catastrophe, when in fact the predictions were shaky in the first place.”

That could produce misplaced confidence in the effectiveness of the social distancing policies, which in turn could produce complacency about what might be needed to keep the epidemic from blowing up again.

If you’re not already aware of it you may find this informative:

There are two tried-and-true ways to model an epidemic. The most established, dating back a century, calculates how many people are susceptible to a virus (in the case of the new coronavirus, everyone), how many become exposed, how many of those become infected, and how many recover and therefore have immunity (at least for a while). Such “SEIR” models then use what researchers know about a virus’s behavior, such as how easily it spreads and how long it takes for symptoms of infection to appear, to calculate how long it takes for people to move from susceptible to infected to recovered (or dead).

“The fundamental concept of infectious disease epidemiology is that infections spread when there are two things: infected people and susceptible people,” Lipsitch said.

Newer, “agent-based models” are like the video game SimCity, but with a rampaging pathogen: using computing power unimagined even a decade ago, they simulate the interactions of millions of individuals as they work, play, travel, and otherwise go about their lives. Both of these approaches have often nailed projections of, for instance, U.S. cases of seasonal flu.

The IHME model doesn’t use either approach.

If I were made of money and had nothing but time, I would try feeding every shred of information we have on people who’ve contracted and people who’ve died of COVID-19 on a day by day basis into a neural net. What you’d get from the exercise would probably not be actionable but it would be interesting if it were better able to predict incidence and outcomes than the “tried and true” approaches.

I suspect that none of the models are actually much use as a guide to policy-makers because they depend so greatly on their assumptions. A reality of model creation is that what a model produces for you depends on what you put into it, particularly on the model’s assumptions. For example:

how many people are susceptible to a virus (in the case of the new coronavirus, everyone)

is something we don’t necessarily know is true or false. It’s an assumption. It may be that some previous pathogen is enough like SARS-CoV-2 that those who contracted that previous pathogen and survived have some level of immunity to the virus we’re facing now.

Additionally, “susceptibility” has more than one component. The component referred to above is those who have immunity because they’ve already had the disease and that’s given them some level of immunity over some period of time. That, too, is an assumption. It may be true or false.

Some people are immune to some pathogens by virtue of hereditary or congenital immunity. For example, some are immune to HIV. There are some forms of cancer which strike people who have certain genes. We simply don’t know enough to claim with any confidence that everybody is susceptible to SARS-CoV-2. We don’t know that everybody is susceptible to the flu.