Today's Revision to IHME Projections Is Good News for NC and Shows How Tough It Is to Predict COVID

Many news reports relied on the University of Washington's Institute for Health Metrics and Evaluation (IHME) for projections on the hospital and ICU demand and deaths expected from COVID. 

If you've read about hospitals and ICUs expected to be overrun or when COVID is expected to peak in your state, it pretty much comes from this source and this model.  They are continuing to evolve the model as new data comes in.  In NC, the news and Observer reported IHME projecting 2411 deaths in NC and a shortage of hospital beds.  Two days later, they revised the projection to 1721 deaths and hospital needs within the state's resources (though with a shortage of ICU beds).

The IHME site lets you choose a state and see when the virus will peak and what the range of hospital bed, ICU and ventilator demands will be through August, as well as projected deaths.  They are very open with the data and make it available for downloads, so I thought I'd spot-check the model for a few states yesterday.  I chose NY, LA, and NJ because NY is the biggest hot spot ant LA and NJ are predicted to peak in the next couple days per the model so it could give us an idea of how accurate their projections are for state's they are projecting to peak weeks later. Hospitalization data is not easy to come by, but these numbers come from pretty reliable sources:

Projections of hospitalizations for April 5, 2020:

State	IHME Hosp	IHME Min	Actual	Difference
North Carolina	1,300	1,083	261	398.08%
New York	69,185	48,161	16,479	319.84%
Louisiana	6,879	2,918	1,803	281.53%
New Jersey	8,657	6,189	4,000	116.43%

These projections had been revised in the past week, but still were well off from the actual figures. IHME offers a minimum and maximum amount around their projections but in all the cases I reviewed, the results for yesterday were well below even the minimum projections. 

This morning,  IHME posted a major update to their model which changed many of their numbers. The overall projected deaths for the US dropped 12.5% from 93,000 to 81,000. The revisions were pretty dramatic for North Carolina:

Here were there projected hospital demands for North Carolina as of yesterday:

Click to Enlarge

And here is their revised projection:

Projected deaths went from this:

To this:

Note that our peak has also shifted from late April to mid-April.  Also, IHME now expects NC's ICU demand from COVID to be less than available resources in the state. 


One of the big effects on these projections is that we're finally getting some more examples of where countries reached their peak new deaths curve.  Until now, the only country we had data on that expanded like we had was China, so the assumption was that the effect of social distancing and other measures would work like they did there (I have concerns about the accuracy of China's current reporting on their COVID situation, but that's another matter).  We've not seen curves peak in communities Spain and Italy that show the social distancing worked much faster than IHME expected in their model. 

While the adjustments to IHME's model are encouraging, they still paint a bleak picture for many other locations where hospitals and ICUs are expected to be overrun. I will try to track their new projections against the peaks this week and hope to share an update in the future.

While this post points out the inaccuracy of some of IHME's projections, they are doing an incredible job pulling together a wide range of data to build their model and offer national, state, and local governments at least something to work with in a very tough situation.  IHME is being up front and honest about their projections and constantly looking for ways to improve it.  If I had any complaint, it would be why we only have one projection. 

Those of us in NC are used to hurricane season where we look at spaghetti maps like this one:



This map shows the projected path of last year's hurricane Dorian from over a dozen different models developed by different groups. They all have access to the same data but come to very different conclusions once you get a day or two out on their projections.

IHME is trying to project the path of pandemic COVID-19 a couple weeks out, has no prior "storms" like it to test their model on, and has few or no other groups developing other models to help them sharpen their work.  Let's hope others emerge either for COVID or at least for the next one.