Since the CPS Food Security Supplement ended, that sentence is everywhere: state agencies, food banks, funders. It deserves an honest ledger. I build these models myself. Advantages first.
A model can produce a number for every place. The flagship survey gave most states one number; a model fills the map, county by county, and in finer detail. It makes small groups visible where direct surveys leave blank cells. It is cheap, fast, hard to cancel: built from data already collected, it does not vanish when a federal survey does.
Now, the other side, and one picture carries most of it.
In 2014, USDA-ERS published a model that predicts national food insecurity from three numbers: unemployment, inflation, and the relative price of food. Over 2001–2012, it was excellent — R-squared 0.94, average miss a third of a point. As good as this kind of model gets.
Freeze its coefficients and run it forward. It drifts a point high before anything dramatic happens. For 2020, it says 18.5% — worse than any year on record. The survey measured 10.5%, the lowest in nearly two decades. Eight points apart. The model knew that unemployment like 2020’s had always meant food insecurity like 2009’s. It could not know the rules had changed: expanded benefits and emergency programs broke the relationship it had learned. By 2022–2024, the error changes character: the model runs high on its inflation term while actual food insecurity climbs for reasons it cannot see. One equation, wrong two ways inside four years. That is what a structural shift does to frozen parameters.
We only know the model missed because the survey kept score. From 2025 on, no official observed line. The dashed line keeps going, and nothing checks it.
The quieter problems compound. Every number a model produces is a prediction of food insecurity, not a report of it. The prediction leans on the measurement it learned from — already two years stale. A yes/no modeled rate erases the severity tiers in which food insecurity goals are written. Margins widen in small places; look-alike areas get look-alike numbers.
Worst is the assistance trap: feed a model SNAP enrollment or pantry visits, and a benefit cut reads as food insecurity going down.
The honest conclusion is not that models are useless. They are the reach layer, not the anchor: they extend a measurement, they cannot replace one. The government works this way elsewhere: the Census’s small-area poverty estimates and the CDC’s neighborhood health estimates both rely on surveys that still exist. Any state funding a modeled map should fund the direct measure that disciplines it — and demand it in writing: what goes in, what is assumed, how wide the uncertainty runs, how it is validated.
A model without a survey is a forecast without a scoreboard.
Five questions for any vendor of modeled food-insecurity estimates:
(1) What direct measurement is the model calibrated to, from what years, and what is the recalibration and validation plan now that the CPS-FSS is gone?
(2) Will the full input list, data vintages, and model documentation be published, so outsiders can evaluate how it was built?
(3) What does the model assume about relationships staying stable, about which places resemble which, and what kind of shock would break those assumptions?
(4) What are the margins of error for the smallest geography you will publish, and will they be printed next to every estimate?
(5) Will every number be labeled as a modeled prediction, not a measured rate of food insecurity?
A good answer to (1) exists: put the USDA food-security module on a survey that the state already runs — several states field a food-hardship item on the CDC’s BRFSS today — and recalibrate to that.
A vendor with good answers is selling the reach layer. A vendor without them is selling the dashed line.