There is a forecasting audit that costs nothing, takes a week, and is more informative than most vendor evaluations: compute the naive forecast, same as last year adjusted for trend, for every item you plan, and measure your actual forecasting process against it, over the last two years of history you already possess.
Few organisations ever run this test, and the published evidence suggests why the silence is comfortable. The forecast value added literature, and the field's long-running competitions, keep producing the same finding: simple methods are embarrassingly hard to beat, and a large share of enterprise forecasts, after the statistical model, the planner's adjustments and the consensus meeting have each had their turn, land worse than the free baseline. The human half of that chain has its own uncomfortable evidence base (half the forecast is a person, and nobody scores that half).
Half the forecasting effort in industry is spent losing to a number that costs nothing to produce.
What the naive test actually measures
The naive baseline is not a straw man. It is the operational definition of "no forecasting process at all", what you would know with zero software, zero analysts and zero meetings. Every step of your actual process, model, override, consensus, exists to add accuracy on top of it, and forecast value added is simply the audit of whether each step does:
- The model versus naive: does the statistical layer beat last year plus trend, item by item? For stable, fast-moving items it usually does. For intermittent, lumpy and short-history items, frequently not, and knowing which is which redirects both the modelling effort and the software spend.
- Each human step versus its input: did the planner's adjustment improve on the model? Did the consensus meeting improve on the planner? Measured per step, the process reveals where it adds value and where it ritually subtracts it.
- The whole chain versus naive: the final, uncomfortable number. If the end-to-end process loses to the free baseline for a third of the portfolio, that third is being planned at a premium price for negative value.
Accuracy that does not beat the naive baseline is not accuracy. It is expensive noise with a review cadence.
Why this precedes any technology decision
The current market moment makes the test urgent rather than merely hygienic. AI forecasting claims are everywhere, vendor benchmarks are self-administered, and accuracy improvements of 8 to 20% are routinely cited, usually without stating the baseline they are measured against. A buyer who has never computed their own naive benchmark has no way to price any of those claims: an "85% accurate" system may be brilliant or may be losing to last-year-plus-trend, and the sales material will not volunteer which.
Run the audit first and every subsequent conversation changes. You know which segments of your portfolio have forecastable signal and which do not, so you can demand segment-level proof rather than portfolio-level averages. You know what your current process genuinely adds, so a vendor must beat your reality, not your anxiety. And you have the discipline that should govern the system after purchase: every model, every adjustment and every meeting continuously scored against the baseline, with the steps that fail retired without sentiment, because the test that justified the purchase keeps running in production.
One more outcome deserves preparation: some demand is not forecastable beyond naive, by anyone, with any technology, because the signal is not in the data. The right response to those items is not a better model but a different operating posture, faster response, more flexible supply, honest buffers, and only the naive test tells you which items those are.
Before you buy a forecast, find out what forecasting nothing would achieve. Everything you pay for is measured from there.
Continuous forecast value added, per item and per process step, with the naive baseline enforced as the floor, is standard in the Prophesee Supply Chain Suite's demand module. Run the audit on your own history. Start here.