forecast-accuracy-review

Measure forecast accuracy with WMAPE (sum of absolute errors divided by sum of actuals) as the volume-weighted headline metric, plus bias to detect systematic over- or under-forecasting. Always compare against a naive last-period baseline using a rolling-origin backtest rather than a single train/test split.

Forecast Value Added (FVA) equals WMAPE of the naive benchmark minus WMAPE of the candidate forecast, computed per segment and overall. A negative FVA means the forecasting process performs worse than simply shipping last period's number and is destroying value.

MAPE is undefined on zero-actual periods, so silently dropping those periods fakes precision on intermittent SKUs. It is also asymmetric, capping under-forecast errors at 100% while over-forecast errors are unbounded, which rewards systematic under-forecasting.

At least 18 periods per SKU are needed for a meaningful rolling-origin backtest, with one-step-ahead forecasts over the last 6 or more periods using an expanding window. SKUs with less history should be flagged rather than scored.

When WMAPE exceeds roughly 100% on lumpy demand segments, the honest recommendation is an inventory-policy answer such as safety buffers or make-to-order, not a better model. Forecasting cannot fix structurally unpredictable demand.