The largest forecasting experiment ever run
In 2020, the M5 forecasting competition ran on Kaggle, organised by INSEAD, the University of Nicosia, and Walmart. It asked participants to forecast 42,840 time series of retail sales data at the SKU-store level, roughly 30,000 data points per series, across 10 stores and over 3,000 products. The prize pool was real. The data was real. The evaluation was rigorous. It was, at the time, the largest controlled forecasting experiment ever conducted.
The results were published in 2022 in the International Journal of Forecasting. They are publicly available. They have been cited hundreds of times. And yet, four years later, most enterprise forecasting tools still ship a single statistical model per SKU and call it a forecast. The gap between what the research shows works and what the market actually uses is wider than it should be.
This article covers what the M5 competition actually showed about ensemble forecasting, why single-model approaches underperform, and what it means for a demand planner choosing a tool in 2026.
What the M5 measured
The M5 had two tracks. The accuracy track measured point forecast accuracy using RMSSE, the Root Mean Squared Scaled Error, which normalises error against a naive seasonal random walk baseline. The uncertainty track measured probabilistic forecast quality using a weighted pinball loss across multiple quantiles.
Both tracks were won by ensemble approaches. The top five entries in the accuracy track were all ensembles that combined multiple base models, typically including gradient-boosted trees (LightGBM, XGBoost), deep learning models (N-BEATS, DeepAR), and statistical baselines. No single-model entry placed in the top ten.
The paper, authored by Makridakis and colleagues, concluded that the single most consistent finding across the M competitions (M1 through M5, running since 1982) is that combining forecasts from multiple methods produces more accurate results than any single method on average. This is not a marginal effect. The winning ensemble in M5 outperformed the best single model by a meaningful margin across the full dataset.
Why ensembles win
The intuition is straightforward. Different models make different errors. A seasonal exponential smoothing model captures regular seasonal patterns but misses trend breaks. A gradient-boosted tree captures non-linear interactions between calendar features and promotions but overfits if the training window is short. A neural network like N-BEATS captures complex temporal patterns but requires substantial data to train, which intermittent SKUs do not have.
An ensemble combines these models so that the errors cancel. Where the statistical model over-forecasts, the tree model might under-forecast, and the average is closer to the truth. The mathematics of this are well established. Bates and Granger proved in 1969, in the journal Operational Research Quarterly, that a weighted average of two unbiased forecasts is always at least as accurate as the worse of the two, and usually more accurate than either. This result has been reproduced in every major forecasting competition since.
The weighting matters. A simple average is better than the worst model but usually not better than the best. Inverse-error weighting, where models with lower recent error get higher weight, consistently outperforms simple averaging. The M5 winners used sophisticated weighting schemes that adapted per series, so a model that was strong on fast movers got higher weight on fast movers, and a model that was strong on slow movers got higher weight on slow movers.
Why most tools still ship single-model
If ensembles are demonstrably better, why do most enterprise planning tools still ship a single statistical model per SKU? Three reasons.
First, the M5 results are recent in enterprise software terms. The paper was published in 2022. Enterprise planning software release cycles are 12 to 24 months. Many tools in market today were architected before the M5 results were available, and ensemble support requires architectural changes, not just a parameter toggle.
Second, ensembles are computationally expensive. Running three models per SKU instead of one triples the compute cost of a forecast run. For a manufacturer with 50,000 SKUs running monthly forecasts, this is meaningful. Cloud-native tools can absorb it. On-premise tools from the 2010s cannot.
Third, ensembles are harder to explain. A planner who sees a forecast of 1,247 units wants to know where the number came from. With a single exponential smoothing model, the answer is "the model tracked a level of 1,250 and the last observation nudged it down slightly." With an ensemble of three models weighted by inverse MAPE, the answer is a paragraph of methodology. Many vendors avoid the explanation problem by not shipping the ensemble.
What this means for choosing a tool
If you are evaluating forecasting tools in 2026, the ensemble question is one of the most discriminating questions you can ask. Specifically:
Does the tool run a single model per SKU, or multiple models blended per SKU? If single, it is a generation behind the published research. If multiple, ask which models, how they are weighted, and whether the weighting adapts per series.
Can the tool show you, per SKU, which model contributed most to the ensemble forecast? This is the explainability question. A good ensemble implementation shows you the per-model contributions and the weighting rationale. A bad one hides the ensemble behind a single number.
Does the tool expose the accuracy leaderboard per product family? This is the M5 pattern. The ensemble should be weighted by which model is winning on each product family, and the leaderboard should be visible to the planner so they can see, for example, that the neural network is winning on fast movers and the statistical model is winning on intermittent items.
If a vendor cannot answer these questions, or answers with "we use a proprietary algorithm," treat it as a single-model tool until proven otherwise. The M5 results are public. Any vendor with a genuine ensemble implementation will reference them.
The honest caveat
Ensembles are not a free lunch. They require more compute, more tuning, and more careful monitoring than single-model approaches. They can also overfit if the ensemble weighting is tuned on the same data used to evaluate it. The M5 winners used careful cross-validation to avoid this.
For a manufacturer with 500 SKUs and stable demand, a well-tuned single statistical model may be perfectly adequate. The ensemble advantage is largest on large, diverse datasets where different SKUs have different demand patterns. If your dataset is small and homogeneous, the marginal gain from an ensemble may not justify the complexity.
But for any manufacturer with thousands of SKUs, mixed demand patterns (fast movers, slow movers, intermittent, seasonal, promotional), and a planning cycle that runs monthly or more frequently, the M5 evidence is clear. Ensembles win. The question is not whether to use one. The question is whether your tool supports one.
Further reading
The M5 competition results paper is open access in the International Journal of Forecasting (Makridakis et al., 2022). The N-BEATS paper by Oreshkin et al. was published at ICLR 2020 and is available on arXiv. The original Bates and Granger paper on forecast combination is in Operational Research Quarterly (1969). The Hyndman and Athanasopoulos textbook "Forecasting: Principles and Practice" (3rd edition, freely available online) covers ensemble and hierarchical forecasting in depth. These are the sources to read if you want to go deeper than this article.