4 Demand Forecast Accuracy KPIs You’ll Actually Use

Demand Planning vs Demand Forecasting


Demand Planning is arguably the most important part of Supply Chain Management. It is the first step in the SCM process. Only after the demand planning process is done, people from production can start working on their operations and plan for raw materials, and logistics can focus on details of transportation planning and various warehouse activities.

Also it is worth mentioning that demand planning ≠ demand forecasting. Demand planning is the process of forecasting demand for a product or service and executing an operational strategy across the supply chain in order to meet it. Demand forecasting refers only to the process of predicting demand/sales numbers, and as such is a part of the demand planning process.


About Demand Forecasting KPIs


We can only improve what we can measure. Demand planners are always on the lookout for THE metric that will help them improve forecast accuracy and solve one of the most persistent problems for any supply chain company - increasing the visibility across the supply chain. There are numerous metrics used in this context, but in this post, I’ll focus only on KPIs that are most commonly used in tracking and improving forecast accuracy:


  • Forecast Accuracy/Error

  • Bias (Mean Forecast Error)

  • Mean Absolute Percentage Error (MAPE)

  • Weighted Mean Absolute Percentage Error (WMAPE)


No matter what metrics you use, your priority is choosing the right forecasting method/model. There is no “one size fits all KPI”. In other words, you have to be able to make quality assumptions about your business by using the perfect mix of qualitative and quantitative demand forecasting methods. More about those in another post.


Forecast Error


The most straightforward KPI. It provides insight into the difference between the actual value (Dt) and the forecast value (Ft) for the given period.

How to calculate:


Forecast Error: 1 – [ABS (Dt – Ft) / Dt]


Dt - The actual observation or sales for period t


Ft - The forecast for period t


Forecast error numbers range from 0 - 100%.

100% forecast accuracy is perfect, obviously, but if your data is right, you won’t be seeing it very often, and this is OK. Depending on the selected period and other operational factors, anything north of 70% can be perceived as acceptable. This goes without saying, but using the Forecast Error as the only KPI can be problematic for two reasons: it only provides information about the absolute value of the error, and as it’s limited when working with multiple time series, which, let’s face it, is always.


Bias (Mean Forecast Error)


Bias is a simple metric providing the information about the tendency of forecasts to persist in one direction - over or under-forecasting. It is a fairly reliable way to check if your forecasting model works the way it is supposed to. In theory, when the bias is zero, forecasts are not biased. If the bias is greater than 4, for the period of 24 observations, it is safe to say that your forecasting model is on the side of under-forecasting. Naturally, when the bias is less than -4, the model is biased toward over-forecasting.


How to calculate:


Bias: [∑ (Dt – Ft)] / n


Dt - The actual observation or sales for period t


Ft - The forecast for period t


n - The number of forecast errors


As I mentioned earlier, a highly biased model indicates that something is wrong with the way you forecast, but the metric itself is not enough to precisely evaluate the precision forecast.


Mean Absolute Percentage Error (MAPE)


You will probably use MAPE most of the time. And for a good reason: it is simple, everyone understands it, and there are only a few accuracy tradeoffs when using it.

Simply put, MAPE is a percentage of relative error. It expresses forecast errors as a percentage of actual observations.


How to calculate:


MAPE: ∑ |Et / Dt |/n * 100


Dt - Actual observation or sales for period t


Et - the forecast error for period t


n- the number of forecast errors


The tricky thing with MAPE is that it is often miscalculated (and misinterpreted) and that it does not provide the information on how exactly the products and moments in time differentiate. Have a look at a table below:


Source: Baeldung

As you can see, when the sale numbers are lower, MAPE can get deceivingly high. To avoid this it is recommended to use WMAPE.

Another problem you will probably encounter at some point are the time periods where the actual observation is 0 - no products are sold. Dividing by zero is problematic, to put it mildly, so the formula doesn’t actually work in those instances. When this happens, you can use scale invariant metrics such as MASE, or simply ignore these periods, which is my preferred choice.


Weighted Mean Absolute Percentage Error (WMAPE)


WMAPE is more value weighted than MAPE. What this means is that if demand planners know that some products and moments in time are more important to predict properly (for example Monday in the example below), they can attribute them more weight when calculating the error.


How to calculate:


WMAPE: ∑(|Dt-Ft|) / ∑(Dt)


Dt - The actual observation for period t


Ft - the forecast for period t


Let’s have a look at the table once again:

Source: Baeldung


In this case, we decided that Monday carries 80% of the importance. Consequently, this made more sense, and made our weighted error more precise - it went from 36.7% down to 9.1%.


More often than not, you will work with multiple time series and create forecasts for the next twelve months, which means that it is impossible to manually weight the periods, so you will need to find a way to automate this. Otherwise, WMAPE becomes pretty much useless.


Conclusion


There is no perfect measure to solve every forecasting problem. Choosing the right one depends on your specific situation and use case. Your data will never be perfect, and some information will always be missing, but using the combination of these KPIs and deep understanding of your business processes should help you increase your forecast accuracy. When in doubt, it is best to stick with MAPE, though.



Solving Demand Forecasting and Supply Chain problems is easier with an integrated supply chain planning, forecasting, and reporting software tool that helps automate your forecasts and build complex models in the blink of an eye.

Learn more about how Farseer can improve your Supply Chain Visibility.


43 views0 comments

Recent Posts

See All