Weekly/Monthly aggregation is a lossy process

Published on by Joannes Vermorel.

When practionners have a first look at a forecast report produced by Lokad, then tend to stumble upon various oddities. For example, some forecasts may look way too low. Without any observable trend nor any seasonality, Lokad anticipates something rather unexpected. Sometimes it's a by-product of rather advanced correlation analytics, but sometimes it's something both simpler and deeper

The graph on the left represents a typical situation: steady sales for a couple of months, and then, a somewhat inexplicable drop in the forecasts.

Common sense is yelling this can't be right, let's fix this broken forecast; and yet forecasting and common sense do not mix well.

The way we observe sales is deeply misleading. Indeed, we are observing here monthly aggregated sales, not the sales themselves. Many businesses favor monthly forecasts because they feel their sales are too low or too erratic at the daily or weekly level to be of any practical use. Hence, they aggregate sales data over long(er) period of time. By doing so, sales appear smoother and, consequently, more predictable.

This visualization of sales, i.e. thinking totals rather than an endless stream of transactions is so ubiquitous than many businesses fail to realize that aggregating sales primarily means loosing information, that is potentially valuable to perform the forecasts.

Let's illustrate the point with a fresh look at the same sales history, although through weekly aggregation.

The picture is extremely different. We realize that the seemingly steady monthly averages were just resulting from two super-heavy weeks: one in between January and February and a second in March.

Such spikes routinely appear in businesses because of promotions and other various kind of exception events. 

With the second illustration, low forecasts are making a lot more sense: sales include infrequent spikes that should not be accounted for, and, when we mentally discard those spikes, we obtain forecasts that just follow the usual averaging pattern.

A traditional forecasting system would typically be fooled by such a situation, and would anticipate a much higher monthly forecast, which would turn to be much less accurate. 

But Lokad is definitively not your traditional forecasting system. When monthly or weekly forecasts are requested, we keep looking at the most fine-grained data available. This let us identify patterns that would otherwise been lost through the sales aggregation process.

Categories: forecasting, insights, time series Tags: forecasting, insights