Working with uncertain futures

Published on by Joannes Vermorel.

The future is uncertain. Yet, nearly all predictive supply chain solutions make the opposite assumption: they assume that their forecasts are correct, and hence roll out their simulations based on those forecasts. Implicitly, the future is assumed to be certain and complications ensue.

From a historical perspective, software engineers were not making those assumptions without a reason: a deterministic future was the only option that the early - and not so early - computers could process at best. Thus, while dealing with an uncertain future was known to be the best approach in theory, in practice, it was not even an option.

In addition, a few mathematical tricks were found early in the 20th century in order to circumvent this problem. For example, the classic safety stock analysis assumes that both the lead times and the demand follow a normal distribution pattern. The normal distribution assumption is convenient from a computing viewpoint because all it takes is two variables to model the future: the mean and the variance.

Yet again, the normal distribution assumption - both for the lead times and the demand - proved to be incorrect in nearly all but a few situations, and complications ensued.

Back in 2012 at Lokad, we realized that the classic inventory forecasting approach was simply not working: mean or median forecasts were not addressing the right problem. No matter how much technology we poured on the case, it was not going to work satisfyingly.

Thus, we shifted to quantile forecasts, which can be interpreted as forecasting the future with an intended bias. Soon we realized that quantiles were invariably superior to the classic safety stock analysis, if only because quantiles were zooming in on where it really mattered from a supply chain perspective.

However, while going quantile, we realized that we had lost quite a few things in the process. Indeed, unlike classic mean forecasts, quantile forecasts are not additive, so it was not possible to make sense of a sum of those quantiles for example. In practice, the loss wasn’t too great because since classic forecasts weren’t making much sense in the first place, summing them up wasn’t a reasonable option anyway.

Over the years, while working with quantiles, we realized that so many of the things we took for granted had become a lot more complicated: demand quantities could no longer be summed or subtracted or linearly adjusted. In short, while moving towards an uncertain future, we had lost the tools to operate on this uncertain future.

Back in 2015, we introduced quantile grids. While quantile grids were not exactly the same as our full-fledged probabilistic forecasts just yet, our forecasting engine was already starting to deliver probabilities instead of quantile estimates. Distributions of probabilities are much more expressive than simple quantile estimates, and, it turns out that it is possible to define an algebra over distributions.

While the term algebra might sound technical, it’s not that complicated; it means that a simple operation such as the sum, the product, the difference, …, can be defined in ways which are not only mathematically consistent, but also highly relevant from the supply chain perspective.

As a result, just a few weeks ago, we integrated an algebra of distributions right into Envision, our domain-specific language dedicated to commerce optimization. Thanks to this algebra of distributions, it becomes straightforward to carry outseemingly simple operations such as summing two uncertain lead times (say an uncertain production lead time plus an uncertain transport lead time). The sum of those two lead times is carried out through an operation known as a convolution. While the calculation itself is fairly technical, in Envision, all it takes is to write A = B +* C, where +* is the convolution operator used to sum up independent random variables (*).

Through this algebra of distributions most of the “intuitive” operations which were possible with classic forecasts are back : random variables can be summed, multiplied, stretched, exponentiated, etc. And while relatively complex calculations are taking place behind the scenes, probabilistic formulas are not more complicated than plain Excel formulas from the Envision perspective.

Instead of wishing for the forecasts to be perfectly accurate, this algebra of distributions lets us embrace uncertain futures: supplier lead times tend to vary, quantities delivered may differ from quantities ordered, customer demand changes, products get returned, inventory may get lost or damaged … Through this algebra of distributions it becomes much more straightforward to model most of those uncertain events with minimal coding efforts.

Under the hood, processing distributions is quite intensive; and once again, we would never have ventured into those territories without a cloud computing platform that handles this type of workload - Microsoft Azure in our case. Nevertheless, computing resources have never been cheaper, and your company’s next $100k purchase order is probably well worth spending a few CPU hours - costing less than $1 and executed in just a few minutes - to make sure that the ordered quantities are sound.

(*) A random variable is a distribution that has a mass of 1. It’s a special type of distribution. Envision can process distributions of probabilities (aka random variables), but more general distributions as well.

Categories: Tags: insights forecasting envision