While shelf availability is a top concern for customers and retailers alike, even the Tier I retailers today mostly rely on manual checks by store staff. This puts a large burden on employees, and response times to out-of-shelf situations are slow.

However, grocery retailers a crossed Europe and the US have started to explore technology that can help address the problem. With this in mind, we have published a whitepaper on out-of-shelf monitoring technology which gives an overview of

• Objectives of out-of-shelf monitoring systems
• Introduction to the technology
• Definition of performance characteristics
• Quantification of system capabilities and limitations

Are we missing some interesting aspects of this type of technology, or do you have experience with OOS monitoring systems you want to share? Please let us know in the comments.

• Are profits earned by a product versus the capital invested in stocking it (i.e. ROI) similar over the product portfolio?
• If the ROIs of the various products are heterogeneous – how can the overal ROI delivered by the inventory be maximized?

As you will have guessed, the answer to the first question is a clear ‘NO’. The return generated by a product for the euros invested in stocking it differs vastly from product to product. Two aspects have a major impact.

First, the gross margin of a product directly affects the ROI. This is obvious, and product managers naturally try to build a portfolio with high margins in mind. However, many other considerations come into play such as the coverage of the product portfolio for example. Adding more products to the portfolio is often also a strategy for growth.

The second aspect is more subtle, but equally important: The amount of stock that is required to provide a desired availability (i.e. service level) varies significantly among products. The main driver here is the volatility of demand - the higher the uncertainty of the future demand, the higher the inventory level required to assure a given service level.

As an example, lets imagine two products with the same gross margin that sell the same number of units during the year, thus generating the same amount of annual gross profit. However, product A sells the same amount each week with a high certainty, while product B is much more erratic with no sales for weeks and large orders in others. To achieve the same availability or service level for both products, the safety stock for product B will be a lot higher than for product A, where very little uncertainty and therefore safety stock requirement is given. As a result, product B requires much more units in stock than product A to achieve the same annual profit. The ROI for product A is therefore significantly higher.

Businesses continually work on maximizing their return on capital employed (ROCE). Inventory is a significant part of the capital invested by most retailers, and therefore an important opportunity for optimization. The good news is that the potential for optimizing the ROI of your inventory by taking advantage of the heterogeneous ROIs across your catalog is large.

The latter is done by finding the optimal service level at which an SKU produces the best ROI within the constraint of a set inventory budget. As a result availability and revenue can be increased for a given inventory budget, or cash can be released from inventory while maintain the overall availability.

The analysis behind this optimization however is not trivial given the non-linear correlation between service level and inventory levels. Additionally, a set of ‘strategic’ constraints such availability goals for certain products need to be taken into account. 

This is a challenge which we plan to take on in the near future, the goal being a fully automated ROI optimization over the product portfolio for a given working capital sum. As an output, the system will determine service levels per SKU that give the highest availability (and therefore revenues) for a chosen inventory budget.

Excessive inventory reduces the return on invested capital, but too little inventory diminishes profits as well. The optimal inventory level is therefore a trade-off between stock-out cost and inventory cost, and falling too far on either side of the optimum will negatively impact your business. Our plans for service level optimization will leverage our forecasting technology. Stay tuned.

We believe that this single problem alone has prevented most retailers to move toward advance forecasting systems at the store level. As with most forecasting problems, it's subtle, it's counterintuitive, and some companies charge a lot to bring poor answers to the question.

The most popular error metrics in sales forecasting are the Mean Absolute Error (MAE) and the Mean Absolute Percentage Error (MAPE). As a general guideline, we suggest to stick with the MAE as the MAPE behaves very poorly whenever time-series are not smooth, that is, all the time, as far retailers are concerned. However, there are situations where MAE too behaves poorly. Low sales volumes fall in those situations.

Let's review the illustration here above. We have an item sold over 3 days. The number of unit sold over the first two days is zero. On the third day, one unit get sold. Let's assume that the demand is, in fact, of exactly 1 unit every 3 days. Technically speaking, it's a Poisson distribution with λ=1/3.

In the following, we compare two forecasting models:

• a flat model M at 1/3 every day (the mean).
• a flat model Z at zero every day.

As far inventory optmization is concerned, the model zero (Z) is downright harmfull. Assuming that safety stock analysis will be used to compute a reorder point, a zero forecast is very likely to produce a reorder point at zero too, causing frequent stockouts. An accuracy metric that would favor the model zero over more reasonable forecasts would be behaving rather poorly.

Let's review our two models against the MAPE (*) and the MAE.

• M has a MAPE of 44%.
• Z has a MAPE of 33%.
• M has a MAE of 0.44.
• Z has a MAE of 0.33.

(*) The classic definition of MAPE involves a division by zero when the actual value is zero. We assume here that the actual value is replaced by 1 when at zero. Alternatively, we could also have divided by the forecast (instead of the actual value), or use the sMAPE. Those changes make no difference: the conclusion of the discussion remains the same.

In conclusion, here, according to both the MAPE and the MAE, model zero prevails.

However, one might argue that this is simplistic situation, and it does not reflect the complexity of a real store. This is not entirely true. We have performed benchmarks over dozens of retail stores, and usually the winning model (according to MAE or MAPE) is the model zero - the model that returns always zero. Futhermore, this model typically wins by a comfortable margin over all the other models.

In practice, at store level, relying either on MAE or MAPE to evaluate the quality of forecasting models is asking for trouble: the metric favors models that return zeroes; the more zeroes, the better. This conclusion holds for about every single store we have analyzed so far (minus the few high volume items that do not suffer this problem).

Readers who are familar with accuracy metrics might propose to go instead for the Mean Square Error (MSE) which will not favor the model zero. This is true, however, MSE when applied to erratic data - and sales are store level are erratic - is not numerically stable. In practice, any outlier in the sales history will vastly skew the final results. This sort of problem is THE reason why statisticians have been working so hard on Robust statistics in the first place. No free lunch here.

How to assess store level forecasts then?

It took us a long, long time, to figure out a satifying solution to the problem of quantifying the accuracy of the forecasts at the store level. Back in 2011 and before, we were essentially cheating. Instead of looking at daily data points, when the sales data was too sparse, we were typically switching to weekly aggregates (or even to monthly aggregates for extremely sparse data). By switching to longer aggregation periods, we were artificially increasing sales volumes per period, hence making the MAE usable again.

The breakthrough came only a few months ago through quantiles. In essence, the enlightenment was: forget the forecasts, only reorder points matter. By trying to optimize our classic forecasts against metrics X, Y or Z, we were trying to solve the wrong problem.

Wait! Since reorder points are computed based on the forecasts, how could you say forecasts are irrelevant?

We are not saying that forecasts and forecast accuracy are irrelevant. However, we are stating that only the accuracy of the reorder points themselves matter. The forecast, or whatever other variable is used to compute reorder points, cannot be evaluated on its own. Only accuracy of the reorder points need and should be evaluated.

It turns out that a metric to assess reorder points exists: it's the pinball loss function, a function that has been known by statisticians for decades. Pinball loss is vastly superior not because of its mathematical properties, but simply because it fits the inventory tradeoff: too much stocks vs too much stockouts.

In Part 2, we provide some non-technical insights why quantile forecasts outperforms classic ones in three usual situations.

Video summary (7min46):

• High service levels
• Intermittent demand
• Spiky demand

Don't hesitate to post questions in comments.

Video summary (6min25):

• What are quantiles?
• How do quantiles work?
• What are the advantages of quantiles?

Stay tuned for more.

However, it exists situations where the trade-off is more complex in the sense there are more than 2 unfavorable options to balance. When there are 3 unfavorable options to be balanced, the situation is called a trilemma.

Out-of-shelf (OOS) monitoring, ala Shelfcheck, is facing a trilemma when it comes to the quality of the alerts being delivered:

• Sensibility, the percentage of OOS problems being captured.
• Precision, the percentage of true alerts within all OOS alerts.
• Latency, the delay between then start of the OOS problem and the alert.

Pick any two, but you can't have them all. In fact, for any given demand forecasting accuracy, improving any of those 3 metrics come at the expense of the 2 other metrics.

Sometimes though, we make a breakthrough. Today, we are announcing the most significant technology upgrade of Lokad since its launch 6 years ago: the immediate availability of quantile forecasts.

Quantiles are disruptive in the sense that in many situations they make classic forecasts plain obsolete as far inventory optimization is concerned - for retail, wholesale and manufacturing businesses.

We have identified 3 situations where quantiles really shine:

• High service levels at 90% and above.
• Intermittent demand (slow mover's).
• Bulk orders (spiky demand).

In those situations, benchmarks against our own classic forecasting technology indicate that quantile forecasts typically bring either 20% less inventory or 20% less stockouts.

Extraordinary claims require extraordinary evidence. Carl Sagan

However, the many benchmarks that we have made so far with our prospects and clients indicate that our classic forecasting technology is already ahead of the competition; but with quantile forecasts, it's whole new level of inventory optimization that can be achieved.

Don't hesitate to put quantiles to the test.

The story behind the quantile upgrade

Quantile forecasting (also called quantile regression) has been around for decades among academic circles. Then, in finance, analysts have been extensively using quantiles for Value at Risk (VaR) analysis since the late 1980s.

At Lokad, quantiles have been around for a long time as well. For example, back in 2009, Sequential Quantile Prediction of Time Series. IEEE Transactions on Information Theory, March 2011, vol. 57, n°3 has been published by one of us. However, until very recently, quantiles were very mistakenly considered a mathematical distraction (business-wise) rather than a mission critical concept.

What did hold us back was not lacking insights in statistics, but lacking insights in the profound relationship between quantiles and inventory optimization. This insight was triggered, mostly out of dumb luck, when a client did ask us to figure out a formula to compute optimal service levels for her inventory.

A breakthrough yes, but a late one

This quantile breakthrough is only very relative in the sense that quantiles have been already applied successfully for decades in other trades. However, there is one aspect that partially explains this late arrival: quantile models typically require about 10x more processing power compared to classic forecasting models. Without cloud computing, we would not have been able to put quantiles in production, while preserving an aggressive pricing.

One of the problems that comes with being specialists of a subject is that you tend to take for granted what is obscure for anyone but your peers. At Lokad, despite our best efforts, we are no exception, especially when it comes to forecasting...

Recently, we realized that we had never provided any in-depth quantitative assessment of the financial gains generated by an increase of the forecasting accuracy, which is about the raison d'être for the company. Furthermore, after investigating the web, we realized that other forecasting vendors (competitors) were rather fuzzy too about the financial rewards that could be achieved through better forecasts.

However, it's not that complicated. With the following variables:

• $D$ the turnover (total annual sales).
• $m$ the gross margin.
• $\alpha$ the cost of stockout to gross margin ratio.
• $p$ the service level achieved with the current error level (and current stock level).
• $\sigma$ the forecast error of the system in place, expressed in MAPE (mean absolute percentage error).
• $\sigma_n$ the forecast error of the new system being benchmarked (hopefully lower than $\sigma$).

The yearly benefit $B$ of going for the new forecasting system is given by:

$$B = D (1 - p) m \alpha \frac{\sigma - \sigma_n}{\sigma}$$

For the proof of this result, check the full length article.

That's why we have spent a bit of time to craft a formula that gives optimal service levels. Naturally, the optimality is not obtained without assumptions. However, we believe those are reasonable enough to preserve the efficiency of the formula for most businesses.

Then, another subject, that receives too little attention, is the optimal order quantity: the quantity to be ordered in order to minimize the combination of purchase costs, carrying costs, shipping costs, etc. As of January 2012, it's fascinating to notice that most of the industry still relies on the Wilson formula devised back in 1913. Yet, this formula comes with strong assumptions that do not make much sense any more for the supply chain of the 21st century.

Thus, we have designed another economic order quantity formula that emphasizes volume discounts (instead of a flat ordering cost) for larger purchases. The formula (or rather the approach) is fairly general, and could be applied to any pricing structure, including non-linear situations where specific quantities are favored because they matche the size of a crate or a pallet.

Both situations are illustrated with Excel sheets (so you don't even need Lokad to get started).

For any mid-large retail network, the informational value of market basket data simply dwarfs about all other alternative data sources, may it be:

• In-store video data, which remain difficult to process, and primarily focused on security.
• Social media data, which are very noisy and reflect as much bot implementations than human behaviors.
• Market analyst’s reports, which require the scarcest resource of all: management attention.

Yet, beside basic sales projections (aka sales per product, per store, per region, per week …), we observe that, as of January 2012, most retailers are doing very little out of their market basket data. Even forecasting for inventory optimization is typically nothing more than a moving average variant at the store level. More elaborate methods are used fore warehouses, but then, retailers are not leveraging basket data anymore, but past warehouse shipments.

Big Data vendors promise to bring an unprecedented level of data processing power to their clients to let them harness all the potential of their big data. Yet, is this going to bring profitable changes to retailers? Not necessarily so.

The storage capacity sitting on display on the shelves of an average hypermarket with +20 external drives in display (assuming 500GB per drive) typically exceeds the raw storage needed to persist a whole 3 years of history of a 1000 stores network (i.e. 10TB of market basket data). Hence, raw data storage is not a problem, or, at least, not an expensive problem. Then, data I/O (input/output) is a more challenging matter, but again, by choosing an adequate data representation (the details would go beyond the scope of this post), it’s hardly a challenge as of 2012.

We observe that the biggest challenge posed by Big Data is simply manpower requirements to do anything operational with it. Indeed, the data is primarily big in the sense that the company resources, to run the Big Data software and to implement whatever suggestions come out of it, are thin.

Producing a wall of metrics out of market basket data is easy; but it’s is much harder to build a set of metrics worth the time being read considering the hourly costs of employees.

As far we understand our retail clients, the manpower constraint alone explains why so little is being been done with market basket data on an ongoing basis: while CPU has never been to cheap, staffing has never been so expensive.

Thus, we believe that Big Data successes in retail will be encountered by lean solutions that treat, not processing power, but people, as the scarcest resource of all.

Forecasting Technology

Since the very beginning, forecasting accuracy has been the foremost priority of Lokad. Over 2011, we have extensively leveraged Windows Azure (our cloud computing platform) to develop forecasting models that would have been completely out of reach without the capabilities of the cloud.

We have made progress over patterns as simple as seasonality, trends or product life cycles. Those patterns are supposed to be well-known for decades, but the more we learn by looking at the data of our growing customer base, the more we realize that we are only scratching the surface.

In 2012, we are planning to dedicate efforts on forecasts at the point-of-sale level for both retail networks and eCommerce. This effort will boost the development of Shelfcheck.

Then, we will also explore alternative ways to make a better use of tags and events. More and more of our clients are now capable of feeding our forecasting engine with high-quality tags and events, which offer more opportunities to refine forecasts.

Pricing

The Lokad pricing for forecast consumption hasn’t changed since November 2009, and we don’t expect any significant change for 2012 apart from minor adjustments. However Shelfcheck will benefit from a distinct pricing, not directly bound to the forecast consumption.

Salescast

Over 2011, our webapp delivering inventory optimization has grown to a relatively mature and stable product. Contrary to what we announced last year, we have finally opted against the idea of importing data from Excel  Instead, the majority of our users are now using our intermediate SQL format which offers a simple and reliable path to achieve complete automation with a minimum of efforts.

For the year to come, we will polish Salescast further, especially around the intermediate SQL format. Indeed, we feel that database administrators still struggle too much to import their data in Salescast. For example, we will provide better and more explicit errors messages.

Shelfcheck

Shelfcheck is our latest product, only announced a few months ago , that focuses on on-shelf availability optimization for retail networks.

At this point, a beta version of Shelfcheck is already in production on multiple stores in Europe. We plan to bring Shelfcheck out of its beta during 2012. Processing the ongoing sales stream of a large retail network at low costs is a tremendous challenge (even with the cloud). In addition, we want to establish Shelfcheck as the technology delivering the most accurate OOS alerts (out-of-shelf) of the market.

Hub

The “Hub” is the webapp in charge of managing registrations and subscriptions. Over 2011, we haven’t invested much effort on this webapp, and now, it feels somewhat antiquated, especially when compared to the more polished user interface of Salescast. Thus, in 2012, we plan to extensively refactor the Hub to simplify the management of users and subscriptions.

This roadmap isn't carved in stone. Don't hesitate to voice your opinion.

With the launch of Shelfcheck, our on-shelf availability optimizer, we have started to process a lot more data at the point of sales level, trying to automatically detect out-of-shelf (OOS) issues.Over the last few months, our knowledge about OOS pattern has significantly improved, and today this knowledge is being recycled into our core forecasting technology.

Let's illustrate the situation. The graph below represents the daily aggregated sales at the store level for a given product. The store is open 7/7. A seven-day forecast is produced at the end of the week 2, but an OOS occurs in the middle of the week 3. Days marked with black dots have zero sales.

In this situation, the forecast is fairly accurate, but because of the OOS problem, the direct comparison of sales vs forecasts look like as if the forecast was significantly overforecasting the sales, which is not the case, at least not on the non-OOS days. The overforecast measurement is an artifact caused by the OOS itself.

So far, it seems that OOS can only degrade the perceived forecasting accuracy, which it does not seem too bad because presumably all forecasting methods should be equally impacted. After all, not forecasting model is able to anticipate the OOS problem.

Well, OOS can do a lot worse that just degrade the forecasting accuracy, OOS can also improve it.

Let have a look at the graph to illustrate this. Again, we are looking at daily sales data, but this time the OOS problem starts on the very last day of week 2.

The forecast for the week 3 is zero the whole week. The forecasting model is anticipating the duration of the OOS. The forecast is not entirely accurate because on the last day of week 3, replenishment is made and sales are non-zero again.

Obviously, a forecasting model that anticipates the duration of the OOS issue is extremely accurate as far the numerical comparison sales vs forecasts is concerned. Yet, does it really make sense? No, obviously it does not. We want to forecast the demand not sales artifacts. Worse, a zero forecast can lead to a zero replenishment which, in turn, extend the actual duration of the OOS issue (and increase further the accuracy of our OOS-enthusiast forecasting model). This is obviously not a desirable situation, no matter how good the forecast happens to be from a naive numerical viewpoint.

Bad case of OOS overfitting

We have found that the situation illustrated by the 2nd graphic is far from being unusual. Indeed, with 8% on-shelf unavailability (a typical figure in retail) and a rough 30% MAPE on daily forecasts, OOS situations typically account for 8% x 100 / 30 ≈ 27% ≈ 1/4 of the total forecast error being measured. Indeed, by definition of MAPE, a non-zero forecast on zero-sale day (OOS) generates a 100% error.

Because the fraction of the error caused by OOS is significant, we have found that a simple heuristic such as "if last day has zero sales on a top seller product, then forecast zero sales for 7 days" may reduce the forecast error from a few percents by directly leveraging the OOS pattern. Obviously, very few practitioners would explicitly put such a rule among their forecasting models, but even a moderately complex linear autoregressive model may learn this pattern to a significant extend, and thus overfit OOS.

Naturally, Shelfcheck is here to help on those OOS matters. Stay tuned.

Again, special thanks to Ray Groover for the voice over.

When illustrating seasonality, everyone (Lokad's included) tend to use long time-series, much like the first three series here above. Indeed, it's more visual and more appealing.

However, long time-series do not represent your usual situation. On average consumer goods have a lifespan of no more than 3 or 4 years. Thus, long time-series are typically a small minority in your dataset. Worse, those long time-series might be outliers that do not reflect the behavior of other shorter-lived products.

Here above, the short 4th time-series is a much more representative case with less than 1 year of data. In such a situation, however, it's much less clear how seasonality can be leveraged. The Lokad trick to do that consists of using multiple time-series analysis.

Learn more on our seasonality definition article.

Again, special thanks to Ray Grover for the voice over.