Filtering by Tag: oos

Whitepaper on benchmarking Out-of-Shelf monitoring apps

Published on by Joannes Vermorel.

Testing and benchmarking of competing OOS (out-of-shelf) monitoring systems is an important step in quantifying the value of such systems for a retailer, and in identifying the most suitable vendor. After introducing the OOS domain in a first whitepaper, we take a closer look at how to best setup a trial and benchmark of competing OOS monitoring systems.

Cover of the OOS whitepaper by LokadBenchmarking OOS systems is fairly straightforward, provided that the test is carefully planned. The good news is that key performance characteristics – precision and sensitivity – can be quantified in a test, which leads to objective benchmark results and a fair comparison of system performance.

Our whitepaper on benchmarking OOS monitoring systems covers among other topics

  • Benchmark critera
  • Methodologies for making OOS monitoring systems comparable
  • Quantification of system profibability
  • Project phases and execution

Are we missing some interesting aspects of testing these systems? or do you have experience with OOS monitoring systems in operation you want to share? Please let us know in the comments.

Categories: on shelf availability Tags: oos shelfcheck whitepaper No Comments

Whitepaper on Out-of-Shelf Monitoring Technology released

Published on by Joannes Vermorel.

Few aspects of retailing are as fundamental as fully stacked shelves, and few concerns rank higher with an ever more demanding customer than product availability. Regardless, on-shelf availability remains a huge challenge for the industry. Growing, more dynamic product portfolios, staff and cost cuts combined with an increasingly complex supply chain have increased the challenge.

Cover of the OOS whitepaper by LokadWhile shelf availability is a top concern for customers and retailers alike, even the Tier I retailers today mostly rely on manual checks by store staff. This puts a large burden on employees, and response times to out-of-shelf situations are slow.

However, grocery retailers a crossed Europe and the US have started to explore technology that can help address the problem. With this in mind, we have published a whitepaper on out-of-shelf monitoring technology which gives an overview of

  • Objectives of out-of-shelf monitoring systems
  • Introduction to the technology
  • Definition of performance characteristics
  • Quantification of system capabilities and limitations

Are we missing some interesting aspects of this type of technology, or do you have experience with OOS monitoring systems you want to share? Please let us know in the comments.

Categories: business, insights, retail Tags: documentation oos shelfcheck whitepaper No Comments

Out-of-shelf trilemma

Published on by Joannes Vermorel.

Shopping Cart logo Most people are familiar with the notion of dilemma when two possibilities are offered neither of which being acceptable. Safety stock analysis is a classical mathematical dilemma: you can choose between more stocks or more stockouts, yet both of them generate extra costs.

However, it exists situations where the trade-off is more complex in the sense there are more than 2 unfavorable options to balance. When there are 3 unfavorable options to be balanced, the situation is called a trilemma.

Out-of-shelf (OOS) monitoring, ala Shelfcheck, is facing a trilemma when it comes to the quality of the alerts being delivered:

  • Sensibility, the percentage of OOS problems being captured.
  • Precision, the percentage of true alerts within all OOS alerts.
  • Latency, the delay between then start of the OOS problem and the alert.

Pick any two, but you can't have them all. In fact, for any given demand forecasting accuracy, improving any of those 3 metrics come at the expense of the 2 other metrics.

Categories: insights, on shelf availability, retail Tags: insights oos out-of-shelf retail No Comments

Out-of-shelf can explain 1/4 of store forecast error

Published on by Joannes Vermorel.

The notion of forecasting accuracy is subtle, really subtle. It's common sense to say that if the closer the forecasts from the future, the better, and yet common-sense can be plain wrong.

With the launch of Shelfcheck, our on-shelf availability optimizer, we have started to process a lot more data at the point of sales level, trying to automatically detect out-of-shelf (OOS) issues.Over the last few months, our knowledge about OOS pattern has significantly improved, and today this knowledge is being recycled into our core forecasting technology.

Let's illustrate the situation. The graph below represents the daily aggregated sales at the store level for a given product. The store is open 7/7. A seven-day forecast is produced at the end of the week 2, but an OOS occurs in the middle of the week 3. Days marked with black dots have zero sales.

In this situation, the forecast is fairly accurate, but because of the OOS problem, the direct comparison of sales vs forecasts look like as if the forecast was significantly overforecasting the sales, which is not the case, at least not on the non-OOS days. The overforecast measurement is an artifact caused by the OOS itself.

So far, it seems that OOS can only degrade the perceived forecasting accuracy, which it does not seem too bad because presumably all forecasting methods should be equally impacted. After all, not forecasting model is able to anticipate the OOS problem.

Well, OOS can do a lot worse that just degrade the forecasting accuracy, OOS can also improve it.

Let have a look at the graph to illustrate this. Again, we are looking at daily sales data, but this time the OOS problem starts on the very last day of week 2.

The forecast for the week 3 is zero the whole week. The forecasting model is anticipating the duration of the OOS. The forecast is not entirely accurate because on the last day of week 3, replenishment is made and sales are non-zero again.

Obviously, a forecasting model that anticipates the duration of the OOS issue is extremely accurate as far the numerical comparison sales vs forecasts is concerned. Yet, does it really make sense? No, obviously it does not. We want to forecast the demand not sales artifacts. Worse, a zero forecast can lead to a zero replenishment which, in turn, extend the actual duration of the OOS issue (and increase further the accuracy of our OOS-enthusiast forecasting model). This is obviously not a desirable situation, no matter how good the forecast happens to be from a naive numerical viewpoint.

Bad case of OOS overfitting

We have found that the situation illustrated by the 2nd graphic is far from being unusual. Indeed, with 8% on-shelf unavailability (a typical figure in retail) and a rough 30% MAPE on daily forecasts, OOS situations typically account for 8% x 100 / 30 ≈ 27% ≈ 1/4 of the total forecast error being measured. Indeed, by definition of MAPE, a non-zero forecast on zero-sale day (OOS) generates a 100% error.

Because the fraction of the error caused by OOS is significant, we have found that a simple heuristic such as "if last day has zero sales on a top seller product, then forecast zero sales for 7 days" may reduce the forecast error from a few percents by directly leveraging the OOS pattern. Obviously, very few practitioners would explicitly put such a rule among their forecasting models, but even a moderately complex linear autoregressive model may learn this pattern to a significant extend, and thus overfit OOS.

Naturally, Shelfcheck is here to help on those OOS matters. Stay tuned.

Categories: insights, on shelf availability, retail Tags: insights oos osa 2 Comments