Optimal service level and order quantity

In the inventory optimization literature, one of the most recurring concepts is the service level, i.e. the desired probability of not hitting a stock-out situation. The service level expresses the tradeoff between too much inventory and too many stock-outs. However, experts remain typically vague when it comes to choose service level values; a pattern also followed by most inventory software products...

That's why we have spent a bit of time to craft a formula that gives optimal service levels. Naturally, the optimality is not obtained without assumptions. However, we believe those are reasonable enough to preserve the efficiency of the formula for most businesses.

Then, another subject, that receives too little attention, is the optimal order quantity: the quantity to be ordered in order to minimize the combination of purchase costs, carrying costs, shipping costs, etc. As of January 2012, it's fascinating to notice that most of the industry still relies on the Wilson formula devised back in 1913. Yet, this formula comes with strong assumptions that do not make much sense any more for the supply chain of the 21st century.

Thus, we have designed another economic order quantity formula that emphasizes volume discounts (instead of a flat ordering cost) for larger purchases. The formula (or rather the approach) is fairly general, and could be applied to any pricing structure, including non-linear situations where specific quantities are favored because they matche the size of a crate or a pallet.

Both situations are illustrated with Excel sheets (so you don't even need Lokad to get started).

Big data in retail, a reality check

Cloud computing being so 2011, big data is going to be a key IT buzzword for 2012. Yet, as far we understand our retail clients, there is one data source that holds above 90% of total information value in their possession: market basket data (tagged with fidelity card information when available).

For any mid-large retail network, the informational value of market basket data simply dwarfs about all other alternative data sources, may it be:

• In-store video data, which remain difficult to process, and primarily focused on security.
• Social media data, which are very noisy and reflect as much bot implementations than human behaviors.
• Market analyst’s reports, which require the scarcest resource of all: management attention.

Yet, beside basic sales projections (aka sales per product, per store, per region, per week …), we observe that, as of January 2012, most retailers are doing very little out of their market basket data. Even forecasting for inventory optimization is typically nothing more than a moving average variant at the store level. More elaborate methods are used fore warehouses, but then, retailers are not leveraging basket data anymore, but past warehouse shipments.

Big Data vendors promise to bring an unprecedented level of data processing power to their clients to let them harness all the potential of their big data. Yet, is this going to bring profitable changes to retailers? Not necessarily so.

The storage capacity sitting on display on the shelves of an average hypermarket with +20 external drives in display (assuming 500GB per drive) typically exceeds the raw storage needed to persist a whole 3 years of history of a 1000 stores network (i.e. 10TB of market basket data). Hence, raw data storage is not a problem, or, at least, not an expensive problem. Then, data I/O (input/output) is a more challenging matter, but again, by choosing an adequate data representation (the details would go beyond the scope of this post), it’s hardly a challenge as of 2012.

We observe that the biggest challenge posed by Big Data is simply manpower requirements to do anything operational with it. Indeed, the data is primarily big in the sense that the company resources, to run the Big Data software and to implement whatever suggestions come out of it, are thin.

Producing a wall of metrics out of market basket data is easy; but it’s is much harder to build a set of metrics worth the time being read considering the hourly costs of employees.

As far we understand our retail clients, the manpower constraint alone explains why so little is being been done with market basket data on an ongoing basis: while CPU has never been to cheap, staffing has never been so expensive.

Thus, we believe that Big Data successes in retail will be encountered by lean solutions that treat, not processing power, but people, as the scarcest resource of all.

Roadmap for 2012

For the third year (see the 2011 and 2010 editions), we will share some insights about the future developments of Lokad.

Forecasting Technology

Since the very beginning, forecasting accuracy has been the foremost priority of Lokad. Over 2011, we have extensively leveraged Windows Azure (our cloud computing platform) to develop forecasting models that would have been completely out of reach without the capabilities of the cloud.

We have made progress over patterns as simple as seasonality, trends or product life cycles. Those patterns are supposed to be well-known for decades, but the more we learn by looking at the data of our growing customer base, the more we realize that we are only scratching the surface.

In 2012, we are planning to dedicate efforts on forecasts at the point-of-sale level for both retail networks and eCommerce. This effort will boost the development of Shelfcheck.

Then, we will also explore alternative ways to make a better use of tags and events. More and more of our clients are now capable of feeding our forecasting engine with high-quality tags and events, which offer more opportunities to refine forecasts.

Pricing

The Lokad pricing for forecast consumption hasn’t changed since November 2009, and we don’t expect any significant change for 2012 apart from minor adjustments. However Shelfcheck will benefit from a distinct pricing, not directly bound to the forecast consumption.

Salescast

Over 2011, our webapp delivering inventory optimization has grown to a relatively mature and stable product. Contrary to what we announced last year, we have finally opted against the idea of importing data from Excel  Instead, the majority of our users are now using our intermediate SQL format which offers a simple and reliable path to achieve complete automation with a minimum of efforts.

For the year to come, we will polish Salescast further, especially around the intermediate SQL format. Indeed, we feel that database administrators still struggle too much to import their data in Salescast. For example, we will provide better and more explicit errors messages.

Shelfcheck

Shelfcheck is our latest product, only announced a few months ago , that focuses on on-shelf availability optimization for retail networks.

At this point, a beta version of Shelfcheck is already in production on multiple stores in Europe. We plan to bring Shelfcheck out of its beta during 2012. Processing the ongoing sales stream of a large retail network at low costs is a tremendous challenge (even with the cloud). In addition, we want to establish Shelfcheck as the technology delivering the most accurate OOS alerts (out-of-shelf) of the market.

Hub

The “Hub” is the webapp in charge of managing registrations and subscriptions. Over 2011, we haven’t invested much effort on this webapp, and now, it feels somewhat antiquated, especially when compared to the more polished user interface of Salescast. Thus, in 2012, we plan to extensively refactor the Hub to simplify the management of users and subscriptions.

This roadmap isn't carved in stone. Don't hesitate to voice your opinion.

Out-of-shelf can explain 1/4 of store forecast error

The notion of forecasting accuracy is subtle, really subtle. It's common sense to say that if the closer the forecasts from the future, the better, and yet common-sense can be plain wrong.

With the launch of Shelfcheck, our on-shelf availability optimizer, we have started to process a lot more data at the point of sales level, trying to automatically detect out-of-shelf (OOS) issues.Over the last few months, our knowledge about OOS pattern has significantly improved, and today this knowledge is being recycled into our core forecasting technology.

Let's illustrate the situation. The graph below represents the daily aggregated sales at the store level for a given product. The store is open 7/7. A seven-day forecast is produced at the end of the week 2, but an OOS occurs in the middle of the week 3. Days marked with black dots have zero sales.

In this situation, the forecast is fairly accurate, but because of the OOS problem, the direct comparison of sales vs forecasts look like as if the forecast was significantly overforecasting the sales, which is not the case, at least not on the non-OOS days. The overforecast measurement is an artifact caused by the OOS itself.

So far, it seems that OOS can only degrade the perceived forecasting accuracy, which it does not seem too bad because presumably all forecasting methods should be equally impacted. After all, not forecasting model is able to anticipate the OOS problem.

Well, OOS can do a lot worse that just degrade the forecasting accuracy, OOS can also improve it.

Let have a look at the graph to illustrate this. Again, we are looking at daily sales data, but this time the OOS problem starts on the very last day of week 2.

The forecast for the week 3 is zero the whole week. The forecasting model is anticipating the duration of the OOS. The forecast is not entirely accurate because on the last day of week 3, replenishment is made and sales are non-zero again.

Obviously, a forecasting model that anticipates the duration of the OOS issue is extremely accurate as far the numerical comparison sales vs forecasts is concerned. Yet, does it really make sense? No, obviously it does not. We want to forecast the demand not sales artifacts. Worse, a zero forecast can lead to a zero replenishment which, in turn, extend the actual duration of the OOS issue (and increase further the accuracy of our OOS-enthusiast forecasting model). This is obviously not a desirable situation, no matter how good the forecast happens to be from a naive numerical viewpoint.

Bad case of OOS overfitting

We have found that the situation illustrated by the 2nd graphic is far from being unusual. Indeed, with 8% on-shelf unavailability (a typical figure in retail) and a rough 30% MAPE on daily forecasts, OOS situations typically account for 8% x 100 / 30 ≈ 27% ≈ 1/4 of the total forecast error being measured. Indeed, by definition of MAPE, a non-zero forecast on zero-sale day (OOS) generates a 100% error.

Because the fraction of the error caused by OOS is significant, we have found that a simple heuristic such as "if last day has zero sales on a top seller product, then forecast zero sales for 7 days" may reduce the forecast error from a few percents by directly leveraging the OOS pattern. Obviously, very few practitioners would explicitly put such a rule among their forecasting models, but even a moderately complex linear autoregressive model may learn this pattern to a significant extend, and thus overfit OOS.

Naturally, Shelfcheck is here to help on those OOS matters. Stay tuned.

Video: introduction to Salescast

Since Salescast now benefits from an extensively improved user interface, we have decided to produce a new video introduction as well.

Again, special thanks to Ray Groover for the voice over.

Seasonality illustrated

Seasonality is one of the strongest statistical pattern that can be leveraged to refine forecasts. Below, 4 time-series aggregated at the weekly level (159 weeks). Historical data are in red and forecasts are in purple. Vertical gray markers indicate January 1st.

When illustrating seasonality, everyone (Lokad's included) tend to use long time-series, much like the first three series here above. Indeed, it's more visual and more appealing.

However, long time-series do not represent your usual situation. On average consumer goods have a lifespan of no more than 3 or 4 years. Thus, long time-series are typically a small minority in your dataset. Worse, those long time-series might be outliers that do not reflect the behavior of other shorter-lived products.

Here above, the short 4th time-series is a much more representative case with less than 1 year of data. In such a situation, however, it's much less clear how seasonality can be leveraged. The Lokad trick to do that consists of using multiple time-series analysis.

Learn more on our seasonality definition article.

Video: How the Forecasting Engine works?

Questions about under the hood details of Lokad are frequent. We have recently added a big FAQ to our Forecasting Technology section. Today, we are releasing a new video that give the big picture on how our forecasting engine is working.

Again, special thanks to Ray Grover for the voice over.

Video: accuracy in sales forecasting

Evaluating the quality of a forecasting method can be dramatically complicated. In order to help the community finding out who's delivering the best forecasts out there :-), we have just produced a video concerning accuracy measurements of sales forecasts. Not all metrics are born equal.

 Special thanks to Ray Grover for the voice-over.

Major UI refresh for Salescast

Our flagship inventory optimization product, Salescast, has benefited from a major documentation refresh about 3 months ago. Now it's the turn of the user interface (UI) to be extensively redesigned. Here below is a screenshot illustrating the new look & feel (deployed today).

The previous UI was suffering from several problems:

• not following UI standards, making it harder for everyone to figure out how Salescast was working. In this version, we have paid a lot more attention to make Salescast not only simpler, but closer to the de facto standards (Amazon, Facebook for example). People should not have to think about UI.
• not focusing on changes, complicating the team usage of Salescast. In this version, the most notable UI addition is the Recent Activity listing that tells who has been doing what. Recent activity is a killer feature for teamwork. A lot of teamwork confusion happen job just because Bob did not tell Alice, he had just generated a new report. Now Alice will immediately notice what Bob has done, by simply looking at the recent activity.
• confusion between queries and commands, triggering the generation of new reports (and causing extra charges!) while the intent was only to browse existing reports. Now, the commands are put on the right while the queries (read only) are gathered on the left. By establishing this simple distinction, we hope to siginificantly reduce the number of misclicks.

Since it's initial release 15 months ago, Salescast has steadily improved, and this release is certainly not our final word. More good stuff is coming. Stay tuned.

Side note: a new logo for Salescast is also under way :-)

Bitcoins accepted, 10% off for BTC payments

Bitcoin is a nascent crypto-currency that has attracted a lot of attention lately. For those who've never heard of Bitcoin yet, you can check the good analysis of The Economist.

We believe that Bitcoin is a tremendous opportunity to vastly reduce payment frictions for companies operating online such as Lokad.

Thus, in order to facilitate the bootstrap of the emerging Bitcoin economy, we have decided to accept Bitcoins as payment method.

Moreover, for the time being, Bitcoin payments come with a 10% discount as a favor made to early adopters. Contact us for up-to-date conversion rates.

We anticipate that the Bitcoin adoption will be faster among eCommerce compared to classical retail. Through Salescast, online merchants can forecast their sales and optimize their inventory levels based on those forecasts. We hope to make Bitcoin-powered eShops even more productive through advance inventory optimization.