Ignore prices, only pricing strategies matter

Published on by Joannes Vermorel.

In order to achieve a better pricing in commerce, the whole initiative should start by realizing that prices themselves are irrelevant. Only the pricing strategy itself matters, that is, the logic that crunches all inputs such as the purchase prices and all the other relevant variables in order to produce the final price values.

When asked about the first step to get better prices, many retail practitioners answer: knowing the prices of your competitors. Rubbish. The first step consists of transitioning from implicit pricing strategies to explicit strategies, because only the latter are subject to measurable improvements.

Unless you’re quite familiar with the concept of pricing, this might sound very confusing.

The most difficult challenge of pricing is that you can’t replay the past. Once you’ve set a price, you will never know how many sales you would have got if you had put another price on display.

Oh yes, you can still change the price now and observe the sales for the next month, but are your sales growing because the price is going down or because your web traffic is going up or because your new product picture is more attractive? You will never know for sure. Actually, it’s not just you. Nobody and certainly not us at Lokad, will never ever know for sure.

Technically, we can argue that pricing is not eligible to backtesting.

Focusing on the prices themselves is a defective process in the sense that this process can’t be challenged. Prices can be changed, obviously, but, except for pathological situations where obvious pricing mistakes get corrected, your company won’t be able to decide if the new prices made the situation better or worse.

As the old saying goes, you can’t optimize what you can’t measure.

What can be challenged, however, is the pricing strategy. The pricing strategy is the logic, the set of rules, that processes the input data such as purchase costs, customer acquisition costs, inventory costs, prices of competitors ... and that produces the final public prices to be put on display.

Unlike raw prices, a pricing strategy can be challenged: given two pricing strategies, a strict experimentation protocol can be devised to decide which one of the two strategies is the most profitable one. Designing such a protocol is not a simple task, we will get back to this in a later post.

Intuitively, if you have 1000 items to be priced, you can assign the first 500 to the first strategy, and the last 500 to the second strategy. If the two pools of items are comparable, then it becomes possible to assess the performance of the respective strategies.

In the past, a few very large online merchants tried to display different prices to different customers just for the sake of gaining further market knowledge. In order to be fair, somehow, all customers were offered, at the end of the check-out, the lowest price. This approach stirred controversies, as far we know, it’s not used anymore, at least not at scale. Furthermore, in many countries, customer protection laws prevent retailers from tweaking their prices per customer.

Unfortunately, in most retail businesses, the pricing strategy does not exist anywhere but in the mind of the people in charge of setting the prices. Frequently, a myriad of spreadsheets also contains bits of pricing logic. However, as spreadsheets mix both data and logic, updating those spreadsheets with the latest data is error-prone and time-consuming.

With such a setup, pricing strategies remain implicit and unchallenged, and consequently the performance of the pricing remains stagnant. Worse, any market change that ends up reflected in the prices requires a lot of manpower just to re-enter the revised prices in the system somehow.

Thus, any pricing initiative in retail should start by transitioning toward an explicit pricing strategy that, given the proper data inputs, can be executed by a machine in order to produce the revised price values.

Some practitioners might argue that the machine is pretty dumb and that it will never know the market like they do. Well, this is absolutely true. Having a fully automated pricing logic just happens to be simplest way to make sure that the pricing logic is well-defined (non-ambiguous, conclusive, etc); however, this logic might be nothing more than the formal transcription of the pricing logic as understood by the practitioner herself. The machine is not expected to invent the pricing strategy, merely to execute it whenever refreshed prices are needed.

Priceforge, our pricing optimization webapp, has been designed precisely to let your company write its pricing strategies, because it’s the first step toward a situation where it becomes possible to actually improve the pricing.

Categories: Tags: pricing, priceforge, insights

We're hiring: Junior Account Manager

Published on by Joannes Vermorel.

Lokad is growing, and we need you. As a junior account manager, you will learn how to grow prospects into clients, and then, learn how to support them so that they make the most of our technology.


Your goal will be to establish and grow commercial relationships between Lokad and its clients. As Lokad is reaching a global market, you will be expected to converse by email in English with companies located pretty much anywhere on the globe. You will also be expected to handle phone calls, in English. Note that we don’t do much cold calling - so you won't spend your days being rejected after trying to call people who don’t know or don’t care about Lokad.

The position in our office Paris (13th arrondissement). This job is not eligible for remote work. Salary depends on the experience and subject to negotiation.


We are seeking an enthusiastic and communicative junior. Impeccable command of English is critical because most of our business happens outside France. For practical reasons, we also expect impeccable command of French.

We do not expect you to be knowledgeable about Big Data, but if you happen to be a bit savvy with the web, or software, or ecommerce, it will be appreciated.


  • Impeccable command of English and French.
  • Sharp analytical mind.

To apply, send us your resume to contact@lokad.com. In your message, please explain shortly why you would be a good fit for this job.

Categories: Tags: hiring

Control your service levels, don’t let them control you

Published on by Joannes Vermorel.

In retail, many companies don’t have much control over their service levels. In fact, many companies don’t monitor the service level where it matters the most: the physical store. Indeed, measuring in-store service level is a tedious exercise. Some companies - mostly panelists - specialize in doing this sort of measurements, but the cost is steep as there is no workaround for the extensive manpower involved in the process.

Taking a step back, why do we even need to measure the service level?

Wouldn’t it be more convenient if the service level was something obtained by design and defined through explicit settings within the inventory optimization software? This would certainly be a lot more practical. Service levels certainly don't need to be an afterthought of the inventory optimization process.

It turns out that, historically, the need to measure the service levels came from early inventory optimization methods such as the safety stock analysis that offer about no control on actual service levels. Indeed, the underlying models rely on the assumption that the demand is normally distributed and this assumption is so wrong in practice that most retailers gave up on this assumption in favor of ad-hoc safety stock coefficients.

Those ad-hoc safety stock coefficients are not bad per se: they are certainly better than relying on abusive assumptions about the future demand. However, the quantitative relationship between the safety stock and the service level is lost. Hence, retailers end-up measuring their service levels and tweaking coefficients until inventory stabilizes somehow. At the end, the situation is not satisfying because the inventory strategy is inflexible: safety stock coefficients can’t be changed without exposing the company to a myriad of problems, repeating the tedious empirical adjustments done originally.

However, with the advent of quantile forecasting, it’s now possible to produce forecasts that very accurately drive the service levels, even if the quantile forecasts themselves are not accurate. All it takes is unbiased forecasts, and not perfectly accurate forecasts.

Indeed, quantile forecasts directly and very natively address the problem of producing the reorder quantities it takes to cover target service levels. If a new and better quantile forecasting technology is found, then this technology might be capable of achieving the same service levels with less inventory, but both technologies deliver the service levels they promise by design.

This behavior is very unlike the case of classic forecasting allied to safety stock analysis where an improvement of the accuracy, while being desirable, leads to erratic results in practice. For example, for many low volume products, as observed in stores, shifting to a dumb forecasting model that always returns zero usually improves the accuracy defined as the absolute difference between the actual sales and the forecasted sales. Obviously, shifting toward zero forecasts for half of the products can only end with dismal business results. This example might appears as anecdotal but it is not. Zero forecasts are the most accurate classic forecasts in numerous situations.

Thus, in order to take control of your service levels, it takes an inventory optimization methodology where such a control is built-in.

Categories: Tags: quantiles, inventory, insights, inventory optimization

Prestashop add-on for BotDefender

Published on by Joannes Vermorel.

BotDefender logo A few weeks ago, we released an add-on for Magento, and now it's the turn of Prestashop to get protected by BotDefender against the automated retrieval of all the prices by competitors. Check out our Prestashop add-on for BotDefender (also available from the Xtendify store).

Protecting your store has now its 1-click solution!

This add-on has been co-designed as a joint effort between Xtendify, our Prestashop specialist, and the Lokad team. As we did for Magento, we paid the same attention to the add-on performance.

Meantime, while add-ons are being released, the BotDefender infrastructure has also been upgraded for better performance. Now BotDefender auto-diagnostics the need for SSL in order to revert to plain HTTP whenever it's applicable.

Categories: Tags: prestashop, botdefender, scraping

Priceforge released, pricing optimization for commerce

Published on by Joannes Vermorel.

We are proud to announce that Priceforge, our latest app, is immediately available as a public beta. Priceforge supports merchants to gain better insights in their data - sales, inventory, prices, web traffic - and helps them to elaborate better pricing strategies; and more.

Priceforge is first a dashboarding engine to compose power dashboards where numbers that matter - and only numbers that matter - are gathered into a single page. Unlike widespread BI tools, Priceforge is commerce-native, focusing on stuff that trully matters for commerce.

Second, Priceforge is a pricing engine to design of both very simple or very advance pricing strategies. Let's replace dump prices with smart pricing. Pricing is a message sent to the market, and unlike demand forecasting, it's not the sort of problem that can be addressed by pure numerical optimization. Priceforge embraces this vision and instead of forcing prices upon merchants, Priceforge empowers them to improve the prices they have.

Commerce insights

For years, Lokad had not been delivering any data visualization capabilities. The thinking went: yes, data visualization is critical but with hundreds of Business Intelligence (BI) tools out there, surely there must be some great stuff for commerce.

Our customers proved us wrong.

The market is certainly not short of vendors but every time we ventured into our client's IT, we observed that BI was nothing but painful and expensive ventures.

Enumerating pitfalls would be tedious. Let's say that almost all solutions require at least one full time software developer to be of any use; and solutions that were not requiring a developer felt like toys when compared to Microsoft Excel.

Thus, we decided to venture into business intelligence ourselves. However, delivering yet another jack-of-all-trade solution supposedly equaly suitable for FOREX trading and car renting was an obvious pitfall we were committed to avoid.

Priceforge would be tailored for commerce.

Simplicity, power and reliability

Excel is a fantasically powerful tool, and yet, in the same time, because data and logic end up intrically mixed, it leads to unreasonably fragile processes where every refresh put the company at risk of silently breaking the logic buried in the middle of the data.

Merchants needed an approach that would bring both the power of Excel and the reliability of an industrial-grade process where the logic can be audited in-depth and incrementally improved through trials and errors. We decided to go for a tiny scripting language named Envision.

The syntax of Envision is largely inspired from the Excel formula syntax, and it's orders of magnitude simpler than a general purpose programming language.

Want to know more? Check out our tutorial to devise your first pricing strategy with Priceforge.

Categories: Tags: priceforge, pricing, commerce, data, bigdata

Magento add-on for BotDefender

Published on by Joannes Vermorel.

BotDefender logo BotDefender protects your store from the automated retrieval of all your prices by competitors and now, we have just released a free add-on for Magento.

Protecting your store has now its 1-click solution!

This add-on has been co-designed as a joint effort between Wyomind, our Magento specialist, and the Lokad team. In particular, we paid a lot of attention to the add-on performance. Indeed, the add-on should never to slow down any pages in order to preserve an optimal shopping experience.

The algorithmic strategy used to achieve a near-zero performance impact on the store is available here. In short, we are using a combination of deferred server calls and smart caching to prevent any blocking operation from the add-on, while keeping the extra cache burden very low - below 10MB even for large sites.

Stay tuned for more, BotDefender will soon be supporting other popular commerce platforms.

Categories: Tags: botdefender, magento

New video: introduction to quantile forecasting

Published on by Joannes Vermorel.

Not all forecasting methods are born equal. As far as inventory optimization is concerned, classic forecasting methods tend to make no sense whatsoever. The problem does not lie in the models used to produce the forecasts, or the way those models are tuned; it lies instead in the underlying methodology which misfits the actual business needs of commerce.

Quantile forecasts, in contrast, deeply embrace the commerce viewpoint. In this introductive video, we develop two key insights related to lead time and service level. The video illustrates why those two variables need to be at the core of the forecasting methodology in order to properly address the business needs.

Categories: Tags: quantiles, video, salescast, insights

New Professional Plan for Salescast, flat $225/month

Published on by Joannes Vermorel.

Salescast logo Since our last revision of the Salescast pricing 18 months ago, our webapp has been steadily evolving. In particular, thanks to a growing number of software partners, Salescast is increasingly accessible to very small retailers or distributors.

So far, we had two plans for Salescast:

  • Express, which is free and does not expire.
  • Enterprise, which starts at $750 / month.

Obviously, there is quite a gap between the Express and the Enterprise plans, and filling this gap is the exact purpose of the new Professional Plan which comes with a flat monthly fee at $225.

When upgrading toward the Professional Plan, you get:

  • 2 projects instead of one, which is very handy to support testing vs production environments.
  • support for bundles and kits which represent a specific inventory challenge.
  • access to the REST API of Salescast to streamline the refresh of the inventory reports.
  • email support, albeit with a low SLA, as the Lokad team may take up to 72h to reply.

While, exporting forecasts as flat files for full end-to-end inventory automation remains only accessible to companies under the Enterprise Plan, the Professional Plan is typically best suited for companies under 5 million USD of turnover.

Best wishes for 2014

Categories: Tags: salescast, pricing

BotDefender, new 3min video

Published on by Joannes Vermorel.

BotDefender is our latest app for commerce that protects your prices from being automatically retrieved by competitors on your website. We have now a free extension for Magento and just released a new video. Enjoy!

Anti-scraping service to protect your online store from the automated retrieval of your prices by competitors. Stop being outpriced every time you make a move. Best suited for retailers.

Categories: Tags: magento, botdefender, video

Megaventory integrates Salescast

Published on by Joannes Vermorel.

Megaventory logo

Megaventory is a SaaS solution dedicated to inventory and order management which has made the complicated and time-consuming task of handling inventory as simple and light as it can get.

With this same objective of simplicity in mind, the teams of Megaventory and Lokad have come together to create an easy-to-use integration bridge allowing Megaventory users to extract and upload their data directly in Salescast, Lokad’s forecasting app, in just one click.

So, after a few weeks of effort, and thanks to Megaventory’s development teams, we are proud to announce that Salescast’s forecasts are now just one click away from Megaventory clients.

Find more information on the bridge and how to access it on Megaventory’s website.

Categories: Tags: megaventory, salescast, community, integration

Senior developer job at Lokad

Published on by Joannes Vermorel.

We are hiring! Below, a copy of our LinkedIn job post.

Lokad is a team of talented and passionate developers. Business is growing, commerce is more demanding than ever for innovative technologies. We are committed to delivering such technologies.

As a senior developer, you will lead the development of one of our Big Data app (check www.lokad.com for more insights in what we do). You will be in charge of bringing our technology to the next level, not to clean up technical debt.

Challenges are numerous:

  • total reliability, because nobody likes crashing a 1000 store network,
  • vast scalability, because 1000 stores is a lot,
  • high accuracy, because we deliver the best numbers.

We expect you to bring a significant expertise to Lokad, but you will benefit from a team capable of coaching you toward your next level of craftsmanship in software design.

We happen to use C#/.NET/MVC on top of Windows Azure, combined with event stores and NoSQL persistence strategies. We expect you to be (or willing to become) extremely proficient in this environment.

We are located 50m from Place d'Italie (Paris 13).

Categories: Tags: hiring, job, bigdata

BotDefender protects your prices from competitors

Published on by Joannes Vermorel.

BotDefender logo Over the last decade, competitive intelligence has evolved from an expensive venture reserved to large companies to a commoditized piece of technology where there are apps capable of retrieving your competitor’s prices for no more than a few hundred dollars per month.

This situation is putting online merchants under a considerable pressure as they are now constantly endangered of being outpriced by competitors. Obviously, merchants did not wait for the Internet to have a look at competitor's prices, however, the automated retrieval of prices on the web really changed the the scale of the practice.

Being outpriced comes with massive costs:

  • your leads are still acquired at the same price (ex: through AdWords) but your conversion rates drop because people notice that your competitor is systematically $0.50 cheaper than you.
  • your margins drop because you have to engage in price wars in order to sustain your conversion rates; however, you usually remain a couple days behind your competitors.

Today, we are proud to announce the immediate availability of BotDefender, a service to protect the prices of your online store from the automated retrieval by competitors.

Without getting too much into the details, BotDefender is a lightweight technology that vastly complicates the task of your competitors (or the experts they have hired) to retrieve your prices. From the end customer perspective, BotDefender is invisible. From an SEO perspective, BotDefender is extremely respectful of all the "good" robots, that is, Google and search engines in general.

BotDefender comes with an Express Plan that is free and does not expire. Moreover, the Express Plan is fully functional.

Time to drive your competition nuts! Why are you still being nice and helpful with competitors? Get BotDefender now.

Categories: Tags: pricing, botdefender, scraping, intelligence, release

MoreForecast.com dedicated to Shopify merchants

Published on by Joannes Vermorel.

Screenshot of Moreforecast.com Shopify is one of the great online web store solutions available for small merchants. Shopify is distinctly rooted in a SaaS (software as a service) business model where not only the online store itself is SaaS, but all the add-ons provided by the community are also SaaS.

Today, we are very happy to announce the general availability of MoreForecast.com, an app dedicated to Shopify merchants, and intended as a bridge between Salescast and Shopify. Kudos to the Didisoft team for the development of MoreForecast.

The app supports the automated and scheduled data transfer from Shopify toward Salescast. With MoreForecast, your Salescast reports are always based on the most recent data available.

Categories: Tags: shopify, partners, release, salescast

eCommHub, drop-shipping experts recommends Salescast

Published on by Joannes Vermorel.

eCommHub is one of those platform that has been optimized for drop shipping from day one, as the founder has been running his own drop-ship site since he was 13 years old.

Then, while the goal of drop-shipping is to avoid inventory altogether, sometimes, some level inventory can't be avoided entirely; and that's where Salescast comes into the picture.

A dedicated Salescast export format is now available for eCommHub powered merchants.

Categories: Tags:

QuickBooks and Salescast integration that rocks with Webgility

Published on by Joannes Vermorel.

QuickBooks is the number one accounting software for small- and medium-sized businesses in the U.S. and Canada. For years, we have received on-going integration requests to export QuickBooks data to Salescast, our inventory optimization web app. We tried a few methods; however those initiatives did not turn out practical nor provided the smooth user experience that we were hoping for.

During our exchanges with merchants over the last couple of months, Webgility was mentioned and praised for their QuickBooks integration solution called eCommerceConnector (eCC). We contacted the Webgility team, and worked closely with their developers to incorporate Salescast into eCC. Fast-forward a few weeks later, we’re happy to announce a new service for eCommerce merchants using Salescast and QuickBooks!

Webgility’s eCC now supports Salescast integration with their QuickBooks integration. By connecting their QuickBooks company file and Salescast account with eCC, merchants can automatically export store data from QuickBooks to Salescast to do forecasting. In addition, eCC connects with the online store(s) and automatically records sales transactions as a sales receipt/invoice in QuickBooks. It also synchronizes products, pricing and inventory across the online channels and QuickBooks. Now eCommerce merchants can easily track sales and products, manage their accounting and finances, and grow their business with informed forecasted data.

Categories: community Tags: quickbooks, webgility, partners, integration

Kit, bundles, assemblies and bills of materials

Published on by Joannes Vermorel.


Earlier this year, we discussed about optimizing inventory in presence of kits or bundles. Since that time, Salescast has been steadily moving forward, and we are proud to announce that support for kits/bundles/assemblies/bills of materials is now available.

All those situations have in common that the granularity of the inventory inputs (goods received) do not match the granularity of the inventory outputs (goods served). The bundle feature of Salescast allows to rewrite the historical sales data with the right granularity, focusing on consumed parts rather than consumed bundles.

The bundles can be listed through the optional file named Lokad_Parts. The technical instruction are available in the Salescast section of our website.

Categories: release Tags: salescast, release, features, insights

Whitepaper: Quantitative Inventory Optimization

Published on by Joannes Vermorel.

A few weeks ago, Brightpearl, one of the most impressive commerce management software companies, was posting their success story about online merchants benefiting from better stock levels through Salescast.

If your company happens to use Brightpearl, check-out the recently revised tutorial to get started. In particular, Salescast now benefits from special features that make it much easier to extract your historical data from Brightpearl.

Today, as part of the Commerce Accerelation 101 initiative of Brightpearl, we are jointly publishing a whitepaper about Quantitative Inventory Optimization.

The paper details the notion of quantitative performance of the inventory. Indeed, there is a widespread misconception in supply chain, that stock levels and demand forecasts would be different things. They are not. The combination of stock on hand and the stock on order represent the true "forecast" made by your company. Hence, this is the quality of this anticipation that needs to be measured.

This is a rather opinionated paper. However, we firmly believe that the classic vision (median forecasts, safety stock analysis, etc) is, simply put, nonsense when it comes to the long tail and most of the online commerce.

Categories: partners, accuracy Tags: whitepaper, brightpearl, inventory optimization

Promotion planning in general merchandize retail - Optimization challenges

Published on by Joannes Vermorel.

So far, we covered data challenges and process challenges in the context of promotional forecasts. In this post, the last of the series, we cover the very notion of quantitative optimization when considering promotions. Indeed, the choice of the methodological framework that is used to produce the promotion forecasts and measure their quantitative performance is critically important and yet usually (almost) completely dismissed.

As the old saying goes, there is no optimization without measurement. Yet, in case of promotions, what are you actually measuring?

Quantifying the performance of promotions

The most advance predictive statistics remain rather dumb in the sense that it’s nothing but the minimization of some mathematical error function. As a consequence, if the error function is not deeply aligned with the business, there is no improvement possible, because the measure of the improvement itself is off.

It doesn’t matter to be able to move faster as long you don’t even know if you’re moving in the right direction.

When it comes to promotions, it’s not just the plain usual inventory economic forces:

  • inventory costs money; however, compared to permanent inventory, it can cost more money if the goods are not usually sold in the store, because any left-over after the end of the promotion will clutter the shelves.
  • promotions are an opportunity to increase your market shares, but typically at the expense of the retailer's margin; a key profitability driver is the stickiness of the impulse given to customers.
  • promotions are negotiated rather than merely planned; a better negotiation with the supplier can yield more profits than a better planning.

All those forces need to be accounted for quantitatively; and here lies the great difficulty: nobody wants to be quantitatively responsible for a process as erratic and uncertain as promotions. Yet, without quantitative accountability, it’s unclear whether a given promotion creates any value, and if it does, what can be improved for the next round.

A quantitative assessment requires a somewhat holistic measure, starting with the negotiation with the supplier, and ending with the far reaching consequences of imperfect inventory allocation at the store level.

Toward risk analysis with quantiles

Holistic measurements, while being desirable, are typically out of reach for most retail organizations that rely on median forecasts to produce the promotion planning. Indeed, median forecasts are implicitly equivalent to minimizing the Mean Absolute Error (MAE), which without being wrong, remains the archetype of the metric strictly agnostic of all economic forces in presence.

But how could improving the MAE be wrong? As usual, statistics are deceptive. Let’s consider a relatively erratic promoted item to be sold in 100 stores. The stores are assumed to be similar, and the item has 1/3 chances of facing a demand of 6 units, and 2/3 of facing a demand of zero unit. The best median forecast is here zero units. Indeed, 2 units per store would not be the best median forecast, but the best mean forecasts, that is, the forecast that minimizes the MSE (Mean Square Error). Obviously, forecasting a zero demand across all stores is buggy. Here, this example illustrates how MAE can extensively mismatch business forces. MSE show similar dysfunctions in other situations. There is no free lunch, you can't get a metric that is both ignorant of business and aligned with the business.

Quantile forecasts represent a first step in producing more reasonable results for promotion forecasts because it becomes possible to perform risk analysis, addressing questions such as:

  • In the upper 90% best case, how many stores will face a stock-out before the end of the promotion?
  • In the lower 10% worst case, how many stores will be left with more than 2 months of inventory?

The design of the promotion can be decomposed as a risk analysis, integrating economic forces, sitting on top of quantile forecasts. From a practical viewpoint, the method has the considerable advantage of preserving a forecast strictly decoupled from the risk analysis, with is an immense simplification as far the statistical analysis is concerned.

Couple both pricing and demand analysis

While a quantitative risk analysis already outperforms a plain median forecast, it remains relatively limited by design in its capacity to reflect the supplier negotiation forces.

Indeed, a retailer could be tempted to regenerate the promotion forecasts many time, varying the promotional conditions to reflect the scenarios negotiated with the supplier, however such a usage of the forecasting system would lead to significant overfitting.

Simply put, if a forecasting system is repeatedly used to seek the maximization of a function built on top of the forecasts, i.e. finding the best promotional plan considering the forecasted demand, then, the most extreme value produced by the system is very likely to be a statistical fluke.

Thus, instead the optimization process needs to be integrated into the system, analyzing at once both the demand elasticity and the supplier varying conditions, i.e. the bigger the deal, the more favorable the supplier conditions.

Obviously, designing such a system is vastly more complicated than plain median promotion forecasting system. However, not striving to implement such a system in any large retail network can be seen as a streetlight effect.

A policeman sees a drunk man searching for something under a streetlight and asks what the drunk has lost. He says he lost his keys and they both look under the streetlight together. After a few minutes the policeman asks if he is sure he lost them here, and the drunk replies, no, that he lost them in the park. The policeman asks why he is searching here, and the drunk replies, "this is where the light is".

The packaged technology of Lokad offers limited support to handle promotions, but this is an area that we address extensively with several large retailers, albeit in a more ad hoc fashion. Don’t hesitate to contact us, we can help.

Categories: insights, forecasting Tags: promotion, forecasting, optimization, insights

Promotion planning in general merchandise retail – Process challenges

Published on by Joannes Vermorel.

Illustration In our previous post, we covered data challenges in promotion forecasts. In this post, we cover process challenges: When are forecasts produced? How they are used? Etc. Indeed, while getting accurate forecasts is tough already, retailers frequently do not leverage forecasts the way they should, leading to sub-optimal uses of the numerical results available. As usual, statistical forecasting turns to be a counter-intuitive science, and it’s too easy to take all the wrong turns.

Do not negotiate the forecast results

The purchasing department usually supervises the promotion planning process. Yet, as much haggling can be of tremendous power to obtain good prices from suppliers, haggling over forecasts don’t work. Period. Yet, we routinely observe that promotion forecasts tend to be some kind of tradeoff negotiated between Purchasing and Supply Chain, or between Purchasing and IT, or between Purchasing and Planning, etc.

Assuming a forecasting process exists - which may or may not be accurate (this aspect is a separate concern) - then, forecasts are not up to negotiation. The forecasts are just the best statistical estimate that can be produced for the company to anticipate the demand for the promoted items. If one of the negotiating parties has a provably better forecasting method available, then this method should become the reference; but again, no negotiation involved.

The rampant misconception here is the lack of separation of concerns between forecasting and risk analysis. From a risk analysis perspective, it’s probably fine to order a 5x bigger volume than the forecast if the supplier is providing an exceptional deal for a long lived product that is already sold in the network outside the promotional event. When people “negotiate” over a forecast, it’s an untold risk analysis that is taking place. However, better results are obtained if the forecasting and risk analysis are kept separate, at least from a methodological viewpoint.

Remove manual interventions from the forecasts

In general merchandise retail, all data process involving manual operations are costly to scale at the level of the network: too many items, too many stores, too frequent promotions. Thus, from the start, the goal should be an end-to-end automated forecasting process.

Yet, while (nearly) all software vendors promise fully automated solutions, manpower requirements creep all over the place. For example, special hierarchies between items may have to be maintained just for the sake of the forecasting systems. This could involve special item groups dedicated to seasonality analysis, or listing of "paired" products where the sales history of the old product is used as a substitute when the new product is found having no sales history in the store.

Also, the fine tuning of the forecasting models themselves might very demanding, and while supposedly a one-off operation, it should be accounted for as an ongoing operational cost.

As a small tip, for store networks, beware of any vendors that promise to visualize forecasts: spending as much as 10s per data point to look at them is hideously expensive for any fairly sized retail network.

The time spend by employees should be directed to the areas where the investment is capitalized over time - continuously improving the promotional planning - rather than consumed to merely sustain the planning activity itself.

Don’t omit whole levels from the initiative

The most inaccurate forecasts are that retailers produce are the implicit ones: decisions that reflect some kind of underlying forecasts but that nobody has identified as such. For promotion forecasts, there are typically three distinct levels of forecasts:

  • national forecasts used to size the overall order passed to the supplier for the whole retail network.
  • regional forecasts used to distribute the national quantities between the warehouses.
  • local forecasts used to distribute the regional quantities between the stores.

We frequently observe that distinct entities within the retailer’s organization end-up being separately responsible for parts of the overall planning initiative: Purchasing handles the national forecasts, Supply Chain handles regional forecasts and Store Managers handles the local forecasts. Then, the situation is made worse when parties start to haggle over the numbers.

When splitting the forecasting process over multiple entities, nobody gets clearly accountable for the (in)effectiveness of the promotional planning. It’s hard to quantify the improvement brought by any specific initiative because results are mitigated or amplified by interfering initiatives carried by other parties. In practice, this complicates attempts at continuously improving the process.

Forecast as late as you can

A common delusion about statistical forecasting is the hope that, somehow, the forecasts will get perfectly accurate at some point. However, promotion forecasts won’t ever be even close to what people would commonly perceive as very accurate.

For example, across Western markets, we observe that for the majority of promoted items at the supermarket level, less than 10 units are sold per week for the duration of the promotion. However, forecasting 6 units and selling 9 units already yields a forecast error of 50%. There is no hope of achieving less than 30% error at the supermarket level in practice.

Yet, while the forecasts are bound to an irreducible level of inaccuracy, some retailers (not just retailers actually) exacerbate the problem by forecasting further in the future than what it is required.

For example, national forecasts are typically needed up to 20 weeks in advance, especially when importing goods from Asia. However neither regional nor local forecasts need to be established so long in advance. At the warehouse level, planning can typically happen only 4 to 6 weeks in advance, and then, as far stores are concerned, quantitative details of the planning can be finalized only 1 week in advance before the start of the promotion.

However, as the forecasting process is typically co-handled by various parties, a consensus emerges for a date that fits the constraints of all parties, that is, the earliest date proposed by any of the parties. This frequently results in forecasting demand at the store level up to 20 weeks in advance, generating wildly inaccurate forecasts what could have been avoiding altogether by postponing the forecasts.

Thus, we recommend tailoring the planning of the promotions so that quantitative decisions are left pending until the last moment when final forecasts are finally produced, benefiting from the latest data.

Leverage the first day(s) of promotional sales at the store level

Forecasting promotional demand at the store level is hard. However, once the first day of sales is observed, forecasting the demand for the rest of the promotion can be performed with a much higher accuracy than any forecasts produced before the start of the promotion.

Thus, promotion planning can be improved significant by not pushing all goods to the stores upfront, but only a fraction, keeping reserves in the warehouse. Then, after one or two days of sales, promotion forecasts should be revised with the initial sales to adjust how the rest of the inventory should be pushed to the stores.

Don’t tune your forecasts after each operation

One of the frequent questions we get from retailers is if we revise our forecasting models after observing the outcome of a new promotion. While this seems a reasonable approach, in the specific case of promotion forecasts, there is a catch and a naive application of this idea can backfire.

Indeed, we observe that, for most retailers, promotional operations, that is, the set of products being promoted at the same period typically with some unified promotional message, come with strong endogenous correlations between the uplifts. Simply put, some operations work better than other, and the discrepancy between the lowest performing operations and the highest performing operations is no less than a factor 10 in sales volume.

As a result, after the end of each operation, it’s tempting to revise all forecasting models upward or downward based on the latest observations. Yet, it creates significant overfitting problems: revised historical forecasts are artificially made more accurate than they really are.

In order to mitigate overfitting problems, it’s important to only revise the promotion forecasting models as part an extensive backtesting process. Backtesting is the process of replaying the whole history, iteratively re-generating all forecasts up to the last and newly added promotional operation. An extensive backtesting mitigates large amplitude swings in the anticipated uplifts of the promotions.

Validate “ex post” promotion records

As discussed in the first post of this series, data quality is an essential ingredient to produce sound promotion forecasts. Yet, figuring out oddities of promotions months after they ended is impractical. Thus, we suggest not delaying the review of the promotion data and doing it at the very end of each operation, while the operation is still fresh in the mind of the relevant people (store managers, suppliers, purchasers, etc).

In particular, we suggest looking for outliers such as zeroes and surprising volumes. Zeroes reflect either that the operation has not been carried out or that the merchandise has not been delivered to the stores. Either ways, a few phone calls can go a long way to pinpoint the problem and then apply proper data corrections.

Similarly, unexpected extreme volumes can reflect factors that have not been properly accounted for. For example some stores might have allotted display space at their entrance, while the initial plan was to keep the merchandise in the aisles. Naturally, sales volumes are much higher, but it’s only a mere consequence of an alternative facing.

Stay tuned, next time, we will discuss of the optimization challenges in promotion planning.

Categories: accuracy, insights, forecasting Tags: promotion, forecasting, insights