Factors of success in predictive supply chains

Having supply chains on autopilot through predictive technologies and achieving above human performance at scale remains a distant goal for nearly all companies, except the usual suspects (e.g. Amazon). This state of affairs is all the more surprising considering the vast number of software vendors that promise radical reductions of stocks and stock-outs - among other things. The long standing joke at Lokad has been that the only way Lokad could compete with the claims of our competitors would be to start saying that we cure cancer too.

Factors of success in predictive supply chains

Yet, my casual observations among the past experiences1 from Lokad’s customer base indicate that the vast majority of predictive supply chain initiatives are failing. By failing, I specifically mean that these solutions don’t even manage to score 10 out of 12 on our 5min supply chain performance test. A more stringent criteria for success would be a lasting boost in the supply chain’s overall financial performance, but right now, our modest 5min test is enough to provide a reasonable upper bound on success rates.

It’s hard to put a number on the actual rate, successes are so few and far between that I believe the overall market success rate2 to be below one out of ten. However, like the lottery, the winner (singular) makes the news, while the losers (hordes) are ignored. The problem is amplified as both parties, client and vendor, are strongly incentivized to market themselves as successful, no matter the actual outcome of the project. For the vendor, a success is obviously great PR material. For the client’s employees3, success means better job prospects4. Worse, letting the rest of the company realize that a multi-million investment was wasted is too frequently a recipe to get fired or sidetracked career-wise. Fortunately, quantitatively measuring the supply chain performance is a remarkably elusive goal - mostly due to network effects. Thus, it really takes an epic5 screw-up to not be able to cover up the mess by simply fudging a little with the numbers.

The first notable exception is the “AI” solutions6 - in supply chain optimization - that achieves a spectacular zero percent success rate based on my extensive observations7. Patrick Cousot, one of my former computer science professors, told me back in 2002 that in computer science, a “solution” was only referred to as “AI” as long as we had no clue whatsoever how to make it work. As soon as a practical path is discovered to make it work, the solution takes another name: convex optimization, static analysis, reinforcement learning, etc. Four years later, Mehryar Mohri, my research supervisor at the time, repeated the same thing to me. Two decades later, those insights proved prescient8, and indeed, those AI vendors don’t appear to have the slightest clue on how to make their “AI” deliver anything that would qualify as production-grade from a supply chain perspective.

If it wasn’t such a waste of resources, the situation would be perceived as comical. Let’s take the recent worldwide Walmart demand forecasting competition: out of the two dozen “notable” supply chain vendors as listed by, say, Gartner, none of them is making it to the Top 100 out of 900+ teams. The discrepancy between what objectively works, and what the market is buying or promoting is staggering. Nevertheless, free markets are incredible filters: over time, what doesn’t work well enough gets eliminated. It’s not because people come to their senses and change their mind, but merely because companies stuck with inefficient methods gradually fade away and get replaced by their competitors - the creative destruction as identified by Schumpeter.

The second notable exception is Lokad9. Over the last two years, our success rate has been consistently above three out of four. The risks are still there, but we are now one order of magnitude less risky than our competitors. Historically, it did not start that way. According to the same success criteria listed above during the first three years, from 2008 to 2011, we achieved exactly zero successes. It took us almost a gruesome decade to painfully earn each extra percentage of success, through dozens of gradual improvements. It would be exhausting to try to catalog the whole affair, but let’s review a cherry-picked list of notable insights.

  • We encourage clients to terminate wherever dissatisfied. Period. Since 2008, Lokad has been promoting monthly subscriptions, while our competitors are still pushing yearly or multi-yearly commitments. This is no accident. When a client quits, it gives a clear signal that it isn’t working. It usually boils down to either a faulty tech, or a lack of competence (or both). There is no sugar coating it. It’s tough, but we can learn from it. In contrast, there is usually nothing to be learned from the made-up polite pain points made up one year after the events to make the story respectively look better than it really was10.
  • The right forecasting technology is more important than a merely accurate one. It took us years to realize that classic naked forecasts were downright harmful. We solved this problem through probabilistic forecasts and specialized algebras to assign financial scores to decisions.

  • The right data crunching platform matters more than raw capabilities. Supply chain data are complex, heterogeneous and poorly understood. There are tons of fairly mundane problems to be addressed to avoid the pitfalls of “garbage in, garbage out”. Facilitating the in-place documentation of the data is a good start and avoiding dumb typos through autocomplete rapidly becomes a must-have.

  • To the greatest extent, correctness should be achieved by design. Fail fast and break things is not an option for supply chains. Purchasing or production blunders are exceedingly expensive. It’s already challenging enough to operate a supply chain in a highly chaotic world, a predictive technology should not make things worse by adding its own layer of entropy.

  • Approximately correct is better than exactly wrong. Hard problems like lead time variability, competitors’ price moves, cannibalization within the assortment, self-prophetic effects, … should be embraced rather than dismissed. Furthermore, it’s easy to derail an initiative by focusing on the wrong challenges such as factoring the weather because it’s cool, while dismissing tail risks because planning for the worse requires nerves and fortitude.

Most of the elements that played a decisive role in improving the success rate of our initiatives for predictive supply chains turned out to be basic - fundamental even - concepts, such as revisiting the very notion of what a forecast should be expected to be, and reengineering our technology and our processes from scratch, based on the new understanding as many times as necessary. We will keep doing so in the future. Our commitment is to the resolution of the problem, not to the specifics of the present day solution.

  1. Companies contacting Lokad that achieve more than half a billion of EUR or USD of turnover usually have a series of previous failed attempts at predictive supply chain optimization, spread over the last two (sometimes three) decades. However, those failures are not always not identified as such, because previous iterations were heterogeneous packages of sorts - like the setup or the upgrade of an ERP - and the non-predictive pieces are doing fine. ↩︎

  2. This observation excludes the management side of the supply chain challenges, which tends to have a fairly high rate of successful implementations, such as OMS (order management system), WMS (warehouse management system), PMS (procurement management system), etc. These solutions support the workflows and automate the majority of the mundane clerical tasks generated by the workflows themselves. The absolute lack of any kind of intelligence in these systems but the most mechanical ones goes a long way in letting them achieve higher success rates. ↩︎

  3. In software matters, the interest of the employees and the interest of the company are frequently at odds by design. For employees, there is a strong latent incentive to do resume-building things, such as gaining experience with the buzzwordish tech of the day or the latest “hype” methodology. As the job market dramatically undervalues “boring” and “no-drama” software works, people lean strongly toward the “exciting” and “high-drama” stuff, at the expense of the company’s performance. ↩︎

  4. Based on the job interviews that I routinely carry out at Lokad, it’s clear that most people think that visible success is essential. Candidates willing to admit genuine failures in their past work experience are few and far between. However, only people willing to take action ever make mistakes, and only people capable of introspection can identify their mistakes and improve over time. As a result, those candidates tend to be the most desirable ones from my perspective. ↩︎

  5. For example, Lidl made the newspaper headlines by admittinged back in 2018 wasting 500M€ in their SAP upgrade debacle, which was originally intended to deliver a series of inventory optimizations. ↩︎

  6. I define a supply chain solution of the “AI” class if it is marketed as such by its vendor. Naturally, based on this definition, the specifics of the AI technology vary enormously from one vendor to the next. ↩︎

  7. The absence of evidence should not be confused with the evidence of absence. I am merely pointing out that those AI successes in supply chain optimization, if any, are exceedingly rare, not that they are impossible. ↩︎

  8. As more and more people learn about this issue with AI, vendors have started to shift gears toward alternative buzzwords that, for all intents and purposes, are strictly equivalent to AI in their lack of substance, but less obvious to the layman. As of 2020, demand sensing seems to be one of these buzzwords. ↩︎

  9. Being the CEO and founder of Lokad, my opinion can be dismissed as entirely biased. Yet, I would invoke my personal track record. Back in 2008, I dropped out of my machine learning PhD, years ahead of the hype, to start Lokad. Back in 2010, we were among the first to move to the cloud. Back in 2011, I identified and invested in Bitcoin. Back in 2012, we became the first vendor to deliver quantile forecasts. Etc. I am inclined to think that luck can’t explain away all of that track record. ↩︎

  10. One year after the fact, people would politely assign the failure to a “strategic pivot”, which was unfortunately incompatible with the success of this particular initiative. Or, they will blame “bad data” problems caused by the “legacy system”. Or, they will blame acceptance issues which prevented the solution from gaining momentum, etc. ↩︎