POCs don’t work in quantitative supply chain optimization
Proofs of concept are one of the most frequent requests we get from our prospect clients looking to try out our supply chain optimization service. Yet, we frequently decline such requests; first because they hurt client’s company itself, and second because they also hurt Lokad in the process. Since POCs – or proofs-of-concept – are so widespread in B2B software, it is usually hard to grasp why they can be downright harmful in the specific case of quantitative supply chain optimization (1). In this post, we gather our findings on POCs, considering them to be a supply chain “anti-pattern".
POCs do not cost less
One core assumption behind the POC methodology is that POCs cost less than the real thing. Unfortunately, this assumption is nearly always incorrect.
First, establishing a small scope within an entire supply chain network only barely moves the needle. In the past, software vendors struggled with scalability problems and actual full-scale deployments did typically require heavy upfront hardware investments, potentially bundled with software licenses such as databases. Without these investments, it was not even possible to start processing data. Yet, in today’s age of cloud computing, this constraint does no longer exist, and if an app is designed correctly, nothing extra is required to start processing data. The cloud computing bill will increase only marginally for every additional client, but all in all, this cost is negligible compared to, say, the costs involved in establishing a discussion with the prospect. Second, the bulk of initial efforts consists of qualifying data, followed by a proper identification oed in establishing a commercial B2B relationship with the client.
Worse still, having more data typically makes things easier, not harder, whenever statistical forecasting is involved. Therefore, by restricting the data scope, POCs tend to make things more difficult, and hence more costly, when compared to addressing the full scope of challenges. Our experience indicates that even when POCs focus on only 5% of the entire supply chain network, these 5% typically involve almost the entire complexity of the network as a whole. Actually, it is precisely because POCs embed nearly all the complexity of a full-scale project, that POCs would be expected to make sense in the first place.
Dismissing the complexity is indeed not an option. If your supply network includes container shipments and working with unreliable suppliers, how could a POC possibly be convincing if these elements are not been factored into the initiative? If any specific constraint is ignored, such as MOQs (minimal order quantities), the numerical results end-up unusable.
The costs beyond the POC are driven by the efforts to be put on both sides, both by Lokad and its client, in managing the full complexity of the supply chain. Those costs are driven by the specificities of the business being considered, the scale having only a marginal impact on costs.
POCs increase the odds of failure
When opting for a POC, companies frequently end up trying stuff to improve their supply chain. However, in this specific case, I would like to quote Yoda, Do. Or do not. There is no try. Despite the claims of software vendors, optimizing supply chain is hard. The problem with POCs is that is gives too much leeway for parties to fail.
- Extracting sales history is hellishly complicated. Alas, there is no alternative anyway: one will never succeed at optimizing supply chain without data representing the demand.
- Electronic stock levels are inaccurate. Technology can help auto-detect the most obvious deviations, and help prioritize recounts. However, it is not uncommon for, supply chain managers to deal with phantom inventory too.
- Forecasts remain poor no matter what. Businesses should learn to embrace an uncertain future, instead of wishing this uncertainty away. Probabilistic forecasts are particularly good at capturing future uncertainty.
Complications are as many excuses to drop the ball.
There are situations where solutions are expected to be easy and uneventful: creating a new email account for a new employee for example. However, optimizing supply chain is nearly always difficult: if the company has been around for more than a few years, the “easy” part of supply chain optimization has already been done years ago. The “difficult” part is what remains.
In our experience, most POCs fail at the initial stages of the project, when teams are still struggling with data issues. Yet, this says nothing about the inventory optimization solution itself, because the solution is never put to the test.
POCs sidetrack supply chain optimization initiatives
POCs emphasize a viewpoint that is not exactly the production viewpoint. Executives seek benchmarks to be made or KPIs to be established. However, what if a certain KPI happens to be more difficult to compute than performing the optimization itself? What if the KPI itself, while instructive, does not offer any tractable options to improve anything?
Our experience indicates that POCs routinely get sidetracked by considerations that are simply non-requirements from a production perspective. Trying to address those requirements typically poisons the POC because suddenly the POC actually becomes an even greater challenge than the production itself.
Also, as the main point of a POC is to seek reassurance, most POCs suffer from gold plating anti-patterns where the client company pressures the vendor to be all inclusive in capturing every single aspect of their business, even at the expense of the overall reliability of the solution. The resulting solution is often too brittle to be of any use from a production perspective.
We have seen many POCs fail on “imaginary” problems as well. For example, if the best forecasting model, empirically tested over thousands of SKUs, happens to be non-seasonal and outperforms all other available seasonable models, should this be considered a problem? There is no question whether the business in question is seasonal: it is. But what if the best known way to anticipate future demand is to merely ignore seasonality in this case? Should this be considered a problem? In our experience, this single “problem” has been considered a blocking issue for many POCs while supply chain practitioners themselves were admitting that the ultimate purchase order quantities suggested were sound.
Go for production and revise project if needed
POCs are usually and rightfully perceived as distractions by practitioners who need to keep the business running while the next-gen solution is coming. Our experience indicates that going straight for production is cheaper and less risky. However, this should be done with the proper methodology.
First, failing due to “logistics of data” is not an option. You can’t optimize what you don’t measure. If data is meaningless, then all optimization attempts will be meaningless too. Success is a requirement since otherwise the company may no longer exist a few years from now. It happens that the vast majority of efforts to be invested are associated with this logistics of data, and this investment can be nearly fully separated from the solution being considered for production. And this is a good thing! If the optimization solution was for some reason falling short, the investment is not lost and merely needs to be redirected to a better alternative solution.
Second, while the goal is to shoot straight for production, it does not mean that numbers go unchallenged, quite the opposite. The old and new process should coexist, picking as many low hanging fruits as possible from the old process (2) while the new one gets polished.
Then, dozens of issues typically arise. It is important to sort them out:
- problems that were already impacting the old process, albeit in a more silent way. Good processes and good technologies make problems obvious; this is not a defect but a virtue.
- problems that can’t be fixed by the software being deployed. If the SKU picking is unreliable in the warehouse, don’t expect the demand forecasting module to make it trustworthy.
- mismatch between real problems vs. expectations. Statistical forecasting is deeply counter-intuitive; don’t let your expectations override what quantitative measurements tell you.
- design issues that can’t be solved without significantly redesigning the solution, which usually happens when the software does not have the right angle to tackle the challenge.
The last point requires another solution to be considered. However, as mentioned above, this should not be the end of the initiative, merely the beginning of a collaboration with another vendor.
Abandoning the idea of a POC usually means losing the entire momentum that has been invested in the initiative. Furthermore, most POCs fail for the wrong reasons, which means that the odds of success of future attempts will barely be improved as the real challenges remain mostly untouched.
Going straight for production is actually less risky than it sounds. It helps prevent an entire class of failures that tend to be ignored in the case of POCs, while they should not be. It forces the initiative to adopt a narrow focus on what is actually needed to obtain improvements, and to put wishful thinking aside. When facing a serious vendor failure, a company can still capitalize on its internal momentum and switch to another vendor, without losing the said momentum as it usually happens with POCs.
(1) There are many ways to optimize supply chain: better processes, better suppliers, better transporters, better hiring … This post focuses on quantitative optimization: supply chain challenges that can be addressed through statistical forecasts and/or numeric solvers.
(2) Fixing phantom inventory is of benefit to all inventory optimization processes. The same is true for revisiting and improving inventory valuations.
