Insights on the Lokad tech evolution

Published on by Joannes Vermorel.

The technology of Lokad has evolved so much that people who have had the chance to trial Lokad even two years ago would barely recognize the app as it stands today.

The "old" Lokad was completely centered around our forecasting engine - i.e. what you can see as a forecasting project in your Lokad account today. As a result, our forecasting engine gradually gained tons of features not even remotely related to statistics. About two years ago, our forecasting engine had become a jack-of-all-trade responsible for almost everything:

  • data preparation with the possibility to accommodate a large diversity of data formats
  • reporting analytics with a somewhat complex, and somewhat flexible, Excel forecasting report
  • scheduled execution through a webcron integration or through the API

Then, during the last two years, we have gradually introduced stand-alone replacements for those features that now live outside our forecasting engine. However, calling those new features mere replacements is unfair, because those replacements are vastly more powerful than their original counterparts.

  • We can now process very diverse files, varying in size, in complexity and even in data formats. Plus, we have many data connectors too.
  • The capabilities of our old Excel forecasting report are dwarfed by the newer reporting capabilities of Envision.
  • Scheduling and orchestration are now first-class citizens which encompasse also the data retrieval from other apps.

Because those new features are plainly superior to the old ones, we are gradually phasing out the cruft, that is, phasing out all the non-forecasting related things that still live inside our forecasting engine.

In order to keep the process smooth, we are gradually - but actively - migrating all our clients from the old Lokad to the new Lokad; and when an old feature isn't used anymore, we remove it entirely.

The old Excel forecasting report is a tough case for us. The challenge is not to merely duplicate the report itself within Envision (that alone isn't hard at all) - the challenge is that the underlying thinking that went into this report is now fairly outdated. Indeed, over the years, Lokad has introduced better forecasting technologies - the latest iteration being probabilistic forecasts - which cannot be made to fit within this report. By design, this one report is stuck with a legacy approach to forecasting, which unfortunately is not such a good fit as far as inventory optimization is concerned.

In contrast, combining probabilistic forecasts with business drivers does require more efforts both on the Lokad side and the client side, but business results simply don't compare. The former is about optimizing percents of error while the later optimize dollars of error. Unsurprisingly, once our clients realize how much money they leave on the table not doing the later, they never consider going back to the former.

Then, our data integrations are currently undergoing a similar, and no less radical transformation. When we started developing data connectors, we tried to fit all the data we were retrieving into the framework established by our forecasting engine; that is producing files such as Lokad_Items.tsv, Lokad_Orders.tsv, etc. This approach was initially appealing because it was forcing a normalization on the data retrieved and processed by Lokad.

Unfortunately, this abstraction - like all abstractions - is leaky. All apps don't agree on what exactly is a product or an order; there are tons of subtle differences to be accounted for, and it was simply not possible to accommodate all the business subtleties through some kind of data normalization.

Thus, we have started to take the data integration challenge from another angle: retrieve the app data while preserving as much as possible the original structures and concepts. The main drawback of this approach is that it requires more initial efforts to get results because the data is not transformed upfront to be compatible with all the default expectations of Lokad.

However, because the data doesn't suffer misguided transformations, it also means that Lokad does not get stuck into not being able to accommodate business subtleties because they don't fit the framework. With some programmatic glue, we accommodate the business needs down to the minute details.

Similarly to our old Excel report, the transition toward native data - as opposed to normalized data - follows our experience which indicates that investing a little more in getting the number aligned with the business yields a lot more results.

Categories: Tags: insights