The Covid pandemic brought incredible disruption and uncertainty to industries across the globe. Retailers, in particular, realized traditional forecasting models that use historical sales data, were inadequate in predicting sales during the COVID-19 pandemic. Traditional forecasting models, which rely on two or three years of history to capture seasonality, did not help as demand took a tailspin for many items (some plummeted while others doubled).
Fluctuating demand meant that retailers needed to shift their focus from solely predicting future store sales across longer term planning horizons, to more accurate short-term planning. In addition, they found that there were incredible amounts of external market data like COVID-19 infection rates, mobility indices (Google, Apple), demographics and macroeconomic information, that could be made use of as drivers to explain demand patterns and also improve the forecast accuracy.
Market knowledge improves forecast accuracy and explainability
Retail forecasting is incorporating more external data and market knowledge through publicly available data on consumer demographics, macroeconomic indicators such as Gross Domestic Product (GDP) and interest rates, social media buzz, and global trade. Additionally, leading indicators of demand like news, product reviews, search engine statistics, and website glance views are becoming prominent as demand sensing levers.
This data allows retailers to get very granular―down to the store and zip code level―by bringing in local weather, events near their stores and road conditions, which affect store footfall and consumer purchases. Forecasting techniques are shifting from traditional time series methods and moving towards intelligent forecasting through AI/ML (Machine Learning) and cloud computing, that can leverage a myriad of external market drivers and scale to retail volumes.
These next generation technologies can take leading indicator data and create a view of the forecast that is free of human bias or manipulation. All while constantly learning what leading indicator data best predicts changes for a more accurate forecast, right down to a granular detail, such as store, item, day/hour, and the specific consumer fulfilment option – purchase at store, ship from store or click-and-collect.
AI/ML with robust feature-engineering is no longer the secret sauce
Feature engineering is critical to robust results and is an important part of the process. Features can be created from internal or external drivers― even historical sales streams, such as seasonality, causal lags, life cycle characteristics, and trends. An example of a causal lag feature could be an event (e.g. markdown, promotion) that influences consumer purchases a few days or weeks after initiation. ML has the ability to iterate over multiple combinations of features to create models with superior forecast accuracy at more granular levels. ML with robust feature engineering is delivering robust forecasts that account for different demand patterns at varying levels of granularity and consumer channels including:
- Omnichannel demand that offers separate forecasts for click-and-collect, ship from store, ship from DC and in-store purchases.
- Slow movers where the volume of sales for items may be too small to generate a robust forecast. Hierarchical ML algorithms have to be leveraged to forecast at an aggregate level and then intelligently disaggregate to lower levels (e.g. at store/daily).
- Day of week variations for food items with a short shelf life and replenishment done many times in a week.
- Intraday forecasting for items that get replenished several times in a day such as in the bakery section. This requires more granular level forecasts which can be by the hour or shift.
Empowering Your Retailer Data Science Teams to Leverage AI/ML Algorithms
A key competitive advantage for retailers is the ability to efficiently turn their algorithms and models into production grade deployed applications. Many retailers have data science teams that have developed cutting edge algorithms in critical areas such as store and omnichannel forecasting, labor capacity planning, assortment optimization, promotion and price modeling and out-of-stock analysis. However, a significant portion of these efforts never end up being fully deployed. Deploying AI/ML projects into usable applications remains a principal barrier to delivering business value.
To effectively leverage AI/ML, data scientists need a platform to productize their models that allows iterations safely and securely, as well as scalability to handle retail volumes. The platform should support enhanced cross-functional coordination with role-based access, scenario management, workflows, and flexible reporting capabilities. Retail planners require the ability to not just view reports, but to be able to edit the plans with overrides at various levels in the item hierarchy, change the parameters of algorithms, and combine their business knowledge with algorithm recommendations for better outcomes. State-of-the-art platforms allow plugging in programs that in-house data science teams have created in technologies such as Python, R, PySpark and Gurobi.
Overall, as retailers continue to emerge from the pandemic, many will likely adjust their forecasting models and methodologies accordingly to best capture consumer demand. AI/ML will continue to play a role in building more accurate forecasting capabilities, ultimately driving stronger decision making processes to better meet consumer needs.