How AI and market knowledge are improving retail demand forecasting

The Covid pandemic brought incredible disruption and uncertainty to industries across the globe. Retailers, in particular, realized that traditional forecasting models that use historical sales data were inadequate to predict sales during the COVID-19 pandemic.

Traditional forecasting models, which rely on two or three years of history to capture seasonality, did not help as demand took a tailspin for many items (some plummeted while others doubled).

Fluctuating demand meant that retailers needed to shift their focus from solely predicting future store sales across longer-term planning horizons to more accurate short-term planning. In addition, they found that there were incredible amounts of external market data like COVID-19 infection rates, mobility indices (Google, Apple), demographics, and macroeconomic information that could be used as drivers to explain demand patterns and improve the forecast accuracy.

Market knowledge improves forecast accuracy and explainability

Retail forecasting incorporates more external data and market knowledge through publicly available data on consumer demographics, macroeconomic indicators such as Gross Domestic Product (GDP) and interest rates, social media buzz, and global trade. Additionally, leading indicators of demand like news, product reviews, search engine statistics, and website glance views are becoming prominent as demand sensing levers.

This data allows retailers to get very granular―down to the store and zip code level―by bringing in local weather, events near their stores, and road conditions, which affect store footfall and consumer purchases. Forecasting techniques are shifting from traditional time series methods and moving towards intelligent forecasting through AI/ML (Machine Learning) and cloud computing, which can leverage a myriad of external market drivers and scale to retail volumes.

These next-generation technologies can take leading indicator data and create a view of the forecast free of human bias or manipulation. All while constantly learning what leading indicator data best predicts changes for a more accurate forecast, right down to granular detail, such as store, item, day/hour, and the specific consumer fulfillment option – purchase at the store, ship from store, or click-and-collect.

AI/ML with robust feature engineering is no longer the secret sauce.

Feature engineering is critical to robust results and is an integral part of the process. Features can be created from internal or external drivers― even historical sales streams, such as seasonality, causal lags, life cycle characteristics, and trends. An example of a causal lag feature could be an event (e.g., markdown, promotion) that influences consumer purchases a few days or weeks after initiation. ML can iterate over multiple combinations of features to create models with superior forecast accuracy at more granular levels. ML with robust feature engineering is delivering robust forecasts that account for different demand patterns at varying levels of granularity and consumer channels, including:

• Omnichannel demand that offers separate forecasts for click-and-collect, ship-from-store, ship from DC, and in-store purchases.
• Slow movers where the volume of sales for items may be too small to generate a robust forecast. Hierarchical ML algorithms have to be leveraged to forecast at an aggregate level and then intelligently disaggregate to lower levels (e.g., at the store/daily).
• Day-of-week variations for food items with a short shelf life and replenishment did many times a week.
• Intraday forecasting for items replenished several times a day, such as in the bakery section. This requires more granular level forecasts which can be by the hour or shift.

Empowering Your Retailer Data Science Teams to Leverage AI/ML Algorithms

A critical competitive advantage for retailers is the ability to turn their algorithms and models into production-grade deployed applications efficiently. Many retailers have data science teams that have developed cutting-edge algorithms in critical areas such as store and omnichannel forecasting, labor capacity planning, assortment optimization, promotion and price modeling, and out-of-stock analysis. However, a significant portion of these efforts never end up fully deployed. Deploying AI/ML projects into usable applications remains a principal barrier to delivering business value.

To effectively leverage AI/ML, data scientists need a platform to productize their models that allows iterations safely and securely, as well as scalability to handle retail volumes. The platform should support enhanced cross-functional coordination with role-based access, scenario management, workflows, and flexible reporting capabilities.  Retail planners require the ability not just to view reports but to edit the plans with overrides at various levels in the item hierarchy, change the parameters of algorithms, and combine their business knowledge with algorithm recommendations for better outcomes. State-of-the-art platforms allow plugging in programs that in-house data science teams have created in technologies such as Python, R, PySpark, and Gurobi.

As retailers continue to emerge from the pandemic, many will likely adjust their forecasting models and methodologies accordingly to best capture consumer demand. AI/ML will continue to play a role in building more accurate forecasting capabilities, ultimately driving more decisive decision-making processes to meet consumer needs better.

In a world of volatility, APEX is the answer

The AI-powered operating model that helps enterprises thrive in the VUCA era. The o9 Digital Brain unifies planning, forecasting, and execution — powered by AI.

Learn more