What’s the Story?

This report is part of Coresight Research’s RetailTech series, in which we discuss leading retail-technology companies and the products they offer. Supply-chain technology is becoming an essential tool for retailers to keep pace with competition from e-commerce and other technology-enabled retailers. Pervasive within the supply chain lies demand forecasting (also known as demand prediction or planning), as its accuracy has major implications for sales, inventory, pricing and, ultimately, profitability. Whereas demand forecasting was previously the domain of spreadsheet-armed employees with substantial industry expertise, the acceleration of competition and dramatic increases in the amount of relevant and available data have elevated its challenges beyond human capabilities. Fortunately, there have been advances in data-analysis techniques, raw computing power and computing tools such as AI and ML—providing always-on capabilities to continuously monitor and optimize demand forecasting in retail. Current forecasting techniques include analysis of quantitative data and qualitative information—from internal and external sources—that incorporates weather forecasts, calendar-based fluctuations and pricing and promotion decisions from the retailer. A broad selection of companies—ranging from startups to global technology powerhouses—have developed targeted demand-forecasting tools to help retailers. This diversity in offerings enables retailers to find the solution that interoperates with their existing IT infrastructure and best suits their needs.

Why It Matters

Anticipating and planning for demand lies at the core of every retailer’s operations, since demand planning affects nearly every other business function. The interconnectedness of retail operations from sales and inventory to pricing, promotions and ultimately, margins, is illustrated in Figure 1. [caption id="attachment_114797" align="aligncenter" width="700"] Source: Coresight Research[/caption]   Errors in demand planning—planning for too much or too little product—can result in stockouts or excess inventory, requiring greater discounts or promotions, with consequences for revenues and margins. Data from every operation is fed back into the next round of demand planning to enhance its accuracy.

Demand-Forecasting Software: Deep Dive

Heightened competition among modern retailers and the complexity of their operations are putting an end to predicting a retailer’s future with rules-of-thumb and spreadsheets. There are two main reasons for this: competition from e-commerce and competition from other retailers. E-commerce competitors are steadily increasing their market share—with the forecasting tools and AI/ML capabilities that they already use helping to widen their lead. In addition, retailers are steadily digitalizing and moving their IT operations to the cloud, which enhances their flexibility and capabilities, creating an increasingly competitive environment. Compared to traditional and manual methods, the benefits of rigorous and robust demand forecasting include:

• Better coordination with suppliers
• Improved synchronization with customer demand
• Smoother supply chains
• Coordination with marketing campaigns
• Better coordination with corporate strategy and financial management
• Reduction of excess inventory, especially waste and spoilage in grocery

Role Within the Supply Chain Demand forecasting plays a key role in the modern, technology-powered supply chain—situated in between the product-design phase and the data-collection phase following the sale, as shown in the figure below. This diagram outlines the links between steps in product delivery, showing the product flow. This information is used in making demand-forecasting and other decisions, which flows in the opposite direction, as shown in the outer circle of Figure 2. Supporting the supply-chain elements is a solid foundation of technologies, including AI and ML—as shown at the bottom of Figure 2. [caption id="attachment_114798" align="aligncenter" width="700"] Source: Coresight Research[/caption]   Retailers’ Use of Demand Forecasting Many retailers are benefiting from demand-forecasting software, either as a standalone solution—most likely from an innovator—or as part of the functions provided by a large software platform, such as an ERP platform. The figure below presents examples of retailers and the demand-forecasting software providers they use. [caption id="attachment_114799" align="aligncenter" width="700"] Source: Company reports[/caption]   Demand Forecasting: Market Overview Demand forecasting is a specialized application within the business intelligence and analytics segment, which is estimated to be worth approximately $15 billion in 2020, according to technology market-research firm Apps Run the World. [caption id="attachment_114800" align="aligncenter" width="700"] Source: Apps Run the World[/caption]   [caption id="attachment_114801" align="aligncenter" width="700"] Source: Company reports/S&P Capital IQ/Owler[/caption]   Figure 6 provides profiles of the vendors, with a description of their business and product offering. [caption id="attachment_114802" align="aligncenter" width="700"] Source: Company reports[/caption]   Vendors in the business intelligence, data analytics and data-visualization industry segment also offer tools that can also be used for demand forecasting. However, it is likely that they are not specifically designed for retail demand forecasting and may lack seamless integration into other parts of retail supply chain management platforms. Many of the above vendors offer business-intelligence tools as well as the services detailed. Demand Forecasting: Art or Science? Whereas demand forecasting has historically been an art practiced by experienced employees, the complexity of a modern retailer—particularly grocers—can exceed human cognitive capacity. For example, according the Food Industry Association, US supermarkets carried an average of 33,055 items in 2018, and demand for these items must be individually forecasted and their optimal prices determined. One solution is the application of AI and ML, which leverage the ability of computers to process and find patterns in data quickly and work continuously to optimize forecasts. AI excels in developing forecasts and algorithms that improve over time, and ML excels at recognizing patterns. Compared to manual demand forecasting, the combination of AI and ML and quantitative techniques offers several advantages:

• Faster computational speeds
• Increased accuracy of forecasts
• Real-time updates
• Ability to analyze larger quantities of data
• Ability to locate hidden patterns in data
• Represents a robust system
• Adapts more quickly to changes
• Ability to handle significant complexity

Despite the many advantages of AI and ML, human experience and intuition still add significant value, particularly in designing and tuning the automated systems to make better business decisions. In addition, one particular limitation of AI is the reliance of its algorithms on historical data. In situations where there is no historical data, or a there is a dramatic change outside the initial parameters (such as the arrival of Covid-19), then the algorithms need time to gather enough data or may require human intervention to produce accurate forecasts. Generally, demand forecasting takes advantage of the following categories of data to generate accurate forecasts:

• Demand data—such as seasonality, consumer trends and sales variation by day
• Business decisions—including pricing, promotions and changes in merchandise location within the store
• External data—including holidays, local information and competitors’ data

Forecasts can be enhanced by incorporating unstructured data, such as human language and text, in addition to digital data. Forecasts can be further enhanced by incorporating external data, such as weather forecasts, to predict demand, as shown in Figure 7. [caption id="attachment_114804" align="aligncenter" width="700"] Source: Institute for Business Planning and Forecasting[/caption]   Forecasts also need to incorporate human-driven actions, such as price changes, promotions, new product introductions and holidays. Figure 8 shows base demand alongside additional expected demand from the retailer’s decisions on pricing, promotions and seasonal sales. [caption id="attachment_114803" align="aligncenter" width="700"] Source: Coresight Research[/caption]   Forecasts are typically made according to time frame. Short-term forecasts typically span less than a year—and may be specific to a week or month. They are created to ensure a smooth flow of products or services, meet sales targets, match price forecasts or to managed fluctuations in labor availability. Long-term forecasts are tied to financial planning, including capital plans and extensive business planning. There are several methods of demand forecasting, including the following:

• Time series—The data is a function of a sequence in time
• Linear regression—The data is fitted to a line
• Feature engineering—Domain knowledge is used in data mining
• Random forest—The forecast is based on many decision trees

AI enables forecasting platforms to find the most suitable forecasting method, tune it over time, and then replace it with a more accurate method when circumstances change.

What We Think

In an increasingly competitive environment, retailers can no longer rely on rules-of-thumb and spreadsheets to operate their businesses. Fortunately, a broad array of methodologies and software vendors of various sizes are available to meet retailers’ needs. Furthermore, with the move to cloud-based architectures, software offerings are modular and can interconnect via application program interfaces—increasing the interoperability of software provided by different vendors. Implications for Brands/Retailers

• Brands and retailers can utilize advanced tools to make their businesses smarter and more efficient, enabling them to keep pace with data-fueled e-commerce and technology-savvy competitors.
• Retailers can also use these tools to make sense of the flood of unstructured and structured data from a variety of sources.
• Being software-based, AI can run around the clock to identify new ways to do business better, and machine learning can identify revenue and profit opportunities that are otherwise hidden to humans.
• Demand-forecasting software offers the opportunity for retailers to improve revenues and margins, as well as achieve other benefits such as improved inventory management.

Implications for Real Estate Firms

• As increased accuracy in demand forecasting also creates more accurate inventory forecasts, retailers are likely to be more efficient and require less warehouse space.

Implications for Technology Vendors

• Technology vendors can leverage their core strengths to offer intelligent demand-forecasting solutions.
• The existence of smaller, niche players suggests that there are still market opportunities for new entrants to spring ahead of slower, larger broad-based technology providers.