This article is based on a study funded by the IMA® Research Foundation.
When forecasting demand, the majority of organizations rely on managerial judgment, either exclusively or in combination with statistical forecasting. Sales managers typically provide to production managers a single-point demand forecast that reflects aggregate expected sales for a given product. While sales managers often have access to an initial baseline forecast generated from statistical algorithms based on historical data, they use their judgment to adjust the baseline forecast using information not reflected in historical data.
This information may include observed and expected competitor actions, changing market conditions, and anticipated customer responses to the organization’s sales promotions. Operations then uses these dynamic demand forecasts to plan material, labor, and equipment resources while finance uses them to conduct trend analysis, develop projections based on current business conditions, and aid in variance analysis to help business leaders better understand deviations and make informed decisions.
By the very nature of trying to predict the future, demand forecasting is a challenging endeavor susceptible to unintentional human judgment error and intentional opportunistic biasing. For example, managers may struggle to attend to all demand information. They may overly rely on their recall of recent sources of demand, or perhaps sources of demand that loom the largest, and neglect sources of demand that are sporadic or smaller in magnitude. This phenomenon is known as the “availability heuristic,” a term coined by Amos Tversky and Daniel Kahneman in 1973.
The resulting bias is an aggregate forecast that inadequately incorporates all sources of demand. In addition, managers are vulnerable to unintentional optimism bias where they fail to include unfavorable outliers and assume a higher likelihood of favorable scenarios in their expectations. Research has shown that optimism bias occurs in a variety of forecasting settings, including analyst forecasts, management earnings forecasts, and supply chain demand planning.
In many settings, self-interested sales managers have incentives to generate positively biased demand forecasts. They want to influence production decisions with the objective of ensuring adequate inventory for the specific products and services they anticipate selling. The lack of transparency in an aggregate demand forecast provides sales managers increased opportunity to introduce opportunistic positive bias as a means of influencing production decisions in their favor (see Figure 1).
Click to enlarge.
COSTS AND CHALLENGES
Naturally, the combination of complex information processing and misaligned incentives can significantly reduce demand forecast accuracy. Unfortunately, the costs of inaccurate demand forecasts are large and far-reaching. Most directly, forecast error results in production plan instability that leads to increased labor overtime, material and freight costs, and potentially lower product quality. Typical responses to anticipated forecast error are costly, like excess capacity and/or inventory safety stock.
In addition, inaccurate forecasts can contribute to lost sales and strained customer relationships. Suppliers even adjust their customer support based on the quality of forecasts they receive. Research has shown that poor internal forecasts negatively affect the accuracy of earnings guidance as well as the extent of misreporting, all of which undermine investor confidence. This is why demand forecast accuracy is critical to operational decisions and ultimately to companies’ overall financial performance.
Given these costs, it isn’t surprising that CFOs consider demand forecasting one of their top organizational priorities. Finance and accounting professionals, because of their central role in planning, have a unique vantage point to see and understand the different challenges inherent in the competing interests of sales and operations. Finance professionals are perfectly positioned to drive improvement and innovation in the demand forecasting process that underlies their business planning efforts. Still, despite the extensive costs and acknowledgments for a need to improve, significant progress in forecast accuracy remains elusive for many companies.
DISAGGREGATION AND FORECAST ACCURACY
Our recent academic research provides valuable insights to address this challenge. We collaborated with a large multinational agricultural chemical manufacturing organization. The company’s demand forecasting process was characterized by high levels of uncertainty and complexity due to changing weather, pest, and commodity price conditions. To further complicate supply chain planning, many products shared a small group of active ingredients with long lead times, making the planning process especially sensitive to forecast error.
In addition, some active ingredients were on allocation, meaning they weren’t freely available on the market and thus needed to be allocated across products by the planners. This dynamic created incentives for sales managers to positively bias the demand forecast, especially for those products with shared and scarce active ingredients resources.
During the period of study, the research site introduced a disaggregated demand forecast system. The new system established a reporting tool for sales managers to disaggregate the demand forecast for each product into two parts. The first part was a forecast of expected sales from relatively predictable demand sources (e.g., repeat sales to customers). The second part was a separate forecast of expected sales for each identified source of demand that depended on external events, such as weather events or competitor actions.
For these conditional sources of demand, sales managers provided information regarding the probability of the event occurring, the expected timing of the event, and the expected sign and magnitude of the event’s demand impact. Forecasts for the conditional events judged 90% probable were included in the official demand forecast. Conditional events with a 60% to 90% probability of occurring, while not incorporated into production plans, were visible to production managers in the reporting tool (see Figure 2).
Click to enlarge.
Separating the identification of different expected demand sources and the assessment of their likelihood and magnitude may mitigate both the complexity of information processing of the forecast judgment and managers’ opportunity for self-interested behavior. For example, the process of “unpacking” the various sources of demand from one another better defines each demand source and its distributional properties (i.e., the expected value and uncertainty around that expected value). These improvements in information processing are expected to reduce unintentional forecast error.
The increased transparency of disaggregated demand forecast information can discipline sales managers and reduce opportunistic positive forecast bias. If production planners (the users of the demand forecast) can observe the forecasts for the separate sources of demand, sales managers (the providers of the demand forecast) will rein in their tendencies to engage in opportunistic forecast biasing as a means of influencing production decisions. The additional information in the disaggregated demand can also help production planners facilitate improved inventory and production planning decisions.
BENEFITS OF DISAGGREGATION
To understand the effects of disaggregated demand forecasts, we gathered stock-keeping-unit-level data on demand forecast error, finished goods inventory levels, and production plan changes from before and after the disaggregated forecast system introduction. The results, published in The Accounting Review
(see “The Folly of Forecasting: The Effects of a Disaggregated Demand Forecasting System,” The Accounting Review, March 2021), show that the implementation of the disaggregated forecast system led to a decline in absolute demand forecast error.
There was also a corresponding reduction in finished goods inventory levels. But this beneficial result was observed only for those products that were more difficult to forecast as a result of increased environmental uncertainty and operational complexity and for which the cognitive benefits of forecast disaggregation were the greatest.
The researchers also found that the disaggregated forecast system led to a decline in positive forecast bias. This didn’t happen for products with insufficient production resources. This could be because the benefits of the disaggregated demand forecast system arising from increased transparency aren’t sufficient to overcome heightened incentives of self-interested sales managers risking lost sales owing to limited production resources.
Finally, the research also revealed that the decline in finished goods inventory was accompanied by an increase in work-in-process inventory. Importantly, this shift from finished goods to work-in-process wasn’t accompanied by an increase in costly last-minute production plan changes. The results indicate that the disaggregated demand forecast system facilitated a production “postponement” strategy where final manufacturing and packaging of the product are delayed as long as possible. This mitigates the likelihood of costly rework, like repackaging to meet the specifications of a specific customer order. See “Effects of Disaggregated Forecasting” for the benefits.
IMPLICATIONS FOR PRACTITIONERS
Our findings suggest that disaggregation can be a useful tool in the pursuit of forecast accuracy improvements, but there are several factors to consider for implementation. For example, if a disaggregated process increases the number of steps so much so that task complexity is increased, the disaggregation will actually increase cognitive load and will likely decrease judgment quality. Conversely, disaggregating a relatively simple task may be more trouble than it’s worth as the benefits of disaggregation are greatest with more complex tasks (see “Effective Use of Disaggregation”).
Click to enlarge.
Unmistakably, achieving demand forecast accuracy is a complex problem requiring a multifaceted approach. Indeed, disaggregated forecasting can and should be paired with other improvement efforts to reduce forecast error. For example, many organizations are taking advantage of the volume, variety, and velocity of information now available through the Big Data revolution. Big Data is incorporated into sophisticated predictive analytics models with superior processing capabilities, but even sophisticated analytics models are subject to a number of weaknesses.
Accessing and incorporating Big Data can be costly, but more importantly, these models often exhibit poor prediction accuracy for infrequent events or when conditions are changing rapidly. Google’s blunder in estimating flu trends highlights these shortcomings. After being lauded for the accuracy and speed of its estimates without the need for human judgment, the model, called Google Flu Trends (GFT), failed dramatically in the 2013 flu season, missing the peak estimates by 140%.
PEOPLE IN THE PROCESS
Researchers investigating the failure found that GFT was vulnerable to overfitting to limited sets of data points and was slow to take into account changes in user search behavior over time. In this case and in many others like it, the addition of human judgment to incorporate novel or rare events can improve overall decision quality. In the era of Big Data, though, the decision maker is often presented with large volumes of information. Disaggregation can help reduce the cognitive load on the decision maker, helping the individual focus on the most relevant cues for the prediction.
Another avenue for improvement includes incentives for demand forecast accuracy. Indeed, experimental studies have shown that incentives that penalize positive forecast errors can reduce intentional bias (see Lisa M. Scheel, Ulrich W. Thonemann, and Marco Slikker, “Designing Incentive Systems for Truthful Forecast Information Sharing Within a Firm,” Management Science, August 2018). But these types of incentive schemes are uncommon in practice, likely because many organizations worry about sales managers misallocating their effort toward forecast accuracy at the expense of revenue generation.
In considering these studies, we see a more palatable approach to overcoming these challenges. Namely, repeated interactions between decision makers elicit a greater aversion to lying, resulting in significant reductions in forecast error, even without financial incentives. Pairing a disaggregated demand forecasting system with a governance mechanism that requires decision makers to periodically review the disaggregated forecast together may further enhance the predisposition toward honesty and even reduce intentional bias.
Additionally, many organizations seek to reduce their production lead times to minimize the impact of inevitable forecast errors. These efforts are time-consuming and complicated, typically requiring cross-functional projects within and across organizations. Utilizing the data from a disaggregated demand forecasting system can help identify which types of events and products would benefit the most from lead-time reductions. Ultimately, finance and accounting professionals, at the center of the organizational planning process, can promote disaggregation as a pragmatic demand forecasting tool to improve internal decision making and align interests of cross-functional teams.
April 2021
