Retail Sales Forecasting Methods

Here's an underrated approach to estimate the causal impact of a product change that affects some users but not others. This is common in product settings, so let's take a simple example: measuring the impact on app downloads when a change is made to the iOS version of an app but not the Android version. The three-step approach is: 𝐒𝐭𝐞𝐩 1: In the pre-treatment period, use a Bayesian structural time series function (𝘧) to learn how the treated group's time series relates to the control group's time series. In our example, this means learning the function that recreates iOS app downloads as a function of Android app downloads. 𝐒𝐭𝐞𝐩 2: In the post-treatment period, use the function (𝘧) to calculate the post-treatment counterfactual for iOS app downloads based on the actual Android app download values. 𝐒𝐭𝐞𝐩 3: Calculate the causal impact by taking the difference between the actual post-treatment iOS app downloads and the counterfactual values from Step 2. The visual shows how this works! (Links to the R and Python packages in the comments) Cite: Brodersen KH, Gallusser F, Koehler J, Remy N, Scott SL. Inferring causal impact using Bayesian structural time-series models. 𝘈𝘯𝘯𝘢𝘭𝘴 𝘰𝘧 𝘈𝘱𝘱𝘭𝘪𝘦𝘥 𝘚𝘵𝘢𝘵𝘪𝘴𝘵𝘪𝘤𝘴, 2015, Vol. 9, No. 1, 247-274

Price adjustments are one of the most important marketing levers for boosting sales. The key challenge lies in measuring consumers' price sensitivity accurately. How different are the results when using experiments versus MMM-style analyses? A new study sheds light on this critical question, comparing price elasticities for a US grocery retailer using these different methods: 📉 Non-experimental scanner price data (which we also call observational data), analyzed using OLS regressions. This approach is common in Marketing Mix Models (MMM) and is the most widely used method for obtaining pricing insights. 🔬 Experimental random price manipulations. The findings reveal significant differences in price elasticities (accounting for temporary price promotions) across nine product categories: 📉Standard OLS: -1.08 📊 OLS with control functions (inverse instrument): -0.92 * 🔬Experiment-based (2SLS): -0.32 Summary: While OLS-based analyses suggest values close to unit elasticity (around -1), the experimental findings imply that many products likely exhibit inelastic demand. In other words, demand does not change significantly when prices rise. Key takeaways: 🔍 These results highlight how potentially misleading non-experimental analyses, including traditional MMM, can be. Even typical econometric adjustment tricks (e.g., instruments/control functions) may not sufficiently adjust price elasticity estimates. 🛡️Having said this, given the bias found in the study, one could also argue that MMM-based analyses are rather conservative in many cases (at least when it comes to suggesting price hikes). But we should be careful when considering price reductions based on MMM results. 💰 The good news for marketers is that the study findings suggest many brands may have stronger pricing power than previously thought. Reminder: Products with a price elasticity smaller than 1 (in absolute value) may have room to raise prices and boost revenues. Caveats: ⚠️ The study focused on nine product categories (409 products), 35 weeks, and 82 stores before COVID. Prices have shifted significantly since then. Price elasticities vary by brand, product, time period, and region. Thus, we need more replication and tests to understand when and why consumer price sensitivities differ. Put simply, we need more brave brands willing to experiment—even with prices 💪. The original study, which includes many different analyses and robustness checks, is a masterclass in price elasticity analysis (warning: it's a highly technical read) and can be found here: https://lnkd.in/gE5Y3yxZ Technical notes: * I could not find the exact number in the text, so this is an approximate average derived from Figure 12 of Bray, Sanders, and Stamatopoulos 2024.

A poor demand forecast destroys profits and cash. This infographic shows 7 forecasting techniques, pros, cons, & when to use: 1️⃣ Moving Average ↳ Averages historical demand over a specified period to smooth out trends ↳ Pros: simple to calculate and understand  ↳ Cons: lag effect; may not respond well to rapid changes ↳ When: short-term forecasting where trends are relatively stable 2️⃣ Exponential Smoothing ↳ Weights recent demand more heavily than older data ↳ Pros: responds faster to recent changes; easy to implement ↳ Cons: requires selection of a smoothing constant ↳ When: when recent data is more relevant than older data 3️⃣ Triple Exponential Smoothing  ↳ Adds components for trend & seasonality ↳ Pros: handles data with both trend and seasonal patterns ↳ Cons: requires careful parameter tuning ↳ When: when data has both trend and seasonal variations 4️⃣ Linear Regression ↳ Models the relationship between dependent and independent variables ↳ Pros: provides a clear mathematical relationship ↳ Cons: assumes a linear relationship ↳ When: when the relationship between variables is linear 5️⃣ ARIMA ↳ Combines autoregression, differencing, and moving averages ↳ Pros: versatile; handles a variety of time series data patterns ↳ Cons: complex; requires parameter tuning and expertise ↳ When: when data exhibits autocorrelation and non-stationarity 6️⃣ Delphi Method ↳ Expert consensus is gathered and refined through multiple rounds ↳ Pros: leverages expert knowledge; useful for long-term forecasting ↳ Cons: time-consuming; subjective and may introduce bias ↳ When: historical data is limited or unavailable, low predictability 7️⃣ Neural Networks ↳ Uses AI to model complex relationships in data ↳ Pros: can capture nonlinear relationships; adaptive and flexible ↳ Cons: requires large data sets; can be a "black box" with less interpretability ↳ When: for complex, non-linear data patterns and large data sets Any others to add?

Machine Learning Interview Question with Solution for a Walmart Data Scientist Role Question: Walmart collects data from various sources like sales, inventory, and customer behavior. One of the main goals is to predict product demand to optimize inventory levels. Suppose you are provided with historical sales data, including features like Date, Store_ID, Product_ID, Sales_Quantity, and Promotion. How would you build a machine learning model to forecast sales for the next 30 days? Solution: To solve this problem, we can follow these steps:    ⏩ Understand the Problem: The goal is to predict Sales_Quantity for the next 30 days, which makes this a time series forecasting problem.    ⏩ Data Preprocessing:        - Handling Missing Values: If there are any missing values, we need to fill them appropriately (e.g., using median or forward fill for missing sales quantities).        - Feature Engineering: Create additional features such as:             - Lag features (previous sales quantities)             - Rolling averages (7-day, 30-day)             - Holidays (since Walmart sales may spike during holidays)             - Days of the week (sales patterns may differ between weekdays and weekends)    ⏩ Train-Test Split:        Split the data into a train set (e.g., sales before the last month) and a test set (last month of sales).    ⏩ Model Selection: Some of the models we can consider are:         - Random Forest Regressor: Can handle non-linear relationships and provide feature importance.         - XGBoost or LightGBM: These are tree-based gradient boosting models that work well for structured data like sales forecasting.         - ARIMA (AutoRegressive Integrated Moving Average): A classic time-series forecasting model.    ⏩ Training the Model:        - Use historical data to train the model on sales quantity (Sales_Quantity) as the target variable.        - Ensure to include relevant features like promotions, store IDs, and lagged sales as inputs.    ⏩ Evaluation Metrics:        Use Mean Absolute Error (MAE) or Root Mean Squared Error (RMSE) to evaluate the model's performance on the test set.    ⏩ Hyperparameter Tuning:        Perform Grid Search or Random Search for hyperparameter tuning to optimize model performance.    ⏩ Deployment:        Once the model is trained and evaluated, deploy it to forecast future sales and adjust inventory levels accordingly. Can you think of any alternate solution? Please share in comments! #dsinterviewpreparation #ml #walmartinterview

Yesterday’s sales can’t see tomorrow’s storm, But AI can 😎 Most manufacturers still build demand forecasts based on one thing: 𝐡𝐢𝐬𝐭𝐨𝐫𝐢𝐜𝐚𝐥 𝐬𝐚𝐥𝐞𝐬. Which is fine… until the market shifts. Or weather changes. Or a social post goes viral. (Which is basically always.) That’s why AI is changing the forecasting game. Not by making predictions perfect—just a lot less wrong. And a little less wrong can mean a lot more profitable. According to the Institute of Business Forecasting, the average tech company saves $𝟗𝟕𝟎𝐊 per year by reducing under-forecasting by just 1%, and another $𝟏.𝟓𝐌 by trimming over-forecasting. For consumer product companies, those same 1% improvements are worth $𝟑.𝟓𝐌 (under-forecasting) and $𝟏.𝟒𝟑𝐌 (over-forecasting). (Source: https://lnkd.in/e_NJNevk) And were are only talking 1 improvement%!!! Let that sink in... All that money just from getting a little better at predicting what customers will actually buy. And yes, AI can help you get there: • By ingesting external signals (weather, social, events, IoT, etc.) • By recognizing nonlinear patterns that Excel never will • And by constantly learning—unlike your spreadsheet But it’s not just about tech. It’s about process: • Use Forecast Value-Added (FVA) to track which steps help (or hurt) • Get sales, marketing, and ops aligned in S&OP—not working in silos • Focus on data quality—AI is only as smart as your ERP is clean • Plan continuously—forecasting is not a set-it-and-forget-it task Bottom line: If you’re still relying on history to predict the future, you’re underestimating the cost of being wrong. Your competitors aren’t. ******************************************* • Visit www.jeffwinterinsights.com for access to all my content and to stay current on Industry 4.0 and other cool tech trends • Ring the 🔔 for notifications!

Modeling something like time series goes past just throwing features in a model. In the world of time series data, each observation is associated with a specific time point, and part of our goal is to harness the power of temporal dependencies. Enter autoregression and lagging -  concepts that taps into the correlation between current and past observations to make forecasts.  At its core, autoregression involves modeling a time series as a function of its previous values. The current value relies on its historical counterparts. To dive a bit deeper, we use lagged values as features to predict the next data point. For instance, in a simple autoregressive model of order 1 (AR(1)), we predict the current value based on the previous value multiplied by a coefficient. The coefficient determines the impact of the past value on the present one only one time period previous. One popular approach that can be used in conjunction with autoregression is the ARIMA (AutoRegressive Integrated Moving Average) model. ARIMA is a powerful time series forecasting method that incorporates autoregression, differencing, and moving average components. It's particularly effective for data with trends and seasonality. ARIMA can be fine-tuned with parameters like the order of autoregression, differencing, and moving average to achieve accurate predictions. When I was building ARIMAs for econometric time series forecasting, in addition to autoregression where you're lagging the whole model, I was also taught to lag the individual economic variables. If I was building a model for energy consumption of residential homes, the number of housing permits each month would be a relevant variable. Although, if there’s a ton of housing permits given in January, you won’t see the actual effect of that until later when the houses are built and people are actually consuming energy! That variable needed to be lagged by several months. Another innovative strategy to enhance time series forecasting is the use of neural networks, particularly Recurrent Neural Networks (RNNs) or Long Short-Term Memory (LSTM) networks. RNNs and LSTMs are designed to handle sequential data like time series. They can learn complex patterns and long-term dependencies within the data, making them powerful tools for autoregressive forecasting. Neural networks are fed with past time steps as inputs to predict future values effectively. In addition to autoregression in neural networks, I also used lagging there too! When I built an hourly model to forecast electric energy consumption, I actually built 24 individual models, one for each hour, and each hour lagged on the previous one. The energy consumption and weather of the previous hour was very important in predicting what would happen in the next forecasting period. (this model was actually used for determining where they should shift electricity during peak load times). Happy forecasting!

Machine learning beats traditional forecasting methods in multi series forecasting. In one of the latest M forecasting competitions, the aim was to advance what we know about time series forecasting methods and strategies. Competitors had to forecast 40k+ time series representing sales for the largest retail company in the world by revenue: Walmart. These are the main findings: ▶️ Performance of ML Methods: Machine learning (ML) models demonstrate superior accuracy compared to simple statistical methods. Hybrid approaches that combine ML techniques with statistical functionalities often yield effective results. Advanced ML methods, such as LightGBM and deep learning techniques, have shown significant forecasting potential. ▶️ Value of Combining Forecasts: Combining forecasts from various methods enhances accuracy. Even simple, equal-weighted combinations of models can outperform more complex approaches, reaffirming the effectiveness of ensemble strategies. ▶️ Cross-Learning Benefits: Utilizing cross-learning from correlated, hierarchical data improves forecasting accuracy. In short, one model to forecast thousands of time series. This approach allows for more efficient training and reduces computational costs, making it a valuable strategy. ▶️ Differences in Performance: Winning methods often outperform traditional benchmarks significantly. However, many teams may not surpass the performance of simpler methods, indicating that straightforward approaches can still be effective. Impact of External Adjustments: Incorporating external adjustments (ie, data based insight) can enhance forecast accuracy. ▶️ Importance of Cross-Validation Strategies: Effective cross-validation (CV) strategies are crucial for accurately assessing forecasting methods. Many teams fail to select the best forecasts due to inadequate CV methods. Utilizing extensive validation techniques can ensure robustness. ▶️ Role of Exogenous Variables: Including exogenous/explanatory variables significantly improves forecasting accuracy. Additional data such as promotions and price changes can lead to substantial improvements over models that rely solely on historical data. Overall, these findings emphasize the effectiveness of ML methods, the value of combining forecasts, and the importance of incorporating external factors and robust validation strategies in forecasting. If you haven’t already, try using machine learning models to forecast your future challenge 🙂 Read the article 👉 https://buff.ly/3O95gQp

In today’s fast-paced tech landscape, understanding the true causal impact of business decisions is more critical than ever. Whether you're launching a new feature, running a marketing campaign, or testing operational changes, it’s essential to go beyond correlation and uncover what actually drives outcomes. In a recent blog post, a data scientist from Walmart explains what Bayesian Structural Time Series (BSTS) models are and how they can be used to measure causal impact. BSTS is a flexible time series modeling approach that breaks down data into components like trend, seasonality, and regressors—enabling teams to simulate what would have happened without an intervention. The post does a great job of explaining the methodology with clear, real-world examples. It’s a valuable read for anyone working on experimentation, marketing measurement, or causal inference at scale. #DataScience #MachineLearning #CausalInference #Analytics #BayesianModeling #SnacksWeeklyonDataScience – – –  Check out the "Snacks Weekly on Data Science" podcast and subscribe, where I explain in more detail the concepts discussed in this and future posts:    -- Spotify: https://lnkd.in/gKgaMvbh   -- Apple Podcast: https://lnkd.in/gFYvfB8V    -- Youtube: https://lnkd.in/gcwPeBmR https://lnkd.in/gzSZcSh8

Sales forecasting is a high-impact use case for predictive analytics! Here's what you need to know about it: 𝗨𝘀𝗲 𝗖𝗮𝘀𝗲𝘀 𝗳𝗼𝗿 𝗦𝗮𝗹𝗲𝘀 𝗙𝗼𝗿𝗲𝗰𝗮𝘀𝘁𝗶𝗻𝗴: • 𝗦𝘁𝗿𝗮𝘁𝗲𝗴𝗶𝗰 𝗣𝗹𝗮𝗻𝗻𝗶𝗻𝗴: Accurate forecasts help the business to make better decisions regarding budgeting, resource allocation, and general planning.    • 𝗜𝗻𝘃𝗲𝗻𝘁𝗼𝗿𝘆 𝗠𝗮𝗻𝗮𝗴𝗲𝗺𝗲𝗻𝘁: Helps manage inventory more efficiently by predicting future demand, and avoiding stockouts or overstock situations.    • 𝗥𝗶𝘀𝗸 𝗠𝗮𝗻𝗮𝗴𝗲𝗺𝗲𝗻𝘁: Sales forecasts allow companies to anticipate market trends and adapt their strategies in response to upcoming shifts. 𝗛𝗼𝘄 𝘁𝗼 𝗜𝗺𝗽𝗹𝗲𝗺𝗲𝗻𝘁 𝗮 𝗦𝗮𝗹𝗲𝘀 𝗙𝗼𝗿𝗲𝗰𝗮𝘀𝘁𝗶𝗻𝗴 𝗣𝗿𝗼𝗷𝗲𝗰𝘁: 1. 𝗗𝗮𝘁𝗮 𝗖𝗼𝗹𝗹𝗲𝗰𝘁𝗶𝗼𝗻: Collect historical sales data and external variables influencing sales (like economic indicators, market trends, promotional activities, and weather data).     2. 𝗗𝗮𝘁𝗮 𝗣𝗿𝗲𝗽𝗮𝗿𝗮𝘁𝗶𝗼𝗻: Clean the data by handling missing values, outliers, and anomalies to ensure the quality and reliability of your model.     3. 𝗘𝘅𝗽𝗹𝗼𝗿𝗮𝘁𝗼𝗿𝘆 𝗗𝗮𝘁𝗮 𝗔𝗻𝗮𝗹𝘆𝘀𝗶𝘀 (𝗘𝗗𝗔): Analyze the data to understand patterns, trends, and seasonal behavior. This step is important for choosing the right forecasting model.     4. 𝗠𝗼𝗱𝗲𝗹 𝗦𝗲𝗹𝗲𝗰𝘁𝗶𝗼𝗻: Choose a forecasting model based on the business context and the structure of your data. Common choices include time series models (like ARIMA or Prophet), regression models, or more advanced machine learning models depending on data and business complexity.     5. 𝗠𝗼𝗱𝗲𝗹 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗮𝗻𝗱 𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗶𝗼𝗻: Train your model using historical data and validate it by splitting the data into training and test sets, and using techniques like cross-validation to ensure its predictive power.     6. 𝗜𝗺𝗽𝗹𝗲𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻 𝗮𝗻𝗱 𝗠𝗼𝗻𝗶𝘁𝗼𝗿𝗶𝗻𝗴: Deploy the model to start forecasting and continuously monitor its performance over time, making adjustments as necessary based on feedback and new data.     7. 𝗥𝗲𝗽𝗼𝗿𝘁𝗶𝗻𝗴: Communicate the forecasting results to stakeholders through visualizations and reports on accuracy, changes, and recommendations. By being able to build sales forecasts, you contribute directly to the organization's bottom line. This high-impact work can increase your visibility with management, opening paths to more senior roles. Have you been involved in sales forecasting or plan to work in this field? ---------------- ♻️ 𝗦𝗵𝗮𝗿𝗲 if you find this post useful ➕ 𝗙𝗼𝗹𝗹𝗼𝘄 for more daily insights on how to grow your career in the data field #dataanalytics #datascience #predictiveanalytics #salesforecasting #forecast #careergrowth

Deterministic vs “stochastic” forecasting Talking #supplychainmanagement I am sometimes stunned by the confusion surrounding the use of “point” (or deterministic) forecasts, versus “probabilistic” (or stochastic) forecasting.   Let me use the context of inventory planning in a setting with long lead times (say several months). Uncertainty can be traced to the manufacture of the product, shipping delays, and the weekly demands for a product which can vary from the randomness of customer choice, seasonal and holiday variations, competitor behavior and corporate decisions (pricing, marketing).   Despite all these uncertainties, industry continues to equate “forecast” with “point forecast” where forecast errors are given by the difference between the actual demand and the (point) forecast.   There are two ways to meet service requirements such as meeting 95 percent of demand:   1)   Inventory decisions are made using a stochastic (probabilistic) lookahead model, the heart of which would be a probabilistic (or stochastic) forecast. This is the most familiar approach in inventory textbooks (e.g. using the 95th percentile of the lead time demand). 2)   We can use a parameterized policy (possibly based on a point forecast) that is tuned using a stochastic simulator which captures all the forms of uncertainties.   There are two ways of performing a stochastic simulation:   1)   Development of a computer program to simulate the system (often called a “digital twin”). 2)   Evaluate the policy in the real world.   The use of a simulator to tune (optimize) a policy seems to be completely overlooked in the standard textbooks on inventory problems. It is easy to overlook that *any* policy will eventually be tested in the field, which is a form of simulation which is more realistic, but very slow.