Regression Analysis in Retail

Price adjustments are one of the most important marketing levers for boosting sales. The key challenge lies in measuring consumers' price sensitivity accurately. How different are the results when using experiments versus MMM-style analyses? A new study sheds light on this critical question, comparing price elasticities for a US grocery retailer using these different methods: 📉 Non-experimental scanner price data (which we also call observational data), analyzed using OLS regressions. This approach is common in Marketing Mix Models (MMM) and is the most widely used method for obtaining pricing insights. 🔬 Experimental random price manipulations. The findings reveal significant differences in price elasticities (accounting for temporary price promotions) across nine product categories: 📉Standard OLS: -1.08 📊 OLS with control functions (inverse instrument): -0.92 * 🔬Experiment-based (2SLS): -0.32 Summary: While OLS-based analyses suggest values close to unit elasticity (around -1), the experimental findings imply that many products likely exhibit inelastic demand. In other words, demand does not change significantly when prices rise. Key takeaways: 🔍 These results highlight how potentially misleading non-experimental analyses, including traditional MMM, can be. Even typical econometric adjustment tricks (e.g., instruments/control functions) may not sufficiently adjust price elasticity estimates. 🛡️Having said this, given the bias found in the study, one could also argue that MMM-based analyses are rather conservative in many cases (at least when it comes to suggesting price hikes). But we should be careful when considering price reductions based on MMM results. 💰 The good news for marketers is that the study findings suggest many brands may have stronger pricing power than previously thought. Reminder: Products with a price elasticity smaller than 1 (in absolute value) may have room to raise prices and boost revenues. Caveats: ⚠️ The study focused on nine product categories (409 products), 35 weeks, and 82 stores before COVID. Prices have shifted significantly since then. Price elasticities vary by brand, product, time period, and region. Thus, we need more replication and tests to understand when and why consumer price sensitivities differ. Put simply, we need more brave brands willing to experiment—even with prices 💪. The original study, which includes many different analyses and robustness checks, is a masterclass in price elasticity analysis (warning: it's a highly technical read) and can be found here: https://lnkd.in/gE5Y3yxZ Technical notes: * I could not find the exact number in the text, so this is an approximate average derived from Figure 12 of Bray, Sanders, and Stamatopoulos 2024.

I have spent nearly a decade as a data scientist in the retail sector, but I have been approaching customer spend predictions the wrong way until I attended Gregory M. Duncan’s lecture. Accurately predicting how much an individual customer will spend in the next X days enables key retail use cases such as personalized promotion (determine X in Buy-X-Get-Y), customer targeting for upselling (which customers have higher purchasing power), and early churn detection (customers do not spend as much as they should). What makes this problem particularly difficult is because the distribution of customer spending is both zero-inflated and long/fat-tailed. Intuitively, most customers who visit your store are not going to make a purchase and among those who do, there will be some super customers who purchase an outrageous amount more than the average customer. Some parametric models allow for zero-inflated outcomes such as Poisson, negative binomial, Conway-Maxwell-Poisson; however, they do not handle the long/fat-tailed explicitly. Even for non-parametric models such as decision tree ensembles, more resources (trees and splits) will be dedicated to separating zeros and handling outliers; this could lead to deterioration in performance. Using the real-world dataset UCI Online Retail, we will compare the performance of common approaches namely naive baseline regression, regression on winsorized outcome, regression on log-plus-one-transformed outcome to what Duncan suggested: hurdle model with Duan’s method. We will demonstrate why this approach outperforms the others in most evaluation metrics and why it might not in some. See more: https://lnkd.in/gJwyhvAW

Visiting the racecourses of Mumbai across years, you learn that there is Maths to winning ! And so also in #CX , knowing the principles of regression, a statistical modelling technique of Maths, helps. In the multi faceted landscape of customer satisfaction, identifying the “material“ key drivers of dissatisfaction is an imperative task. Imagine a scenario where there are ten contributing factors causing dissatisfaction— enter regression analysis, a compass that can guide through this complex terrain in prioritising. Regression analysis allows organizations to discern the weightages of importance among these contributing factors. Let's consider a case where a service-oriented business faces challenges in customer satisfaction. Regular analysis unveils that responsiveness, product quality, and pricing significantly contribute to dissatisfaction. What sets regression analysis apart is its ability to move beyond a mere enumeration of these factors; it assigns importance to each, offering a nuanced understanding of their impact. For instance, a retail giant grappling with various elements affecting customer satisfaction—inventory management, checkout efficiency, and staff courtesy—can leverage regression analysis to unveil which factors weigh more heavily on dissatisfaction. This analytical powerhouse allows organizations to prioritize and address the factors with the most significant impact, providing a roadmap for strategic interventions. It's akin to shining a spotlight on the critical elements in a sea of variables, ensuring that efforts are directed where they matter most. In a market where customer preferences and expectations evolve swiftly, the ability to adapt is paramount. Regression analysis empowers businesses to stay agile. By understanding the importance of each dissatisfying factor, organizations can pivot quickly to address the most impactful issues. In conclusion, regression analysis is not just a statistical tool but a strategic ally in the pursuit of profitable business and CX. Read my weekend #CXNotes for insights on CX in various perspectives. Pic : At the Mumbai racecourse, and of course winning 🙂