Time Series Analysis in Ecommerce

The highest ROI marketing message isn't a clever campaign. It's simply recommending the right product at the right time. Most ecommerce brands rely on basic "customers also bought" algorithms that miss the crucial element: timing. We recently analyzed purchasing patterns for a home goods brand and discovered: - Customers who bought sheets had an 82% likelihood of buying pillowcases...but not for 27 days - Customers who bought dining tables looked for chairs within 72 hours - Customers who bought rugs didn't look for furniture until 45+ days later By aligning their cross-sell flows with these natural buying windows, they saw: - 47% higher click rates - 28% higher conversion rates - 18% increase in average order value The key wasn't better creative, copy, or subject lines. It was simply respecting the natural progression of the customer's buying journey. Timing isn't just a factor in cross-selling. It's THE factor. Is your cross-sell strategy based on data or assumptions? #Ecommerce #MarketingAnalytics

Data can’t speak for itself. Here are the 3 pitfalls of that belief — and what you should do instead: 1. Manipulation Everyone has seen examples of manipulation as a result of “letting data speak for itself,” in both personal and professional settings. - This state is hotter than all other states in the country this summer (but Southern states are almost always hotter than non-Southern states) - We should invest more money in this campaign because it’s ROI is 300% (but we can’t — it’s a yearly conference) These things are all technically facts, but they don’t give you the full story, which can lead to misguided conclusions and manipulation. 2. Thinking you need more and more data When data “speaks for itself,” you’re incentivized to get more data. Before you know it, you’re investing huge portions of your budget on measuring whether or not someone raising their eyebrow in a sales call makes them less likely to buy your product. Instead, I recommend going back to the basics — taking a look at your existing data and seeing what you can do with it. It’s the basic questions that are the most impactful. How did our campaigns change over time? Are sales cycles shorter? Are they more efficient? These are the questions that have an actual impact on business outcomes, and these are what you should be focusing your attention on. 3. Asking the wrong questions When you think data can speak for itself, you ask the wrong questions. Data tells you what happened — but without interpretation, it doesn’t tell you why. For example, if you had 55 million folks in the pipeline last week and 57.5 million this week, your delta is 2.5 million. That’s what your data tells you. But that’s not useful for business strategy. Your Head of Sales is going to want to know how that delta was formed. That requires interpretation. Now it’s time to geek out over perhaps my favorite method of data interpretation: time series analysis. When you analyze data in a time series, you are linking your data to timestamps so you can compare different snapshots of time. For example, when you open up your CRM, you can see all your selected columns and rows for July 1, 2024. Then, the next day, you’ll see the entire report for July 2. Wow! Deals exploded on July 1 — why was that? Traffic to our blog dipped intensely — was there an algorithm update? We got 35% more signups on July 2 than on July 1 — was this because of the webinar we hosted? Time series data helps you track changes over time, which is one of the most powerful ways to identify what has an impact on your business outcomes. And, despite its impact, most tools don’t support this kind of analysis — though one in particular comes to mind (😉). Tl;dr: Don’t trust your spreadsheets to tell you the full story. You’ve got to interpret your data before you can use it.

Machine Learning Interview Question with Solution for a Walmart Data Scientist Role Question: Walmart collects data from various sources like sales, inventory, and customer behavior. One of the main goals is to predict product demand to optimize inventory levels. Suppose you are provided with historical sales data, including features like Date, Store_ID, Product_ID, Sales_Quantity, and Promotion. How would you build a machine learning model to forecast sales for the next 30 days? Solution: To solve this problem, we can follow these steps:    ⏩ Understand the Problem: The goal is to predict Sales_Quantity for the next 30 days, which makes this a time series forecasting problem.    ⏩ Data Preprocessing:        - Handling Missing Values: If there are any missing values, we need to fill them appropriately (e.g., using median or forward fill for missing sales quantities).        - Feature Engineering: Create additional features such as:             - Lag features (previous sales quantities)             - Rolling averages (7-day, 30-day)             - Holidays (since Walmart sales may spike during holidays)             - Days of the week (sales patterns may differ between weekdays and weekends)    ⏩ Train-Test Split:        Split the data into a train set (e.g., sales before the last month) and a test set (last month of sales).    ⏩ Model Selection: Some of the models we can consider are:         - Random Forest Regressor: Can handle non-linear relationships and provide feature importance.         - XGBoost or LightGBM: These are tree-based gradient boosting models that work well for structured data like sales forecasting.         - ARIMA (AutoRegressive Integrated Moving Average): A classic time-series forecasting model.    ⏩ Training the Model:        - Use historical data to train the model on sales quantity (Sales_Quantity) as the target variable.        - Ensure to include relevant features like promotions, store IDs, and lagged sales as inputs.    ⏩ Evaluation Metrics:        Use Mean Absolute Error (MAE) or Root Mean Squared Error (RMSE) to evaluate the model's performance on the test set.    ⏩ Hyperparameter Tuning:        Perform Grid Search or Random Search for hyperparameter tuning to optimize model performance.    ⏩ Deployment:        Once the model is trained and evaluated, deploy it to forecast future sales and adjust inventory levels accordingly. Can you think of any alternate solution? Please share in comments! #dsinterviewpreparation #ml #walmartinterview

How to Build a Killer Time Series Forecasting Project for Your Data Science Portfolio If you want to showcase serious machine learning skills in your portfolio, a time series forecasting project is a great way to stand out. It touches multiple advanced areas, from feature engineering to model tuning and error analysis. A perfect playground for this? The M5 Forecasting Accuracy Competition on Kaggle https://lnkd.in/gggecJrQ Let’s break down how to approach it like a pro 👇 Step 1: Choosing and Understanding the Dataset Start with exploratory data analysis (EDA). In the M5 dataset, you’ll find daily sales for thousands of products across multiple stores and categories. Ask - Are there seasonal or holiday trends? How does demand vary across states or categories? Are there anomalies or missing data? Then, do feature engineering - Create calendar features (day of week, month, holiday flags), Add lag features and rolling means to capture trends, Aggregate demand at different levels (store, department, product) Good features often beat fancy models. Step 2: Model Selection and Evaluation You can go two routes - (a) Classical models: ARIMA, SARIMAX, ETS (b)ML/DL models: LightGBM, XGBoost, Prophet, LSTM, Temporal Fusion Transformer. Pick the model based on your data scale and interpretability needs. Then define the right metrics - RMSE or MAE for general accuracy, WRMSSE (Weighted Root Mean Squared Scaled Error) - used in M5 for multi-level forecasting. Choosing the right metric shows you understand the business impact of forecasting. Step 3: Hyperparameter Tuning and Optimization Use GridSearchCV or Optuna for automated hyperparameter tuning. Test how different lags, window sizes, or regularization terms affect performance Step 4: Error Analysis and Model Improvement Don’t stop at scores - investigate why your model fails. Which products or stores are consistently over/under-forecasted? Are errors higher during holidays or promotions? Does the model react poorly to new product launches? Use that insight to - Add better exogenous features (promotions, prices, events), Segment models by category or region, Ensemble multiple models for stability Final Step: Tell the Story Your notebook should read like a case study, not just code. Explain - the business context (“Why forecasting matters”), the modeling journey (“What you tried, what worked, what didn’t”), the insights (“What patterns the model uncovered”) That’s what turns a project into a portfolio highlight that recruiters remember. The M5 dataset is challenging - but that’s the point. If you can handle it, you demonstrate real-world readiness in forecasting, feature engineering, and model optimization - skills every company values. --- 🚶➡️ To land your next Data Science role, follow me - Karun! ♻️ Share so others can learn, and you can build your LinkedIn presence!

In this era of LLMs and generative AI, I enjoy reading about how people are using "classical" machine learning to solve problems and improve existing processes. In fields such as retail, how do you incorporate relatively rare events like holidays into your time series forecasting? As the blog post below -- written by Cagri (Chad) A. and Zainab Danish from the engineering team at DoorDash -- describes, you have a small fraction of the days of the year that are holidays, and each holiday can show very different behavior. So, if you are training, say, a tree-based model, if you have a feature that says "is today a holiday?", that is not quite enough to get accurate results. DoorDash is a food delivery company, and they could see in their historical data that demand drops off significantly more on Thanksgiving, then say on July 4th, compared to non-holiday days. But, Thanksgiving only happens once a year, and how many years of training data do we have to train our model on? And, the change in demand on holidays can also affect the model's predictions in the days after. The team at DoorDash therefore took a cascade modeling approach. They train a gradient boosting machine (GBM) on a time series where the holiday effect has been removed. They do so by first training linear regression models on the holiday data for each holiday in various locales across the country, where the target variable is the week-over-week change in orders on those holidays. This gives the effect of each holiday, which is used to get a time series with holiday effects removed. Then, when making predictions, the holiday effect from the linear regression models can be applied to the results from the GBM when a holiday is happening. Their results showed a decrease in the weighted mean percentage error (wMAPE) from 60-70% down to 10-20% around Christmas. Another challenge is that if one wants to run an A/B test with a new model in a field like this, one would have to wait an entire year to cover each unique holiday. This team instead did A/B testing on just a couple holidays in a span of a month or so, combined with backtesting on historical data. One last point is that the intuitiveness of this approach, training one model for "regular" days, and a separate one for holidays, makes it easier to convey to stakeholders and get their buy-in. Check out their blog post for more details and some plots. Have any of you dealt with a time series problem like this, that contains some relatively "rare" events? How did you approach it? https://lnkd.in/ejbKwiCy