Economic Modeling for Forecasting

Modeling something like time series goes past just throwing features in a model. In the world of time series data, each observation is associated with a specific time point, and part of our goal is to harness the power of temporal dependencies. Enter autoregression and lagging -  concepts that taps into the correlation between current and past observations to make forecasts.  At its core, autoregression involves modeling a time series as a function of its previous values. The current value relies on its historical counterparts. To dive a bit deeper, we use lagged values as features to predict the next data point. For instance, in a simple autoregressive model of order 1 (AR(1)), we predict the current value based on the previous value multiplied by a coefficient. The coefficient determines the impact of the past value on the present one only one time period previous. One popular approach that can be used in conjunction with autoregression is the ARIMA (AutoRegressive Integrated Moving Average) model. ARIMA is a powerful time series forecasting method that incorporates autoregression, differencing, and moving average components. It's particularly effective for data with trends and seasonality. ARIMA can be fine-tuned with parameters like the order of autoregression, differencing, and moving average to achieve accurate predictions. When I was building ARIMAs for econometric time series forecasting, in addition to autoregression where you're lagging the whole model, I was also taught to lag the individual economic variables. If I was building a model for energy consumption of residential homes, the number of housing permits each month would be a relevant variable. Although, if there’s a ton of housing permits given in January, you won’t see the actual effect of that until later when the houses are built and people are actually consuming energy! That variable needed to be lagged by several months. Another innovative strategy to enhance time series forecasting is the use of neural networks, particularly Recurrent Neural Networks (RNNs) or Long Short-Term Memory (LSTM) networks. RNNs and LSTMs are designed to handle sequential data like time series. They can learn complex patterns and long-term dependencies within the data, making them powerful tools for autoregressive forecasting. Neural networks are fed with past time steps as inputs to predict future values effectively. In addition to autoregression in neural networks, I also used lagging there too! When I built an hourly model to forecast electric energy consumption, I actually built 24 individual models, one for each hour, and each hour lagged on the previous one. The energy consumption and weather of the previous hour was very important in predicting what would happen in the next forecasting period. (this model was actually used for determining where they should shift electricity during peak load times). Happy forecasting!

Budget vs Forecast vs Financial Model vs Pro Forma vs Reforecast Most people use these terms interchangeably...but they're completely different tools. Each serves a specific purpose in your financial planning process...and knowing when to use which one can save you hours of confusion. Let me break this down for you. ➡️ THE 5 CORE FINANCIAL PLANNING TOOLS 🔹 BUDGET Your detailed financial plan for expected revenues and expenses over a specific period. Usually runs annually with monthly or quarterly breakdowns...and yes, it's that static document you revise periodically. Management teams and department heads use this for operational planning and expense management. 🔹 FORECAST A prediction of future financial performance based on current trends, market conditions, and business assumptions. This gets updated monthly, quarterly, or annually depending on your needs...it's dynamic and changes regularly. Executives, investors, and analysts rely on this for future performance prediction. 🔹 FINANCIAL MODEL A comprehensive mathematical representation of your company's financial performance. Built for multi-year projections and strategic decision support...think complex, interactive tool. Analysts, investment bankers, and CFOs use this for deep scenario analysis and decision making. 🔹 PRO FORMA Projected financial statements showing what results would look like under specific assumptions or after certain events. 🔹 REFORECAST An updated forecast that reflects new information, changed conditions, or actual performance data. This happens mid-period to provide course correction and updated guidance to management and board members...revised predictions when things change. ➡️ KEY DIFFERENTIATORS THAT MATTER The biggest differences come down to four areas... Flexibility ranges from rigid budgets to adaptive reforecasts that change as your business evolves. Update frequency varies widely. Budgets typically get refreshed annually...while forecasts and reforecasts happen monthly or quarterly as needed. Detail level depends on your needs. Budgets dive deep into operational details...while pro formas focus on specific scenarios or events. Primary use determines everything. Budgets control and plan, forecasts predict trends, models support strategic decisions...and so on. ➡️ THE FINANCIAL PLANNING PROCESS FLOW This follows a logical sequence... 1️⃣ Create Budget → Annual planning process 2️⃣ Generate Forecasts → Regular predictions 3️⃣ Reforecast → Mid-period updates 4️⃣ Build Models → Strategic analysis 5️⃣ Create Pro Formas → Scenario planning Each step builds on the previous one...creating a complete financial planning ecosystem. === The companies that get this right have better decision making, more accurate predictions, and faster responses to market changes. What financial planning tool do you rely on most? Share your experience in the comments below 👇

Machine learning beats traditional forecasting methods in multi series forecasting. In one of the latest M forecasting competitions, the aim was to advance what we know about time series forecasting methods and strategies. Competitors had to forecast 40k+ time series representing sales for the largest retail company in the world by revenue: Walmart. These are the main findings: ▶️ Performance of ML Methods: Machine learning (ML) models demonstrate superior accuracy compared to simple statistical methods. Hybrid approaches that combine ML techniques with statistical functionalities often yield effective results. Advanced ML methods, such as LightGBM and deep learning techniques, have shown significant forecasting potential. ▶️ Value of Combining Forecasts: Combining forecasts from various methods enhances accuracy. Even simple, equal-weighted combinations of models can outperform more complex approaches, reaffirming the effectiveness of ensemble strategies. ▶️ Cross-Learning Benefits: Utilizing cross-learning from correlated, hierarchical data improves forecasting accuracy. In short, one model to forecast thousands of time series. This approach allows for more efficient training and reduces computational costs, making it a valuable strategy. ▶️ Differences in Performance: Winning methods often outperform traditional benchmarks significantly. However, many teams may not surpass the performance of simpler methods, indicating that straightforward approaches can still be effective. Impact of External Adjustments: Incorporating external adjustments (ie, data based insight) can enhance forecast accuracy. ▶️ Importance of Cross-Validation Strategies: Effective cross-validation (CV) strategies are crucial for accurately assessing forecasting methods. Many teams fail to select the best forecasts due to inadequate CV methods. Utilizing extensive validation techniques can ensure robustness. ▶️ Role of Exogenous Variables: Including exogenous/explanatory variables significantly improves forecasting accuracy. Additional data such as promotions and price changes can lead to substantial improvements over models that rely solely on historical data. Overall, these findings emphasize the effectiveness of ML methods, the value of combining forecasts, and the importance of incorporating external factors and robust validation strategies in forecasting. If you haven’t already, try using machine learning models to forecast your future challenge 🙂 Read the article 👉 https://buff.ly/3O95gQp

Without his contribution, we wouldn't have Stochastic process. The Wiener process, named after mathematician Norbert Wiener, is a continuous-time stochastic process that exhibits the properties of Brownian motion. Brownian motion refers to the random motion of particles in a fluid due to their collisions with other fast-moving particles in the fluid. In mathematical terms, the Wiener process is defined as a continuous-time stochastic process W(t) with the following properties: 1) Independent Increments: For any 0≤t1<t2<…<tn, the random variables W(t2)−W(t1),W(t3)−W(t2),…,W(tn)−W(tn−1) are independent. 2)Stationary Increments: The distribution of W(t2)−W(t1) depends only on t2−t1 and not on the specific values of t1 and t2 3)Gaussian Increments: For any 0≤t1<t2, the random variable W(t2)−W(t1) follows a normal (Gaussian) distribution with mean 0 and variance t2−t1. 4)Continuous Paths: The paths of the Wiener process are continuous, meaning that W(t) is almost surely continuous for all t≥0. Mathematics Behind Wiener Process: The Wiener process is often represented by the differential equation: dW(t)= √dtZ, where, Z is a standard normal random variable. This equation implies that the increments(t) are normally distributed with mean 0 and variance dt. Contribution to Quantitative Finance: The Wiener process has played a crucial role in quantitative finance, particularly in the modeling of asset prices and the development of option pricing models. Here are some key contributions: Geometric Brownian Motion (GBM): The Wiener process is a key component in the formulation of the Geometric Brownian Motion, which is widely used to model the continuous-time evolution of asset prices. The GBM model is a fundamental building block for various financial derivatives and risk management tools. Option Pricing Models: The Black-Scholes-Merton option pricing model, a cornerstone in option pricing theory, relies on the assumption of geometric Brownian motion. This model, developed by Robert C. Merton and Myron Scholes, uses the Wiener process to describe the stochastic process of underlying asset prices. Risk Management: The randomness and continuous paths of the Wiener process provide a realistic representation of the unpredictable nature of financial markets. This makes it a valuable tool for risk management, helping financial professionals model and assess various aspects of market risk. Stochastic Calculus: The Wiener process is a fundamental element in stochastic calculus, a branch of mathematics widely used in quantitative finance. It enables the development of sophisticated mathematical tools to analyze and model financial markets, allowing for more accurate pricing and risk management strategies.

🚀 Just Released: ML-Enhanced Stock Prediction using Ito's Lemma! 📈 I'm excited to share my latest project that bridges the gap between traditional financial mathematics and cutting-edge machine learning! 🔬 What makes this special? ✅ Combines Ito's Lemma (stochastic calculus) with LSTM neural networks ✅ Multi-head attention mechanism for complex temporal pattern recognition ✅ Predicts volatility and drift parameters dynamically using ML ✅ 20+ technical indicators including RSI, MACD, Bollinger Bands ✅ Monte Carlo simulations with confidence intervals ✅ Real-time 6-month forecasting with uncertainty quantification 📊 Key Results: 68.3% profit probability (ML) vs 61.2% (traditional) 2.15% reduction in prediction uncertainty Dynamic parameter estimation that adapts to market conditions 🛠 Tech Stack: PyTorch for deep learning architecture LSTM + Attention for sequence modeling Geometric Brownian Motion enhanced with ML predictions yfinance for real-time market data This hybrid approach demonstrates how mathematical finance and AI can work together to create more robust prediction models. The model doesn't just predict prices—it learns market dynamics and adjusts volatility/drift parameters in real-time. 🔗 Open Source: Available on GitHub with comprehensive documentation and examples! Disclaimer: This is for educational/research purposes only. Always consult financial professionals for investment decisions. #MachineLearning #QuantitativeFinance #StochasticCalculus #DeepLearning #PyTorch #FinTech #DataScience #LSTM #StockPrediction #OpenSource What do you think about combining traditional financial mathematics with modern ML techniques? Would love to hear your thoughts! 💭

Most small businesses default to two forecasting methods: top-down or bottom-up. But they both share the same problem. The "why" behind performance isn't explained. These approaches are easy to model and are used all the time. But they can easily fail as companies grow larger and more driver based. (1) Top-down forecasting Many companies favor top-down because it's simple and aligned with strategic goals. But the biggest drawback is it's often completely disconnected from an operational reality. I use it for high-level financial forecasting and hardly ever for operational planning. • Leadership sets growth or margin targets • The P&L is segmented into business units • These targets cascade down the statements • Line-items are forecast on high-level assumptions (2) Bottom-up forecasting Bottom-up forecasting is based upon detailed inputs such as sales to customers, sales by SKU, hiring plans by individual versus job category or department, expense budgets, etc. The benefit of bottoms-up is it's detailed and grounded in operations. But it's usually time-consuming, fragmented, and hard to roll up consistently. • Individual contributors come up with their numbers • They share it with an accountant or financial analyst • The accounting/finance person puts it into a model • The model is updated constantly with new details (3) Driver-based forecasting Rather than come up with high-level assumptions that don't tie into operations, or granular detail that doesn't separate signal from noise, driver-based combines the best of both. In this example for a professional staffing company, we can tie future revenue to placements per recruiter, contract duration, markup percentage, bill rates, and recruiter headcount. This allows FP&A the ability to flex operating assumptions, test them, and quickly see what can be done on the ground to influence. Differences between the 3 methods matter: Top-down may set revenue at $50 million based upon an 8% growth rate. We can ask "how do we increase growth?" Bottoms-up may set revenue at $50 million based upon a monthly forecast of 200 customers. We can ask "what do we expect from each customer?" Driver-based planning may arrive at the same $50 million but ask "what operational levers can we press to truly move revenue and margin?" The result is forecasts that are faster, more explainable and easier to update. 💡 If you want to explore next-level modeling techniques, join live with 200+ people for Advanced FP&A: Financial Modeling with Dynamic Excel Session 2. https://lnkd.in/emi2xFdZ

Here are 5 machine learning algorithms used for FP&A and #finance time series analysis: ✅ ARIMA/SARIMA: Forecast future revenues and expenses by identifying trends and seasonality. ✅ LSTM: Analyze complex patterns in cash flow or sales data to improve financial planning. ✅ Prophet: Handle unpredictable markets and still make reliable forecasts. ✅ GARCH: Assess and predict market volatility to make more informed investment or budgeting decisions. More detail below ↓ 1. ARIMA (Auto-Regressive Integrated Moving Average) ARIMA helps predict future values by analyzing past data to identify patterns like trends or seasonality. For example, you can use ARIMA to forecast next year’s monthly revenue by recognizing historical trends and seasonal variations, such as higher sales during holiday seasons. 2. LSTM (Long Short-Term Memory) Networks LSTM is an artificial intelligence technique that learns from past data and remembers long-term patterns. It can be used in FP&A to forecast cash flow by identifying recurring inflows and outflows over time, like specific project payments or seasonal cash patterns. 3. SARIMA (Seasonal ARIMA) SARIMA extends ARIMA by incorporating seasonality, making it ideal for forecasting data with regular patterns. For example, you can predict quarterly expenses more accurately if certain quarters have consistently higher costs due to contracts or seasonal demand. 4. Prophet Prophet, developed by Facebook, handles missing data and outliers well, making it useful for complex datasets. To get the code and example for implement it, go here: https://lnkd.in/eJKcHzqU You could use Prophet to forecast annual sales even when your data is incomplete or affected by irregular events like economic shifts. 5. GARCH (Generalized Autoregressive Conditional Heteroskedasticity) GARCH models volatility and is great for predicting how much financial data varies over time. You can apply it in FP&A to assess and predict the volatility of stock prices in your investment portfolio, helping in risk management and budgeting.

It's common to see cumulative revenue curves for freemium digital products fitted across groups of users to singular point estimates at various cohort days, eg., Day 90. This can be helpful for guidance, but there are a few drawbacks to approaching the aggregation in this way: - it masks variation in payer behavior and the vast differences in values across the two classes of users; - the singular point estimate is misleading and may inspire too much confidence for certain use cases (eg., setting advertising bids). Another way of approaching the analysis is to segment users by cohort in some defined time window, truncate their individual cumulative revenue curves so that they're all of the same length, and to treat each individual curve as a time series that a curve can be fitted to. Those fitted individual cumulative revenue curves can then be projected and bootstrap sampled, and confidence intervals can be constructed on the means. I published the first in a multi-part series about this concept this week. In the next part, I'll explore using fixed effects to control for hierarchical variation across cohorts or acquisition sources.

Predicting #financialmarket stress has long proven to be a largely elusive goal. Advances in artificial intelligence and #machinelearning offer new possibilities to tackle this problem, given their ability to handle large datasets and unearth hidden nonlinear patterns. In the BIS paper , the authors have developed a new approach based on a combination of a recurrent neural network (RNN) and a large language model. Focusing on deviations from triangular arbitrage parity (TAP) in the Euro-Yen currency pair, our RNN produces interpretable daily forecasts of market dysfunction 60 business days ahead. To address the “black box” limitations of RNNs, our model assigns data-driven, time-varying weights to the input variables, making its decision process transparent. These weights serve a dual purpose. First, their evolution in and of itself provides early signals of latent changes in market dynamics. Second, when the network forecasts a higher probability of market dysfunction, these variable-specific weights help identify relevant market variables that we use to prompt an LLM to search for relevant information about potential market stress drivers.  - Source Bank for International Settlements – BIS

Honey, I Shrunk the Sample Covariance Matrix - Research Paper "Honey, I Shrunk the Sample Covariance Matrix" by Olivier Ledoit and Michael Wolf addresses a fundamental issue in portfolio optimization: the instability of the sample covariance matrix when the number of assets is large relative to the number of observations. This instability can lead to poor portfolio performance, as the sample covariance matrix tends to overfit the data. Key Points 1. Problem with Sample Covariance Matrix: When the number of assets (p) approaches the number of observations (n), the sample covariance matrix becomes unreliable. This is because it tends to capture noise rather than the true underlying relationships between assets. The problem worsens as the ratio of p/n increases, making it harder to estimate the covariance matrix accurately. 2. Shrinkage Estimator: The authors propose a "shrinkage" method to improve the estimation of the covariance matrix. The idea is to combine the sample covariance matrix with a well-structured target matrix. By introducing a shrinkage factor, the estimator is a weighted average of the sample covariance matrix and the target matrix. The shrinkage reduces the impact of sampling noise while retaining essential information about asset relationships. 3. Optimal Shrinkage: The authors derive an optimal shrinkage coefficient that balances bias and variance. This is done using a rigorous statistical framework, minimizing the mean-squared error of the estimator. 4. Benefits: The shrinkage estimator improves out-of-sample performance in portfolio optimization by providing more stable and reliable covariance matrix estimates. It helps prevent the overfitting problem associated with using the raw sample covariance matrix, leading to better risk-adjusted returns. 5. Applications: This approach is widely applicable in portfolio construction, and optimization. It is particularly valuable in high-dimensional settings where the number of assets exceeds or is close to the number of observations. In essence, the paper offers a practical and theoretically sound solution to the problem of noisy covariance matrix estimates in portfolio optimization by "shrinking" the sample covariance matrix toward a more stable and robust estimator. I've attached a comprehensive research paper. I highly recommend reading it for anyone interested in portfolio optimization. #covariance #portfolio #optmization #shrinkage