Executive Summary
The low volume of data available for individual retail products strongly encourages the adoption of hierarchical demand modeling, because such models provide the statistical power necessary to estimate demand parameters reliably. This challenge is well-recognized in both academic and applied literature on markdown optimization and demand modeling.
1. The Problem of Low-Volume, Low-Signal Data
When trying to estimate demand for a specific item at a given store or customer segment, the available data is often sparse and noisy. Several compounding factors make the estimation of parameters such as price elasticity or promotion lift particularly unreliable:
- Low sales volume — many SKUs sell only a few units per week, yielding limited information.
- Low signal-to-noise ratio — random variation in sales often dominates the true price–demand signal.
- Infrequent or small price changes — if prices are rarely changed or only within a narrow range, it becomes statistically impossible to detect meaningful elasticity.
- Insufficient statistical power — the number of observations is too small relative to the number of parameters being estimated.
- Unstable parameter estimates — item-level regressions or models tend to produce highly variable or implausible estimates, especially for elasticity coefficients.
These limitations make it difficult to fit reliable demand functions for each item individually — even with modern machine learning methods.
2. The Solution: Hierarchical Modeling and Partial Pooling
A hierarchical (multi-level) model addresses these problems by introducing a structure that pools information across related items, categories, or stores. This "borrowing of strength" stabilizes parameter estimates while preserving item-specific variation.
Hierarchical Structure in Retail Data
Retail data naturally forms multiple hierarchies:
- Merchandise hierarchy: Division → Department → Category → Class → Item
- Location hierarchy: Region → District → Store
These hierarchies provide the foundation for statistical pooling.
How Pooling Improves Reliability
By combining information across similar entities, hierarchical models achieve greater precision:
- Shared parameters: Coefficients such as price elasticity, promotion lift, or seasonality can be estimated at higher levels (e.g., class or category), where data volume is sufficient.
- Item-specific intercepts: Parameters like baseline demand () remain specific to each item, capturing its unique sales level.
- Empirical Bayes shrinkage: Item-level parameters are "shrunk" toward their group means, balancing bias and variance for optimal predictive accuracy.
- Cross-location robustness: The same item sold at multiple locations contributes to joint parameter estimation, improving generalization and reducing overfitting.
Example
For instance, suppose we model log-demand as a function of price and promotions:
Here:
- — base demand, item-specific
- — price elasticity shared at the category level
- — promotion lift shared at the class or department level
This structure ensures that elasticity and promotion effects are estimated where the data is dense enough to be statistically meaningful.
3. Impact on Optimization and Decision-Making
By leveraging hierarchical estimation:
- Optimization inputs (e.g., elasticities, cross-effects) become more reliable and less noisy.
- Downstream pricing or inventory policies become more stable and interpretable.
- Scalability improves, allowing consistent estimation across thousands of SKUs and stores.
In short, hierarchical modeling provides the bridge between sparse item-level observations and robust, actionable optimization inputs for markdown and pricing systems.
Solutions
About the author

Cyril Noirot
Lead Data Scientist
Freelance data scientist. I design and ship decision systems — forecasting, pricing, marketing measurement, optimization.