The low volume of data available for individual retail products strongly encourages the adoption of hierarchical demand modeling, because such models provide the statistical power necessary to estimate demand parameters reliably. This challenge is well-recognized in both academic and applied literature on markdown optimization and demand modeling.
When trying to estimate demand for a specific item at a given store or customer segment, the available data is often sparse and noisy. Several compounding factors make the estimation of parameters such as price elasticity or promotion lift particularly unreliable:
- Low sales volume — many SKUs sell only a few units per week, yielding limited information. - Low signal-to-noise ratio — random variation in sales often dominates the true price–demand signal. - Infrequent or small price changes — if prices are rarely changed or only within a narrow range, it becomes statistically impossible to detect meaningful elasticity. - Insufficient statistical power — the number of observations is too small relative to the number of parameters being estimated. - Unstable parameter estimates — item-level regressions or models tend to produce highly variable or implausible estimates, especially for elasticity coefficients.
These limitations make it difficult to fit reliable demand functions for each item individually — even with modern machine learning methods.
About the author
Cyril Noirot
Lead Data Scientist
Freelance data scientist. I design and ship decision systems — forecasting, pricing, marketing measurement, optimization.