Project Name
0.86 Pooled Geo R² Achieved for a Multi-Retailer Enterprise With Hierarchical Bayesian Marketing Mix Modeling
A multi-retailer enterprise managing marketing investment across national and retailer-specific channels, including retail media networks, social, search, video, and offline. Relied on a single national Marketing Mix Model that averaged away meaningful differences in media performance and baseline demand across geography, leaving budget decisions blind to local variation. Historical planning optimized only at the retailer-national level, so there was no operational path to reallocate spend toward the markets where incremental returns were strongest, and stakeholders routinely confused raw sales efficiency with true incremental ROI. Ksolves designed and validated a hierarchical geo-level Bayesian MMM, producing geo-week and national diagnostics, geo-scoped contribution and ROI views, and two production-ready budget optimization modes, while building the validation framework needed for executive trust in the outputs.
- National Models Masked Local Performance: A single national series could not explain why some markets responded strongly to media while others showed weak or unstable fit, forcing one-size-fits-all allocation.
- Uneven Geo Signal and Model Fit: Individual time-series R² varied widely across tens of geos — from roughly 0.65–0.70 in strong markets to far lower elsewhere — even as pooled geo-level accuracy stayed high, with no clear framework for interpreting the difference.
- Budget Optimization Was Retailer-National Only: Historical planning optimized channel mix for the whole footprint, with no path to optimize per geo or for a selected set of priority geos under shared budgets and date windows.
- Incremental ROI Confused With Total Sales Efficiency: Stakeholders compared raw revenue-to-spend ratios (6–7x in validation) against model ROI (1.1 for media-driven incremental revenue), undermining trust until the distinction was documented.
- Single-KPI, Single-Grain Pipeline: Training and artifact storage assumed one model per retailer, so supporting multiple KPIs and a future geo_mode would have required duplicating Lambdas and Step Functions graphs.
- Data Constraints for Geo Panels: Meridian requires a balanced time index across geos; staggered start dates failed at data build, requiring clear standards for panel construction, population scaling, and geo quality filters.
Ksolves treated this as a measurement architecture problem rather than a modeling tweak, building geo hierarchical models that share national structure while allowing geo-specific baselines and media response, paired with a validation framework designed to build executive trust in the outputs.
- Hierarchical Geo-Level Bayesian MMM: Built geo models sharing national time baseline, adstock, and saturation structure, scaled by population so large and small markets remain comparable.
- Dual-Granularity Diagnostics: Implemented predictive accuracy reporting at geo-week, national roll-up, and per-geo levels, documenting pooled vs. individual-geo R² as expected hierarchical behavior rather than a defect.
- Geo-Scoped Insight Suite: Extended Analyzer APIs beyond national defaults to produce geo-filtered spend-vs-contribution, ROI, response curve, and summary views.
- Two-Mode Budget Optimization: Operationalized Meridian's BudgetOptimizer for per-geo optimization and shared-window, multi-geo group optimization under fixed budgets and ±30% channel constraints.
- Validation Framework for Trust: Established that model budget should approximate historical paid spend while model ROI should never be conflated with total revenue-to-spend, using incremental media revenue as the correct numerator.
- Config-Driven Multi-KPI Training Design: Rebuilt the AWS pipeline around a (retailer, KPI, geo_mode) unit of work with a flattened Step Functions map, KPI-scoped S3 artifacts, and DynamoDB keys, enabling new KPIs via configuration alone.
Technology Stack
| Category | Technology |
|---|---|
| Marketing Mix Modeling | Hierarchical Bayesian Geo MMM (Meridian) |
| Statistical Modeling | Bayesian inference, adstock, Hill saturation, geo random effects |
| Programming | Python |
| Cloud / Orchestration | AWS (S3, DynamoDB, Lambda, Step Functions) |
| Analytics | Geo diagnostics, contribution & response analysis, budget optimization |
| Data | Weekly geo × channel panels (spend, impressions, KPI, population, controls) |
- Geo coverage: Hierarchical models were evaluated across up to ~40 geos, with focused experiments on 5-geo subsets for rapid iteration.
- Pooled accuracy: Geo-week R² ranged ~0.77–0.86, with national roll-up R² and MAPE reported separately to preserve planning trust.
- Per-geo transparency: Individual geo R² was ranked (example range ~0.25–0.69) to guide which markets support local optimization versus national reliance.
- Channel system: Multi-channel paid media, organic, and controls/non-media treatments were unified in a single Meridian specification.
- Optimization modes: Two production-ready patterns, per-geo and multi-geo (selected_geos) fixed-budget optimization, were delivered with explicit constraints.
- Insight automation: Repeatable geo loops for contribution, ROI, response curves, and summary tables replaced manual notebook one-offs.
- Decision clarity: The gap between raw sales efficiency and incremental media ROI was documented (validation example: ~6.8x raw vs ~1.1 model ROI).
- Platform readiness: The training pipeline design supports multi-KPI modeling per retailer and geo_mode without duplicating orchestration.
A multi-retailer enterprise whose national-only Marketing Mix Model averaged away local performance differences and offered no path to geo-level budget optimization was transformed through Ksolves’ marketing analytics services. A hierarchical geo-level Bayesian MMM with dual-granularity diagnostics, a geo-scoped insight suite, and two-mode budget optimization now gives planners local ROI visibility alongside national roll-ups. Pooled geo-level R² reached 0.86. Geo coverage scaled to 40 markets. Incremental ROI was clarified at 1.1x against a previously conflated 6.8x raw efficiency figure. A config-driven, multi-KPI training pipeline on AWS now supports additional retailers, KPIs, and geo expansion without redesigning the orchestration layer.
Ready to See Where Your Marketing Spend Is Actually Working, Market by Market?
