I’ve built 20+ Marketing Mix Models, and it’s always the same thing that blows the model’s credibility with decision-makers. The assumption that marketing performance doesn’t change! Nobody believes all their hard work was for nothing. Uber’s ‘Orbit’ model solves that problem.
Marketing Mix Modeling commonly uses OLS Linear Regression, which assumes the coefficient for marketing spend doesn’t change over the time period...
I just don’t get it
Is everyone that’s using MMM making this assumption?
Of course marketing performance changes over time! Otherwise we could all just retire early instead of working on optimizing the accounts…
Let’s search for a way to let marketing performance flex up and down
If we can’t do that, I’m not sure I can condone using MMM, it just won’t be something I can get behind. What if one of our investors makes the same ‘discovery’?
This is top priority
This course is a work of fiction. Unless otherwise indicated, all the names, characters, businesses, data, places, events and incidents in this course are either the product of the author's imagination or used in a fictitious manner. Any resemblance to actual persons, living or dead, or actual events is purely coincidental.
One of the big assumptions everyone is making when doing marketing mix modeling, is that marketing performance doesn’t change over time. Most aren’t even aware that’s the assumption they’re making. Of course MMM usually accounts for changing in spend over time with diminishing returns and adstocks, but the baseline coefficients don’t change in most models. If you shorten the date range or update the model, you tend to find that the coefficients vary significantly from time period to time period. This is because now the model doesn’t “know” about those previous time periods and can’t remain consistent with them. For most models making the assumption that marketing performance doesn’t change very much isn’t as bad as it seems. For simple models and small businesses, that assumption largely holds. It’s the red queen effect: we do constantly optimize our accounts, but so do our competitors, so the net result is usually pretty stable performance over time.
However this hidden assumption is a ticking timebomb. It sounds so obviously wrong, that when someone non-technical like a CEO discovers this insight, it can ruin the credibility of marketing mix modeling in general as a method. The solution to this problem is an advanced, but growing technique, called “Bayesian Time-Varying Coefficients” (BTVC): used by companies such as Recast and Uber. One unique benefit of using Bayesian methods, is that you can use priors to allow marketing coefficients to change over time in a stable way. Today’s performance is likely to be like yesterday’s, but if the model has strong evidence today is different, it’ll be allowed to evolve away from yesterday’s results. The end result of this model is that coefficients for each variable change over time, letting marketing performance increase or decrease with seasonality, optimization and competition.
Complete all of the exercises first to receive your certificate!
Share This Course