Can we try Uber Orbit?

One of the big assumptions everyone is making when doing marketing mix modeling, is that marketing performance doesn’t change over time. Most aren’t even aware that’s the assumption they’re making. Of course MMM usually accounts for changing in spend over time with diminishing returns and adstocks, but the baseline coefficients don’t change in most models. If you shorten the date range or update the model, you tend to find that the coefficients vary significantly from time period to time period. This is because now the model doesn’t “know” about those previous time periods and can’t remain consistent with them. For most models making the assumption that marketing performance doesn’t change very much isn’t as bad as it seems. For simple models and small businesses, that assumption largely holds. It’s the red queen effect: we do constantly optimize our accounts, but so do our competitors, so the net result is usually pretty stable performance over time.

However this hidden assumption is a ticking timebomb. It sounds so obviously wrong, that when someone non-technical like a CEO discovers this insight, it can ruin the credibility of marketing mix modeling in general as a method. The solution to this problem is an advanced, but growing technique, called “Bayesian Time-Varying Coefficients” (BTVC): used by companies such as Recast and Uber. One unique benefit of using Bayesian methods, is that you can use priors to allow marketing coefficients to change over time in a stable way. Today’s performance is likely to be like yesterday’s, but if the model has strong evidence today is different, it’ll be allowed to evolve away from yesterday’s results. The end result of this model is that coefficients for each variable change over time, letting marketing performance increase or decrease with seasonality, optimization and competition.