Can we try Uber Orbit?

I’ve built 20+ Marketing Mix Models, and it’s always the same thing that blows the model’s credibility with decision-makers. The assumption that marketing performance doesn’t change! Nobody believes all their hard work was for nothing. Uber’s ‘Orbit’ model solves that problem.


Time-Varying Coefficients

Marketing Mix Modeling commonly uses OLS Linear Regression, which assumes the coefficient for marketing spend doesn’t change over the time period...


Mike Taylor

Built a 50-person growth agency.
💪 Useful 0
😓 Difficult 0
🎉 Fun 0
😴 Boring 0
🚨 Errors 0
😕 Confusing 0
🤓 Interesting 0
Free access for email subscribers.
Python experience recommended.
Usually completed in 22 minutes.
1. Scenario
GOOLYBIB OFFICES – 6 months into a marketing mix modeling project
You’ve been building marketing mix models for the past six months, but an uncomfortable question just came up in a meeting. The CEO came to the realization that our MMM assumes marketing performance doesn’t improve! What’s more when we update the model with new date ranges the coefficients are wildly different. Your credibility is in question, and they might decide to abandon MMM entirely (but what else is there?).
Gustav Gieger
at GoolyBib

I just don’t get it

Is everyone that’s using MMM making this assumption?

Of course marketing performance changes over time! Otherwise we could all just retire early instead of working on optimizing the accounts…

Let’s search for a way to let marketing performance flex up and down

If we can’t do that, I’m not sure I can condone using MMM, it just won’t be something I can get behind. What if one of our investors makes the same ‘discovery’?

This is top priority

This course is a work of fiction. Unless otherwise indicated, all the names, characters, businesses, data, places, events and incidents in this course are either the product of the author's imagination or used in a fictitious manner. Any resemblance to actual persons, living or dead, or actual events is purely coincidental.

2. Brief

One of the big assumptions everyone is making when doing marketing mix modeling, is that marketing performance doesn’t change over time. Most aren’t even aware that’s the assumption they’re making. Of course MMM usually accounts for changing in spend over time with diminishing returns and adstocks, but the baseline coefficients don’t change in most models. If you shorten the date range or update the model, you tend to find that the coefficients vary significantly from time period to time period. This is because now the model doesn’t “know” about those previous time periods and can’t remain consistent with them. For most models making the assumption that marketing performance doesn’t change very much isn’t as bad as it seems. For simple models and small businesses, that assumption largely holds. It’s the red queen effect: we do constantly optimize our accounts, but so do our competitors, so the net result is usually pretty stable performance over time.

However this hidden assumption is a ticking timebomb. It sounds so obviously wrong, that when someone non-technical like a CEO discovers this insight, it can ruin the credibility of marketing mix modeling in general as a method. The solution to this problem is an advanced, but growing technique, called “Bayesian Time-Varying Coefficients” (BTVC): used by companies such as Recast and Uber. One unique benefit of using Bayesian methods, is that you can use priors to allow marketing coefficients to change over time in a stable way. Today’s performance is likely to be like yesterday’s, but if the model has strong evidence today is different, it’ll be allowed to evolve away from yesterday’s results. The end result of this model is that coefficients for each variable change over time, letting marketing performance increase or decrease with seasonality, optimization and competition.

3. Tutorial

Hey, let's walk through how to use Uber orbit, which is their time series forecasting tool, or we're using it for marketing mix modeling.

Uber Orbit.ipynb
4. Exercises
5. Certificate

Share This Course