Goolybib Offices – Brand Planning Session

The CEO is questioning long held KPI targets for the brand team, and withholding budget until they can be connected to future sales. 

Times are tough, I’m having to cut budgets across the board, and we can’t continue having a budget that’s based on a fluffy metric like “brand awareness”
I know brand must have some long term effect
But without knowing how much, how can we justify carving off half of our budget?
The performance guys can tell me to the penny how much revenue we’re driving
It can’t be a two-track system
We need to find a way to link brand awareness to future sales, or I’m going to have to reallocate the budget
There must be something we can do to account for brand awareness in our marketing mix model
Can you look into this as a matter of urgency?


Conversations around brand versus performance are fraught with politics because the difficulty of measuring long term effects leaves open a chasm of uncertainty around what the right budget split should be. If you believe in the value of brand advertising you may over-allocate to long term activities at the expense of more dependable short term returns. Alternatively allocating everything to short-term performance marketing will cause issues from under-investment. Ideally you capitalise on fulfilling demand in the short term, while also investing in generating demand over the medium and long term. How you make that split will make a big difference: either the brand or performance team will be short-changed if you get it wrong, and there is no standardised or widely accepted way of doing this type of analysis. To make matters worse, the average tenure of a CMO is 4 years, meaning they usually won’t be around to see if their campaigns succeeded or failed in driving a long term benefit.
The work of Binet and Field in “The Long and Short of it” recommends a 60:40 split between short term activation and long term branding activities, after doing a meta analysis of IPA award winning campaigns. However this dataset suffers from selection bias (only award winning entries) and even the author’s admit this is a guide not a rule: the optimal amount will be different by brand, company stage, and sector. Many attempts have been made to estimate the long term impact of brand campaigns, and some of this effect is covered by adstock rates in marketing mix models: transforming the data so some of the effect of brand spend carries on to future time periods. However adstocks are rarely sufficient for long term brand awareness, as high adstock rates make the impact hard to distinguish from the constant term in the linear regression. Literally: where does the baseline sales end and the impact of the brand campaigns begin?
These are difficult questions to answer, but a recent session at the Meta Marketing Mix Modeling summit by GfK and and Nepa, entitled “Measuring long term effects with MMM”, shows some promise in an elegant approach for bringing brand and performance together. The technique centers around embedded modeling, essentially building a model that predicts brand awareness over the long term, and embedding the outputs of that model in your base sales model as an input. That way you can predict long term drivers of brand awareness, while also forecasting forward to what the total impact of a brand campaign today might have in the future, as well as what the payback period might be.

Embedded Models

00:01 Hey, I'm gonna show you how to incorporate brand awareness into your marketing mix model. So I'm just running this notebook.
00:09 It's in Google, CoLab. If you just click into the cell, you can press first and enter. It'll run and get the results.
00:16 So you just need to do that in order to follow along. You can click play on here as well. So this just loads the libraries and then this authenticates with Google.
00:25 It'll pop up if you're not authenticated, and it will get you to log into Google account. Once you've done that, then you can pull all this data here.
00:33 So if you've done everything right, and then it should come through here like this. We have the sales, how much we spend on brand and then how much we spend on performance, as well as the brand awareness.
00:45 And, and brand awareness is you know, you can see that we have a plot here. Brand awareness is recorded monthly, but then we've used interpolation to connect the dots.
00:57 So all that means is, you know, if next month brand awareness was 30% and, and this month it was like 29%, it just kind draws a line between those two.
01:08 Which is why you kind of, you get we have values for every single day. Cool. So that's brand awareness. We also have you know, the brand spend, if you just kind of take a look so you can see how these are related, you can clearly see that, you know there are times where we do brand spend and then awareness drops off, like people forget who we are, and then we do more brand spend and then, you know, that seems to have an effect, you know, and so on and so forth.
01:36 So it doesn't seem to be the only thing driving sales, but it does seem to be at least correlated with these periods.
01:42 So that's what we're trying to tease out. Now one of the ways that you typically would do this is you use ad stocks.
01:50 So we can actually plot this if, if it's useful. Let's see, let's see, brand ad stock. And let's also plot it with the brand awareness I'm sorry, brand spend.
02:13 Let's see that. So here we go. So you can see that basically what, what we're doing in ad stock is we're giving it a a, a carry a carryover rate.
02:28 So basically like well this, this is a very long ad stock. It's saying that because this is a very kind of brand awareness type campaign like, you know, you can imagine emotional TV ad for example.
02:41 You know, this is this is saying that basically if I spend a dollar today, 99 sense of that is still felt tomorrow and, you know, 99 cents of that is still felt the next day and so on and so forth.
02:54 The reason why we have a long ad stock here is that we want kind of of long term effects, cuz this is kind of driving memory, it's driving brand awareness rather than directly driving sales.
03:06 The other thing we're using here is the this is what we're taking it to, the power of 0.6. This is the saturation effect.
03:15 So what it's saying is like if we have the ad stock build up too much then we're not getting much for an additional dollar spend, if that makes sense.
03:26 So if we didn't have this these ad stock rates would, would accumulate you know, accumulate quite high. So that's what that does.
03:34 We can see those next to each other. You can see that like the impact of the brand TV campaign is highest towards the end kind of builds, and then, and then it starts to drop off until the start of the next TV campaign.
03:45 Cool. So that's ad stocks right now. Let's just run let's just run our model. So we create a marketing mix model and we have a constant term.
03:55 We have our performance spend we have brand awareness, and then we have brand ad stocks. So we're kind of modeling the impact of the brand campaign separate from the impact of brand awareness.
04:05 And we also have our performance marketing, which is, you know, kind of fairly regular. And we can work out the contributions with any marketing list model.
04:15 Like obviously this is go go, right? But it gives you the coefficients and what you can do in order to get a forecast is to say if you multiply that coefficient by the input value then then you get basically the the, the total number of sales that, that, that variable contributed.
04:39 So we're just gonna go through looping through the list, and then we're multiplying the coefficient by the variable value.  And then just depending that contribution so, and then we're just transposing it and then summing it and rounding it.
04:52 So what that gives you after all that is basically how many, how many sales did each contribute? So the constant, basically the baseline sales, like we didn't spend anything on marketing.
05:01 We would still gotten 882,000 sales over the three year period. Performance marketing drove 465,000 and brand awareness drove another 457,000.
05:13 And it seems like brand ad stock basically the brand campaign didn't drive any, any real sales, like just 2000 over the three years directly, right?
05:22 Cool. So this is showcasing a problem and we know it's a problem because I actually made this data, right? I simulated it so I know what the real values are.
05:32 I know that the constant is actually, you know, the baseline is actually way lower than than than what we're saying.
05:38 And this could cause us to make a really bad decision because if we think that we could 10 all our marketing spend and still get 882,000 sales, we're wrong.
05:46 Like, we would actually only get 328,000. And this is a problem whenever you're doing long term effects, is that the constant term becomes kind of indistinguishable from from brand ad stocks or brand awareness because they're relatively kind of fight over time, more like, you know, similar to the constant, which is constant every day, right?
06:07 Then you know, your marketing mix model can, can get the two confused. And sometimes that means that we make brand seem to be more valuable than it actually is, right?
06:17 Because it's stealing something from the constant. But in this, this particular case the opposite is happening. We're actually, you know, the constant term is stealing something from you know, from the real value of brand awareness.
06:30 When I made this data, I did a couple of things. One is I had the value of brand awareness drive sales but then also you know, performance was separate in terms of driving sales.
06:40 And then I had this kind of halo effect performance times awareness. So the theory is, and, and you know, in practice I've seen this a bunch of times, if you have more brand awareness, then people are more likely to click on your ads and therefore your performance spend does better when you have high brand awareness.
06:57 So that kinda stripped out separately, that actually drove 131,000 sales. Cool. So how do we start to correct for this?
07:05 And the way that we do this is to build an embedded model or a multilevel model. There's a few different names for it but essentially what it is, is you're building one model inside of another.
07:13 And the embedded model in this case we're gonna use to predict brand awareness. So there's a few things we're doing.
07:22 One is we're adding a trend line. So this is literally just and like I already know the value of this, but typically what you would do you know, just like I already knew the value of the of the adoc typically what you do is you'd test lots of different values of this line and then, and then find the one that fits the best.
07:40 But that's a whole separate topic, <laugh>. And, you know, there's different algorithms for doing that or you can do it manually but let's just assume we, we know what the trend rough trend is, right?
07:51 And then what I'm doing is I'm building the model instead of predicting sales and predicting brand awareness. So everything else is the same and except the coefficient is telling you how much brand awareness you're getting you know, for every every kind of increase in spend or, you know or, or like for, for each kind of uptick in the trend line.
08:17 And so what that, what has done, so like if I run this, what it's done is actually predicts brand awareness pretty well.
08:27 And there we go. You can kind of see the brand prediction is in blue and the brand, actual brand awareness is in you know, in this yellow orangey color.
08:36 And you know, it might not be this accurate, like when you're not simulating the data but this is useful in and of itself.
08:42 So, you know, even if you don't go on to predict sales and it doesn't work that well for sales, actually predicting what brand awareness will be help you measure your performance in terms of, you know, should we be spending more on,  Our TV advertising goals?
08:56 Like, you know, did our last TV and creative work or did it not, right? Because if it, if it drives brand awareness to the level that you would've predicted with this model then it's done a good job, right?
09:09 And if it is lower than expected, then you know, mid-flight that actually it's not working. The other thing that allows us to do is to spot anomalies.
09:19 So like if, you know, we see this normal trend and then suddenly like a spike, like who is here then maybe there's something wrong, right?
09:27 Maybe you know, maybe like, or maybe something, right, right? Like maybe an influencer started talking about you like Kim Kardashian share your product or something like that, right?
09:36 So you know, that would drive brand awareness and that's something you can then let add into the model, kind of understand a bit more about what drives it.
09:44 Cool. So now we've done that, and again, I'm just showing kind of sharing the basics, like to actually build, this is a toy version of the problem, to actually build a real model, it would be much, much harder.
09:56 Like you want to probably use be statistics because that will allow you to get better results and, and more plausible results.
10:04 But we can see actually that, you know, when we run this model, like not only have we improved os squared, os squared isn't a great measure of fit <laugh> in and of itself, but it's like one thing to look at but also we're getting closer to the reality, right?
10:19 So so the previous model we thought you know, the constant term had 882,000 sales. And actually now we've kind of brought it down a bit.
10:31 So we've reattributed some of those things that were going to the baseline and we've gotten closer to the actual there's a couple of problems with the model though.
10:40 You know, even though we have gotten closer to reality for brand awareness we went from four, five, 7,000 sales to five, five, 5,000 sales.
10:48 That's good. But like the real, the truth was 6 34, right? And same for here. Like we are still pretty under on performance 5 55 versus seven 13, but it's an improvement for what we were before.
11:01 And one of the issues that we have is that the performance times awareness is actually showing us negative. And this is a problem that we have with, you know, multi colinearity, like one multiple terms or kind of relate to each other.
11:13 The model does wacky things, right? So what it's done is it's given too much credit to the constant not enough to brand awareness, not enough to performance spend, and then it's to kind of like make the numbers work.
11:24 It's like deducted 93,000 from this performance times awareness, even though we know this is a positive term, the model built it as a negative term.
11:33 So that's one of the reasons why you'd wanna use like a Beijing marketing mix model where you can tell the model, no, no, no, this is definitely gonna be positive.
11:40 We just dunno how positive you can kind of give it ranges to look inside which you can't do with a simple model like this.
11:46 Cool. So like what the takeaway should be here is that if you're following along that rather than trying to put brand awareness directly into your model, you could build a model that forecasts brand awareness and like forecasts what what your brand campaign does to brand awareness.
12:08 And then use the result of that embed that as a variable in your model, right? So this brand pre is a variable in our model now and therefore, you know, when we're forecasting forward we can have some assumptions about what that brand pre would be based on that other model.
12:24 So if kind of, we've built a whole model just to predict this one line, and then we use this one line to encompass a lot of those assumptions in the main model that we're using to predict sales.
12:35 Cool. So that's how that works. You know, you can get better or worse results with this. It's more complicated than a normal model.
12:44 And there are actually packages, you know, that will do this for you. Like with Beijing statistics, you can build this all into one model rather than kind of having two separate which is one of the other big advantages.
12:55 So I, you know, recommend you checked out, but that's the way outside the scope of, of this lesson.


Why is it important to measure brand advertising performance?

What does Binet and Field recommend as a split between long and short term campaigns?

Why is it difficult to incorporate brand in a marketing mix model?

Traditional brand marketers can’t justify the impact of their campaigns, and yet performance marketers have a short term bias that can wreck your company. They both fight over budget, so how do we reconcile the two?

Vexpower Learning Certificate of Completion

Congratulations Anna Cosentino 💪