The Unwanted Child: Marketing Campaign Evaluation
Listen to the audio version here
Did the last campaign work? Isn't that why we do Marketing Mix Modeling (MMM)? Yes and no. MMM does not measure at which touchpoints a channel's spend actually hits eyeballs. Worse, it focuses on short-term impact and misses the biggest contribution of advertising, which is brand. Brand trackers (if you have one), on the other hand, are not designed to evaluate campaigns. Why has no one solved this puzzle? Some say it is an "unwanted child" .... I love to share a story where I not only learned why, but was able to develop a solution that works amazingly well.
This year, Anna and I took time to look into the future and brainstorm. Anna is the Insights Lead for a multinational apparel brand. This year, she rejected all of my suggestions for the agenda and instead added "campaign evaluation" to the list.
I ask Anna, "You do marketing mix modeling every year. Isn't the purpose of this tool to evaluate campaigns? She said, "Yes, it was supposed to do that, but we found out many things that it does not do well. So she gave me a long list.
MMM tries to explain short-term sales. It does not explain how spending drives the brand. "Why haven't you included your brand tracking data in the modeling?" I asked. Anna explained that the company had implemented a new brand tracker, so there was not enough history to feed the two years of data with a consistent data source.
Instead, if you look at the time series of the brand tracker, it is not obvious that this year's campaigns drove the metrics. The impact of one campaign is small compared to all the other influences on the brand measurement. That's why you can barely see it. Although the ROI could be as large as 10x and you still have a hard time seeing it.
“Although the ROI could be as large as 10x, you may still have a hard time seeing it.”
But things got more complicated. "The summer campaign is completely different from our winter campaigns," says Anna. Putting different creative strategies together can be a big problem. The reason is simple. Science has shown that 70% of the impact of a campaign is not how you allocate the marketing budget, but how the creative is made.
“70% of a campaigns impact is not how you distribute the marketing budget but how the creative is made.”
It dawned on me that an MMM is way too general and the Brand Tacker is not only not granular enough, but also does not look at the specific impact of an individual campaign. These insights are needed to review what the campaign is really doing for customers and how to do it better next time.
During our discussion, we developed a 7-point checklist for a proper campaign tracking tool.
Granular: The campaign tracker should run daily. We do not need a large sample size per day, but the distribution over time should be in line with the distribution of spend.
Timing: Starting the measurement before and ending it after the campaigns allows us to filter out confounding influences on the impact in the analysis phase and serves as a baseline measurement.
Implicit: We agreed to measure brand as willingness to consider the brand, but we need to complement the measure with an implicit measure of willingness to buy the brand's products. Scientific studies have shown that in some cases the same person has only a 50% chance of answering the same question the next day with the same why. With implicit measurement, we can control for this noise.
Imagery: We measure how key product attributes are perceived, and we implicitly measure how the brand is perceived qualitatively. We use a brand archetype perception tool to do this.
Touchpoints: We also measure if and where people had a touchpoint with the brand. We measure this implicitly, as science shows that most touchpoints can hardly be consciously remembered, but still change the attitudinal (implicit) responses. This has many advantages: It is very easy and quick to do in a survey. Also, we can mix any type of touchpoints from TV, billboard, taxi ad, event sponsorship, social, newspaper, news, WOM/Friends, and so on.
Drivers: Measuring perceptions and results is of little value if we just start correlating them and end up with false results. That's our core philosophy at SUPRA and that's why we apply Causal AI in almost all cases.
Cost-effective: The exercise should cost a tiny fraction of the total campaign spend.
Fortunately, the SUPRA Brand and Touchpoint Optimizer already provided most of the tools. With some customization, the tool was ready.
Getting back to the original intent of this 10xInsights article: how do we create insights that have the potential to have 10x impact?
10x Insights with modern campaign tracking
The status quo in advertising optimization is this
Ads are tested and accepted if they meet certain metrics. There is NO real quantitative research on why some ads are better than others. This is all about "optimizing" 70% of the ad impact.
Many companies do Marketing Mix Modeling, but with this they only have a very rudimentary understanding of the ROI of channels, without being able to be specific per campaign, and while often completely missing 65% of the impact type: Brand.
Brands run brand trackers that mostly just measure how awareness and consideration change over the years - there is typically little insight into how to improve that brand consideration.
On average, the industry agrees that advertising typically delivers a 2X ROI. This varies by channel and creative. What is striking is that out of every 100 ads aired, only about 10-15% "kill it". Some creatives can easily show 10x or more ROI.
“Most ads are bad. Just one in ten is a money machine”
What does it tell you if most ads are bad, but a few ads in the category have ROIs that are many times higher? It tells you that this is a winning strategy: Create a process and methodology that helps you continually learn about the causal factors of success in your industry. Be relentless and never satisfied: Create, learn, repeat.
The numbers available to everyone provide clear direction: Thorough research creates a flywheel that can lift marketing ROI from a risky 2x to a whopping 10x.
The campaign evaluation scheme outlined here is an important piece of the puzzle.
It helps us understand the true impact of creative execution. Perception measurement helps us see the qualitative creative impact. Causal driver analysis tells us which perceptions move the needle and which do not. The tool measures long-term impact and traces it back to touchpoints. Causal driver analysis, in turn, lets us see the impact of each touchpoint. This allows us to calibrate the marketing mix.
Here comes the afterburners: With the same tool, you can spy on your fiercest competitors. You can learn how the touchpoints you are not using today are performing. You can learn how their creative strategies are paying off.
Create an insight flywheel based on continuous tracking, causal inference and implicit measurement.
THIS ... is how you will 10x your insights.