Imagine you have spent years improving a mailing. You have tested headlines, pictures, arguments and calls to action. You have a control that is genuinely difficult to beat.
The next challenger loses. So does the one after that.
It is tempting to conclude that there is little room left for improvement. Perhaps you have reached the ceiling.
But have you reached the ceiling of what your customers will respond to? Or the ceiling of the ideas your team has considered?
Your A/B test cannot tell you the difference.
It can identify the strongest version among those you put into the experiment. The most powerful version might never have entered it.
The part that remains guesswork
A/B testing brings discipline to marketing. With a properly designed randomized test, we can establish whether a change improves results.
But look at what happens before the test.
Somebody proposes a stronger headline. An agency recommends a different image. A colleague suggests shortening the copy. AI generates twenty more alternatives.
Where do these ideas come from?
Experience. Creative judgment. Best practices. Previous winners. All useful inputs. None automatically tells us what is holding this particular audience back from acting.
We can become extremely rigorous about evaluating ideas while remaining surprisingly uncertain about where to look for them.
Every live test then consumes a scarce resource: customers, time and an opportunity to communicate. Generating another hundred variants does not give us another hundred meaningful testing opportunities.
The question becomes: how do we improve the quality of the next challenger?
Customers cannot explain what moves them
Consider a fundraising appeal.
A team might reasonably assume that making the problem look more urgent will increase donations. Yet urgency could also make the situation feel overwhelming. Showing a practical solution might instead make a donation feel worthwhile.
Both explanations sound plausible. Which matters more for this audience?
Now consider an email to existing customers of a nutrition brand. Should it explain more science? Make the product feel easier to use? Emphasize the reassurance of choosing something familiar?
These are hypotheses, not universal recipes.
Asking customers which email they prefer does not settle the question. Judging a piece of communication is a different task from deciding whether to buy. People may also struggle to explain the associations that make an offer feel right—or easy to ignore.
This is why I focus on the intuitive response behind the decision.
The useful research question goes beyond whether people like the headline. We need to understand what the communication makes them feel and associate about the offer, and which of those perceptions are likely to influence action.
Give the next idea a better starting point
At SUPRA, we put a research and modeling step before the live A/B test.
We start with the real target audience. Deep Implicit research measures intuitive reactions and associations, helping us investigate motivations and barriers that direct questions can miss.
We then use those data to build an causal AI-based audience model and investigate the relationships between perceptions and the response we want to improve. The aim is to establish priorities: which perceptions deserve more emphasis, and which may be working against us?
That gives the creative work a more specific brief.
Instead of asking for a more persuasive mailing, we can develop alternatives around the drivers identified in the research. We assess and refine those alternatives against the model before choosing the challenger for the live test.
The model is a guide. Its conclusions depend on the data and the assumptions behind the analysis. The live experiment still decides whether the finished communication improves commercial results.
Research helps us choose a more promising direction. Testing checks whether we got it right.
What happened in a donor mailing
In a SUPRA project for Kindernothilfe, an optimized mailing to existing donors was compared with the conventional approach in an A/B test.
3.5% → 5.1%
Conversion rate
~46% relative increase
€59 → €75
Average donation
~25% relative increase
The first change is a relative conversion increase of approximately 46%. The second means that the people who responded also gave more on average.
The full case has been already published in the Fundraiser Magazine:
One case does not establish the uplift another organization should expect. But it raises a useful question for anyone working with an established customer or donor base:
How much opportunity might remain in the way you communicate with people who already know you?
An existing audience can still respond differently when the message connects with a motivation that previous versions failed to address.
Put that question to work
This is the practical application we are now offering: take one existing direct mailing or customer email, research the audience behind it, and develop a challenger for your own A/B test.
The pilot takes four weeks from briefing to a test-ready asset. Your involvement is one briefing, access to the current material and one approval round. You receive the revised communication and the reasoning behind the changes.
WHO: Any brands or NPOs sending out 6-figure postal or e-mailings every year to drive revenue
WHEN Pilot offer available until November 30, 2026
FEE: Performance-based agreements on request
DELIVERY: Four weeks from briefing to a test-ready asset
We agree on the test design and the basis for calculating incremental revenue before starting. The optimized asset can then be reused without another fee for that asset. For direct-mail projects, we agree the pilot terms around the specific campaign.
The first step is a 30-minute conversation about one concrete mailing or email: what it currently achieves, what you have already tested, and where research could reveal a more promising direction.
… and bring the version that has been hardest to beat.