CX Analytics is like “Teenage Sex”
Listen to the audio version here
CX analytics is the teenage sex of the last decade: Everyone's talking about it. Few actually do it. And no one is good at it.
I used to run the world's biggest CX Analytics masterclass, have written two books on the subject (CX Insights Manifest & The CX Insights Playbook), and this is what I've heard from talking to hundreds of CX professionals about their CX ambitions and failures.
The customer experience hype started about twenty years ago with the rise of the NPS system, promoted by BAIN Consulting. Methodologically, it was neither new nor innovative. Rather, it was an simplification of earlier approaches (customer satisfaction in the 70/80s and relationship marketing in the 90s) that had been perfected decades earlier. In the 2000s, CX emerged, and with the rise of online research and software, it became an industry.
CXM barely delivers
The industry promised a lot.
Improving CX would boost brand trust, sales and profits by leaps and bounds
Customer Experience Management would improve the customer experience
Both turned out to be overpromised. Let's start with the impact on CX. There have been many studies on the impact on revenue and profit. In fact, I don't know of any that make sense methodologically.
Let's take Qualtrics XMinstitute, which publishes a study every year interviewing nearly 30,000 customers about CX in different industries and countries. Surprise! They find a strong correlation between CX, brand and purchase intent.
Just because you interview large numbers of customers does not make the analysis true. But XMinstitute is not alone. Many others have followed the same false causal inference. Other studies, such as the American Customer Satisfaction Institute, even find correlations between CX and stock market valuations.
What's wrong with that?
One of the milestones of evidence-based marketing began in the 1960s and has almost been forgotten: PIMS (Profit Impact of Market Strategies). The programme collects holistic data from strategic business units over at least 5 years per unit and now has tens of thousands of records. One of the puzzling findings was that market share was a key causal driver of sales and profits.
Another study later provided an eye-opening explanation. The Ehrenberg-Bass Institute, the world's largest scientific marketing institute, became widely known through Byron Sharp's influential book How Brands Grow. A key insight from the institute's extensive research is that customer loyalty is largely driven by market share. This reflects a fundamental pattern: customers typically buy from multiple brands, but are more likely to choose larger brands simply because they are more familiar with them. Familiarity, in turn, positively influences perceptions of the customer experience.
As a result, big brands outperform on CX. Not because they have a better customer experience, but because they attract those customers who are more likely to rate the CX as sufficient or even great.
CX correlates with success because of a long-established law-like phenomenon. But correlation is not causation. Brand-building advertising can have a greater impact on customer loyalty than CX initiatives.
At Supra, we once analysed a dataset measuring NPS for all insurance brands in Switzerland. Our client had a question: Why does the market leader have such an excellent NPS? What CX issues do we need to improve on to reach this level?
We carried out a proper CX AI analysis, described below, and found something astonishing: NOTHING can lead to catching up with the market leader. In fact, the market leader outperformed the brand on most of the issues the customers mentioned. The model predicted that our client could only close the gap by 40 per cent if they matched the leader's CX performance.
Same law-like pattern.
But the good news is that Customer Experience Management CAN still make a difference.
How to make CXM work?
Practitioners were enthusiastic about customer experience management. Software has been bought. Customer journey maps are everywhere. Yet most companies can barely move the needle. NPS goes up and down.
Why is this? Besides inflated expectations, there is another phenomenon: questionable analysis of customer feedback. Companies cling to correlations and counting feedback. Software vendors sell what clients want, not what they need.
(Side note: That's what's different at Supra. We focus on what enterprise brands need. Yes, it's not the best business strategy, but it's more fulfilling and fun).
Here are the four things companies should rethink to fix their CX analytics.
Standard sentiment/text analysis is not enough
CX trackers tend to use open-ended questions because they are convenient for the customer and allow them to learn about new issues that arise.
However, the feedback is so large that no one can read it manually. This is why text categorisation through sentiment analysis or text AI is used. The problem: Companies use standard coding schemes implemented in their software for their industry. They miss important granular differences that arise for different countries, customer segments, products and brands.
Recommendation #1: Build your own codebook.
Counting feedback is not enough
When measuring NPS or any other metric, companies ask "why did you rate this way". Then they count how many customers said what based on the prior categorisation. Because counting responses sounds reasonable, everyone does it.
But customers have neither the incentive nor the awareness to pinpoint exactly what is driving their behaviour. What you need is a proper modelling approach that finds the hidden causal link between perception (feedback) and outcome (rating).
Recommendation #2: Use multivariate statistics, ideally proper causal inference to understand the impact of topics.
Classical driver analysis can lead astray
When companies try to understand the impact of issues, they start to correlate. What do promoters say compared to detractors? Sounds reasonable, but it is not. Many of these correlations turn out to be misleading. Then companies build strategies on them and are surprised when they do not work.
Sometimes promoters are more open to sharing issues that need improvement than detractors. Sometimes issues are related and therefore correlated (such as "good service" and "friendliness"), but only one of them is the immediate driver. The regression is confused by this "multicollinearity". Often ratings are influenced by contextual variables (e.g. a more critical customer segment), and themes are correlated because that critical segment mentions other themes - but NOT because the theme drives loyalty, just because of the different nature of the customers.
To disentangle the unique impact of each issue raised, you need to use proper causal but exploratory inference to understand the impact of issues and control for contextual factors. So why use a half-baked, oversimplified approach like regression analysis when you are talking about multi-million dollar impacts?
The right insights can make all the difference.
Recommendation #3: That is why we believe it makes perfect sense to use Supra Causal AI for this. With SUPRA CX AI we have standardised the process: custom-built codebooks using leading text AI coupled with SUPRA Causal AI to understand the impact of topics. All reported in an established interactive dashboard.
Predict the financial impact needed to move the business
Knowing what's important and what really moves the needle is the key information you need to focus your time, money and resources effectively.
To be fully aligned with the organisation and ROI-based decision making, you need to link potential actions to predicted financial outcomes.
You want to know what the typical increase in customer value will be if your NPS improves by 1 point. You get this link through a one-off causal modelling project that looks at customer data to predict next year's churn and up-sell, and pinpoints the causal contribution of NPS measurement.
You also want to know the NPS impact of an improvement in theme perception. And you want to be able to plan which actions will achieve these improvements and what the costs of these plans are. When you have all this, you have a framework that predicts the ROI for each potential CX initiative or action.
Sounds futuristic? It's all there, in practice, and it works. With Supra CX AI we even have a software for it. Yet few companies do it this way. Why not? It's not part of what Qualtrics, Medallia, InMoment and the like offer. But nobody stops you from putting a dinghy in the water.
From “Teenage Sex” to Tantra
Sorry, the teenager in me couldn't help but write that shallow sub-headline.
To sum up: CX is overhyped, justified by spurious correlations. But it can still drive value. Value depends not only on proper management and implementation (an argument on which everyone agrees). Value depends on the accuracy of customer insights. Wrong insights mean ineffective initiatives with low ROI.
Do this instead:
Build custom codebooks for your business, trained with high-end text AI
Use exploratory causal AI to understand what really drives the customer experience
Enable effective decision making with a planning tool that links the costs of potential actions to their predicted financial outcomes.
THIS is how you 10x your CX impact.