‘What Do NPS Scores Really Mean?’ By Clare Muscutt
Personally, I've never been a massive fan of NPS (Net Promoter Score) for a number of reasons, which I shall outline in this article. However, when I discovered that it is the only industry benchmark for online communities, we decided to use it in our Voice of Our Members (VoOM) survey.
When our results came in, I found myself asking... What does it really mean? I hope it's useful that I share my inquiry with you, too!
Firstly... Here's our result for Women in CX: +88!*
*I should note that this is only one question asked in the survey (with the follow-up of ‘Why did you give that answer?’), where we also ask far more interesting things!
What Is a Good NPS?
Apparently, any NPS score above 0 is good. It means that your audience is happier than not. Anything above 20 is considered favourable. Bain & Co, the source of the NPS system, suggests that above 50 is excellent, and above 80 is world-class.
+88 appears to be wonderful news indeed, then... But I was reticent to celebrate too much until I dug a little deeper.
For anyone new to this metric, I'll start by explaining what NPS is.
What is NPS?
Net Promoter Score (NPS) was invented by a chap called Fred Reichheld, in partnership with Bain & Company, and Satmetrix Systems, in 2003 to determine the engagement and loyalty of customers.
The score is determined by simply asking users, “How likely are you to share {x} with your friends/family and colleagues?” and presenting them with a 0-10 scale to answer.
NPS is based on the principle that every company’s customers can be divided into three categories: those who score 0-6 are Detractors, 7-8 scores are Passives, and 9-10 scores are Promoters.
NPS is simply calculated as the % of promoters - % of detractors.
Since 2003, NPS has grown to be widely measured across all industries, and now, two-thirds of Fortune 1000 companies use this metric.
Possibly because of the brilliant marketing job done by Fred (et al), in hailing it ‘The One Number You Need to Grow’.
But is NPS just another 'Big Consultancy' commercial con?
What Was the Logic?
All companies seek to grow. And growth—profitable, sustainable organic growth—occurs most often when customers and employees love doing business with a company.
Most leaders want customers to be happy; the challenge is how to know what customers are feeling and how to establish accountability for the customer experience.
“Most leaders want customers to be happy; the challenge is how to know what customers are feeling and how to establish accountability for the customer experience.”
Fred and the Bain team launched a research project to determine whether a new metric would prove fruitful. Working with data supplied by Satmetrix, they tested a variety of questions to see how well the answers correlated with customer behaviour.
As it turned out, one question worked best for most competitive industries:
What is the likelihood that you would recommend ‘Company X’ to a friend or colleague?
High scores on this question correlated strongly with repurchases, referrals and other actions contributing to a company’s growth. In 11 of the 14 industry case studies the team compiled, no other question was as powerful in predicting behaviour.
To test the link between Net Promoter Scores and growth, research teams compiled scores for leading companies in a wide range of industries. What they found was compelling. Though the scores varied widely by industry, they found Net Promoter leaders, on average, grew at more than twice the rate of competitors.
Here Lies My First Issue with NPS
Can a measure of reported behaviour actually be trusted as an indicator of actual future behaviour?
The primary objective of the Net Promoter Score methodology is to infer customer loyalty (as evidenced by repurchase and referral) to a product, service, brand, or company on the basis of respondents' responses to a single survey item.
It doesn't mean they actually will repurchase and refer.
However, proponents claim the use of the NPS score in addition to revenue retention rates and customer retention rates may offer valuable customer insights and may offer a better 'predictability' of customer loyalty rates.
How Reliable Is the Score at Predicting Behaviour?
As it represents responses to a single survey item, the validity and reliability of any NPS score ultimately depend on collecting a large number of ratings from individual users. However, market research surveys are typically distributed by email, and response rates to such surveys have been declining steadily in recent years.
(Our response rate was 41% and apparently anything over 30% is good!)
In the face of criticism of the Net Promoter Score, the proponents of the Net Promoter approach claim that the ‘recommend’ question is of similar predictive power to other metrics, but that it presents a number of practical benefits to other more complex metrics. They argue that the practical benefits of the approach (including a short survey, a simple concept to communicate, and corporations' ability to follow up with customers) outweigh possible statistical inferiority to other metrics.
That feels like something I can agree with.
What Do the Critics Say?
While the Net Promoter Score has gained popularity among business executives and is considered a widely used instrument for measuring customer loyalty in practice, it has also generated controversy in academic and market research circles. Scholarly critique has questioned whether the NPS is at all a reliable predictor of company growth.
Other researchers have noted that there is no empirical evidence that the ‘likelihood to recommend’ question is a better predictor of business growth than other customer-loyalty questions (e.g., overall satisfaction, likelihood to purchase again, etc.), and that the ‘likelihood to recommend’ question does not measure anything different from other conventional loyalty-related questions. Several studies have shown that there is little statistical difference in reliability, validity, or discriminating power between the NPS and other metrics.
“It's no better or worse, but simpler and more likely to receive a response than traditional long survey methods.”
So essentially, it's no better or worse, but simpler and more likely to receive a response than traditional long survey methods.
But what does our score of 88 mean in context?
What Do You Have to Benchmark Against?
I found it fascinating to understand that different industries, cultures and countries have widely different behaviours when it comes to 'recommending' anything!
Bain & Co, the source of the NPS system, suggests that above 50 is excellent, and above 80 is world-class.
However, these are only general guidelines. A ‘good NPS’ will depend on the industry and country a business is in. As a global community, this seems important to understand.
“A ‘good NPS’ will depend on the industry and country a business is in.”
What Is the Impact of Location on NPS?
In Europe, for example, customers tend to be more conservative with their ratings. A high score, a 9 or a 10, is rare. Japan has a similar reaction, as it is considered poor etiquette to provide a company with a perfect score. In the US, on the other hand, it is far more common to see higher scores, with low scores being just as rare as high ones are in conservative countries.
In other countries, scores are more extreme, with dissatisfied customers liberally using the lower end of the scale and even mildly impressed clients honouring a business with a full 10/10.
“In the same organisation, there can be an enormous difference in its average NPS from country to country.”
For example, in the same organisation, there can be an enormous difference in its average NPS from country to country. In one SaaS (Software as a Service) example, in Singapore, the company scores 20 points lower than their global average. In the United States, it scores 10 points higher.
This is a single company offering similar products and services from country to country to similar people. But the average NPS varies wildly. This might be for legitimate reasons, such as poor customer support in some areas, but it will still be heavily affected by the likelihood (or lack thereof) of people rating the same across countries.
Knowing what a ‘good NPS’ is the world over quickly becomes difficult.
What Is the Impact of Industry on NPS?
There is a wild disparity in benchmarks between B2B, B2C and sub-industries.
Your industry may naturally lend itself to lower NPS as a whole.
Property management and debt collection, for example, tend to score in the negatives. Check your score against NPS benchmarks for your specific industry to ensure this isn't the case.
If you do find that your industry scores poorly across the board, you have been presented with an opportunity. If you can impress even a small number of clients/customers, you can outpace the competition in customer experience.
“If you do find that your industry scores poorly across the board, you have been presented with an opportunity.”
If NPS data isn't readily available, the advice is to benchmark yourself against your previous performance instead. If you are improving by about 10 per cent a quarter, you can assume you are heading in the right direction. This is true even if your recent score is still negative – it just has to be less negative.
Also, according to the 2023 Consumer NPS Benchmarks report, the UK average is +14, which makes our +88 look positively spectacular! But there are no comparables in that list for communities.
For online communities, the world's leading community for community professionals (CMX) published its score, so it's all we have to go on... and theirs was 50! So now I'm thinking we are doing something pretty special.
My Main Gripe with NPS Isn't About NPS but How It Is Used
Having pulled together this article, I now feel more comfortable using NPS as a health metric for our community and feel proud that we are succeeding in putting our members at the heart of everything we do.
Having reflected, I think my experience of NPS in big business jaded my view.
I can see now my gripe was less with NPS as a statistical methodology and more about how it is applied:
As an oversimplified way of judging CX
Being used as a replacement for proper CX thinking
Applied in inappropriate places/industries
Driving 'chase the measure' behaviour in employees
Not being used alongside other data sets
Not being used to actually listen to customers or take meaningful action
And ultimately, because the score became more valuable than the insight behind 'why' customers gave that score.
Ok, NPS, we can be friends again for now....
But given that the way businesses gather NPS is via surveys, between survey fatigue, low response rates, sample bias, and businesses struggling to extract actionable insights, the bigger question might be… In the age of artificial intelligence, should we be looking for better ways to understand the customer experience?
