Measuring Net Promoter Score
2015-01-07
If you've ever rented a car, purchased something from Apple, or called AT&T with a billing question, you may have even responded to an NPS. It's a single question, "How likely are you to recommend [product name] to a friend or colleague?"
NPS & Product Development
We have feedback coming into GitHub from many places: customer interviews, usability studies, sales threads, support tickets, tweets, etc. Organizing and making sense of this mountain of feedback is a daily challenge. Last year, we began experimenting with Net Promoter Score (NPS).
NPS is a metric that can help create a shared vocabulary across departments to understand customer loyalty. Loyalty is a person's willingness to put their reputation on the line to recommend a product like GitHub to a friend, family member, or colleague. Customer loyalty is a reliable predictor of growth; it also helps humans persevere through less-than-ideal conditions that may include hiccups in product quality, price, and convenience.
However tempting, don't take the score at face value instead, analyze results through different lenses:
Revenue-weighted analysis – Learn if customers spending the most are the least happy and if the inverse is true.
Time-weighted analysis – Learn if newer customers are less likely to recommend the product than tenured customers.
We can see some measure of loyalty with product renewals and upgrades, but to rely solely on those metrics alone would be a flawed research approach. For example, some people might stick with a product like GitHub because it's the system they inherited or it's too much of a hassle to move to another service. That sounds less like loyalty and more like baggage. We want to do better.
Setting Up Your NPS
1. Ask the single required question:
How likely are you to recommend GitHub to a friend or colleague?
Answers range from 0 to 10, and are measured on an 11-point scale with one neutral point (promoters = 9 - 10, passives = 7 - 8, and detractors = 0 – 6).
Note: Consider experimenting on the 11-point NPS instrument with a 5- and 7-point scale. The 11-point NPS scale is widely used and accepted across many industries. However, there is solid research that 5- and 7-point scales result in fewer measurement errors.
2. Ask an additional optional open-ended question:
What is the most important improvement that we could make that would make you more likely to recommend us?
3. This is optional, but I recommend including a few additional multiple-choice questions to help you look at the data with additional dimensions (e.g., Is your life at work better or worse with [product name]?).
4. Calculate the percentage of customers who respond with nine or ten (promoters) and the percentage who respond with zero through six (detractors). Subtract the percentage of detractors from the percentage of promoters to arrive at NPS. However, don't be satisfied with this first output; do a revenue-weighted and time-weighted analysis to learn more about your customers.
5. Share some high-level insights that you learn back with your customers, which should help incentivize them to continue to take surveys, share feedback, and ultimately become more loyal.
6. Measure the score over time. Continue to sample people who have completed an NPS to see if their score improves, remains neutral, or decreases, and sample new customers. Are you improving with newer audiences?
GitHub-flavored NPS
We created our NPS-flavored survey that asked a few additional questions to provide more information about demographics and product features. Some examples:
How much does GitHub help you with your daily job? (5-point likert scale)
Would your job be better or worse without GitHub? (We looked at how passives and detractors answered this question.)
How easy is it to work with your teammates on GitHub? (multiple choice)
How many people do you interact with inside of GitHub? (multiple choice)
Designing Your Approach to NPS
These are some of the questions we discussed before putting our first NPS together:
Where do you want to experiment first?
How do you want to prioritize customer groups for an NPS experiment?
Do you want to survey the product, the company, or a particular team (e.g., sales or support)? (why)
What's the best context for including a survey for each particular group? (where can you best reach your audience?)
Measuring Over Time
As a one-off exercise, it can be interesting. Still, NPS is most beneficial to your organization when you measure it over time and when paired with other qualitative and quantitative research efforts.
Much like we keep an eye on our financial data, customer satisfaction is a form of currency that should be monitored regularly to see if we're losing, maintaining, or gaining goodwill. Most customer satisfaction surveys are long, complicated, have low response rates, and don't yield valuable information that product, sales, and customer success teams can effectively act on.
NPS doesn't tell you what's happening or why something is happening. People use them because they are simple and familiar, help set baselines, and, when paired with other types of research, are influential in helping organizations identify and solve problems quickly.
Think of NPS as part of a more extensive monitoring system that helps you see what's happening and can act as a proxy for what will happen.
