Measuring Net Promoter Score

Posted by Chrissie Brodigan on January 7, 2015

If you’ve ever rented a car, purchased something from Apple, or called AT&T with a billing question, you may have even responded to an NPS. It’s a single question, “How likely are you to recommend [product name] to a friend or colleague?”

NPS & Product Development

We have feedback coming into GitHub from lots of places: customer interviews, usability studies, sales threads, support tickets, tweets, etc. Organizing and making sense of this mountain of feedback is a daily challenge. Last year, we began experimenting with Net Promoter Score (NPS).

NPS is a metric that can help to create a shared vocabulary across departments for understanding customer loyalty. Loyalty is broadly defined as person’s willingness to put their own reputation on the line to recommend a product like GitHub to a friend, family member, or colleague. Customer loyalty is a reliable predictor of growth, it also helps humans persevere through less-than-ideal conditions that may include hiccups in product quality, price, and convenience.

However tempting, don’t take the score at face value, instead analyze results through different lenses:

  1. Revenue-weighted analysis – Learn if customers spending the most are the least happy and if the inverse is true.
  2. Time-weighted analysis – Learn if newer customers are less likely to recommend the product than tenured customers.

We can see some measure of loyalty with product renewals and upgrades, but to rely solely on those metrics alone would be a flawed research approach. For example, some people might stick with a product like GitHub because it’s the system they inherited or it’s too much of a hassle to move to another service. That sounds less like loyalty and more like baggage. We want to do better.

Setting Up Your NPS

1. Ask the single required question:

How likely are you to recommend GitHub to a friend or colleague?

Answers range from 0 to 10, and are measured on an 11-point scale with one neutral point (promoters = 9 - 10, passives = 7 - 8, and detractors = 0 – 6).

Note: Consider experimenting on the 11-point NPS instrument with a 5- and 7-point scale. The 11-point NPS scale is widely used and accepted across many industries, however there is solid research that 5- and 7-point scales result in fewer measurement errors.

2. Ask an additional optional open-ended question:

What is the most important improvement that we could make that would make you more likely to recommend us?

3. This is optional, but I recommend including a few additional multiple choice questions that can help you look at the data with additional dimensions (e.g. Is your life at work better or worse with [product name]?).

4. Calculate the percentage of customers who respond with nine or ten (promoters) and the percentage who respond with zero through six (detractors). Subtract the percentage of detractors from the percentage of promoters to arrive at NPS. However, don’t be satisfied with this first output, instead do a revenue-weighted and time-weighted analysis to learn more about your customers.

5. Share some of the high-level insights that you learn back with your customers, which should help incentivize them to continue to take surveys, share feedback, and ultimately become more loyal.

6. Measure the score over time. Continue to sample people who have completed an NPS to see if their score improves, remains neutral, or decreases, and sample new customers. Are you improving with newer audiences?

GitHub-flavored NPS

We created our own NPS-flavored survey that asked a few additional questions to provide a bit more information about demographics and product features. Some examples:

  1. How much does GitHub help you with your daily job? (5-point likert scale)
  2. Would your job be better or worse without GitHub? (We looked at how passives and detractors answered this question)
  3. How easy is it to work with your teammates on GitHub? (multiple choice)
  4. How many people do you interact with inside of GitHub? (multiple choice)

Designing Your Approach to NPS

These are some of the questions we discussed before putting our first NPS together:

  1. Where do you want experiment first?
  2. How do you want to prioritize customer groups for an NPS experiment?
  3. Do you want to survey the product, the company, or a particular team (e.g. sales or support)? (why)
  4. What’s the best context for including a survey for each particular group? (where can you best reach your audience?)

Measuring Over Time

As a one-off exercise it can be interesting, but NPS is most useful to your organization when you measure it over time and when it’s paired with other qualitative and quantitative research efforts.

Much like we keep an eye on our financial data, customer satisfaction is a form of currency that should be monitored with a regular cadence to see if we’re losing, maintaining, or gaining goodwill. Most customer satisfaction surveys are long, complicated, have low response rates, and simply don’t yield useful information that product, sales, and customer success teams can effectively act on.

NPS doesn’t tell you what’s happening or why something is happening. People use them because they are simple, familiar, help set baselines, and when paired with other types of research powerful in helping organizations identify and solve problems quickly.

Think of NPS as part of a bigger monitoring system that helps you see what’s happening right now and can act as a proxy for what will happen in the future.

Photo credit: my brilliant colleague Matthew McCullough