Say my name (A story about metrics)

joako
7 min readFeb 16, 2017

It’s hard when you are so scared of something that you don’t dare to even pronounce its name. You see the boots under the curtain, and you know there’s someone behind… with a knife. You are so terrified to unveil the curtain, that you keep acting as if nothing happened at all… but you know for certain this cannot end well. Do you guess which is the metric with the knife?

NPS.

Yes, I said it. You keep using that figure to understand how well your company is doing but, correct me if I’m wrong, lately you get the feeling that more than anticipating what the market needs, you are just trying to ‘please’ a single numeric indicator that pretends to express how your customers think about your company. Just as ancient people tried to please volcano gods to spare their lives.

Net Promoter Score (NPS) is an indicator that someone thought summed up everything you needed to know about what your customer thinks of your company. Would they recommend our product/service to other people?

It’s clear that if they do, you have a Promoter, someone who liked so much your company that is willing to spread the word. Of course, if they don’t you have a detractor, someone who maybe tries to stop others from approaching your brand. And then, there’s this grey middle ground of people, neutrals, who are interpreted just as: “meh…”

One number, more than one lecture

So, your NPS is the number you obtain by subtracting the percentage of detractor from your percentage of promoters. It ranges from a 100 (all promoters) to a -100 (all detractors), with plenty of intermediate states that can mean many things… An NPS score of zero can be 50% promoters - 50% detractors or 100% neutrals (and all possible intermediate states). These are completely different places to be and yet they would be expressed using the same number. Isn’t that weird?.

Check out all the possibilities for a fancy 42 NPS score

Truth is that, when you dig deep enough into most companies cultures, you get to find that “the number” is treated as an absolute. Maybe the people who craft the NPS stats in your company have a larger understanding of it, but the rest of the crew get a single digit I.E: “42”, with a sparse explanation I.E: “This is a really good number, trust us”. But what you get is a graph that welcomes you from a monitor each time you get out of your company’s elevator that tells you if this is going to be a good or a bad month.

“The Answer to the Great Question Of Life, the Universe and Everything is Forty-two,” said Deep Thought, with infinite majesty and calm.”
Douglas Adams, The Hitchhiker’s Guide to the Galaxy

What started as a prediction indicator on sales increase or decrease, has now transformed into and goal. A number that MUST increase… or else.

Ineffable and Ubiquitous

In the end, the NPS has become an invisible overlord that mandates the destiny of every initiative your company dares to launch. Even more, every interaction with your client now needs to have an NPS score attached. And the NPS is the KPI to rule all the satisfaction KPIs. It decides not only the future of all your products and services but also the fate of the people who are in direct contact with your customers.

One KPI to rule them all, One KPI to find them, One KPI to bring them all, and in the darkness bind them

Spicing it up

So, NPS is not used anymore as a tool but as a goal, we measure every interaction with our customers with this KPI, it can be used to evaluate us directly, and we ourselves are responsible to analyze the results…

It’s not weird to think that, being our own designed Damocles sword… we tend to cook the numbers. If your place in the company or your yearly bonus depends on how the NPS scores you will tweak anything that can be tweaked to fake the numbers that you need.

Let’s Cook

And when we mean cooking the numbers we mean even before making the survey; You are a teleoperator, and they have told you to score high or face the consequences, so you warn the people you attend about the incoming call… and how to score you. As in: “You’re going to receive a call asking about how well I attended you. If you score me less than 9, it will mean you are dissatisfied and it would not help me at all”. (In the hard version of this speech I’ve been warned they will be fired).

And then there's us as clients. Here we are, just having solved this annoying problem with a company we expected to have no problems with, and having wasted a great deal of time in doing so, and we receive yet another call asking us to qualify the service. Just the icing on our cake.

Summing it up

  1. KPIs work as tools, not as goals. And if the NPS is actually guiding your destiny, prepare for a travel to mount doom, because it’s a KPI difficult to get rid off. KPIs are a necessary tool, but they’re just that.
  2. It’s not useful to measure every interaction with a customer, and it’s driving your company in the wrong way. You’re becoming obsessed with meeting the demands of your customers rather than addressing their real needs. As Henry Ford would put it; You are trying to breed faster horses instead of thinking about the next steps. Satisfaction is something that goes beyond a single question or number.
  3. Measuring everything also affects your client´s perception of the quality of your service. It’s not only that you are bothering your client with yet another meaningless interaction that doesn’t add any value to them. It’s also that by the very act of measuring your client satisfaction in such an impersonal and intrusive way, you are altering the measure you intend to obtain.
What’s my name if I’m talking about altering the measure I want to make by observing it?

So, what can be done?

Sometimes getting help from the outside (did I say I work at a strategic Design consultancy?) can be useful to launch some internal processes. It helps you start things, and sometimes an outsider perspective is very useful. But transforming how your company operates is not sustainable if it’s not done internally, it has to be done from the inside. So, where to start?

  1. Identify where your KPIs are being used as a tool and where they have turned into goals by themselves. Once you identify which ones are making more harm than good… let them go. It won't be easy, lot’s of research strategy, planning and decision making will be involved, but it’s worth the try. Don’t worry, you are not the first to go through this ordeal but trust me, you have to.
  2. Get some help, create a team, involve other departments. You are not alone and your department is not the only one suffering this problem. In my experience these problems are systemic. You need to raise awareness about what’s happening, create a cross-departmental team, and involve the right level of management. Because you will need to be empowered.
  3. It’s not only making the changes, but it’s also fostering the conditions for the changes to growth. Any new measurement system/ metric will fail to be implemented if you don’t think about how it will be adopted, assimilated and evolved by your teams and your users.
  4. Keep moving. During your journey, one of the dangers you will find will be to think you have found the perfect solution, something that really allows you to substitute NPS. There’s no such thing, there’s only transitional design.
Thankfully, in your case, there’s no Sauron involved… (right?)

To close things up with one tip I’d recommend you to follow no matter what… start your journey today. Remember, no matter how long a journey is, every journey starts with a first step.

“It’s a dangerous business, Frodo, going out your door. You step onto the road, and if you don’t keep your feet, there’s no knowing where you might be swept off to.”

J.R.R. Tolkien, The Lord of the Rings

P.S: Its fun to think that most of the arguments given in this article can apply to other metrics. For some of you, your silent murderer will be Churn or Time to Market. Just remember, tools not ends.

Also, I found this reference which for me is an adaptation of Heisenberg uncertainty principle to statistics, it’s the Goodhart’s law: Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.

--

--

joako

Narrative designer for a living if that makes sense to you. If not I’m a jack of all trades and a storyteller who loves felines.