Friday, November 10, 2006

Metrics

http://www.joelonsoftware.com/items/2006/11/10b.html
I love Joel's send up of management consulting applied to software development through the use of metrics.

My question is a real one though. The CEO of one of the places where I work as a "working" consultant (in terms of actually doing productive work, not just analyzing what other people do) just downsized a whole bunch of smart people. He wants to look at our metrics- including things like customer satisfaction. That's going to be tough to measure. How do you measure customer satisfaction in such a way as to rate people on it?

I think step one for me is figuring out who my customers are...I am taking over a project that one of the downsized guys was running, and I haven't even met some of the key stakeholders. When I do, I am going to be sure to remind them to send a monthly note to the CEO reminding him of how satisfied they are! If he gets 5 or 6 of those notes per month, that would be a good metric.

Of course, then the only way to show continuous improvement is to increase the number of messages, or the intensity of the satisfaction they convey. So, I guess I should only tell one person the first month, and then two people the next month. And, I'll need a satisfaction score in order to quantify the satisfaction level. It should probably be an integer, so that it can go negative to indicate dissatisfaction, not that I would tell any of the dissatisfied to send email, they would just do it on their own anyway, and could include a little number. Then he just sum the satisfaction level to get total satisfaction index. With multipliers added to gauge the relative importance of the people that are being satisfied, maybe the multiplier would be their salary divided by 100k.

Hmmm, I don't know how else to do it, but I am afraid if I pose this, I will look like a sarcastic jerk...even though it's just the best I can do sometimes.

2 comments:

Clarke Ching said...

Hi, Take a look at the "Net Promoter Score" book by Fred Reicheld. NPS measures loyalty, not satisfaction, by asking custoers to rate on a scale from 0 to 10 how they would answer the following question "would you recommend company x to your friends and colleagues". A 0 means something like I'm actively telling my friends not to use them. A 10 means, I love 'em and would drive them to the shop they're so good. It's suppposed to be one of the best ways of measuring customer loyalty. Some of our clients are using it to measure their success with external customers.

It's not so easy to use for internal customers but you can reword the question to something like "how loyal do you feel to our team?". You need to frame the question by first asking them to think of a company or even a boss who they feel no loyalty to vs. a company or boss they feel very loyal too, then asking you to rate you and your team from 0 to 10 accordingly. The important thing is the conversation that goes on around this discussion, not the question or number itself. You can ask every so often, though, if you're getting better or worse, and ask them what would make them rate you better or worse. The important thing is the conversation.

There is a hbr paper about NPS which tells you everything you need to know.

Matt M said...

Clarke-

Thanks so much for the leads on this information. I think loyalty is actually a good model here for an internal IT support group. I can measure this by seeing if the internal customer loyal enough to us to prevent them from outsourcing the project to someone else.

I am checking out the HBR article now.

Matt