18 November 2008

Comfort in numbers

The quest amongst those who can afford them is about gathering metrics. In a risk adverse culture, metrics are like a warm blanket. They are comforting when you are working alone, and if things get rough you can pull them over your head and hide from the truth… or the scary monsters.

In scrutinizing the use of metrics I often wonder, is the goal to make the decision process simpler and less risky, or to make a better decision. In that spirit I would offer a couple of suggestions.

The terms research and metrics would seem to be hand and glove. Research is an annoyingly costly and time intense process to find out what you already know, right? But I digress and that would not really be my main point. Outcomes from research can only be useful as guidance. To expect research, whether it is the building of personae (qualitative), or the use of A/B testing (quantitative), to solve the problem is a bit naiveté. In the end a person must do the analysis and a person must make a judgment (more on this in a second). The research is just one component of due diligence and along with other work sets the stage for resolve - it must not and cannot make the decisions for you.

In a hierarchal culture we are often saddled with solving problems that as assigned to us. As those problems are given to us to solve or resolve, it’s worth bearing in mind that the consequences of our decisions may not fit the situation. A critical component of problem solving is asking the right question. Is it appropriate to question the validity of the problem you have been asked to solve… well, yes - if only to set appropriate expectations for your own impact. Will your answer improve the situation? It might, but only, if the right questions are being asked.

Lastly, judgment is much more that choosing between what’s behind curtain number one or the box that Carol is showing us. Judgment requires due diligence, it requires answering the right question, but also requires looking into the future, selling the decision, and most important – follow through. The relief you feel when you finally make that all-important decision is often accompanied by anxiety and second-guessing. The best cure for those troubling side effects is to carefully guide your decision into reality by managing a solid plan.