GovernIT

Observations on IT Metrics

I recently read an article on IT measurement titled, “IT Metrics: Feast or Famine.” It is an interesting piece, emphasizing that metrics should exist for a reason and it makes sense to periodically check if that reason is itself still valid!

As a long time designer of measurement systems, I have a few observations on IT Metrics, which I share with you below.  Some of them are humorously written, but true nevertheless:

• A useful set of metrics cannot be defined in one go. It takes a minimum of n² iterations to arrive at a stable set of metrics, n being the number of stakeholders who will report the metrics
• Metrics that measure the positive side of a transaction are more easily accepted. For example, instead of the number of test failures, the number of tests passed stands a better chance of being accepted
• The reporting format is usually the most debated part of a framework. Since this is what everyone will finally see, there will be much angst on how the information is represented. In many cases, the success or failure of a measurement system is measured by the reporting framework
• A basket of metrics, with freedom for managers to select different metrics goes down better than a small set of core metrics that are inflexible
• Senior management always want comparison – between projects, projects in other departments and against industry standards (specifically in the same type of business). Without this “referential”, no metrics program will be accepted, even though everyone tells you to start small
• Someone will invariably ask for traffic light colors in your reporting, even though you stress a thousand times that no metric should be looked at in isolation
• Someone will invariably ask for a “Dashboard”, no matter the contents

No related posts!: Ask Sridhar to write some more :)