“What’s measured improves”
― Peter F. Drucker
The discussion about metrics is not a discussion about what we want to measure. It’s a discussion of what we want to improve.
Before a project or a program embarks on defining metrics, it must have a clear definition of what they want to improve. Metrics are a way of reflecting if the improvement is happening at all or in the right direction.
We need to ask the right questions to get the right answers.
For example, in case of a large program that is delivering with Agile, a few questions come to my mind:
- What is our definition of productivity? Is productivity in itself a goal ? or its productivity at a sustainable pace the goal?
- How do we define predictability ?
Does it mean stable throughput ? Does it mean the teams meet their commitment consistently sprint over sprint ? Does it mean the ability to predict what we can or cannot do, given the constraints? - Are our teams performing well? What does it mean “well” ?
- Are the teams getting better over time? Are we okay with a slow start ? Or need all engines firing from day 1?
- What is our definition of quality ? Is it building defect-free code or finding more defects early?
Is it getting frequent feedback from the customer thereby improving quality ? What is frequent ? - Are we delivering value ? How do we define value ?
These are a few representative questions, the list could be longer. These questions represent the end that we have in mind. Metrics are a means to that end.
Metrics communicate to the teams what’s important. Right metrics offer an opportunity to make sure that the program is in a healthy upward spiral, not a vicious downward spiral. They help us know if our vision of success is indeed being realized.
Metrics need focus. That will happen when there are few. A long list loses the charm and the focus. It becomes mechanical, routine, a bureaucracy. We cannot measure and improve everything – at the same time. So the value discussion must happen of what’s important at any given time. The importance of certain metrics may change as the program evolves and that’s fine. However at any given point of time, we need to focus on a few parameters – the things where attention is needed “now”.
Metrics decide the direction and success of the program. Choose wisely.