Velocity & Retrospective Mean We Don't Need Process Metrics

The concept here is simple.  Teams will get better over time.

I agree on the surface.  A team repeating it's efforts iteration after iteration will get better over time.  But we want smarter, not just practiced.

This is the era of big data.  If you want to know something you need to measure it or, in the case of big data, collect the data.  Are velocity and the retrospective process enough that we don't need to measure or collect anything?  No.  Not only is there still a need for metrics, the difficulty of creating software demands better metrics. We may as well make it into a slogan: Metrics or die!

Retrospectives offer the perfect venue to use metrics and decide how to act on their analysis. The suggested pattern of a retrospective is: Set the stage -> Gather data -> Decide -> Close. Why not gather some of the data before the retrospective?

Retrospectives shine at the social aspects of improvement, but not so much at the process aspects.  Experiments about process are hard to come by and even harder to stick to.  Suggestions such as "speak up more" are easy enough to encourage.  Learning whether you could make your estimates better, well that's complicated.

One of the things that has always bothered me about the scrum burn-down chart is that it's based on "hours remaining."  I love the chart, but not the data. In many tools the recording of "hours remaining" throws away actual-hours-worked.  Your entering hours, why not collect actual along with remaining? I've always considered it a shame.

If we are going to measure we should know why we are doing it. Before you measure, know what you'll do with the data. If you don't know, you risk wasting everyone's time.

Big data changes that recommendation somewhat. Big data is about collecting data through automated means, which helps avoid waste.  The corollary to "know why you're looking for before you measure" is "have an idea of how you want to analyze before you collect data."  If you want to grope in the dark for meaning don't waste your team's time with it.  Find the hidden value then apply it, but until then don't take time away from the team.

Which metrics we collect and how is an open question for the team to discuss.

...more