Numbers considered harmful – Part One

Something strange happens to us when we hear a number. Hardly surprising, if we consider that we are likely to have been associating numbers with targets and achievement for many years. How many steps we can take, how high we can count, how far we can jump; how fast we can run; how ‘good’ we are at any particular school subject and at what age we can marry, drive a car or have our first legal sip of alcohol.

All very useful, yet sometimes we strip away the context in which we hear a number and automatically hold it up as a target*. Here is a true-story of misconstrued numbers from a few years ago, different flavours of which I am still noticing today:

Directors at an organization want to know how productive their development teams are. The teams have recently started using Scrum and XP, however the rest of the organization maintains their previous way of working. Developers work on Product Backlog Items in pairs and estimate in comparative points along a classic simplified Fibonacci sequence.

The department manager understands that one team’s velocity is not comparable to another, yet he needs to provide a number for reporting. A good approach, he thinks, would be to add up the number of points completed per sprint (the velocities) of all of the teams in the department and divide it by the number of pairs to give an average number of points per pair per week. When he does this calculation the answer is 5 – the average pair completes 5 story points of work each week.

Now somehow this number trickles down to the teams themselves, and somewhere along the line this number 5 that was merely a historical average becomes an expectation. A target. A stick. Fast-forward to the next Sprint planning, where we see some interesting behaviours.

Team A has 6 members (3 pairs). They begin taking items from the Product Backlog to work on in the sprint. They reach 15 points worth and are happy. They are confident that they could achieve more but they don’t want to rock the boat by taking more than the 5 points per pair that they perceive they have been ‘told to do’. This team has ignored the information they have and their own knowledge and drastically under-estimated.

Team B has 8 members (4 pairs). They now also believe that they SHOULD be aiming for 5 points per pair per week (a total of 20 points). Rather than using historical evidence of how fast they can go, they peel off 20 points worth of work from the top of the Product Backlog. It immediately feels like too much and they realize that they cannot commit to it. Yet the team now believes that they must be seen to be capable of completing 20 points per sprint, so rather than taking some work out of the Sprint, they feel honour-bound to take 20 points of work. A discussion takes place about which pieces of work cannot be included in the Sprint. These items are removed. The remaining items are ‘scaled up’ to make a total of the 20 points that the team perceives they have been told they have to achieve. This team has broken their comparative estimating.

Beware the power of numbers.

*John Seddon’s “I want you to Cheat” is a brilliant read on the subject.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s