Note: This post has been updated based on feedback in the comments. I was confusing lead and cycle time. Thanks Paul!
If measuring productivity is bad for software then what on earth should we measure? Initially I argued with Dan that it must include something about value delivery. His take is: that is a problem for the whole business, but if you want to look at software development in isolation then the key metric has to be about how quickly the team deliver stuff, but we know we don’t actually care how productive they were, so what does it mean?
It’s simple – lead time – how long does it take from when you identify your customer needs something to when they get it. That’s it!
If you team can deliver what the customer wants as close as possible to when they want it you will win.
So how do you measure this?
Here are 3 simple ways.
1. If you use a ticketing system like Jira, lead time is simply the time the issue was logged up until the time it was resolved. In my experience as soon as you do this you will realise half the stuff in your Jira system is out of date, and so the data is no good. Try approach number 2.
2. Whenever you add an item to the product backlog add the date (if you are using physical card, just write it on one corner). Whenever you ship a feature add the date. Now just subtract the two to get lead time. Of course that might not work for everything on your backlog today. Try approach 3 for that.
3. If you have a ton of stuff on the backlog you have no idea when it was added, and generally new stuff is not added, then try looking at just the cycle time. When a story is started in a sprint note the date, and when a story is release to customers note the date. Subtract the two to get your story cycle time. Remember this is just a piece of the lead time because it excludes the wait time it sat on the backlog for, but it’s a good place to start. See this link for a great explanation.
How can it be gamed ?
If you know anything about metrics, you will know they can and will be gamed as soon as you start measuring them. So how can you game lead or cycle time?
The two patterns I’ve seen are:
a) It’s not in scope. Team members can start to nitpick around scope, if their focus is on getting things done as soon as possible. The PO wants the text worded differently and the team request a new story for that because “it’s not in scope”. While in general this is not a bad idea, you probably want to avoid extremes. As a counter metric you could track how many new stories arise because of ‘scope creep’ and how big they are. If you’ve got a lot of very small stories being added every sprint, cycle time is probably being gamed.
b) High failure rate. In an effort to finish things quicker teams can unconsciously drop quality. We all know this, and so of course it can happen if teams are chasing a short cycle time. The best way to balance this is with a failure load metric that tells you how many things come back because of errors.
c) Some people think that to game this teams will want to break stories down as small as possible, particularly if you are only measuring cycle time in the sprint (option 3 above). However in my experience that’s a good thing, I have rarely seen a team break down stories too small. If you find this happening and the team have >20 stories in a sprint, maybe try measuring the full lead time from customer request to delivery.
How can we reduce our lead time?
The simplest possible way (which can be mathematically proven using queueing theory or demonstrated with the airplane game), is to reduce work in progress. Stop starting things and start finishing things. Here are a few ways you can do that to improve lead time:
- For Product Owners this means culling your backlog. Let’s be honest. Those features that have been sitting in your backlog for 2 years are never going to be important enough to get done. Be bold and delete them. Think about what your ideal lead time is for your product, and then delete anything older than that in your backlog. i.e. if you’d like a lead time of 3 months, take everything older than 3 months and delete it from your backlog. Trust me, if it is important it WILL come back.
- For teams this usually means working on only one or two stories as a time, and keeping stories small – it helps get them ‘done’ sooner. This means collaborating and having multiple team members work on the same story. If you think that’s impossible, you might need to look at your version control strategy or your system architecture and find ways you can make this possible.
- For releasing this means smaller more frequent releases. I guarantee you have a process to get a hot fix to a client if they encounter a sever bug in production. Why can’t that process be the way you release all software? The worst wastage in lead time is having a feature done and not making it available to the client.
— Update October 2016
If you enjoyed this blog post, you will love our book Kanban Workbook: A Practical Guide to using Kanban.
The book will guide you through implementing your own Kanban board and all the principles of Kanban, including metrics like lead and cycle time, in your own environment.