During AgileCE conference I had a discussion with Robert Dempsey about measuring lead time. I never really thought much about the way we count lead time in our team and talk with Robert triggered some doubts.
What We Measure
As you already know on our Kanban board we have backlog, todo queue, several steps describing our development and deployment process and finally done station.
OK, so when do we stamp the starting date for the feature? We do it when the card goes from todo queue into design station, which is the very first column in our development process.
When do we stamp ending date then? You may take your best guess and um… you’ll be wrong. No, not when the sticky note is moved to done column. Actually we mark ending date when a feature makes its way to live column which is third station from the right.
And this is different from what you may have heard from Kanban gurus out there. Don’t blame me, I’ve already told you thought-leaders don’t know it all.
What we measure is time which passes before we start actual work on feature to the moment it goes live.
What We Don’t Measure
What is left behind then? First and the most important thing is the time feature spends in todo queue waiting for some developer to become free and start working on it. If you were trained by the father of Kanban – David Anderson – you’ve probably heard something different but stay with me, I have a good explanation. Or so I guess.
Another thing which is left outside is the last part of our process. There is documentation station where we (surprise, surprise) update documentation. This is done after pushing new version to production.
It looks like we cut something on both ends to make our lead times look better, doesn’t it? Anyway what we gather as lead time doesn’t really describe time which passes from the moment of decision to build the feature to the moment it is done-done. Thus another question arises.
Why, Oh Why?
The way we measure lead time came in natural way but chat with Robert forced me to make some explanation up to justify our practice.
Time Spent in Todo Queue
We left time spent in todo queue out basically because content of this station is changing pretty often. Sometimes feature lives there just for a day or so just to go back to backlog when priorities change. And believe me they do change every now and then. Sometimes a feature stays in todo queue for a longer time as it is always pushed to the second or third place because, well, priorities change.
There is another reason too. The basic reasoning for adding time spent in todo queue to lead time is that you should be able to tell your customer how long it would take from the day 0 (when they tell you they want the feature) to the moment they get it in production. It is pretty rare case when developers are able to start working on a new feature immediately so it is natural that some delay will appear when feature is waiting for a free developer.
I’m not convinced though. Actually very much depends on additional circumstances. The feature the client is asking for may be the only high priority thing to do at the moment but it is pretty unlikely. If the client asks just for one feature lead time will be different than if they asked about a list of ten features. If you have limit of 3 in todo queue you would need to put 7 features in backlog anyway and time they spend in backlog won’t be measured.
If you have a few clients and you need to balance your workload among them it becomes even more complicated since product owner (or whoever is your priority setter) has to decide which feature can be put on the top of the todo queue, which can occupy second or third place and which has to go into backlog.
Basically in any situation but the simplest measuring time spent in todo queue wouldn’t help us much and that’s why we decided to exclude it from lead time.
Time Spent on Documentation
With documentation the situation is a bit different. Documentation isn’t a real part of our product. We create it for our internal reasons – to make life of our sys admin easier and to avoid problems if he was hit by the bus. This means that, from client perspective, our product is complete as soon as it goes live. Even though we have to do some housekeeping later it doesn’t affect our ability to deliver.
Theoretically it might be a problem if documentation phase would be time-consuming and would steal time we’d prefer to spend for something else, namely deployment or testing. However looking at the past in the team it isn’t a problem so we may safely throw documentation out of lead time.
The next thing is using lead time to create some estimates but that’s the subject for another post.
If you liked this post you should like the whole Kanban Story too.
Subscribe RSS feed
Follow on Twitter
Subscribe by email
8 comments… add one
Hi,
I guess it just depends what you want to optimize. Obviously you care more about the development process, which might be fine in your case. Then it makes sense to measure how you do it, I simply wouldn’t call it lead time (as this is reserved for the time from ‘ordering’ until ‘delivery’. You could call it development time or cycle time or whatever.
But once you are focused on optimizing the whole (organization) it might make sense to really measure the whole lead time, as it gets everybody (in the organization) focused on reducing the lead time and thus to think twice before prioritizing and reprioritizing, as it costs, yes … lead time.
Cheers
Markus
Markus,
You’re right – it depends on what one want to optimize. If there was a problem with wrong features pushed into development, i.e. R&D features instead of ordered by client, we might have measured the whole cycle. Fortunately we don’t have any issues here.
By the way: you made me aware of one more thing. To measure lead time by the book I should start counting at the moment client calls me/sends me an order even if that means setting starting date for a feature which is in backlog. At the same time lead time for features which weren’t ordered by the customer should start… when?
Having the system described above in place I’m able to be pretty precise to tell how long it would take to deliver a feature (or a set of feature for this matter) for a customer and this is the primary reason to measure lead time. So even if my lead time isn’t a real lead time I can derive the latter from the former.
And of course I am aware that I tell just a specific story about specific team working in specific environment. But that’s exactly what my Kanban Story was intended to be.
Hi,
I enjoyed reading about your thinking about what to measure. My question is what benefit do you think you are getting fro
these measures? How do they help you understand how you are doing and how you might improve?
Regards,
Benjamin
Benjamin,
Basically there’s one goal of measuring lead time for us which is estimation. There will be follow-up post on lead time-based estimating next week where I’ll describe in detail how we do it. Anyway to make the long story short basing on average lead times I can get coarse-grained estimates as soon as we break work down to standard-sized features. And when I say “standard” I think “typical for our team.”
There’s option to derive precise estimates too, but it is yet another story to be shared.
Measuring lead time doesn’t really help us in improving our process or making problems visible. There is much more about that in Kanban board itself where you have process definition (columns) and constraints. Limits help us to keep finishing tasks, and to improve flow of work and of course limit WIP. Columns bring some order and have auto-alert mechanism: whenever someone doesn’t know where to put a sticky note it is time to adjust the board.
Self-improvement is mainly driven by informal ad-hoc discussions since we follow no-meeting culture so we don’t organize formal meetings to run retrospectives.
Actually lead times show us how we’re able to deliver average feature in less than a half of time we used to need but measuring lead time wasn’t the reason why it happened. It just shows the effect of actions triggered somewhere else.
Hi Pawel,
It was great speaking with you at AgileCE. I look forward to getting to speak with you again soon.
When I attended the Kanban Coaching Workshop with David Anderson the definition of “cycle time” was debated between he and another attendee. While having industry standard definitions is great, as long as everyone is on the same page in your company, which it sounds like they are, then you are in a great place. Kanban is meant to be highly flexible, and a system resulting from collaboration between business and IT folks. It sounds like you’ve achieved that as well.
If it’s working for you, and you are measuring what is important to you, and you can tweak the system, then I’m very happy it’s working for you. Keep doing it.
Robert,
It depends much on what you use lead time for. If, and this is our case, just in sake of estimation it isn’t even crucial to have everyone on the same page. It’s enough when folks who produce estimates for client/sales team understand what cycle time or lead time really means.
One of things I love Kanban for is its flexibility. I don’t say people should count lead time in a way we do. Actually, as you point, they should do it in a manner which work for them and is understood by them. And of course allows them to derive reasonable estimates.
I understand David’s points on the subject but I still don’t have a clear answer what how, in David’s scenario, lead time should be counted when you have more features that you can put in todo shortlist. Probably the answer is “do what works for you” but still I’d eagerly hear to a story of someone who did it in practice.
We have a story which analysis has been completed, so we moved it from ‘Analysis Pipeline’ to next state ‘Development Buffer’.
Then the Product Owner realized that some definitions were missed, so.. should the story be moved back to ‘Analysis Pipeline’?
We have a property which is ‘Analysis Completed On’ (that we use for metrics.. to calculate the time stories spend ‘In Analysis’), that would be reset. Is that ok? Or should we not move the story back to ‘Analysis Pipeline’ and complete those pending definitions leaving the story where it is.. in ‘Development Buffer’ state?
Kevin,
Well, the question is more for you than for me. There is no hard rules here. We don’t move features back in any situation (besides moving them back from todo queue to backlog). If we realize we forgot to do something on earlier stages the sticky note stays where it is while we update missing parts.
However you say you have a development buffer. It means it can make more sense to move the card back to analysis station as, if I understand correctly, you haven’t yet started development of the feature.
We generally don’t move sticky notes back since it might generate some problems, like breaking limits etc. If it did make sense and in specific situation would be source of new issues I probably wouldn’t object. Well, we actually might have done it once.
As a rule of thumb: if the situation happens by exception it doesn’t really matter much which path you’d choose. If it happens more often you may want to establish more specific rule. However that would mean you have a problem with analysis, not with the board, and you might want to fix the real problem instead of preparing standard workaround.