Pawel Brodzinski on Leadership in Technology

Whatever it takes to lead a team, build a product, and run a business

Tag: #NoEstimates

  • Story Points and Velocity: The Good Bits

    You get what you measure. The old truth we keep forgetting about so often.

    Story Points and Velocity

    One relevant context to remember this is when we measure progress of project teams. The set that was wildly popularized along with Scrum is story point estimation, most typically with Planning Poker as a method to come up with the estimates, and measuring velocity. In such a set velocity, which simply is a number of story points completed in an iteration, is a primary measurement of pace.

    I don’t say the whole set is evil. What is evil though is how it is frequently used. Story point is pretty much meaningless – the same story can be estimated 2 or 8 and both are perfectly valid sizes. This means that the moment someone starts expecting specific velocity they will get it. In fact, continuous improvement in velocity is as easy as pie. It’s known as story point inflation.

    The same thing will happen when someone starts comparing teams basing on velocity.

    And then there’s expectation for velocity to be predictable, which translates to low variability. If that’s the goal story point estimates will be gamed so velocity looks predictable.

    How much does it have to do with any real sense of sizing?

    OK, I hear the argument that these all are dysfunctions related to velocity. Fair enough. Let’s assume for the rest of this article that we are doing it right.

    Measuring Progress

    The problem is, that the whole activity of estimating story points doesn’t provide much value, if any. What Larry Maccherone’s research shows is that the best measure of throughput is simply counting the stories or features done.

    Let me stress that: it doesn’t matter what size we initially though a story or a feature would be. What matters is that it’s either completed or not. That’s it.

    Larry knows what he’s talking about. The data sample he had was from ten thousand agile teams, vast majority of them being Scrum teams. If there had been a quality signal in story point estimations and measuring velocity he would have seen it. He didn’t.

    So even if we do the whole thing right it’s just a complete waste of time. Or is it?

    Estimation

    One part of Planning Poker, or any other discussion about story point estimates, is validating all sorts assumptions about a feature or a story. Another is addressing gaps in knowledge about the task.

    These discussions can provide two valuable bits of data. Sometimes we realize that the chunk of work hidden behind a story is simply too big and should be split it. Typically it’s after someone provides input about complexity of a scenario.

    The outcome of such a scenario would be simply splitting a story.

    A different case is when we realize we simply don’t know enough to come up with any meaningful sizing. It may be either because we need more input or simply we are discovering a new land and thus our level of uncertainty is higher. The reason doesn’t matter.

    What matters is that in such a case we deal with more uncertainty than normally thus we introduce more risk.

    In both cases we get additional valuable information. Beyond that a discussion whether something is worth 5 or 8 story points is irrelevant.

    No Bullshit Estimation Cards

    That’s basically rationale for no bullshit estimation cards. I like to think of it as of “Story Points and Velocity: The Good Bits.”

    Instead of focusing on futile discussion and potentially driving dysfunctional behaviors we get a neat tool that keeps few valuable parts of the approach. And you get a little bit of sense of humor for free.

    By the way, there’s a more serious a.k.a. politically correct version too.

    It saves time. It provides value. And most of all, it makes you focus on the right challenges.

    And in case you wondered, you can get your own deck (or as many as you want really).

    esimtaion cards

  • Estimation Quality

    OK, so I am on yet another agile event. I’m sitting there in the last row and a guy on the stage starts covering estimation. That’s interesting, I think, maybe I’ll learn something. After all estimation is something that bothers me these days.

    Planning poker, here it comes. Yawn. People would pull the card with their estimate, yadda, yadda, yadda, they’d discuss and agree on a story point value. Yawn. The distribution of estimates would be normal.

    Wait, now the guy has my attention.

    So apparently he knows the distribution of the estimates. Good. How about checking what the distribution of lead times for historical data is. Because, you know, there are people who have done that, like Troy Magennis, and it seems that this distribution isn’t normal (Troy proposes Weibull distribution as the closest approximation).

    In short: if your estimates have a different distribution than historical data for the tasks that the team has completed the estimates are crap.

    Back to the presentation. The guy points how estimates for big tasks are useless in his context. Well, yes, they are. All the estimates in his context are crap from what I can tell. Then he points that we should be splitting tasks to smaller ones so planning poker makes sense and produce more reasonable estimates.

    Um, sorry sir, but you got it wrong. If you are using estimates that are distributed differently than historical lead times you are overly optimistic, ignorant, or both.

    How about this: I will not poke you in the eye with my pen but you will check the distribution of the estimates and the past lead times. Then you will recall some basic stuff from statistics course and stop selling crap that we can’t possibly answer the question: “when will it be done?”

    OK, rant aside. I don’t say that coming up with the idea how long it will take to build something is easy. Far from that. I just say that there are pretty simple things that can be done to verify and improve the quality of the estimates.

    Once you got the estimates look at their distribution. If it is different than what you know of historical data (because you’ve checked historical data, haven’t you?) the estimates don’t bear much of value.

    In fact, you can do better. Once you know the distribution of historical lead times you may use that to model estimates for a new project. You don’t really need much more than simply basic work break down. You can take the worst case scenario from the past data and assume the best case scenario is 0 days and let the math do its magic. No guesswork whatsoever needed.

    The credit for this post should go to Troy Magennis, who opened my eyes to how much use we can make out of limited data we have at our hands.

  • No Estimates Middle Ground

    The idea of no estimates (or #NoEstimates) is all hot these days. People would choose different parties and fight a hell of fight just to prove their arguments are valid, they are right and the other party got it all wrong. I’d occasionally get into the crossfire by leaving a general comment on a thread on estimation in general, i.e. not steering a discussion for or against #NoEstimates.

    And that’s totally not my intention. I mean, who likes to get into crossfire?

    What No Estimates Mean

    A major problem of no estimates is that everyone seems to have their own freaking idea what that is. Seriously. If you follow the discussions on the subject you will find pretty much anything you want. There are crusaders willing to ban all the estimates forever as they clearly are the source of all evil in the software world. There also are folks who find it a useful tool to track or monitor health of projects throughout their lifecycles. You definitely can find people who bring to the table statistical methods that are supposed to substitute more commonly used approaches to estimation.

    And, of course, anything in between.

    So which kind of #NoEstimates you support, or diss for that matter? Because there are many of them, it seems.

    Once we know this I have another question: what is your context? You know, it is sort of important whether you work on multimillion dollar worth endeavor, an MVP for a startup or an increment of established application.

    My wild-ass guess is this: if every party getting involved in #NoEstimates discussions answered above questions they’d easily find that they’re talking about different things. Less drama. More value. Less cluttered twitter stream (yeah, I’m just being selfish here).

    Is this post supposed to be a rant against discussion on no estimates?

    No, not really. One thing is that, despite all the drama, I believe that the discussion is valuable and helps to pull our industry forward. In fact, I see the value in the act of discussing as I don’t expect absolute answers.

    Another thing is that I think there is #NoEstimates middle ground, which seems to be cozy and nice place. At least for me.

    Why Estimating Sucks

    Let me start with a confession: I hate estimating. Whoa, that’s quite a confession, isn’t it? I guess it is easily true for more than 90% of population. Anyway, as long as I can get out avoiding estimation I’d totally go for that.

    I have good reasons. In vast majority of cases estimates I’ve seen were so crappy that a drunken monkey could have come up with something on par or only slightly worse. And last time I checked we were paying drunken monkeys way less than we do developers and project managers. Oh, and it was in peanuts, not dollars.

    It’s not only that. Given that kind of quality of the estimates the time spent on them was basically waste, right?

    It’s even worse. It was common when these estimates were used against the team. “You promised that it will be ready by the end of the month. It isn’t. It’s your fault.” Do I sense a blame game? Oh, well…

    And don’t even get me started with all the cases when a team was under pressure to give “better” estimates as the original ones weren’t good enough.

    Why We Estimate Then

    At the same time, working closely with clients for years I perfectly understand why they need estimates. In the case of a fixed-price contract we have to come up with the price somehow. That’s where estimates come handy, don’t they? There also is a million dollar question: so how much will I spend on this thingamajig? I guess sometimes it is a million dollar question literally…

    So as much as I would prefer not to estimate at all I don’t hide in a hole and pretend that I’m not there when I’m asked for an estimate.

    All Sorts of Estimates

    Another story is how I approach the estimation process when I do it.

    I would always use a range. Most of the time pretty broad one, e.g. the worst case scenario may mean twice the cost / time than the best case scenario. And that’s still only an estimate meaning that odds are that we would end beyond the range.

    Whenever appropriate I’d use historical data to come up with an estimate. In fact, I would even use historical data from a different setup, e.g. different team, different project. Yes, I am aware that it may be tricky. Tricky as in “it may bite you in the butt pretty badly.” Anyway, if, basing on our judgment, team setup and feature sizing is roughly similar I would use the data. This approach requires much of understanding the dynamics of different teams and can be difficult to scale up. In my case though, it seems to work pretty fine.

    I’m a huge fan of Troy Magennis and his work. By the way, despite the fact that Troy goes under #NoEstimates banner, he couldn’t possibly be farther from folks advising just to build the stuff with no estimation whatsoever. One of most valuable lessons we can get from Troy is how to use simulations to improve the quality of estimates, especially in a case where little data is available.

    Finally, I’m also fine with good old guesstimation. I would use it on a rather general level and wouldn’t invest much time into it. Nevertheless, it works for me as a nice calibration mechanism. If the historical data or a simulation shows something very different than an expert guess we are likely missing something.

    Interestingly enough, with such an approach having more details in specifications doesn’t really help, but that’s another story.

    On the top of that, whenever it is relevant, I would track how we’re doing against initial estimates. This way I get early warnings whenever we’re going out of track. I guess this is where you think “who, on planet Earth, wouldn’t do that?” The trick is that you need to have quite a few things in place to be able to do this in a meaningful way.

    A continuous flow of work gives us a steady outcome of delivered features. An end-to-end value stream means that what is done is really done. At the same time without continuous delivery and a fully operational staging environment end-to-end value stream is simply wishful thinking. Limiting work in progress helps to improve lead time, shortens feedback loops and helps to build up pace early on. And of course good set of engineering practices allows us to build the whole thing feature by feature without breaking it.

    Quite a lot of stuff just to make tracking progress sensible, isn’t it? Luckily they help with other stuff too.

    Nevertheless, I still hate estimation.

    And I’m lucky enough to be able to avoid it pretty frequently. It’s not a rare case when we have incremental funding and budgets so the only thing we need is keeping our pace rather steady. And I’m not talking here about particularly small projects only. Another context where estimation is not that important is when money burn-out rate is so slow (relatively) that we can afford learning what the real pace is instead of investing a significant effort into estimating what it might have been.

    No Estimates Middle Ground

    To summarize the whole post I guess my message is rather straightforward. There’s value in different approaches to estimation so instead of barking one at another we might as well learn how others approach this complex subject. For some reasons it works for them pretty well. If we understand their context, even if ours is different, we might be able to adapt and adopt these methods to improve our estimation process.

    That’s why I think the discussion is valuable. However, in terms of learning and improving our estimation toolbox #NoEstimates notion doesn’t seem to be very helpful. I guess I’ll stay aside in the middle ground for the time being.

    By the way, if we are able to improve our cooperation with the clients on the estimation I couldn’t care less whether we call it no estimates or something different.