≡ Menu
Pawel Brodzinski on Software Project Management

Good Waterfall Is Better Than Bad Agile

Agile

Lech brought an interesting subject recently: isn’t running an R&D project with heavy-weight, structured approach extremely difficult?

We use to think that we need very flexible approaches for projects which aren’t defined very well, thus agile being frequently a tool of choice for R&D projects, which bear a lot of unknowns by definition. After all can you really produce reliable plan for a research project? It is research; you don’t really know what you’re going to end up with.

But we fall in the trap of simplifying things. When we think about heavy-weight approach to project management we instantly see BDUF waterfall project with no checkpoints on the way carried through without taking changing conditions into consideration. Yes, there are still a lot of projects done this way but this isn’t the only way of running projects non-agile way.

The trick with R&D projects is that you wander through unknown areas. It doesn’t mean you can’t plan you journey though. It doesn’t mean you can’t assess your progress and change your course along the way. It doesn’t mean you can’t measure where exactly you are or compare that against expectations which were set up front.

What more, to some point agile, when done well, enforces us to do exactly that. But then every reasonable approach, no matter whether it’s heavy- or light-weight, does the same. We can have inspect-and-adapt attitude with pretty much any project management approach used these days.

The real problem is we often misuse tools we have. Project management methodologies aren’t an exception here. When we fail to apply our project management approach reasonable it doesn’t really matter which one we choose – the result will always be poor.

So my answer for the problem from the beginning of the post is: as long as your project management approach is well-organized and reasonable your R&D project will go well. It doesn’t matter much whether the method is heavy- or light-weight. However if you fail to apply the approach right way don’t expect much of success in your project even if the label on you’re a tool of your choice says “agile.”

In other words good waterfall is better than bad agile.

Since I expect “but good agile is better than good waterfall, especially for R&D projects” kind of argument I’ll answer it up front: it depends. It depends on many factors, starting with your domain, going through organizational culture and ending with people you work with. And yes, we can discuss each case separately.

in: project management, software business

19 comments… add one

  • Glen B Alleman January 26, 2011, 8:02 pm

    We work a $300M US Government BioPharma research program that is managed using standard DoD procurement Earned Value. The science guys are as happay as they have ever been, They know what to do during the coming period of performance for their work packages, they get tangible feedback on their physical percent complete weekly and formally monthly, they know their budget to the penny, they know their expenditures to date to the penny, they know their Estimate to Complete, their Estimate At Completion, and their To Complete Performance Index.

    The term “waterfall” is both a red herring and not allowed in US DoD procurement. Is this “agile” not in the way emerging requirements are. But self organizing and actually self directed – yes. The Statement of Work say what done looks like 18 months from now. There are intermediate deliverables and close cost accounting – it is the publics money.

    But not Waterfall and not pure emergent agile. But in BioPharma R&D this is a sweet spot for government money.

    At the same time they are doing cutting edge virus therapy research for world changing products.

    The Performance Measurement Baseline has work packages no longer than 44 days, with 100%/0% complete measures at the Task level, risk adjusted duration estimates, alternative planning paths as the technology emerges.

  • Mark Phillips January 26, 2011, 8:13 pm

    There is a common trap of confusing the marketing message of Agile with the actual practice.

    For R&D projects or other projects where there are high degrees of unknown, you can use iterative methods for defining work packages over time with “Waterfall.”

    Rolling wave planning gives you this flexibility while operating in a framework that provides control points, clear metrics, direction and the measurement capabilities Glen talks about.

  • Mike Ramm January 27, 2011, 12:30 am

    The methodology is just a tool, like a knife – you can slice your bread or you can cut off your finger. It’s not pancea. If you don’t understant the basic principles no methodology could save your project. So I agree with Pawel – if you are proficient in the methodology you use, you have better chances to succeed no matter what methodology it is.

  • Pawel Brodzinski January 27, 2011, 1:02 pm

    Glen,

    We actually discussed your BioPharma research program with Lech and it is one of situations which catalyzed the post.

    I used waterfall and agile in the post as general labels, as they are usually understood. As you probably know I’m not attached to names. However waterfall is widely used as infamous label for variety of heavy-weight approaches and this is what I refer to.

  • Pawel Brodzinski January 27, 2011, 1:05 pm

    Mark,

    And that’s exactly how I see R&D projects. Since you deal with much uncertainty you regularly need to check how are you doing and whether there are any adjustments needed. And yes, these decisions should be made basing on facts and not gut feelings.

    The thing people seem to forget very often is that agile is not the only way to do this.

  • Pawel Brodzinski January 27, 2011, 1:08 pm

    Mike,

    This reminds me one more thing – you won’t implement a new approach unless you have team buy-in. If people are comfortable and proficient with the method they use I’d be far from enforcing new one only because it might yield better results.

    I believe there’s enough screwed projects and screwed teams to fix that we don’t have to start with those which work pretty damn well.

  • David Wright January 27, 2011, 10:19 pm

    “Rolling wave planning …” I call it a Cascade.
    http://practicalanalyst.com/2009/10/01/cascade-qa-with-david-wright/

  • Craig Brown February 3, 2011, 3:16 pm

    Another point in picking and working methods; we are delivering in part of a larger context. We are a step in a value chain or process.

    And that takes us back to rule 1 for effective projects; play as a team.
    Which con-incidentally is also rule 1 in process analysis; look at the handover points. And if I were a programmer I’d say the complexity of a system is in the interfaces. (Any more similes?)

    Anyway, the point; Working effectively means working well at communications and handover points.

  • Chuck S March 20, 2011, 10:59 pm

    The term “Waterfall” was invented as a strawman. See http://www.idinews.com/waterfall.html.
    The quantity of upfront design is really not the issue, but rather the nature of it. Identify a spectrum of what is least likely to change in the project to what has the greatest risk of change. Focus your design efforts along that continuum. Design with modularity as a first principle – that will go a long way towards inoculating against change.
    If there is radical change in the project definition and you have spent time doing up-front design is more lost? You throw away design instead of code. I’m not sure I see the problem.

  • Jeff M April 16, 2012, 11:03 am

    @Chuck S

    Usually the changes only come after coding and usually during testing even. And by then, it’s not just a change in up-front design that you throw away with a change in project definition. you’ll be throwing away a whole lot more.

  • Derek Davidson May 2, 2012, 3:52 am

    An interesting post and I certainly agree with the headline that good waterfall is better than bad agile!

    As a methodology, agile has much to commend it but unless you implement it well, you’re subject to failure, as with most things in life.

    Overall though, your article seems to work on the presumption that agile is better than waterfall, under equal circumstances. It’s a view I tend to share but when I went to look on the Internet for why anyone might want to use Scrum instead of waterfall, I came up short so, I wrote a post about it. I’m hoping that it might prove useful for others, too:

    http://www.webgateinternational.com/2012/05/top-10-reasons-to-use-scrum-instead-of-waterfall/

  • Gregory July 23, 2012, 7:46 am

    Waterfall is better than agile, period.

  • Pawel Brodzinski July 23, 2012, 7:58 am

    @Gregory – Any arguments to support such a bold statement?

  • Grego July 23, 2012, 8:31 am

    @Pawel – I worked for a company for 7 1/2 years was #1 at what they did using the waterfall method. Projects were done on or under time and under budget. Good design prior to building is very important. In no other business do we run design and implementation in paralell. Imagine trying to build a house without a blueprint. No method is without problems, but in my opinion waterfall has proven itself over time.

  • Pawel Brodzinski July 24, 2012, 12:12 pm

    @Gregory – You’ve just given an example of 1 company that is successful doing waterfall. Glad you work for successful company but it seems you’re missing something — go read how often waterfall fails and how often more flexible, adaptive or agile methods are more suitable. You’ll find tons of such stories.

    Actually I know a story of successful waterfall approach too. One story. At the same time I know a dozen successful appliances of agile methods. And when I say successful I mean really successful — delivering value to customers at expected rate with predictable financing. Of course there are many, many more that are close to this and, again, there are more of them from agile side of the world.

    What more, even such conservative clients as Department of Defense seem to adopt agile methods too.

    Is this just a coincidence?

    BTW: do you care to share the name of the company, or at least describe what kind of projects you work on?

  • Gregory July 24, 2012, 1:06 pm

    My experience with agile is that it tends to disorder rather than order. Like I said before, imagine trying to build a house without a blueprint? You wouldn’t. We do not do this in any other industry.

  • Pawel Brodzinski July 24, 2012, 1:24 pm

    @Gregory – Imagine painting without a blueprint. Um, wait, bad example. Maybe, imagine writing a song… no. Imagine developing a garden, organizing holidays, furnishing a house…

    In construction, most of the time, we do have blueprints, yet I know people who built houses without of them. And when we are at it, there are lots of changes that happen during building a house that weren’t included in blueprints whatsoever. You change where the walls are, you change all the installations, you change the height of the house, you even change where the house is placed exactly (all the examples form the house I know very well — mine) .

    Besides I don’t exactly buy the comparison between construction and software development, but that’s a different story.

  • Gregory July 24, 2012, 1:31 pm

    @Pawel – Obviously you have the right to disagree. I don’t have a problem with that. I still say waterfall is better, hands down. Thank you for allowing me to post my opinion :).

  • Codewarrior February 15, 2013, 2:01 pm

    This sums up how awfully bad Agile is.
    http://www.dilbert.com/2013-02-15/

Leave a Comment