≡ Menu
Pawel Brodzinski on Software Project Management

More on Waterfall

More on Waterfall post image

My recent post on usage of waterfall caused some stir and expectedly so. One of its outcomes was an interesting discussion on Twitter with Markus Andrezak followed by his comment under the post. As I promised to continue discussion on the blog, here is a follow-up.

Markus points that there’s no point in resigning from using method which is superior in terms of building software, period. What more, as I pointed out that using waterfall could help to decrease cost in discussed case, Markus question was: how the hell larger batch sizes could do so? Well, that’s not exactly the wording he used, but that’s what he meant.

I should have expected that. There are a couple of things. First, to some point I wasn’t precise when I used the infamous W-word. I mean, my idea wasn’t to build the whole damn thing in one batch. Actually I didn’t think much about the way development phase should be organized in. My point about using waterfall was more about having isolated analysis, development and acceptance test phases separated with formalized milestones, accepted by the customer. This way the vendor doesn’t start building anything until the whole app (part of the app actually) is defined. Then, after it is built it is verified against specs and not against emergent requirements.

Second, and this should have been stated more clearly, for me using waterfall in this case was more of a way of fixing broken vendor-client relationship than a way of fixing development process, which wasn’t broken in the first place by the way.

OK, this requires a bit more elaboration, I know. Here’s more of a context. In this case vendor-client relationship is kind of long and big history. The discussed project is tiny, teeny part of it. For both sides it isn’t an option to crush a lance on such project. It also means going away from the project isn’t considered as a feasible solution. The project is fixed-price and the price is not going to change even by a penny. The scope was defined very vaguely and with no details whatsoever. Folks on client’s side naturally want as much as possible, even though they don’t precisely know how the app should look like. For whatever reasons they don’t want to, or can’t, be engaged in development process on the fly – they don’t verify biweekly demos, etc. They just want to see final prototype.

In other words, being on client’s side I would eagerly treat every delivered version of software as a prototype, see what is nice and what is not and then define requirements on the fly. At the same time I would probably enhance my domain knowledge so I can better answer what exactly I do need.

It would have basically worked great. It would have if it had been a classic agile contract where cost is based on real vendor’s effort. Unfortunately it is not.

If the money wasn’t a problem at all I would definitely advise using agile methods as with vague requirements and changing scope it is kind of obvious choice. The problem is with such approach cost sort of skyrockets.

And this is where we come back to good, old, hated waterfall which tells us to fix the previous phase before we start the next one. Fix the analysis/design before rushing to build anything. Yes, I do know that this method doesn’t end up building the best possible app. Yes, I do know that it leaves the problem of vague requirements solved suboptimal way, e.g. we define what done means at the beginning when we have the most limited knowledge. However, considering that walking away is not an option for any of sides, the best option is trying to make it win-win.

What does win-win mean in this situation? Well, it means reasonably good app within given cost. It means defining what is to be built, estimating the effort, cutting corners when needed or adding stuff when possible, fixing all those and building the thing according to specs. And then, verifying the app against specs relentlessly.

The goal isn’t to build the app which solves the problem in the optimal way, or to build the optimal app, or to build the app optimal way. It’s not the software development problem. It is relationship issue.

Now, I don’t say anything about the software development itself. Actually, despite the situation I would advise small batches even though with detailed specs and no expected changes bigger batches aren’t that painful. This part however will be neither seen nor appreciated by the client. It is kind of inside baseball thing from a perspective of the whole problem.

OK, this time I hope a long version of my explanation does a better work but, same as the last time, I’m ready for discussion.

Waterfall FTW!

OK, just joking…

in: project management

13 comments… add one

  • Markus Andrezak August 22, 2011, 1:03 pm

    Hi Pawel,

    sorry for you to wright this long posting. But I guessed that situation already from your first posting. And no – from my PoV no stir, no irritation ;)

    To clarify I’d like to tell you the points I still don’t get. Chronologically: I do get your point that you don’t want to deliver in one batch, but in – say – four. But I still do not see or read any reason why this should serve YOU better than, say 20 batches. OK, if you wouldn’t know how to manage iterations, I would understand why it would be cheaper/easier for you to work in 4 batches. But I guess you are familiar with handling small, frequent batches.

    What advantages and cost savings does any of you have if you decouple Analysis from Development from Testing from shipping – if you know how to work agile? Basically you are saying Waterfall is cheaper than small batch size, iterations etc. And yes, I do understand that I have to leave the client out of the equation.

    My point is: If the client does not care until the end – do what you want until the end. If you’re (internally) good in agile: Go agile and deliver the beast in the end. The client obviously doesn’t even care. And: if you are good in agile, how on earth would you tell your developers to decouple Analysis from Design from Development from Test from Deploy etc. (My guys wouldn’t even know how to do it anymore.)

    And then: Again, even if the client doesn’t care – you are going fix price. How do you manage risk for your company in a fixed price project in Waterfall? Right: In the middle of development you realize that you are soon out of time. But you have analyzed and designed the whole thing. Great achievement.

    What I’m saying is simply: if you are good at agile – go for it. If not, leave it be. If the client really doesn’t care it also shouldn’t have impact on you. But why you would change what you are doing best because the client doesn’t follow and care and how that would save money or how you would manage risk – I just don’t understand that.

    There are some more things I don’t understand: You are talking about managing the client, basically. Where does Waterfall help you better in doing this than X, Y or Z agile approach? You can be quite formal in X, Y and even Z! But: It’s your responsibility to do it – just as in waterfall. No excuses please ;)

    Another point I don’t get is why you are hammering along the “best quality” thing. You can happily discuss the expected delivery quality w/ the client and deliver to that expectation.

    In general I see the danger that you are selling “agile deliver” as some gold plating here rather than the pragmatic approach that it is. Agile is not intended to deliver best quality at higher cost, as you are implying. (Or do I get you wrong?)

    (BTW as a side note: when I was still in your business I was selling fixed price projects via Scrum all the time. And I loved it. It bought me all the trust in the world. I even go as far as saying that I think Scrum (or Kanban) are near perfect fix price delivery models!)

    But what I really think is that the Problem you describe is invariant of whatever approach you are using. If you don’t have trust you – well don’t have trust and then both sides don’t accomplish much. That seems to be fine. But the whole discussion regarding methodology doesn’t solve your problem. It’s a funny hook for some nice clicks ;) but actually the discussion ‘Waterfall would be cheaper’ doesn’t have a bit to to w/ the issue at hand. Except … you can show me how to save money w/ Waterfall …

    Cheers

    Markus

  • Pawel Brodzinski August 22, 2011, 2:10 pm

    “I guess you are familiar with handling small, frequent batches.”
    Um, how can you know??
    Sorry, couldn’t resist :)

    “Where does Waterfall help you better in doing this than X, Y or Z agile approach? You can be quite formal in X, Y and even Z! But: It’s your responsibility to do it – just as in waterfall.”
    This is where I believe we’re coming closer to understand better each other.

    Waterfall, well any formal method really as I used waterfall just as a rather well-known example, has some formalisms built-in which makes it easy/natural to introduce them into the process. It works the same way as pulling work in Kanban. You can perfectly do it using whatever method but somehow it is a native concept for Kanban but for the method X or Z not so much.

    Of course, I agree that formalizing approach in specific parts to cover vendor’s interest plus using the agile approach would work. Same as using formalized approach, which is a part of waterfall, along with the rest of waterfall. Actually the key thing, which you point by the way, is going formal where you need to go formal.

    You have my full support where you propose to use whatever software development approach team is most familiar with. However, specifying analysis, build and verification phases as separate instances and isolating them with formal milestones, like analysis acceptance etc., usually isn’t without an impact on the way team works. I mean when the goal is to build whatever is specified in a blueprint, well, that’s what you build. You don’t ask what would be the best possible option – you just read the freaking blueprint.

    My general feeling here is, that we more agree on the approach here than not. If that’s true, the main point for discussion is: how formalized approach (here: waterfall) can save you money.

    Consider two situations:

    1. You approach the project assuming the changes will come and you’re going to embrace them. So you build the app which is rejected once it is presented to the customer. So you redo it according to what the customer said. Unfortunately after a second demo it still seems that you’re far from satisfying the customer. They say you didn’t understand what they really meant the first time. So you redo the app… etc.

    2. You build the specs first. You beat the dead horse until you have something which is defendable. I mean you rule out most of the interpretation part out of the deal. Yes, it costs money. Yes, it takes time. But then you build the thing once, go through probably painful process of redirecting the client to the specs and acceptance test list etc and you’re done.

    If you know that the option number 1 would be so, and would not look like the other (nice) flavor of it where you have a reasonable and helping customer seeing much value in joint building their app you spend less effort going through the second path.

    Note: I neither used agile or waterfall words describing these situations because it’s not really about names but about attitudes or approaches we have. Yes, most of people usually label them with names and that’s why I used the names in the first place, but I could either way avoid them. The only problem would be I’d end up defining what I meant a number of times. It seems I do so either way, don’t I?

  • Markus Andrezak August 22, 2011, 2:51 pm

    Shortest comment: ;)

    You reduce it to two options. None of them will work. 1) is obvious. 2) You can not tell if the specs are right – only after you’re done.Just as in 1)

    See what I mean? It’s not down to the method. And it has nothing to do with money.

  • Pawel Brodzinski August 22, 2011, 3:11 pm

    Actually does it even matter whether the specs are right or wrong? What does it mean exactly?

    What I expect in this very case is to see the specs (meaning: scope) formally accepted. In details. Not like in “embrace change” but like in something totally opposite. Like in “if you define the wrong thing we will build the wrong thing, sorry.”

    From my perspective it is about money. Pay my $100 and you’re going to get a document which describes the work worth $75, considering I spent $15 writing the damn thing and $10 is my buffer/profit. Now, pay me the same $100 and tell my to build to the app which satisfies your needs and I’ll either compensate additional work which I might eventually do with one of our other deals or I accept the fact I won’t earn any money on the deal, etc.

    In the first case the main risk is my estimates are wrong and somehow I take this risk into account. In the second I don’t even know whether it’s feasible to build the thing within budget as it’s hard to define “the thing.”

  • Steffen Lentz August 22, 2011, 3:19 pm

    Pawel
    I don’t think this is a waterfall vs agile discussion.

    Let’s be a bit pragmatic and look at the facts:
    The client doesn’t know what he actually needs. Fair enough, that’s not unusual. In fact it’s even good if both sides recognize this.

    The thing is: If it’s really so much unclear, then there’s no point in developing any kind of full-blown software system, no matter which method is used. Even one agile sprint is at too high a risk of being thrown away afterwards.
    Waterfall will change that only in the way that the discussion a) is deferred and b) will not be about the software but about how to interpret the specification. Where do you see the win-win? Do you think the deferral is the way to fix the relationship? Or do you think forcing the client by formal means to accept a possibly useless solution is adding value for any party?

    I think if you really care about the relationship and about the money spent, then you should focus on sorting out the solution and not so much on the contract. Design-to-cost is still possible once you understood the problem.
    Why don’t you do a couple of moderated brainstorming and prototyping sessions? Start with a flip chart, that’s the quickest way to explain ideas and draft solutions. This is way cheaper then formalizing specifications or developing real software, and the direct interaction and strong focus on the solution will create trust between the parties.

    Regards
    Steffen

  • Pawel Brodzinski August 23, 2011, 12:27 am

    @Steffen – I agree it isn’t really agile versus waterfall discussion. It isn’t as vast majority of such discussions aren’t either.

    The axis of the discussion one of the concepts which is tackled differently in different method. Here, it is approach to specification. However I don’t feel bad about calling names as people exactly know what to think about specs in waterfallish methods compared to agileish methods.

    Anyway, waterfall doesn’t defer the discussion in this case. It kind of forces the client to decide: they go this way or the other and live with the decision. I know that most of the specs out there are highly interpretable but knowing that you can limit this risk vastly investing enough time to go into fine-grained details.

    Forcing the client to go formal bears a risk that the app won’t be perfectly adjusted to the real needs but it definitely won’t be useless. The world isn’t black and white – the world is gray. Fully useful and fully useless aren’t the only options. We talk here about a bit better and a bit less adjusted to emerging requirements. What more, as we discuss the software which is sort of related to tax law regulations, most of these emergent requirements will touch usability stuff.

    By the way: the vendor and the client tried prototyping you suggest twice already. As far as I know it didn’t end up with a result being much closer to what they had in the beginning as client’s requirements kept changing each time they saw anything.

  • Markus Andrezak August 23, 2011, 2:32 am

    Hi Steffen,

    brilliant comment, you actually said a lot of things I was looking for :-)

    The general logic structure of the posting is a bit misleading I guess, which makes it hard to get to the point.

    The issue of not ‘getting specs’ is invariant to any method, but the posting creates a method discourse around it. Logically, obviously you can derive any ‘truth’ from a false ‘statement’.

    Sp you reacted brilliantly by mentioning that neither of the methods is the solution but caring – or not – about the client relationship. Which is simply an economic decision outside of the agile / non agile topic.

    Cheers

    Markus

  • scot August 25, 2011, 4:08 am

    If the cost is fixed but the scope isn’t why not just propose to the client: the cost is $100, developers cost $1/hr, you get $100 of developer time. You will get $100 of developer value. If the client wants to putz about and change their mind every 30 hours of developer time, that’s their call. Make sure you are upfront about that risk, in writing.

    Waterfall does NOT force the client to ‘decide’ what they want. With a client like that they will sign off on the requirements, the design, and then in UAT decide they want something else and force you to build it *for free*.

    Seriously, the client is asking you, the vendor of programming talent, to take a massive hit for a “relationship”. That’s just exploitative. Walk away.

  • scot August 25, 2011, 4:20 am

    If the client has two choices, A and B, which direction to choose? What you want is a spike, as Steffan alludes to when he says to do ‘moderated brainstorming and prototyping sessions’ . Build a bare bones solution A and another for B. Don’t spend more than a couple of days on each, but charge time-and-materials for it. The client needs this “consulting” in order for them to work out what they really need. Ask the client to choose which one they like better, throw away both the spikes, and then build a proper solution based on its approach, fixed price.

    Oh, and here’s a big risk to *you* if you don’t do this. Reading the original description it smells like a classic death-march project. If it’s longer than a month or two, you could lose half the development team (i.e. they will become demoralised and quit). Even if they don’t quit, they are demoralised and much less productive, and they don’t think much of the company anymore because they put them on a shitty project. And if you don’t put *good* people on this project, it will fail. And if you do, you have lost expensive-to-replace talent.

  • Vin D'Amico August 25, 2011, 6:14 am

    Pawel, you are absolutely right in calling this a “relationship” problem not a software development problem. Thus, the key to success is mapping the development approach to the client relationship. In the end, you won’t settle on pure waterfall or pure Scrum (etc.). You’ll create an approach that works in the sense that it strengthens the relationship. The resulting software application may not be optimal but as long as the client is happy, it’s a win-win.

  • Pawel Brodzinski August 25, 2011, 6:32 am

    @scot – It’s funny how easily we offer our sure-shot advices not taking context into consideration. Make them sign time&material. Walk away.

    Well, let me tell you once more: neither one is an option here. Most of the time building software doesn’t happen in vacuum and we never build software just for the sake of building it. If the best, most proper, ultimate, super-sexy, you name it method (whichever that might be) doesn’t pay the rent I won’t go that way.

    Actually I wish I saw all these silver-bullet sellers advising companies to walk away from make-or-break deals or customers which basically feed the whole companies. It’s always about the context.

  • Vukoje Milan August 31, 2011, 6:23 am

    Hi Pawel,

    Excellent article. I came to similar conclusions from everyday experience.

    Customer doesn’t know what he wants, wants to know exact price and date of UAT. UAT will be formal and executed by specs. Customer won’t give you feedback in smaller iterations and will give vague and slow answers to tough (read crucial) questions. These are the facts that won’t change!

    Our goal is to make grate software and make money.

    From my experience only whey to survive this is heavy analysis before any development. This analysis is charged by hour and not estimated upfront. After the analysis, detailed documentation and detailed estimates are produced. And when I say heavy analysis I mean that it will be done by our best people (usually team leads), and they will sweat a lot while doing it. In this process of gathering information, we send loads of mail, and resend them few times until someone answers, we build simple prototypes, and we give detailed alternatives with pros and cons and implications to other parts of the system. We do all this to simplify and speedup Customer’s decision making.

    While doing all this and especially while UAT, we gather many interesting info and ideas that we later propose to Customer as new fixed price projects.

    So is this waterfall? Maybe, who cares, that’s not the point. Buy the way, we still do one week Scrum-BUT. :)

  • Pawel Brodzinski August 31, 2011, 3:43 pm

    @Vukoje – I’m far from generalizing. Not every client requires such treatment. What more, over time as adoption of agile methods progresses more and more clients understand agile principles and are willing to play their role in the process.

    Anyway, I know clients who require process similar to what you describe even though it usually ends up with suboptimal product. But then again: we usually don’t get points for building optimal products. At least not when we build custom software (products, and apps dedicated directly for end-users are totally different story).

Leave a Comment