≡ Menu
Pawel Brodzinski on Software Project Management

Flaws of The Chaos Report

Jim Highsmith criticizes the methodology standing behind The Chaos Report provided by Standish Group. His main points are:

• Wrong qualification of projects. While The Chaos Report treats all projects which overrun budget and/or time as challenged, Jim would like to see them rather as successful. He sends us to business background, different goals projects have to achieve and constantly changing environment.

• Wrong priorities on recipe for success. Jim would like to see competent staff as one of the essentials, while Standish Group focuses on executive support, user involvement etc.

I agree with the latter, but The Chaos Report was never a manual of achieving a success for me. It was rather a knowledge base about current state of IT projects. From that perspective the more important is the first argument Jim makes – the one which is hard to accept for me. When I look at projects which are finished I usually ask myself several questions. Was that a success? Could the time/budget overrun was smaller? Is the 3-month delay in the final acceptance justified with changes of business environment?

Using common sense and not giving excuses answer usually is: “We could have done it better. We should have done it better.” Running away from statistics, that’s my definition for challenged projects. With that definition statistics aren’t far away from what The Chaos Report brings.

Yes, you will find projects described by Jim Highsmith, which should be qualified as successful even though time or budget has been overrun. However, from my experience they’re much less often than situations where overrunning one of constrains is a sign of something wrong happening in projects. In my opinion The Chaos Reports bring quite real overview of quality of IT projects. Trends which are confirmed by the last report from 2006 don’t differ greatly from what you can see every day. Unfortunately, that’s nothing to be very happy about – there’s constant improvement, but barely slight.

in: project management

3 comments… add one

  • Craig June 11, 2007, 5:21 am

    A similar point is this;

    If a project is say 20% over budget or over time is that a failure of the project? of project management? Or of the initial estimates and planning?

    If the project still makes you a god return on investment (or whatever it is trying to do) how does that equate to failure?

  • Pawel Brodzinski June 11, 2007, 5:48 am

    From a business perspective even project which was delivered 100% over time and 200% over budget can still be successful. But it is not when you look from project management perspective.

    The Chaos Report focuses not on profits software project bring, but on the process of developing and implementing projects. From that perspective significant budget or time overrun makes the project “challenged” (not “failed”).

    Yes, we could discuss what exactly “significant time or budget overrun” means, but there’s no strict definition here. I know projects with half-year time span where overrunning deadlines for two weeks (which is less than 10%) would make them “challenged.” On the other hand you can find projects where 20% over time isn’t a real issue (e.g. because schedule was rather loose from the very beginning).

    I believe the Standish Group knows all of that and methodology of research addresses those issues.

  • Steve Hovland March 11, 2010, 8:49 am

    In the construction world, a project that exceeds budget may fatally damage the contractor, even though the customer will have a successful building. The poor reputation of IT in the business in part comes from poor performance on budget and schedule.

Leave a Comment