Pawel Brodzinski on Leadership in Technology

Whatever it takes to lead a team, build a product, and run a business

Category: project management

Portfolio Management: The Search for Value

I am talking about the cost of multitasking pretty frequently. This discussion gets even more interesting when we are in portfolio management context. Why? It is so because we are dealing with extremes more frequently on that level.

Cost of Multitasking

Let’s imagine a situation when a software developer is dealing with three tasks concurrently. We know that it isn’t efficient. It may be based on our knowledge how multitasking affects our work but it may as well be our intuition.

Now, would we have the same intuition if we changed the context and we were discussing a team working concurrently on three different projects? Interestingly enough, we’d be looking for arguments why in such a context it isn’t that much of a problem. Stuff like: part of the team would work on one project and another part on another project. Sounds familiar, doesn’t it?

Thing we typically forget about is how team members would interact with each other. They wouldn’t think of themselves as of isolated sub-teams. They will be frequently looking for colleagues’ help thus they will make other people switch projects for a while every now and then. Finally, we have the coordination effort that has to done. Who is working on what, what is the status of everything, etc.

This gets even worse when the distribution of people across the projects isn’t fixed. Then people would be thrown to one or the other initiative depending on the current situation. Why is it costly? Zeigarnik Effect describes that we, as humans, have intrusive thoughts about stuff that we left unfinished.

In other words, if I change a project but I haven’t wrapped up my part in the current initiative I will likely be interrupting myself thinking about the old tasks. In fact, I don’t need any external factors to incur the context switching cost to my work.

There’s more than just efficiency penalty though. The teams that are working on more than one project deliver lower quality results, as Larry Maccherone points in the results of the research run across thousands of agile teams.

Cluttered Portfolios

Things get even more interesting when look at the big picture – not a single team but all the teams within one organization. It’s enough to ask two questions. How many teams are there? How many active projects or initiatives are run concurrently?

The answers with the ratio in a range about 10:1 (ten projects per team on average) aren’t uncommon. It means that our forces are spread very thin, the coordination effort is significant and the frequency of people switching projects is fairly high.

Another question may be: what is the percentage of people assigned to more than a single project?

Either way, in some cases we’d find that not only do we have more projects than teams but also we have more projects than people working on them. Some extreme examples I know of would show that there are 4 times more ongoing projects than there are people available to work on these projects.

How efficient is that?

No reasonable person would do such a thing on purpose. Yet, this is happening pretty often.

Project Attractiveness

If we assume the goodwill of people managing project or product portfolio, there must be something wrong with the approach we typically use to manage the portfolio. Let’s think for a while data we use to inform our decisions on starting new initiatives.

Obviously we know the client and, at least roughly, the scope of the project. On that basis we come up with the idea how much the project would cost and what revenue we can get out of it. How do we do this? Well, we estimate.

One problem is that, as Daniel Kahneman points, we are biased toward the most optimistic scenarios. Another one is when we do these estimates. Johanna Rothman phrases it:

If you fall for estimation as your way of valuing projects in the portfolio, you are doomed to fail. Why? Because you are trying to predict the cost or the date when you know the least about the project.

Typically, we base on insufficient data and use our flawed estimation skills to come up with the measures that are supposed to help us assess value of projects. That doesn’t sound like an extremely useful approach if you ask me.

The problem is that pretty frequently that’s all we have in our toolboxes: an unreliable total cost estimate and expected income based on that very estimate. If we are really lucky we can say a bit about high-level risks too. However, if nothing else is in place the only application of risks assessment on this stage would be tweaking the income expectations so it includes potential screw ups.

All in all we end up looking at projects through attractiveness glasses. It’s all about how much profit we can potentially earn doing this or that project. Given that we model income on expected total cost almost all of the projects will look reasonably profitable, thus all of them will be considered attractive.

There’s a pattern our brain follows, called What You See Is All There Is (WYSIATI). It basically says that whenever we are judging something we take into consideration only evidence that we have at hand and omit data that we could potentially gather to inform our decisions better. In other words, if what we see is cost and profit we won’t automatically be looking for other data points that can tell us something about value of the project. We will just do our judgment on information we have at hand.

If we put WYSIATI on the top of our limited and unreliable data almost every initiative would look appealing. It is no surprise that we end up with heavily overloaded portfolios.

Where Value Is

The question we should be addressing is not the one about expected profit, but about expected value. The problem is that the latter is more difficult to answer. On different occasions, during my presentations, I’ve had a chance to ask audiences whether they know what the value of features they build for their clients is. Few people do.

We simply don’t think about our products and projects in such terms. Even when we do though, it solves only a part of the puzzle. One oversimplification I often notice is that we boil down the discussion about value to the users or clients only. The problem is that both parties will define value differently. Let me give you an example.

At Lunar Logic we build web applications for our clients. We can build a product for a client that delivers valuable features in a predictable fashion. We can even help the client shape the product so we avoid the non-value adding functionality. The client would be happy – they’d got what they define as value. At the same time though, the client may simply be a toxic person who would make the team’s life miserable. Now, on our end, depending on a situation, it means that overall value of such a project may as well be negative: we would have gotten the money (one aspect of value) but lose a part of the team (another aspect of value). It is likely that the former wouldn’t compensate for the latter.

I don’t have all the answers for the questions about value. The cases when I’d disagree with how our clients define value aren’t rare. On my end, while I can say what we value as the company, sometimes it is very difficult to assess value for initiatives we undertake. It’s not as bad as it looks though.

We don’t need to define value in absolute numbers. If the goal is to improve the way manage portfolio most of the time we will need to answer only relative questions, e.g. which project is more valuable given the criteria that are important in a given context?

In fact, the most valuable outcome of the discussion will not be the exact assessment. It will be a discussion itself as it will likely cover the areas we typically ignore, thus will yield in the better understanding of the situation.

It won’t happen though unless we understand the limitations of our decision-making process on a portfolio level. Otherwise we will keep doing even more of the same thing make our teams even more inefficient and miserable.

December 13, 2013
Multitasking Teams

There’s one question I ask pretty frequently during my presentations or trainings: do you think that context switching comes for free? I’m yet to see a single hand up after the question.

Not that I believe that this is universally accepted point of view. In fact, there is a follow-up question which is: who has a boss who thinks that context switching comes for free? I do get positives for the latter.

I’d also assume that some people, even if they believe in free context switching simply wouldn’t raise their hands. Peer pressure, you know.

I’m happy with the assumption that awareness of costliness of context switching isn’t ubiquitous and definitely there are different assessments of how costly it is exactly. Still, I’d say that basic understanding that we pay a context switching tax is pretty common.

After all, it oh so obvious to predict how texting on the mobile while walking through a crowded street is going to end. Or playing Angry Birds while driving. Or driving, having a call, lighting a cigarette, overtaking another car and shifting gears, all at the same time (a true story; I’m glad that the passengers made it alive).

It gets a bit less obvious when it comes to our workplaces. On occasions you’d hear things like “do it in the meantime” or “can’t you work on both of these things concurrently?” (Sure, I can. As long as you want both of them to be late, that is.)

The real fun starts on yet another level though. Concurrent projects. When we aren’t discussing individuals but teams and not atomic tasks but projects, suddenly the assumptions that concurrent work on them doesn’t hurt becomes surprisingly common.

The reasoning is that one team member will be working on one project and another team member on the other project. I’m always astonished that this thinking pattern is there even when there are more projects than team members… Anyway, let’s assume the situation is not that dramatically bad.

There is still a problem of multitasking on a few different levels. First, there’s planning and coordination. Who should do what for how long? Even if team members typically do have comfort of being able to focus on a single thing there are people on the team who constantly switch context between all the projects that are run.

Then, there’s regular communication. Typically ad-hoc communication can be a distraction. We willing pay the price for the distractions because we get value of those discussions. They are relevant for people because they touch the matter of the project they run. Well, as long as it is about the project they run. That isn’t necessarily true if a team run a few projects.

Finally, there are situations when people would change the context of the project even if we don’t plan they would. What happens when someone is stuck? They’d ask for help. Whom? The person who is likely to help them. Does it mean only people from the same project? Not really.

Obviously we’d get some of that even if a team works on a single endeavor, but such cross-team interactions are usually way less frequent, thus way less costly, than those within the team.

This is sort of a gray area that is often forgotten even in organizations that are aware of the cost of multitasking. This is one of the reasons why visualization of project portfolio is so important. Each case where a team is coping with a few different projects or products at the same time should at least spring a discussion, as this is an obvious inefficiency.

Multitasking on a team level is no less painful than on any other.

November 12, 2013
Estimation Quality

OK, so I am on yet another agile event. I’m sitting there in the last row and a guy on the stage starts covering estimation. That’s interesting, I think, maybe I’ll learn something. After all estimation is something that bothers me these days.

Planning poker, here it comes. Yawn. People would pull the card with their estimate, yadda, yadda, yadda, they’d discuss and agree on a story point value. Yawn. The distribution of estimates would be normal.

Wait, now the guy has my attention.

So apparently he knows the distribution of the estimates. Good. How about checking what the distribution of lead times for historical data is. Because, you know, there are people who have done that, like Troy Magennis, and it seems that this distribution isn’t normal (Troy proposes Weibull distribution as the closest approximation).

In short: if your estimates have a different distribution than historical data for the tasks that the team has completed the estimates are crap.

Back to the presentation. The guy points how estimates for big tasks are useless in his context. Well, yes, they are. All the estimates in his context are crap from what I can tell. Then he points that we should be splitting tasks to smaller ones so planning poker makes sense and produce more reasonable estimates.

Um, sorry sir, but you got it wrong. If you are using estimates that are distributed differently than historical lead times you are overly optimistic, ignorant, or both.

How about this: I will not poke you in the eye with my pen but you will check the distribution of the estimates and the past lead times. Then you will recall some basic stuff from statistics course and stop selling crap that we can’t possibly answer the question: “when will it be done?”

OK, rant aside. I don’t say that coming up with the idea how long it will take to build something is easy. Far from that. I just say that there are pretty simple things that can be done to verify and improve the quality of the estimates.

Once you got the estimates look at their distribution. If it is different than what you know of historical data (because you’ve checked historical data, haven’t you?) the estimates don’t bear much of value.

In fact, you can do better. Once you know the distribution of historical lead times you may use that to model estimates for a new project. You don’t really need much more than simply basic work break down. You can take the worst case scenario from the past data and assume the best case scenario is 0 days and let the math do its magic. No guesswork whatsoever needed.

The credit for this post should go to Troy Magennis, who opened my eyes to how much use we can make out of limited data we have at our hands.

October 25, 2013
5-Minute Board Test

I discuss different Kanban boards or task boards with their teams pretty often. It’s not uncommon for me to see a board of a team that I know nothing of. I like these moments. I typically challenge myself to make sense out the visualization as I stare at it.

Some parts are rather obvious. Typically the process isn’t that difficult to figure out. Usually I can tell who is working on what at a glance too. On the other hand something that should be obvious, but often isn’t, are classes of service – what they are and which items are of which class. And then there are lots of small different artifacts like blockers, comments, subtasks and what have you. Those do require some additional explanations.

One of common sins of teams adopting Kanban is making visualization too complex. We do say visualize everything, but obviously it means “as long as it adds value.” Boards tend to get more and more complex over time as we typically want to track more information as we understand better how the work gets done. We really don’t need to complicate things more than necessary on the day one.

So what makes a board a good one? The board is good when it does its job as an information radiator. This happens only when it is easy to comprehend and understand for the team members.

This is a core of my 5-minute test to verify whether the board is designed well. The test is simple.

A random team member should describe what is happening on the board to an outsider – someone who neither is a part of the team nor works with it regularly. If the outsider has any questions about specific things in the board they should ask and get the answers from the team member. The whole activity is time boxed to 5 minutes (thus the name).

The board passes the test if at the end of the task the outsider can tell what exactly is happening in the team: who works on what, what are the problems (if any), what is the progress of work, how the team knows what to do, etc.

One could argue that for bigger, more complex boards 5 minutes is not enough. I don’t agree. When we are interacting with the boards we rarely have more than just a couple of minutes. Of course team members would know part of the design without any explanation. That is why I’m giving an outsider full 5 minutes.

The more stuff there is to take into consideration when making routine, atomic, everyday project decisions the bigger the chance someone would misinterpret part of that. That’s the point where understanding the board, and the process, isn’t fully shared across the team anymore. That’s the point where people start acting differently despite identical input.

This is exactly the problem we tried to address when introducing information radiators in the first place. If the board doesn’t solve that problem it simply doesn’t work.

Would your Kanban or task board pass the 5-minute test?

October 22, 2013
What Does Project Management Mean to Me

I was poked to answer the question on meaning of project management by Shim Marom. Since my work, and this blog, evolved away from covering what can be called traditional project management approach long ago I thought it may be a good occasion to restate the purpose of the blog as well.

Se here it is.

Getting the stuff done

The simplest answer, from the top of my head, is that project management is all about making it happen. Of course it’s not a single person’s effort. It’s more orchestration of all the stuff happening around but at end of the day the discussion always starts with what got delivered.

But wait…

“There is nothing so useless as doing efficiently that which should not be done at all.”

~Peter Drucker

Getting the right stuff done

The discussion on the right stuff is primarily product management / product ownership thing but most definitely it’s not beyond the scope of what I consider project management. After all, if you’re doing the wrong thing there’s no credit to anyone. A project manager isn’t an exception here.

So yes, it starts with grasping the business case, understanding how the project corresponds to it and actively working on that match.

When we are on the right thing though, it’s inevitable that doing right thing versus doing thing right argument will pop up…

Getting the right stuff done right

Getting the thing right is about the quality. If there’s no quality built-in it’s going to come back to you and bite you in the butt. Painfully. While I’m as far as possible from introducing oppression tools like quality procedures and such stuff, quality doesn’t automagically happen. One needs to create environment where high quality thrives and is encouraged.

What exactly the high quality means is obviously contextual but from my experience it’s not that difficult to describe the quality in any given context.

If nothing else you can always make the next project better than the last one.

Getting the right stuff done right and improving

This brings me to one of the areas I sunk into when I started flirting with Kanban – continuous improvement. There should be no pride in doing same stuff again and again. Unless you’re a freaking genius and know it all about project management, that is.

Personally, I couldn’t be farther from that. That’s why I have an ambition to improve. Improve the way I work but also improve the way everyone around works. It’s not that easy as it sounds though. The prerequisites are: understanding how the work gets done and learning like hell.

Without understanding work we are like children in the fog – clueless and lost. Learning means that we broaden our horizons and go beyond that carrot and stick method our first managers were using all the time.

Getting the right stuff done right, improving and building trust

This one is a classic last but not least. In fact, I believe that without trust you will struggle across the board. And I mean trust here in a very broad sense. One, it is about building trust within a team. Without that people wouldn’t become vulnerable, thus would restrain to become transparent. Without that a project manager would always have limited information of what’s happening in an endeavor they’re supposed to lead.

Second, and more importantly, it’s about building trust with a client. That’s where the real fun starts. It’s really rare when a client would take the first step and will start to be open, transparent and vulnerable. The good part is that they will likely do so once they see it on the other side. But this is fear that every project manager, and every team member, has to overcome by themselves.

The example, as usually, should go from leaders.

So this is it. I believe this definition works well for me right now and the stuff I deal with fits the picture neatly. It doesn’t matter whether I talk about Kanban, organizational culture or team management. It doesn’t matter whether I deal with a portfolio level, a project level or a task level. It’s all there.

Project management is about getting the right stuff done right, improving and building trust.

September 24, 2013
Options, Options, Options

I think the first time I’ve heard that we should consider that a project backlog is simply a set of (unexercised) options it’s been from Ellen Gottesdiener It took me some time though to translate this, otherwise attractive, concept to something more down-to-earth in my own context. The catalyst in this case was one of Kanban Leadership Retreat sessions that covered portfolio management.

Let’s imagine a company working on multiple projects. The organization size may vary, but let’s make it at least a couple hundred people strong. Such a company would have a bunch of project in progress but will also continuously work on selling more work to keep the business running.

Every now and then, one of ongoing projects will get finished. What typically happens then is the company has some free capabilities. At the same time sales department is busy closing the deals. If the organization is extremely lucky and we are living in a fairy tale the pace of project completion will be exactly equal to the pace of new project arrivals.

I have bad news though. We don’t live in a fairly tell. It’s even worse – most of the time we are not extremely lucky either.

What typically happens is one of two things: there either are some free capabilities and no work that can fill this gap or the opposite – there is more work that should be started than can be handled with available capabilities. In fact, the situation dynamically changes from one to the other and back again.

I want to focus here on a scenario with more incoming work that can naturally be absorbed by the org.

One, sadly very common, approach is taking whatever comes from sales funnel and then pushing hard to make it fit, no matter what available capabilities are. Guess what – wishful thinking that people would handle more concurrent work doesn’t work at all. Conversely it only makes the problem worse as it forces teams to multitask which never comes for free.

We can do better, can’t we?

The opposite approach would be taking available capabilities into account and committing only to whatever the organization can handle. In other words it means saying no to some of available projects.

It is like treating every incoming project not as a commitment by default but as an option. Well, it seems we have options again.

So let’s picture our portfolio backlog as a bunch of options.

Now, what happens when we start a project? Let’s keep it consistent with the Ellen’s concept. We fill the project backlog with a number of options. Except this time the options are more fine-grained. We may build these features or those features. Well, odds are we will build them all eventually, but the order is kind of important here.

By the way, I’m yet to see a project where the scope doesn’t change. This basically means that every time we decide to build a feature we exercise an option of building this very feature while keeping the options of building other features still open.

The option backlog will get replenished on occasions while we discover more and more details of the project. It will live and evolve the same way as our portfolio option backlog is alive and changing.

The interesting part here is that commitment on one level (launching a project) springs a bunch of new options on another level (features or stories).

In fact we can take this model even further. Once we as a team commit to build a feature we generate a bunch of options to build the feature in this or that way. We have an option to write tests. We have an option to write tests before the code. We have an option to write acceptance tests. We have an option to review code and so on. At the same we also have options to implement the feature in a few different ways functionality-wise. Some of them will be mutually exclusive, some of them won’t.

Basically, starting work on a feature springs a handful of options on yet another level of granularity.

If you imagine that building stuff in the company is happening on a few different levels – each of them dealing with more fine-grained stuff – this model will work throughout the board. Commitment on a higher level generates options on a lower level.

In fact, you can take the model way beyond the three levels I proposed here. For organizations that deal with product lines, programs and stuff it will be the same. There always is a commitment to more high-level stuff before we choose how to do low-level items. There simply will be more layers to take into account.

That’s why considering backlog as a set of options seems so attractive to me. It’s not only about a project or an iteration backlog. It’s about any backlog on any organizational level.

What’s more, having the Real Options theory at hand we may use the model to manage the stuff we build wiser. Again, it doesn’t matter whether we think about managing portfolio, prioritizing features or deciding on implementation details.

“Don’t commit early unless you know why” may as well apply to launching a new project or getting more work in progress than we really need to have. And knowing why should include understanding of all the costs attached. The bit that is too often forgotten.

September 23, 2013
Portfolio Visualization

I’ve been lucky enough that throughout my career I’ve had occasions to work on different levels: from personal (that’s pretty much everyone’s experience, isn’t it?), though team and project to program / PMO / portfolio level. Not only that. In most of my jobs I’ve been involved in all levels of work concurrently. This means I’m schizophrenic.

Um, actually this means that I’ve been pursuing goals that require focusing on different granularity of work.

Granularity of Work

Let me give you an example from my company – Lunar Logic. If I have to complete a task to close a new deal for the company, e.g. to have a call with a potential client, that’s a personal level. It is something that has to be done by myself and it involves only my work. Most of such stuff would be rather straightforward.

At the same time I run a project where I’m supposed to know what’s happening and for example share my estimates when the work is supposed to be finished with the client. That’s a different kind of work. I mean it’s not only me anymore – there’s a team. Also there are a lot of dependencies. Unless bugs are fixed we don’t ask a client for acceptance, this task has to be finished before the other, etc. Of course, tasks will be bigger – we don’t want to run 30-minute long tasks through our task or Kanban board. At least not as default.

Then there is whole effort of coordinating different projects run in the company. It is about understanding which are about to be finished making people available to work on something new, how we can cover for unexpected tasks, what kind of free capabilities we have, etc. At this level a user story or a feature is meaningless. We’re talking about big stuff here, like a project here and a project there.

Depending on what I do I may be interested in small personal tasks, user stories or whole projects.

Actually, there may be more than just three levels of stuff. It is a pretty frequent case. Imagine an org that employs a few thousand people. They will have teams, divisions, projects, programs and product lines. On any level beyond personal there will be a few different granularities of work.

There may be a user story as the smallest bit of work that a team gets done. It would be a part of an epic. The epic would be a part of one of these huge thingies – let’s call them saga stories. Sagas would build up a project. A set of projects would make a program. The program will be developed within a product line… Complicated, isn’t it? Well, I guess it’s still far from what Microsoft or Oracle has.

Focus

Now, the interesting part. On every level leaders are likely to be interested in understanding of what’s going on. They will want to visualize stuff that’s happening. Wait, what that “stuff” is exactly? I mean, haven’t I mentioned at the beginning that work items that interest me may be very different?

Um, yes. However, in each context, and at any given moment, there’s only one granularity of work that will get my attention. When I wear a hat of a company leader I don’t give a damn about a user story in a project XYZ. I just want to know whether that project is progressing well, what are the major risks attached to it and how it can impact other projects and our available capabilities.

When I go down to wear a hat of a project leader, I’m no longer interested in timelines of other projects. I have project tasks to be finished and this is my focus. Scope of risk management would be limited to only a single project too. This means that I will be paying attention to a different class of risks.

Then I go even further down to wear a hat of a leader of my own desk (not even a real one) and granularity of stuff changes once more.

Good news is that vast majority of people don’t have this switching dilemma – since they’re always working on the same class of stuff there’s always the same level of work that they pay attention to. Always a single level.

Well, almost…

Second Level

One level of stuff will always be a primary focus. In many cases there will be a secondary focus as well. It will be either one level up or one level down. For example, when I was a team leader my primary focus was the features we were building. However, on occasions I was going down to development tasks and bugs to understand better the progress we were making. Not that I needed the view on all the development tasks and all the bugs all the time – it would make information cluttered and less accessible. Sometimes I needed that data though.

Another example. In my previous job I led more than a dozen development teams. Obviously user stories or features were far beyond my focus. My primary area of interest was projects on PMO level. I was expected to juggle projects and teams so we build everything on time, on budget and on scope (yeah, right). Anyway, the atomic unit of work for me was a project. Occasionally I was going a level up though, to program management if you will. After all, we weren’t building random projects. Teams had business knowledge specialization. I had to take it all into account when planning work.

You can think of any role out there and it’s very likely that this pattern will be true: main focus of one level of work and possibly secondary focus on work that is either happening a level up or a level down. The secondary focus is likely to be sometimes on but sometimes off as well.

Why is this so important?

Visualizing Work

My work around Portfolio Kanban and discussion on the topic made me thinking of what we should be visualizing. Of course it is contextual. But then, again, in any given context there is this temptation to see more. After all, given that we visualize what’s happening with the projects, it’s not that hard to visualize the status of the epics within these projects. And of course we definitely want to see what’s happening on program level too…

No, we don’t.

Again, in any given role (or hat) you have only one primary area of interest. If these are projects, focus on them and not on programs or epics.

Otherwise the important bits of data will be flooded in all the rest of information that is needed only incidentally, if ever.

Portfolio Visualization

For whatever reasons we get that intuitively most of the when we manage work on a team level. I rarely see task or Kanban boards that attempt to map the whole universe (and more). Maybe it is because we typically have some sort of common understanding what an atomic task on a team level is.

At the same time we go on a project level and things get ugly. I mean, what is a project anyway? For one person it will be a Minimal Viable Product (MVP), which is smallest increment that allows verification of a product hypothesis and enables learning. For the other it would be defined by a monolithic scope that is worth 200 man months of work. And anything in between of course.

Unless we remember the rule of focusing on only one level of work, any meaningful visualization of that work would be impossible. This is one of the reasons why visualization in Portfolio Kanban is so tricky.

Instead of creating one huge wall of information and throw all the different bits of data there, think about your primary focus and deal only with this. And if you keep switching between different roles no one forces you to stick to a single board.

Right now I’m using portfolio board, personal one and a bunch of different project boards. For each of them, there is just one level of work visualized. So far I never needed to combine any two of them or put stuff from one at the other. It seems that even for such a schizophrenic guy as me the single focus rule applies as well.

August 28, 2013
What Makes Project Attractive

Throughout years of my professional career I’ve heard all sorts of ideas what makes a project attractive for people working on it. A favorite technology. Greenfield work with no legacy code. Scalability challenges. Number of users. Potential to change the world. A genuine idea. Big money involved. Freedom to choose any tools. Code quality. Probably a dozen different things too.

Since I joined Lunar Logic a number of projects I’m involved in a given time period went significantly up. It is so since on the top of a couple bigger endeavors we run quite a bunch of small projects. It’s great as I have an occasion to see, and learn from, many different environments.

This experience influenced my thinking of the factors that make a project attractive. In fact I’ve seen a project that scored really high in most of areas mentioned above, yet still everyone hated it (and vice versa).

Why? I think we focus on the wrong thing. No matter how cool is technology, scale or the idea if people involved in a project suck big time nothing is going to save the experience.

If you’re like “yeah, I should have thought about the team too” I’m not heading this direction really. In fact conflicts within teams are kind of rare and typically people learn how to cope with each other quickly.

However if a client is toxic, well, that’s an extreme case of bad luck.

I mean, extreme in terms of severity, not in terms of frequency. The fact is such clients are pretty damn common.

This is exactly why I pay so much attention to the people on client’s site when we consider a new project. Are folks who we’re going to cooperate with on a daily basis OK? Whether collaboration is going to be smooth? If so, I don’t care that much about how cool the code, the product or the technology is.

Even if there is legacy code with poor test coverage and the idea is boring like shit but the guys on the other side are great people would still like to work with them. Actually, it is likely that there isn’t any other side at all – we’re all in it together.

The opposite is also true. Take the coolest thing ever and work on it with a micromanager as a client and you will hate his guts in less than a week. With reciprocity, of course. And it’s not about micromanagement. Take any flawed client: a sociopath, an “I don’t give a damn” guy, a bureaucrat, or what have you. Same thing.

Actually when I go through all the stuff teams liked or hated to work on I see the same pattern – show me the client and I will tell you how happy the team is. In Lunar Logic this is seen even more vividly because given our extreme transparency we simply make ourselves vulnerable in front of our clients. That’s an awesome tool to build trust. At the same time it can be abused to bring misery upon us.

Fortunately, for whatever reasons, the latter is kind of rare.

August 26, 2013
No Estimates Middle Ground

The idea of no estimates (or #NoEstimates) is all hot these days. People would choose different parties and fight a hell of fight just to prove their arguments are valid, they are right and the other party got it all wrong. I’d occasionally get into the crossfire by leaving a general comment on a thread on estimation in general, i.e. not steering a discussion for or against #NoEstimates.

And that’s totally not my intention. I mean, who likes to get into crossfire?

What No Estimates Mean

A major problem of no estimates is that everyone seems to have their own freaking idea what that is. Seriously. If you follow the discussions on the subject you will find pretty much anything you want. There are crusaders willing to ban all the estimates forever as they clearly are the source of all evil in the software world. There also are folks who find it a useful tool to track or monitor health of projects throughout their lifecycles. You definitely can find people who bring to the table statistical methods that are supposed to substitute more commonly used approaches to estimation.

And, of course, anything in between.

So which kind of #NoEstimates you support, or diss for that matter? Because there are many of them, it seems.

Once we know this I have another question: what is your context? You know, it is sort of important whether you work on multimillion dollar worth endeavor, an MVP for a startup or an increment of established application.

My wild-ass guess is this: if every party getting involved in #NoEstimates discussions answered above questions they’d easily find that they’re talking about different things. Less drama. More value. Less cluttered twitter stream (yeah, I’m just being selfish here).

Is this post supposed to be a rant against discussion on no estimates?

No, not really. One thing is that, despite all the drama, I believe that the discussion is valuable and helps to pull our industry forward. In fact, I see the value in the act of discussing as I don’t expect absolute answers.

Another thing is that I think there is #NoEstimates middle ground, which seems to be cozy and nice place. At least for me.

Why Estimating Sucks

Let me start with a confession: I hate estimating. Whoa, that’s quite a confession, isn’t it? I guess it is easily true for more than 90% of population. Anyway, as long as I can get out avoiding estimation I’d totally go for that.

I have good reasons. In vast majority of cases estimates I’ve seen were so crappy that a drunken monkey could have come up with something on par or only slightly worse. And last time I checked we were paying drunken monkeys way less than we do developers and project managers. Oh, and it was in peanuts, not dollars.

It’s not only that. Given that kind of quality of the estimates the time spent on them was basically waste, right?

It’s even worse. It was common when these estimates were used against the team. “You promised that it will be ready by the end of the month. It isn’t. It’s your fault.” Do I sense a blame game? Oh, well…

And don’t even get me started with all the cases when a team was under pressure to give “better” estimates as the original ones weren’t good enough.

Why We Estimate Then

At the same time, working closely with clients for years I perfectly understand why they need estimates. In the case of a fixed-price contract we have to come up with the price somehow. That’s where estimates come handy, don’t they? There also is a million dollar question: so how much will I spend on this thingamajig? I guess sometimes it is a million dollar question literally…

So as much as I would prefer not to estimate at all I don’t hide in a hole and pretend that I’m not there when I’m asked for an estimate.

All Sorts of Estimates

Another story is how I approach the estimation process when I do it.

I would always use a range. Most of the time pretty broad one, e.g. the worst case scenario may mean twice the cost / time than the best case scenario. And that’s still only an estimate meaning that odds are that we would end beyond the range.

Whenever appropriate I’d use historical data to come up with an estimate. In fact, I would even use historical data from a different setup, e.g. different team, different project. Yes, I am aware that it may be tricky. Tricky as in “it may bite you in the butt pretty badly.” Anyway, if, basing on our judgment, team setup and feature sizing is roughly similar I would use the data. This approach requires much of understanding the dynamics of different teams and can be difficult to scale up. In my case though, it seems to work pretty fine.

I’m a huge fan of Troy Magennis and his work. By the way, despite the fact that Troy goes under #NoEstimates banner, he couldn’t possibly be farther from folks advising just to build the stuff with no estimation whatsoever. One of most valuable lessons we can get from Troy is how to use simulations to improve the quality of estimates, especially in a case where little data is available.

Finally, I’m also fine with good old guesstimation. I would use it on a rather general level and wouldn’t invest much time into it. Nevertheless, it works for me as a nice calibration mechanism. If the historical data or a simulation shows something very different than an expert guess we are likely missing something.

Interestingly enough, with such an approach having more details in specifications doesn’t really help, but that’s another story.

On the top of that, whenever it is relevant, I would track how we’re doing against initial estimates. This way I get early warnings whenever we’re going out of track. I guess this is where you think “who, on planet Earth, wouldn’t do that?” The trick is that you need to have quite a few things in place to be able to do this in a meaningful way.

A continuous flow of work gives us a steady outcome of delivered features. An end-to-end value stream means that what is done is really done. At the same time without continuous delivery and a fully operational staging environment end-to-end value stream is simply wishful thinking. Limiting work in progress helps to improve lead time, shortens feedback loops and helps to build up pace early on. And of course good set of engineering practices allows us to build the whole thing feature by feature without breaking it.

Quite a lot of stuff just to make tracking progress sensible, isn’t it? Luckily they help with other stuff too.

Nevertheless, I still hate estimation.

And I’m lucky enough to be able to avoid it pretty frequently. It’s not a rare case when we have incremental funding and budgets so the only thing we need is keeping our pace rather steady. And I’m not talking here about particularly small projects only. Another context where estimation is not that important is when money burn-out rate is so slow (relatively) that we can afford learning what the real pace is instead of investing a significant effort into estimating what it might have been.

No Estimates Middle Ground

To summarize the whole post I guess my message is rather straightforward. There’s value in different approaches to estimation so instead of barking one at another we might as well learn how others approach this complex subject. For some reasons it works for them pretty well. If we understand their context, even if ours is different, we might be able to adapt and adopt these methods to improve our estimation process.

That’s why I think the discussion is valuable. However, in terms of learning and improving our estimation toolbox #NoEstimates notion doesn’t seem to be very helpful. I guess I’ll stay aside in the middle ground for the time being.

By the way, if we are able to improve our cooperation with the clients on the estimation I couldn’t care less whether we call it no estimates or something different.

July 23, 2013
Cumulative Flow Diagram

One of charts that give you a quick overview of what’s happening in a project or product work is Cumulative Flow Diagram (CFD). On one hand in CFD you can find typical information about status of work: how much work is done, ongoing and in backlog, what is the pace of progress, etc. This is the basic stuff. On the other hand, once you understand the chart, it will help you to spot all sorts of issues that a team may be facing. This is where Cumulative Flow Diagram shows its real value.

Before we move to all the specific cases let me start with the basic stuff though (feel free to scroll down if you’re familiar with this part).

The mechanism of Cumulative Flow Diagram is very simple. On a vertical axis we have a number of tasks. On a horizontal one we have a timeline. The curves are basically a number of items in any possible state shown in a time perspective. The whole trick is that they are shown cumulatively.

If the green curve shows stuff that is done it will naturally grow over time – that’s simple. If the blue line shows tasks that are in progress, and we have stable amount of work in progress it will still go up as it adds to the green line. In other words work in progress would be represented by the gap between the blue and the green lines… We’ll come back to that in a while.

Any line on CFD represents a specific stage. In the simplest example we’d have items that are to be done, stuff that is ongoing and things that are done.

For the sake of the rest of this article I’m going to use a simple process as a reference.

We have a backlog, items that are in development or testing and stuff that is done. For the sake of Cumulative Flow Diagram examples it doesn’t matter whether tasks in development are ongoing or done and waiting for testing. However, as we will find later, there may be some indicators that would make tracking these two stages separately valuable.

With such a workflow our Cumulative Flow Diagram may look like this.

First, the meaning of the lines. The green one shows how many items have been delivered over time. Everything that is between the blue and the green curves is stuff that is in testing. The area between the red and the blue lines shows how much stuff is in development (either ongoing or done). Finally, the top part below the orange line is the backlog – how many items weren’t yet started.

In a glimpse we can find a few important bits of information about this project. First, after slow start a pace of delivery is rather stable. Pretty much the same can be said about work that is in progress – the pace is stable and things go rather smoothly. We know that the scope has increased a couple of times, which we can tell looking at jumps of the orange line. Finally, comparing where the green line (done) and the orange line (scope) are on a vertical axis right now we can say that we’re not yet halfway through the project.

Quite a lot of information for the few seconds, isn’t it? Well, there is more.

On this CFD a few things have been shown explicitly. One is a scope change. We’ve discussed it on the previous chart too. Another one is the space between the red and the green lines. It represents work in progress (WIP). Note, basing on Cumulative Flow Diagram only you can’t learn how much work in progress you have precisely; it is some sort of approximation. Pretty good one, but only approximation. It is a very good indicator how WIP is changing over time though. There also is an arrow labeled “prod. lead time” where “prod.” stands for production. It roughly shows how much time we need to complete an item. Again, it shouldn’t be used as the ultimate lead time indicator but it shows pretty well what lead time we’ve had and how it changes over time. Finally, we can approximate slope of done curve to roughly estimate the delivery time. Of course if the scope changes the delivery time will change as well thus the scope line (the orange one) is also approximated.

Now, we have even more information. Yay!

You will rarely see such nice Cumulative Flow Diagrams though. And that’s good news actually. I mean if CFD looks plain and nice all the time you can only learn that much from it. The real CFD magic is revealed when things don’t go so well.

Let’s go through several typical cases.

In this situation the spread between the red and the green lines is growing over time. It indicates a really bad thing – we have more and more work in progress. That sucks. Increased WIP means increased lead time as well. Not only is time to market longer but also it is more and more difficult to deliver anything fast when we need it.

That’s not the worst thing. The worst thing is that with increased amount of work in progress we also increase multitasking thus we incur all the costs of context switching making the team less efficient.

Do we know that for sure? Um… no, not really. I make here an assumption that the team setup hasn’t changed, meaning that we have the same people spending similar amount of time on a project, etc. If it was Cumulative Flow Diagram for a team that is constantly growing and then it would be just OK. The chart may also present an increasing number of blocked tickets which definitely would be a problem but a different one then described above.

In either case such a situation is a call for more analysis before jumping to conclusions. The potential reasons I offer you with this and following charts are simply the likely ones; not the only available.

By the way, please treat all the following remarks keeping that in mind.

One more interesting observation about this Cumulative Flow Diagram is that we have no clues where the root cause for increasing WIP lays. Neither development nor testing part seems to be steadily attached to any other line over time. A further investigation is a must.

There are charts where we get some clues which stage of a process is problematic.

Whoa, this time the development part compared to the testing part is really heavy. What can we learn from it? We don’t have problems with testing. Also, if a definition of testing is “testing and bug fixing,” which is a typical approach, it doesn’t seem that quality of work is much of an issue either. If we are to point fingers we’d point them to development part, wouldn’t we?

And we might have been wrong. Of course one thing that may be happening here is a lot of items in development but few of them ready to test. Another issue though may be that there is a lot of stuff waiting for testing but availability of testers is very limited and when they’re available they focus on finishing what they started.

How can we tell? We can’t unless we have more data. In fact, another line on the chart – one that distinguishes items in “development ongoing” from those in “development done” – would help. Without that the CFD is only an indicator of a problem and a call for a deeper analysis. After all, that’s what Cumulative Flow Diagrams are for.

Another flavor of a similar issue is on the next CFD.

We can find two things here. Let’s start with the more obvious one – the shape of the green line. It looks like stairs, doesn’t it? Stairs are typical when the last stage, which commonly is some sort of deployment, is done in cadences, e.g. weekly, biweekly, etc. Building on that, a stairs-shaped delivery line mean that work in progress and lead time would vary depending on a moment of release cadence you’re in. Maybe it’s time to make a step toward continuous

There is one more thing here though. There is pretty significant, and increasing, number of items that are in testing but don’t get released. The gap between the blue and the green line is growing with each consecutive release.

This one is a real issue here. It may mean that we have a problem with quality and we can hardly reach a state when an item has all the bugs fixed. It may mean that developers simply don’t pay much attention to fixing bugs but tend to start new stuff; at the same testers would follow up on new stories as they wait for bug fixes for the old ones anyway. It may mean that a code base is organized in a way that doesn’t allow releasing everything that is ready. Once again, the root cause is yet to be nailed but at least we know where to start.

It seems we have more questions than answers. If you think that I’m not helping it will be no different with the next example.

This would happen occasionally in almost every team. All the lines flatten out. What the heck? The first thing I do when I see that is I check for public holidays or company-wide event happening during that time. It may simply be time when no one was actually working on a project and there is a perfect explanation for that.

Sometimes it is not the case though. This is when things get interesting. If everyone was at work but the chart still indicates that no one got anything done it most likely tells a story about serious problems. A staging environment could have gone down so everyone has been focusing on bringing it back alive. Another project could have needed help and virtually everyone has been sucked there. There could have been a painful blocker that has forced everyone in the team to refocus for a while.

In either case, whatever it was it seems to be solved already as the team is back on track with their pace.

Another flavor of such a scenario would look a bit differently. It would give more hints too.

There are two important differences between this and the previous Cumulative Flow Diagrams. One is that, in this case, only two lines flatten out; the development line keeps the healthy progress. The other is that ends of both the green and the blue line are as flat as a table top.

The latter suggests that whatever is the problem it isn’t solved yet. What the problem might be though? It seems that the team has no problem starting development of new items. They can’t, however, start testing, thus they clearly can’t deliver anything either. One of probable hypothesis would be that there is something seriously wrong with either the testing environment or the testers.

In the first case it just isn’t technically possible to verify that anything works as intended. In the second it seems something bad happen to our only tester (if there were more than one there would likely be some progress). There is another hint too. Developers don’t seem to care. They just start, and possibly complete, their stuff as if nothing happened.

I’d say that these guys have to deal first with the issue and then discuss how they collaborate. I sense a deeper problem here.

The same way as the previous example indicates an issue in the way people cooperate, the next one suggest a quality problem.

Development line goes up in a stable and predictable manner. The testing curve? Not so much. And we better not mention the done line. Obviously we have more and more work in progress here over time – we’ve covered this one before.

But wait, then suddenly the magic happens and everything goes back on track. At the very end we have decently small amount of work in progress and much stuff delivered. The smell here is how the done (and testing to some point as well) curve skyrockets at the end.

How come that earlier such pace was impossible? I’d challenge the idea that the team suddenly become so fast. Of course they might have not kept the board up-to-date and then, out of the blue, have realized that they’ve had way more finished items that they’ve thought they had.

More likely scenario is that under pressure they just deployed whatever seemed at least remotely close to working. If that’s true the problem isn’t solved at all and it’s going to come back to bite them in their butt. A curious reader may try to draw a picture how further part of Cumulative Flow Diagram would look like in this case.

The next one is one of my favorites. I wonder why it is so far down the list. Oh well…

This Cumulative Flow Diagram is surprisingly common. Let’s try to list a few things that we can find here. The development curve goes up aggressively. Halfway through more than 80% of items are started. Testing doesn’t go nearly that well. And delivery? Well, the start was crappy, I admit, but then it simply went through the roof. And it isn’t only a single day that would suggest delivery of uncompleted stuff. Odds are that these items are properly done. Wouldn’t bet the real money on that but wouldn’t be surprised if that it was so either.

Of course we have very high WIP in the middle of this CFD but at both ends the gap seems to be significantly smaller.

Ah, one more thing. It seems that at the end of the day we’ve delivered everything that was in the backlog. Yay!

Now, what would be the diagnosis in this case? Time boxing! This is one of classic visualizations of what typically happens over the course of an iteration. If a team is comfortable with planning and has rather stable velocity it’s likely that they’d fill backlog with reasonable amount of new features.

Then, given no WIP limits within the time box everyone does their own thing: developers quickly start many features having no pressure other than the end of the iteration to finish stuff. Eventually, the backlog is cleared so the team refocuses to finish stuff, thus the acceleration at the latter stages of the process.

If you pictured a series of such Cumulative Flow Diagrams attached one to another you’d see a nice chain going North-East. You’d find many of these in Scrum teams.

Another chart, despite some similarities to the previous two, suggests a different issue.

In this case almost everything looks fine. Almost, as the done line barely moves above the horizontal axis. However, when it finally moves it goes really high though. What does it mean?

My guess would be that the team might have been ready with the stuff but, for whatever reasons, they wouldn’t deliver. In fact this is one of typical patterns in fixed price, fixed date projects, especially in bigger ones. Sometimes a basic measure that is tracked is how many items are done by the production team. No one pays attention whether it can possibly be deployed in production or even staging environment.

Eventually, it all gets deployed. Somehow. The deployment part is long, painful and frustrating though. Cumulative Flow Diagram representation of that pain and tears would be that huge narrow step of the done curve.

Talking about huge and narrow steps…

Another chart has such a step too. We’ve already covered its meaning at the very beginning – it is the change of the scope. In this case it is not about the fact that such change has happened but about its scale and timing.

First, the change is huge. It seems to be more than a half of initial scope added on the top of it. Second, it happens out of the sudden and pretty late in the project. We might have been planning the end date and now, surprise, surprise, we barely are halfway through again.

Now, this doesn’t have to be a dysfunction. If you were talking with the client about the change or it is simply a representation of expected backlog replenishment that’s perfectly fine. In either case you it shouldn’t come as a surprise.

If it does, well, that’s a different story. First, if you happen to work on fixed prices contract… man, you’re screwed up big time. It isn’t even scope creep. Your scope has just got on steroids and beaten world record in sprint. That hurts. Second, no matter the case you likely planned something for these people. The problem is it’s not going to happen as they have hell lot of work to do in the old project, sorry.

So far lines on Cumulative Flow Diagrams were going only up or, at worst, were flat. After all, you’d expect that given the mechanism of creating the chart. That’s the theory. In reality a following chart shouldn’t be that much of a surprise for you.

Whoa! What happened here? A number of stories in testing went down. The red line representing stuff in development followed but don’t be fooled. Since the gap between the red and the blue line is stable nothing really happened to the items in development; it’s only stuff in testing that was affected.

Now, where did it go? Definitely not to done bucket – the green line didn’t move. It didn’t disappear either as the total number of items (the orange line) seems to be stable. A few items had to go from testing to backlog then.

What could it mean? Without an investigation it’s hard to say. I have good news though. The investigation shouldn’t be long – such things don’t happen every other day. For whatever reasons stuff that was supposed to go past code complete milestone was marked as not started.

I sense a major architectural or functional change. What’s more, it’s quite probable that the change was triggered by the tests of aforementioned items. Unfortunately it also means that we’ve wasted quite some time building wrong stuff.

Another flavor of that problem looks a bit scarier.

Again, the total scope didn’t change. On the other end every other line took a nosedive. Once again amount of stuff in development doesn’t seem to be affected. This time the same can be said about items in testing. It’s the delivered stuff that got back to the square one.

It means that something that we though was done wasn’t so. One thing is that we were building wrong stuff, exactly as in the previous example, only we discovered it later. We likely pay the order of magnitude bigger price for the late discovery.

There’s more in it though. This Cumulative Flow Diagram shows that we likely have problems with acceptance criteria and / or collaboration with a client. I mean how come that something that was good is not so anymore? Either someone accepted that without checking or we simply don’t talk to each other. No matter the case it sucks big time.

Would the orange line never move down then? Oh yes, it would.

I mean, besides an obvious case where a few items are removed from backlog and the only line that moves down would be the orange one, we may find this case. Using the technique perfected in the previous examples we will quickly find that a few items that were in testing are… um, where they are actually?

Nowhere. They’ve disappeared. They haven’t been completed, they haven’t been moved back. These items are no more.

What does it mean? First, one more time we’ve been working on wrong stuff (fools we are). Second, we’ve figured it out pretty late (but could have been later). Third, the stuff doesn’t seem to be useful at all anymore.

It’s likely that we realized that we don’t know how exactly build this or that and we asked the client just to learn that they don’t need either of those anymore. It’s also likely that we’ve encountered a major technical issue and rethought how we tackle the scope possibly simplifying the whole approach. Whatever it was, if we had figured it out earlier it wouldn’t have been so costly.

Finally, one more Cumulative Flow Diagram I want to share with you.

Think for a while what’s wrong with this one.

When compared to the previous charts it seems pretty fine. However, by now you should be able to say something about this one too.

OK, I won’t keep you in suspense. In the first part of this CFD work in progress was slowly but stably growing. However, it seems that someone noticed that and the team stopped starting new stuff. You can tell that seeing how relatively flat the red line has become somewhere in the middle of the chart.

Given some time testing and delivery, even though their pace hasn’t changed, caught up. Work in progress is kept at bay again and the team’s efficiency is likely improved.

As you can see, despite past several examples, you can see the effects on improvements at Cumulative Flow Diagrams too. It’s just that CFD is more interesting in terms of learning that you have a problem than finding confirmation that you’ve solved it. The latter will likely be pretty obvious anyway.

Congratulations! You made it through to what is probably the longest article in the history of this blog. Hopefully you now understand how to read Cumulative Flow Diagrams and what it may indicate.

I have bad news for you though. You will rarely, if ever, see such nice CFDs as shown in the examples. Most likely you will see an overlapping combination of at least a few patterns. This will likely make all the lines look like they were tracing a rollercoaster wagon.

Fear not. Once you get what may be happening under the hood of the chart you will quickly come up with the good ideas and the right places to start you investigation. After all, Cumulative Flow Diagram will only suggest you a problem. Tracking it down and finding an antidote is a completely different story.

However, if you’re looking for a nice health-o-meter for your team Cumulative Flow Diagram is a natural choice.

July 15, 2013