Those of you who read Software Project Management regularly know for sure that I have sort of experimentation attitude. I like to try different things, see how they work and, if they sort of do, share my experience here with you. I’m particularly happy with a bunch of such concepts, one of them being ad-hoc retrospectives.

This story is more than a year old, but recently Bernd and Ilja from it-agile reminded me about it with their comments under the post.

A short version of the idea is that, instead of doing regular retrospectives bi-weekly, monthly, or whatever your cadence for retrospective is, you do a very short and focused ad-hoc discussion on a single problem raised by any team member. The outcome should be similar to what you get from a regular retro – an action point, or a bunch of them, aimed at solving the issue. In my case the whole mechanism proved to work very effectively. If this teaser sounds interesting I strongly encourage you to read the full story.

Both Bernt and Ilja pointed that I shouldn’t call it a retro. As Ilja puts it:

Retrospectives are more than just meetings where you solve problems you are already aware of, anyway.

Well, I guess we are in full agreement on this one. The only difference is I actually believe that even with ad-hoc retrospectives we are (usually) solving problems that we aren’t yet aware of. At least we aren’t when we launch the retro. Pretty often we start with just a symptom. Someone spots something that they think is worth discussing. So here we are – at ad-hoc retro.

First, we don’t have to agree that this is a real problem and often, initially, we don’t. Thus a discussion. Even though we actually are focused on this single starting point, we dig deeper trying to find some common denominator for our views.

Second, as I’ve already told you, we usually start with just a symptom. Quite often we are yet to discover, and address, a root problem.

Finally, we try to come up with some action points to fix a root cause we agree on.

Now, I happen to facilitate a bunch of regular retros recently and all of them seem to follow similar pattern, no matter the team or the organization. We start with a handful of things we like or don’t like, try to cluster them somehow, look for root causes and address them. However, it all starts with observations people make.

It’s not some kind of magic that tells us what we are doing wrong. It’s not an outsider who comes and blesses us with an epiphany. It’s not a scientific process which makes us come up with the right solution. It’s the team that finds it out by themselves.

In both cases: regular and ad-hoc retros the outcome is similar, the process is similar and even the same people do the job. The main difference is granularity. While during a regular retro we try to cover things which happened during a specific time-box, on ad-hoc retrospective we start with just a single idea. An idea we would write down on the on a sticky note at the very beginning of the next regular retro anyway. In other words we would bring it up as well, no matter the process.

I just wanted to add that another difference is, that during typical retros, not every problem is acted on. We usually try to focus just on a bunch of most important things at the moment. However, on a second thought it works the same with ad-hoc retrospectives. Sometimes after a brief discussion we just decide to leave a thing alone for the time being. We either don’t share views of a person who proposed the issue or don’t see any action points that we could agree on or whatever else. So similarly to regular retrospectives the outcome can be “do nothing” too.

If nothing so far convinced you that both concepts are surprisingly similar I have a question about a purpose of retrospectives. It is improvement, isn’t it? We want to do better in future that we’ve just done. If so, we can perfectly do this job using both approaches. Of course, depending on a team’s maturity I would go with one or another, but that’s a totally different story.

Of course we can bring it down to some orthodox discussion over definitions, but don’t count in. I’m more into solving problems than arguing over definitions. It just feels more useful.

Software estimation. Ah, a never-ending story. Chances are good that, whenever you’re talking about building software, this subject will pop up soon. You can be pretty sure that basically everyone around has problems with estimation or simply struggles with it. And that’s virtually obvious that there would be a new sexy method of estimation every year or so. The method which is claimed to solve an unsolvable puzzle.

Recently there was yet another question on estimation on Project Management StackExchange. This sole fact isn’t probably worth writing a whole blog post about that, but there was one side thread which is worth focusing at.

One of advices I shared was that whenever you can you should avoid estimation at all. This sprung sort of objection. OK, so the subject definitely is worth wider discussion.

First things first. Why do we estimate at all in the first place? Well, we usually want to know how much time it’s going to take to build this damn thing, don’t we? Um, have I just said “usually?” Meaning, “not always?” Actually yes. It’s not that rare when we either don’t need any estimate at all or we just want to have a general insight whether the project will be built in hours, days, weeks, months or years. In either of these cases just a coarse-grained wild-ass guess should be fine. If it’s needed at all.

OK, but what about majority of cases when we need some kind of real estimate? For example all those fixed price projects where estimates are basically a part of risk management, as the better the estimate is the smaller are chances that the project goes under water. I can’t deny that we need to have something better than wild-ass guess then.

Yet, we still can avoid estimating quite often.

Let me start with one of obvious things about estimation: if you base on historical data, and you apply them in a reasonable way of course, you can significantly improve your estimates. In other words, no matter the method, if you are just guessing how much something is going to take, you will likely to end up with way worse results when compared to a method, which uses your track record. And yes, I just dared to name planning poker “guessing.” It is collective, involves discussion, etc but usually it is just this: guessing.

Cool, let’s use historical data then. What’s next? My next question would be: how precise must your estimates be? Seriously, what kind of precision you aim for? My point is that we need very precise estimates very rarely. This is by the way the reason why I don’t use Evidence Based Scheduling anymore.

Anyway, ask yourself a question: how much you would pay for bringing your estimates to the next level of precision. Think of it like being correct in terms of estimating in years, months, weeks, days, hours, etc. Let’s take just an average several-month-long, fixed-priced type of project.

If I’m wrong with years I’m totally screwed, thus I’d pay significant part of project budget to be correct on such level. If I’m wrong with months it might be a hit on our reputation and also it may consume our whole profit we get of the project, so I’d be ready to invest something around the profit to be correct with months. Now weeks. Well, a week here, a week there, does it make such a difference in this kind of project? Can’t I just assume there is some variability here? Unless of course our deadlines are written in stone, e.g. you adjust your software to law changes. In most cases I’d invest just a handful of bucks here at best. Days? Hours? Are you kidding? Does it even make a difference that I spend a day more on such project?

Now you know what kind of precision you expect from your estimates. Would it be possible for you to come up with estimates of such precision basing purely on historical data? I mean, can’t you just come up with a simple algorithm which automatically produces results reasonable enough that you can forget about all the effort spent on estimation?

Whenever we come to discussing estimation I like to share the story from one of my teams: basing on a fact that we were collecting data on our cycle times and we reduced variability in task sizes coming up with the idea of standard-sized features we were able to do a very good job with estimates not estimating at all. We were simply breaking work down so we could learn how many features there are to build and then we were using very simple metrics basing on our track record to come up with the numbers our sales department requested. By the way: a funny thing is, almost all of that appeared as an emergent behavior – something we started doing as a part of continuous improvement initiative.

Either way, even though we were capable of providing reasonably precise and reliable estimates we didn’t really estimate. I was surprised how easy it was to get rid of estimation, but then I can come back to the point from the beginning of the article: what is the point of estimation? You don’t do it just for the sake of performing the task; you do it for a reason. As long as you achieve your goal, does the method really matter? It means that you can get rid of estimating even if you do need some kind of estimates.

Recently many of my discussions on process optimizations come to a point where we focus on hand-offs. When I say about hand-offs I think about every situation when a work item, feature, user story, requirement, or however you call those gizmos you build, is handed from one person to another. Think a business analyst handing requirements to a developer or a developer handing updated version of an app to a quality engineer, etc.

Now, hand-offs are bad because I say so. You have to believe me because I am a Jedi.

OK, being serious, I can show you why it is so. Every hand-off which is incorporated in your process is a place where work can pile up, and it usually does. Lots of tasks waiting to be picked up by another team member – doesn’t it sound familiar? Basically, one of things we are trying to do whenever we’re implementing Kanban is to limit work in progress, and one of very first effects of it is limiting a number of tasks waiting in these hand-off stages.

There is more however. Another reason why hand-offs aren’t cool at all is it means making knowledge chain longer. It means adding another link between client/user and people who actually build the thing. You are virtually asking for more of unspoken assumptions and interpretations while making it even more difficult to sort things out whenever they’re vague. You will go through all these people to get the answer for your question. Or, more likely, you will just make one more assumption which is likely to be wrong.

Then, we have responsibility. It’s funny to look at these huge corporations adding new roles and teams to their development process. Somehow they believe that each team which is more downstream will verify what a team which is more upstream has just done. At the same time they’re dissolving responsibility for building good software, and building good software is I guess what is what they aim for. I don’t say that mentality like “I have a whole team of quality engineers so they will catch any bugs I make” is something they consciously incorporate to their process but that’s exactly what they typically get. They should really look at those small companies that are built purely from software developers. Somewhat magically they can deliver high-quality software. How come? How is it even possible without a crowd of business analysts, quality engineers, release managers and whatnot? I have a hint: they all feel damn responsible for quality as they hand their product off only once – right to their client.

I am well aware that, most of the time, we can’t get rid of all hand-offs from our development process. I don’t even say that we have to avoid them by any means.

I just say that we should be aware of everything that comes along with them and…

• Limit a number of hand-offs to lowest reasonable level. Think about any shortcuts in your process that you can make without quality loss. Think about simplifying the process.

• Avoid adding new hand-offs whenever possible. Sometimes we want, or need, or have to add another link to our chain. Sometimes we want to have a messenger between clients and users and development team. But please, don’t add more messengers that it is absolutely necessary. As a rule of thumb: the closer development is to the client/user the better.

• Measure and control hand-off stages you already have. Learn how big piles of idle tasks there are. Limit work which is waiting for someone. Organize your process in a way which supports quick consumption of work items which hit one of hand-off stages.

• Make hand-offs more informal. The more informal hand-off is the more smoothly it will usually go. People sitting in one room, cross-functional teams, little formalisms all help to keep hand-offs informal.

In terms of improving team’s productivity it is really a low hanging fruit and the one that may teams don’t even think of.

If you don’t think it is a problem in your case, perform a little experiment. Measure how much time your work items live within the process and then split it into two numbers: how much time someone is actively working on them and how much time they are waiting for someone to take care of them. Simplifying things a bit, the latter number is the price you pay for all your hand-offs. And yes, I pretty much expect that, statistically speaking idle time is way bigger than active time.

OK, so I landed in this fine, fine job, leading a crowd (almost 150 actually) engineers who work on, well, software projects. Not a surprise, eh? With such a big team your job is mostly orchestrating things. You just have to keep the machine running and performing well.

What you basically need is an overview of stuff. All kinds of stuff. For me personally the easiest part of catching up was people. Well, of course no matter how hard you try you won’t have the same relations with 100+ team as you had with 25 or, heaven forbid, 5 people. No surprise here, it was totally different from that neat tiny team I had previously. Anyway learning who I work with was sort of easy.

However to make any reasonable decisions in such team you need to know, at least on the high level, who does what and how exactly you define different “whats.” I mean, you need to know all the ongoing projects their importance, relationships with clients, experience needed to successfully complete specific types of projects, and so on and so forth. In short: you need to learn hell lot of things about dozens projects.

And you better be quick as the first person knocking to your door to ask for people for a specific project is coming next week. Early next week. Namely Monday 9 am.

OK, first days, well, first months you spend in a fog. That’s natural. Then you start to actually know what all these teams do on a general level. Except you keep forgetting all these details: when this project starts, when another ends, how many man hours/weeks/months/years we plan for the other and which of those two are somehow connected with the other one you’ve just discussed.

It was exactly this sort of situation I found myself in. I could have started a rugged friendship with our budgeting application but it isn’t a friendship you dream of, if you know what I mean.

On the other hand I knew Kanban enough to play with it in completely different area and I heard there are folks applying it on portfolio level successfully. Sounded like a perfect idea for an experiment.

To be honest I didn’t make a deep research on the subject. The basic concept is simple if you already know Kanban – you treat projects the same as you treated features and then the magic happens. Well, I might be oversimplifying a bit, but only a bit.

After all I tackled the problem with a proper mindset: “what I start with is wrong so I will be changing my approach as I learn what works and what not.”

My expectations were simple: visualization would help me to have control over what is happening. At the beginning I didn’t even set any goals how I wanted to adjust our project portfolio. I just wanted to start and see the early results. Knowing Kanban for a longer time I felt pretty sure there’s something good waiting for me on the way.

Finally, I accepted the possibility that I would eventually abandon portfolio level Kanban if it doesn’t give me any value. So that’s how it all started.

In the next post in the series you I will share what the first version of my portfolio level Kanban board was.

Project Portfolio Kanban Story is the place where the whole series will be aggregated once further chapters are published so keep an eye on it.

The Kanban Story was the idea which came to my mind soon after we I started playing with Kanban with my team back then in early 2009. I thought that sort of live journal of our Kanban adventure can be an interesting thing to read. I knew that we probably got some things wrong initially but after all Kanban encourages you to have experimentation mindset and also I’m a strong believer that failure is actually a good thing.

By the way: now that I read my story, sometimes I deeply disagree with the way we approached Kanban, yet still it was a valuable experience and hell of a lesson.

The Kanban Story ended up as the most popular series of posts on this blog of all time and a few chapters are also high on the top 20 list. Sort of success I’d say. So when I started playing with portfolio level Kanban I decided to go the same way. I let myself a few months of experience just to have some content for the beginning and here it is: the very first post of The Project Portfolio Kanban Story.

It will follow similar pattern as The Kanban Story which means I won’t try to address all the problems of the world but will rather focus on the specific situation and explain what we did and why. Expect much of the context and much of experimenting to find the right solution. Expect wrong decisions and changes. It’s going to be some sort of experience report. In chapters. I hope you will enjoy it as much as its predecessor.

Here’s the list of all posts of The Project Portfolio Kanban Story:

• Project Portfolio Kanban? Why?
• A Basic Approach
• First Changes
• Why Standard Board Design Is Not a Good Choice
• Better Board

Sometimes Twitter is a mine where you can dig thought-provoking gems. A couple of days ago Alan Shalloway said, as a part of a heated discussion, that:

People who say Kanban isn’t about collaboration forget that one of the core practices is improve collaboratively (using models/scientific methods).

I both agree and disagree.

I mean, yes, this is true – collaborative improvements are one of Kanban principles. And yes, the word “collaboratively” is there for a reason. And triple yes, whenever you are setting up the process or improving it you work collaboratively.

At the same time I can’t agree with boiling collaboration in Kanban down to “improve collaboratively” only.

One of most canonical examples of improvement introduced by Kanban is on-demand swarming. When either a feature is blocked or we just have bottleneck resulting with full work-queue people swarm over it to get the flow, well, flowing again. I know Kanban doesn’t tell you that you should collaborate in such situations but actually introducing limited WIP results in many similar behaviors. People are helping each other. Last time I checked it counted as collaboration, but well, maybe we use different definition of collaboration.

Another, pretty typical, situation I see in Kanban teams is a result of visualization – when team members see that one of their colleagues, for whatever reasons, is coping with multiple work items, they tend to help instead of just starting another feature from backlog. Sounds like collaboration, isn’t it?

Then we have complexity of processes we try to cover with Kanban. Despite simple Kanban board we see here and there software development is usually pretty complex process. At the same time we are advised to start simple with Kanban. It means that we often come up with a board which is pretty simplified model of what we are doing. For example, usually testing/quality control stage covers both testing and bug fixing, meaning that most likely at least a couple of people work on a feature at the same time. Collaboratively. With no formal hand-offs.

And when we are at hand-offs, with Kanban we make them visible on Kanban board. We also have incentives to optimize hand-offs, both in terms of a number of them and in terms of time features spend waiting to be picked up by the next person. Fewer hand-offs mean improved collaboration as people need to cover removed formalisms (hand-offs) somehow. And believe me, they do.

We also work collaboratively whenever we are changing the process. I’m yet to meet a team which had this part of Kanban setup just enforced by someone. I’m sure there are such teams out there but, at least now, it is total minority. What more, usually these discussions are launched be team members, e.g. when they don’t know where to put an index card, which basically means that the process might need some improvements. This collective decision making process can be called collaboration I guess.

I could probably go further but, by this time, you get the point. Despite Kanban doesn’t say collaboration over processes or something alike, thanks to its rules, it really fosters everyday collaboration. Usually it is all about small things: swarming over this blocked feature, quick feedback loops with that one being tested, launching a discussion about adding a column here or changing a limit there.

Collaboration in Kanban isn’t about big words and big ideas. It is about tiny everyday actions. But please, don’t tell me that Kanban isn’t about collaboration. That’s just bullshit.

The other day I facilitated a retrospective for a fellow team. My goal, as a facilitator, was basically to help them to suck as much value out of the meeting as possible.

Now, before we move on, a picture from a past. I recall a bunch of retrospectives which looked like this: a whole project team met for a longer time and everyone was asked what was good about the project and what needed improvements. Then, one of project leaders wrote it down in a document uploaded the document to a server and finally everyone could just happily forget about the whole thing.

Does it sound familiar? It probably does for many of you. Does it add any value? Um… next question, please. Isn’t it a complete waste of time? Oh well… If you don’t plan to make any use out of retro, don’t even start it.

So the question is: what makes a retrospective valuable?

The answer is actually pretty simple.

Value of retrospective can be measured in terms of changes sprung by it.

It basically means that the team decided to act, to try something new, to deal with a problem. It doesn’t necessarily mean that it will be an overnight success. Most likely it won’t. But at least they gave themselves a chance. They might even totally fail with the first approach, but they kept trying.

Note: when I say changes I think about things which are really changing, not about those we just say we’re going to change but don’t do so.

Anyway, another problem pops up. We want changes, but how to make them happen?

Um, that’s sort of easy. Remember about a few simple rules:

• Don’t chase too many goals. It’s usually tempting to cover each and every issue we spotted. After all we have all the enthusiasm and we want to improve. The problem is that when we commit to too many tasks we’re going to fail at many of them. Then, we’ll get discouraged that we don’t see any results of retro and our enthusiasm won’t be that enthusiastic next time. If there’s going to be the next time, that is.

• Assign people to tasks. Task with no individual attached to it isn’t really assigned. A decision that the team would do something means that, well, someone else can do it, not necessarily me, right? Tasks assigned to everyone most likely end up not being done by anyone.

• Have deadlines. Ask when you’re going to be done doing this or that. Keep your deadlines possibly short, yet definitely reasonable and achievable. Stating that something will be done in 6 months is meaningless. In 6 months I can work in the other place of the planet. A couple of weeks are a time frame we understand way better than a few months. If tasks don’t suit short time frames, chop them to smaller ones.

• Verify outcomes. When deadlines pass remember to discuss with the team what was done, what was the outcome, what else, if anything, has to be done about discussed issues. Again, I don’t assume that all the problems are solved. You may end up with a solution, which didn’t work, and will to try something different. You can also end up with solved problem but the least you should do is saying so. Starting the next retro with such a summary of outcomes from the previous one is a good practice.

• Repeat. One retro is just a quick fix. If you need sustainable change do retrospectives regularly. I don’t believe you are so perfect that one retro is enough to solve all the issues you might possibly have.

In short you want to end up with a short list of actionable work items assigned to people and then check how you’re dealing with them.

Of course sometimes it just sounds that easy. Sometimes you need to work hard to avoid blame game, get focused on specific issues, cut out longish but pointless discussions, learn to accept things you can’t change etc. Sometimes you will need to try different formats to animate communication or build basic trust between team members or change their attitude to anything positive. Sometimes it may be damn hard work to do.

But as long as you aim for the goal and your actions help in achieving it, you should do pretty well.

It started with Twitter discussion. Then I followed with a post on standard Kanban board designs. Dominica DeGrandis added her perspective as well. By the way, there’s one lesson I have to take from Dominica and from discussion on Twitter – we need to show people different Kanban board designs so they don’t get fixated on a standard and just stick with it.

In the meantime I was selling Kanban to my wife, as we both believed it may help her in coordinating projects, which is what she does at her workplace. An interesting part is that she works in land surveying industry – something completely different than our everyday reality in software development business.

And after the discussion she comes back home and tells me that she has to show me something. It appears that it is a picture of her very first Kanban board. Out of curiosity I’m asking her about meaning of different artifacts on her board and it strikes me that she’s done exactly what we want people adopting Kanban to do: she’s used Kanban board examples I drew as, well, just examples. And then she’s come up with her original design reflecting her own constraints, which are pretty different than in typical software projects.

First, the board.

Now, that you ask, no, it doesn’t seem like a “standard Kanban board” whatsoever. Where is the process? Where are the limits? If these questions are your first thoughts than you are fixated at least a bit.

You should rather ask how the process looks like. It’s a bit tricky as for each big task there are a few stages. There are these which have to go one after another, but there also are those which can be done in parallel. In short: process isn’t linear.

What more, depending on a task process can be built out of different stages. Imagine that for one task you do the whole production, from idea definition to deployment, for other you do only development, and for another only testing and deployment. OK, now you have an idea how it looks like. In short: process isn’t homogenous.

So the basic idea is that process is defined per task. It can be safely assumed that project manager, or whoever coordinates the work, knows how to cope with each task. What is the role of columns then? Priorities! The old idea of classes of service. The closer the task is to the left side of the board the more important, or urgent, it is to deal with the task.

Different colors of index cards are used to mark different clients as it is somehow important.

Then, we have a trick, which I like the most. As process depends on a task it is defined on the fly for each task separately and represented by small stickies attached to index cards.

Stickies show stages, subtasks, you-name-it which have to be done to complete a task. Green one means the subtask is completed, yellow marks ongoing one, while red color is used for blocked work (which is sort of typical situation). Labels on stickies say what kind of stage/subtask it refers to. In other words they define the process.

Instead of defining complex graph of all possible states with dedicated Kanban board for each state we just add needed stages as we go. Isn’t that agile?

Another nice thing about this board is how limiting work in progress can be handled here. We neither have old-school homogenous process where we limit number of tasks per stage nor dedicated boards assigned to different people where limiting WIP is done per board. However we can easily say which tasks/subtasks are ongoing: yellow stickies tell us that!

We can limit WIP by limiting the number of yellow stickies which are on the board. In a given situation it is actually something which comes very handy, as a number of people involved in doing work is changing pretty rapidly and limits can be adjusted on the fly depending on the current situation.

What I really like about this Kanban board design is that is addresses all the specifics of the situation in a neat way, yet it is perfectly aligned with all the Kanban principles.

And this is why I love to work on Kanban with people from outside of IT industry. Since their constraints are so different than ours they come up with fresh Kanban board designs as our standard solutions just don’t work for them.

As I’m lagging a bit with my reader I read recent Jurgen Appelo’s article on networked Kanban yesterday. An interesting reading, especially that it ignited a few thoughts (and a great discussion on Twitter with Jabe Bloom, Jim Benson and a few others).

First, Jurgen’s experiments with Kanban boards are perfectly coherent with Kanban principles. We are told to visualize workflow, not to use a specific Kanban board design.

Second, value stream mapping, which is used as one of the basic techniques by many Kanban proponents, is limiting and it brings you back to a standard Kanban board design. David Anderson avoided using the term in his writings and it wasn’t without a reason.

Third, we use examples of typical board designs as it is so hard to explain the whole thing without showing these examples, yet it makes people fixated on typical boards and eventually their board designs are suboptimal.

Finally, I sometimes feel we actively harm people showing them standard Kanban board designs. We prompt them to use one of our well-done solutions instead of looking for the one which is adjusted to their context and reflects their process well.

Now, this is tricky, we’d like people to start with a clean whiteboard and totally open mind and to come up with something which reflects the way they work, presents important information in accessible and understandable way. It is pretty open-ended task, isn’t it? Unfortunately also the one which may make people totally confused.

I’m not sure if I’d even be on the Kanban land now if I was starting with such task. I was more than happy to use the most common board design as a starting point. Now I don’t have problems using uncommon designs (just wait to see my portfolio level Kanban boards) but yes, it took some time to get here.

Either way, my message is: no matter whether you’re just starting with Kanban or are rather advanced don’t stick to standard Kanban board designs. If you have no better idea, use them – that’s what they are for. But please, please, please, don’t treat them as the one and the only way to visualize workflow.

On a side note: I find it amusing working on visualization with people from industries different than software development. They aren’t constrained with the good old process of building things, which was being hammered into our heads for years, namely: analyze, design, build, test, deploy. And they often come up with uncommon designs on the very first approach. Quite a lesson for us I would say.