When I first learned about the D. Edwards Deming’s idea that 95% of performance should be addressed to the system and only 5% to the people it seemed totally counter-intuitive to me. Not even the idea itself, but its consequences. I mean a simple thought that you can basically stop working individually with the people on their performance as it just doesn’t make any sense. The impact of suboptimal system improvements should be tenfold or better than the best possible people-related changes.
Then, after researching the subject a bit, I swayed to the other side. The examples shared by Deming or Eli Goldratt are undeniable. I can also find a lot of examples from my own experience how the system was impacting performance heavily. Not that I had the numbers, but it resonated well with me.
Now, finally, you know what I think? Both approaches are crap.
Yeah, I’ve just tried to piss of both: guys who strongly believe in Deming’s work and those who totally disagree with him with a single attempt.
What Deming Really Said
I guess this story started when I was looking for precise Deming’s quote on one occasion. It seems that the meaning people attach to his words wasn’t there initially. The original 95/5 thingy was not about performance literally but about variability in performance.
Let me use a sport analogy to explain that. Let’s take a basketball player. If he scores anything between 5 and 30 points a game, and neither of extremes is an extraordinarily good or an extraordinarily bad game for him (meaning: it happens on occasions) variability of his performance is high. As a comparison we can have another guy, a second-tier player, who regularly scores anything between 0 and 5 points. Variability of his performance is significantly lower, even though his average performance is significantly worse.
Now, it’s not a surprise that the system governs variability. I mean, the more repeatable the situation the less variability you can expect. If the team plays are consequently set to create situations for one player his performances will be sort of repeatable. He won’t be suddenly disappearing from the game. At the same the system also governs variability of performances of a role-player who joins the game only under special circumstances, e.g. when the team needs more three-pointers or maybe when one of important players is injured. Variability of performances of such a player will be significantly bigger.
Of course, to some point, the system also governs performance itself. Think what happens when the star sits on the bench throughout the whole game or when a guy who is typically a sub plays the whole game. It doesn’t mean, however, that you tell a third-tier player to substitute a team leader and expect them to score 30 points a game. It’s not going to happen in a consistent manner. (For basketball fans: yes, I know Jeremy Lin’s story, but that’s an exception.)
What Deming was talking about was basically that the system governance is responsible for creating a stable and repetitive environment so people performances are controllable. Again, it’s not about how exactly people perform, but how their performance varies from day to day.
In fact, the above statement is mostly an interpretation too. Deming was talking about the system being, or not being, in statistical control which isn’t exactly performance variability, but for the sake of this argument let’s leave it as it is and not complicate things even further.
Anyway, Deming Is Still Wrong
So far I made an attempt to convince you that: first, Deming was basically talking about performance variability and second, the impact of the system on performance itself, while existent, isn’t anywhere close to 95%.
However, we shouldn’t forget Deming’s background, which is a factory floor. In other words it’s a place where workers are supposed to produce same things in a repeatable manner. I could exploit the basketball analogy further pointing that every game, heck, every play is different, but I guess in software development we simply don’t need that analogy anymore. It’s enough to ask: how many times have you built exactly the same feature?
And for all three of you who accidentally had this chance: how many times have you built the feature second time in exactly the same way you did the first?
I bring this argument to show that in knowledge work, or sports, we naturally have significantly higher variability to deal with. And, unlike on a shop floor, that is good. What’s more, there is a different value attached to every single thing we develop. In fact, it’s even worse than that. Very often we don’t know exactly what value is attached to a work item we’re going to build.
This means that in such a situation we should use a very different statistic mechanism than Deming based on. We need to take into account both expected cost and expected payoff. The latter, naturally, isn’t taken into consideration on a shop floor as in this situation payoff for each part is well known. In knowledge work, on the other hand, payoff function is extremely important and never flat.
Coming back to the basketball analogy: if you wanted to bet which player is going to score 30+ points during the next game would you rather choose this reliable guy who averages 15.0 and who always scores double-figure but rarely, if ever, exceeds 20.0 or rather that crazy shooter who (surprise, surprise) averages 15.0 as well but, depending on a day, can score either nothing or 40+ points? Bravo! You’ve just chosen higher variability, which is totally against what Deming taught us.
This is because you’ve taken the payoff into account. In the former case any payoff is almost impossible as, because of low variability, it’s highly unlikely that the guy is going to score 30+ points. The other case is a different story though.
Now, especially when we think about product development we are in a betting business. We bet that this or that feature will bring a ton of new users or a truckload of money. We can’t ignore the payoff function. We can’t assume the payoff function is flat. In either case we’d be throwing the baby out with the bathwater.
By the way, if you want to dig deeper around this one I strongly recommend watching this Don Reinertsen’s session. Twice. Seriously.
Besides, Who Does Create the System?
I’m not done yet. I don’t challenge the idea that the system affects performance. I simply challenge the so-called 95/5 rule or, to be more precise, I challenge the way some people interpret the so-called 95/5 rule.
The question we should be asking is who (co-)creates the system. If we start with Deming he points shaping the system as a management responsibility. This is probably very true on a factory floor. We are not at a factory floor though.
Let’s stay for a moment within software development perspective and think what exactly an equivalent of a production line is.
Obviously, one part would be organizational culture. What are general ways things are done in the organization? Does the org use a lean / agile or rather a heavy-weight, formalized approach? How do your clients act? These general rules will be difficult to change.
However, when we consider a lower level it’s a different game. Let’s try to translate low-level manufacturing process to software development. Production line would likely be a sort of a process: analysis, coding, testing, deployment, etc. What’s interesting though is what is happening on each of these stations. How exactly the work is done? Who makes decisions how the work is done?
Take coding for example. Who decides whether unit tests are there or not? How many of them? What kind of tests? Who decides on a specific coding style? A manager? Um, no, not really. Most, or even every, of those decisions are made by a coder – a worker.
Now, the important question: do such decisions define the system which a developer operates in? Obviously! Just think what is going to be the outcome of work of a cowboy coder and what happens when a worker is an eXtreme Programming fan. These are different systems indeed.
The conclusion of this story is that in our world workers can influence the system heavily, thus completely change the Deming’s equation.
One might point that leaving people freedom and empowering them is still the part of the system design which is still the management job. I’d disagree. One thing is that adopting a technique or a practice isn’t comparable to moving the machines on a factory floor. I can perfectly start writing unit tests and it doesn’t instantly require everyone else to change the way they work.
Another thing is that it’s way easier to be a Grace Hopper follower in our industry. We can easily change the part of the system that goes through our desks, no matter whether we are empowered to do so or not. Eventually, if anyone notices (which isn’t that obvious by the way) we should have enough information to have meaningful discussion about the impact of how we’ve done our work.
All in all, in knowledge work there’s no clear line that separates the system from the people, thus the whole notion of addressing specific part of influence to one or the other is flawed.
It’s the People, Stupid!
The last piece of this puzzle is a story. Not that long ago I’ve joined the organization that is exceptional when compared to its surroundings. I’m far from saying that it is a perfect system already, but it’s definitely the most mature company that I’ve ever been working with.
In fact, it’s not only maturity of software development process we have but also this touchy-feely stuff we call the atmosphere. Not a dream job (yet) but pretty damn close to it when compared to everything else I’ve had a chance to experience.
Let’s try to translate this to a system definition then. It would be pretty effective. It would be delivering high-quality stuff. It would be a great environment to work at. It would help everyone to grow. How could one possibly not like it?
The only problem is that I’ve lost one third of technical stuff over the course of few months. The system? No, the people! Their individual needs, frustrations and dreams. Their unique contributions to what the system is, or has been.
Because at the end of the day, in our world, the system is not machinery with replaceable humanoid parts; it is inseparably connected with the people who operate within the system. Change a single person and the whole system is different. Suddenly the rules of the game have changed and everyone else operates in a very different system. And the only thing that has changed is a single person has left.
My message in all that is that we should first understand our work, how it gets done and what kind of system we operate in. Because when we think about that it becomes obvious that there’s no simple way of separating the people from the system in knowledge work domain, especially when we’re talking about a small scale, e.g. dozens of people, not thousands of them.
At the same time I’m challenging the common notion of Deming’s rule because it’s nothing close to what Deming really said and it is simply flawed in our domain anyway.
At the end of the day you should understand the system (obviously!) but focus on the people too if you want a significant change.