Category: product management

  • Conway’s Law Teaches a Grim Lesson About AI in Product Development

    Conway’s Law Teaches a Grim Lesson About AI in Product Development

    When I first stumbled upon Conway’s Law, it was anything but intuitive to me.

    Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.
    Melvin E. Conway

    Why would an organizational structure of a company have anything to do with software architecture? After all, there are whole bodies of knowledge covering high- and low-level concepts of software development. Aren’t these things properly planned and executed?

    I mean, sure, in the details, there will always be a mess. However, in a grand scheme of things, the high-level design should be far more controllable than the law suggests.

    In retrospect, Conway’s Law is one of those things that, once you’ve seen it, you can’t unsee it.

    Conway’s Law and Spotify Model

    Whenever Conway’s Law emerges in a discussion, my favorite example is Spotify. I’m a user since the beta. I know several people who worked there in leadership positions. Most of all, Spotify, at some point, was very vocal about their organizational approach, the so-called Spotify Model. It’s like I have enough data to connect the dots.

    As popular as it was at the time, if you look at the Spotify Model, it’s hard not to see it as a glorified matrix organization. Yes, there’s more autonomy across the board, but the communication paths? They all scream “Matrix!”

    What should we expect from their product design if Conway’s Law is true? My best guess is a set of features interconnected in non-obvious ways, with a common issue of lack of alignment, and a lot of local optimizations. And that’s precisely what we get.

    Spotify UX

    While Spotify’s codebase is not open, its UX is quite telling. By now, it’s a clutterball of misaligned ideas fighting for your attention.

    spotify home screen
    Playlist sidebar, announcements, recently played, made for me, music video, related music videos, now playing… Welcome to the Spotify home screen.

    From a user perspective, it’s fascinating, although not in a good way, how I struggle to find the same options that I use regularly (like, multiple times a week, for years). Managing playlists? Oh, boy. Search? Every other time, it’s in the wrong context. Inconsistencies between mobile and web? Don’t get me started.

    But maybe it’s just grumpy me. Fine. I’ve just literally googled “who loves spotify ux?” and among the top results are:

    Doesn’t sound like a love wave, really. It’s either one: Google reads my mind, or the UX is problematic.

    All that comes from a product praised (and copied) for some product innovations, such as Discovery Weekly or Spotify Wrapped. Most definitely, not everything is wrong here. It’s just a matrix of somewhat misaligned ideas, good and bad. That isn’t really a recipe for customer delight.

    lighsaber seek bars spotify

    By the way, did you know that Spotify has a custom seek bar for Star Wars songs that looks like a lightsaber? And you can change the looks of the specific lightsabers, no less. There could hardly be a better illustration of “a matrix of somewhat misaligned ideas, good and bad.”

    Communication, Alignment, Autonomy

    If you consider Conway’s Law, it’s hard not to see how Spotify UX is a precise map for the Spotify Model. Wherever communication between teams (squads, guilds, chapters, or whatever the hell they call them) was messy and faced multiple conflicting goals, so did the interconnections between features.

    To make things harder, Spotify sprinkled high autonomy over their feature teams. So the matrix organization made alignment a challenge, yet decision-making was pushed down the hierarchy nonetheless.

    A high-autonomy-limited-alignment setup is not a recipe for effective work. And I say that from experience. At Lunar, we walked that path. The lesson? As much as I’m a huge fan of distributed autonomy, I’d always consider alignment first.

    In the late 2010s, Spotify had a few dozen relatively independent feature teams responsible for specific parts of the product. Small wonder that the “mobile playlist” team had different ideas from the “web playlist” team. Communication paths that might have fixed that were watered down by an overly complex organizational model.

    By now, the engineering headcount is already in the thousands, and the team count is thus in the hundreds. Just imagine the product mess that the Spotify Model would create. No wonder they largely abandoned it.

    OK, but what does it have to do with AI?

    AI Product Management

    Product people are encouraged almost as strongly as developers to use AI extensively. It comes as a mixed blessing. Early intensive prototyping is a viable path, and it opens up whole new avenues for validating product desirability.

    LLMs make it just so easy to create extensive specifications. We can attach all the existing product descriptions as context, let AI do its own research, analyze the codebase, and more. It will produce a nice, detailed description, and the engineers will nail it.

    Except we misinterpret the ease of creation for the value of the output.

    A sidenote: It’s the same aspiration we had when we tried to model systems with UML diagrams, and it didn’t work either. It’s not about the tools. It’s about the iterative exploratory nature of designing software.

    Still, AI can create the same illusion we followed many times before—that we can specify software upfront in detail from the outset. The output looks good. Better than ever, even. And we get it effortlessly. It’s the AI model that does the heavy lifting.

    It takes some time to realize that product development doesn’t work like this. It never has. And it has nothing to do with the tools we use to create specs.

    AI Kills Communication

    In the past, product specifications were brief. Save maybe for some Business Analysts, no one fancied writing long forms describing all the feature details. We relied on just enough context and communication between engineers and product people.

    However, now, generating detailed specifications with AI is easy and cheap. Initially, we might even verify whether the output is correct. Eventually, we’ll give up. One, often they’d be good enough. Two, LLMs are great at creating outputs that sound sensible. Three, honestly, read a 4-pager with a feature description and tell me you understood everything.

    At some point, and sooner rather than later, a product manager will stop carefully verifying AI output. Soon, the developers will follow. That is, given they read the extensive specs at all in the first place. It quickly evolves into the “you give me the specs, I’ll get them built, no questions asked” kind of scenario.

    dev building up to ai specs meme

    The key part here is: “no questions asked.” It’s like going all the way back to the 90s, way before Agile happened. We build up to the specs, whether it makes sense or not. The only difference is that we deal with the development so much faster. Does it matter, though, when we build the wrong thing?

    Conway’s Law Meets AI Product Management

    The most important change happened in between the lines, though. A one-sentence-long feature description was, by definition, full of holes and required the team to discuss. A detailed specification creates the illusion that a feature has been thought through from all angles.

    The former invites communication. The latter discourages it.

    As we close down communication paths, Conway’s Law kicks in. We’re bound to design architectures that copy organizational communication structures.

    Less peer-to-peer communication and less collaborative exploration mean a more fragmented and less coherent architecture. As each individual is treated more and more as an isolated island, so will be the code that individual develops.

    The effect will affect both a technical level (think code architecture) and a product level (think UX). Give such a way of working a couple of years, and we’ll start praising Spotify for its exceptional product design in comparison.  

    AI Is on a Collision Course with Conway’s Law

    Dev: The last feature is on staging.
    PM [checks the feature out]: It doesn’t work as specified! Here’s what should be different.
    Dev [checks the code, checks the specs, 2 hours have passed]: Well, actually, it works as specified. Here are the specific parts that prove my point.
    PM: Oh, my Claude must have hallucinated that part.

    That’s an actual conversation that I heard that inspired this article. When I consider the impact of AI on communication, the paths connecting product and engineering are most affected. Yet, similar effects emerge across the board.

    • A single developer may produce more code with AI tools, so they are more independent and, even under time pressure, they don’t need to collaborate with other engineers as often. We reduce the need for communication.
    • Coding agents running independently step on each other’s toes way more carelessly, creating merge conflicts and triggering rework (the worst kind of rework, actually). Operators of said agents are thus better off when they isolate their active work areas. Which means less coordination and less communication.
    • The asynchronous nature of working with AI agents incentivizes the “throw over the fence” hand-offs. My agent’s output may be ready when I’m already off, but let that not stop you from doing whatever you need to do with it. Again, less human-to-human communication.
    • The sheer amount of documentation we can generate on different levels automates away collaborative activities (or parts of them). Code review? Let an agent handle that. Even less communication, perchance?

    If Conway’s Law holds, we may be up for a rude awakening. As an industry, we are hyped about all sorts of “my agent talks to your agent” scenarios. It’s easy to see the upside—automating away the mundane, tedious, and routine. It’s hard to see the long-term cost of deteriorating communication paths.

    So, we either learn to navigate the new realities of collaboration better, or we accept that the products we use will increasingly be crap. Which one will it be?


    These posts are not generated. 웃 https://okhuman.com/nf1GGg

  • The Most Underestimated Factor in Estimation

    The Most Underestimated Factor in Estimation

    We were preparing yet another estimate. It was a greenfield product, nothing too fancy. We used our default approach, grouped work into epic stories, and used historical data to produce a coarse-grained time estimate per epic.

    We ended up with a 12-20 week bracket. Unsurprisingly, our initial hip shot would probably be close to that.

    The whole process took maybe half an hour. Maybe less.

    Then we fell into an AI rabbit hole. Should our estimate be lower since we will generate a good part of the code?

    AI in Early-Stage Product Development

    We could discuss the actual impact of AI tools in established and complex code bases. Even more interestingly, we could discuss our perceptions.

    Yet, for a greenfield project and not-very-complex functionality, generating swaths of code should be easy enough.

    After all, it seems that’s what cutting-edge startups do these days (emphasis mine):

    The ability for AI to subsidize an otherwise heavy workload has allowed these companies to build with fewer people. For about a quarter of the current YC startups, 95% of their code was written by AI, Tan said.

    Garry Tan is the CEO of Y Combinator, so most definitely a highly influential figure in the startup world. And probably quite knowledgeable of what YC startups do, let me add.

    If that’s what the best do, we should follow suit, right? That’s why we got back to our initial estimate and tried to assess how much we can shave off of it, thanks to the technology.

    It’s Not About Coding Speed

    A lot of the early-stage work we do at Lunar Logic has already shifted to the new paradigm. The code is generated. Developers’ jobs have evolved. It’s code-review-heavy and typing-light. That is, unless you count prompting.

    Yet, it’s possible to generate entire features, heck, entire apps with AI tools. So we should be faster, right? Right?

    One good discussion later, we decided to stick with the original estimate nonetheless. The gist of it? It was never about coding pace.

    writing code fast was not the bottleneck

    Yes, you can generate a lot of code with a single prompt, and with enough preparations, you can make its quality decent. But AI is not doing the discovery part for you. It does not validate whether what you’re building works.

    It won’t take care of the whole back-and-forth with the client whose vision is most definitely somewhat different from what they’re going to get. And even if they were able to scope their dream precisely, the First Rule of Product Development applies.

    our clients always know what they want. until they get it. then they know they wanted something different

    It’s a completely different experience to imagine a product and to actually interact with it. No wonder people change their minds once they roll up their sleeves and start using the thing.

    The Core Cost of Product Development

    After building (partially or entirely) some 200 software products at Lunar, we have enough reference points to see patterns. Here’s one.

    What’s the number one reason for the increased effort needed to complete the work? Communication.

    Communication and its quality.

    • Insufficient clarity before starting a task triggers rework down the line.
    • Waiting for feedback increases context switching and thus makes the team inefficient.
    • Inadequate knowledge of the business context results in building the wrong thing.
    • Lack of focus in communication is a direct waste of everyone’s time.

    Should I go on? Because I totally could.

    In practice, I’ve seen efforts where poor communication added as much as 100% to the workload. It went down to all the rework and inefficiencies triggered by a lack of clarity.

    When such a thing happens, we might have been wrong about the actual number of features or the size of some of them, and it wouldn’t have mattered. At all. Any such mistake would be covered many times by the bad communication overhead. And then some.

    AI Does Nothing to the Quality of Communication

    Before we move further, a disclaimer: I understand that there are many AI tools designed around human-to-human communication.

    AI summary of conversation between developers
    A “helpful” Slack AI conversation summary

    While there’s still work to catch up with regular technical conversations between developers, things like meeting summaries can be useful. Although I’d love to see usage data, how many of these summaries are read? Like, ever.

    The communication I write about is a different beast, though. It’s not notetaking. It’s attentive listening, creative friction, and collective intelligence. It’s experience cross-pollination.

    With that, AI is of little to no use. And yet, this is the critical aspect of any effective software project.

    What’s more, there’s little you can know about the quality of communication before the collaboration starts. Sure, you get early signs. But you know what it really is once you start working together.

    Start Small

    One of the reasons why I’m a huge fan of starting collaboration with something small—like a couple of weeks kind of small—is that we learn what communication will look like.

    It’s a small risk for our clients, too. After all, how much can you spend on a couple of people working for two weeks?

    Once we’re past that initial rite of passage, we know how to treat any later estimates. Should we assume there’s going to be a significant communication tax? Or rather, we could shave some time here and there because we all will be rowing in the same direction.

    One of our most recent clients is a case in point. Throughout the early commitment, he actively managed stakeholders on his end to avoid adding new ideas to the initial scope. He helped us keep things simple and defer improvements till we get more feedback from the actual use.

    The result? Our estimate turned out to be wrong. We wrapped up the originally planned work when we were around 75% of the budget mark.

    Communication quality (or lack thereof), as much as it can add a lot of work, can remove some, too. That’s why it’s the most underestimated factor in estimation (pun intended).


    A post on estimation is always a chance to share our evergreen: no bullshit estimation cards. After a dozen years, I still hear how they get appreciated by teams.


    If you like what you read and you’d like to keep track of new stuff, you can subscribe on the main page.
    I’m also active on Bluesky and LinkedIn too, with shorter updates.
    I also run the Pre-Pre-Seed Substack, where I focus on early-stage product development (and, inevitably, AI).

  • Lateral Skills Are Core Skills

    Lateral Skills Are Core Skills

    We can consider so many things both in our professional and personal lives as value exchange. The most obvious case: I exchange 40 hours of my time each week for a set amount of money that lands in my bank account each month.

    As a business, we do the same thing. We commit our engineers to spend their time on whatever our clients need them for, and in exchange, we receive an agreed-upon rate for each hour of that effort.

    In fact, I often use that frame when describing how we perceive our contributions. We aspire our clients to be happy with the value we deliver for the price they pay for the whole team.

    What Is Value?

    Now, the tricky part in these examples is value. While we can easily assess how many hours anyone spends on a task, the time doesn’t automatically translate to value. What’s more, it’s not even purely a matter of how one will spend that hour.

    I can work on an activity that has only certain odds of success. If the outcome ultimately emerges as a failure, no value will have been delivered. The reasons for that may be entirely outside my sphere of control.

    As an example, think of all the effort I invest into helping a potential client improve how they will approach building their new product just to see them choose a different partner. In this case, my work has not yielded any value for Lunar.

    However, even if I work on something that we assume to have intrinsic value, it’s still tricky. Let’s look at adding a feature to a product. (And yes, I know that not every feature is value-adding. In fact, there are some whose net value would be negative.)

    Assuming the feature I’m building will have a positive effect, the big question is how big of an impact and how much our client values it. That question is almost universally highly challenging.

    Most of the time, we can’t know the answer upfront. We need to build the thing to be able to validate the outcome. Most of the time, however, we don’t try to check that anyway. Even if we did, most of the time, the early validation will now give the complete picture.

    Think of The Shawshank Redemption. Its theatrical release was largely a flop. And yet, over the years, it gathered a strong fanbase, still keeps the #1 position as the best movie ever at IMDB, and eventually, it at least paid off production costs. Not a bad outcome for a flop, eh?

    While there obviously is a correlation between the opening weekend box office and the eventual financial success of a movie, we can’t precisely know the financial success/failure just after the first few days.

    The same goes for value assessment in most of the knowledge work. You could just as easily consider whether paying an employee any specific salary yields a valuable (enough) outcome. And the smaller chunk of work you look at, the more the mileage will vary.

    Making Value Exchange Work

    We use two basic coping mechanisms to work around uncertainty about value.

    The first one is ignoring the actual value altogether and sticking with our early assumptions of what we expected it to be. It’s like deciding to build that feature because “it will bring us so many new subscribers” and never looking back. Sure, before committing to the development, we might have asked for an estimate. If it was acceptable enough (and the work didn’t take much longer), it was the last time we did any assessment of the work.

    We thought it would bring us a ton of subscribers, and it was a sound decision to pay $10k for that. Oh, and since we already paid for that work, who cares whether it actually brought a single new soul to our solution?

    Well, I’d say this means giving up on a ton of learning here, and that’s of huge value by itself, but I clearly must be wrong, as very few product organizations do any post-release validation.

    The second way to dodge the uncertainty of value is by looking at a bigger whole. I don’t try to assess whether an engineer has been productive during every single hour or day. Heck, I’d be perfectly fine with a slower week, too. However, I want to know whether their long-term performance justifies their salary.

    By the same token, I want the whole team working for a client to deliver good value for money in the long run. So, if we look at the weeks or months of work of the whole group, we are happy with the value of everything they deliver (of course, in the context of what that effort costs, again, in total).

    In extreme cases, you could look at movie studios or VCs investing in startups. They are happy to weather plenty of bad investments as long as that one movie or that one startup yields a 100x or more return.

    Lateral Skillsets

    We all make one subconscious assumption here. That is, even though we exchange different things, they are of similar value for both parties. If we ask for $90 an hour for our developers, that hour would be priced roughly a similar way, whether by Lunar’s client or me. Or rather, we would price the skills our client rents for that hour similarly.

    This assumption, though, often doesn’t hold true.

    To stick with the engineering/software development context. When our potential clients want to assess our team’s skill set, it will almost always be heavily skewed toward technical skills. After all, when you hire developers, they will do development, right?

    But let me redefine the situation here. Imagine you seek someone to turn your idea into an MVP. You have two candidates with very similar technical skills. However, one builds their experience by working on many different small engagements, including several very early-stage products. The other spent big chunks of their time working on very few large and complex solutions.

    Which one would you choose?

    I bet most of us would go with the former over the latter, as we’d deem their experience more relevant. This relevancy, however, stems from a lateral skillset. It’s not an ultimate value. It’s value in the context.

    Product management skills may actually emerge as the most crucial for that role, even if we don’t consider it so upfront. At the early stage of product development, building the wrong thing is rarely salvageable. Building the thing wrongly, on the other hand, can typically be saved.

    If we had considered a different gig, we might have chosen differently.

    This is but one example of lateral skills that may make or break any endeavor. A broad range of people skills, project management savvy, business context understanding, and more may be similar game-changers here.

    And yet, without the specific context, the broad market would largely ignore those traits. Even within the context, they are most often omitted.

    Asymmetric Value Exchange

    The lateral skills are what change the economy of the whole value exchange. The job market would value two similarly technically skilled developers, well, similarly. After all, across a wide variety of challenges, they will provide comparable value.

    However, within the early stage product development context, the first one will deliver something extra. They wouldn’t be working extra hours, their code wouldn’t objectively be better quality, they wouldn’t be faster.

    Yet something that costs nothing extra to that developer–their lateral early product management skills–would be highly beneficial for the product company.

    That’s what breaks the simple economy of value exchange. I add to the mix something that’s of little to no cost for me but gives the other party a big upside.

    In other words, I give up nothing while they gain a lot.

    Suddenly, the whole deal has so much more wiggle room, which can be used to make it more attractive for both parties.

    Not only that, though. It also generates additional options for value delivery. Our developer may use their time building a feature that ultimately will not help. However, thanks to their experience, they can also suggest (in)validating the whole part of an app prior to committing to development. That, in turn, may lead to much more significant savings.

    In this case, exploiting lateral skills makes the value exchange asymmetric. Why is it important? It’s because whenever you can find a partnership with an asymmetric value exchange, it’s a plain win-win scenario for both parties.

    Since lateral skills typically create these scenarios, we should pay much attention to them.

    Side Skills Are Core Skills

    If I looked for a technical partner to help me with an early-stage product, I would primarily be looking for stories about discovery work, building MVPs, validation, etc.

    As a matter of fact, I’d be explicitly asking for failure stories. I mean, we know the data. New products do fail. So, if the only thing someone has to show is a neat streak of successes, you can be sure they’re in a fantastic realm.

    One of my mantras is, “Many of our past clients paid us to fail, so you don’t have to. You get all the experience we got from that as a part of the package.”

    These lessons do not fall into what’s commonly perceived as “core skills” for software development teams. And yet, we do consider them core. We shine most when we’re able to utilize those side skills.

    So, go figure out what constitutes those lateral skillsets in your context. These are the core traits you should be looking for, whether hiring or choosing a partner in your endeavors.