October 30, 2024

the death of the architect

Once upon a time, every project began with the creation of a canonical design document. This was called the system architecture, because it "rightly implie[d] the notion of the arch, or prime, structure."¹

Then, documents would be written for each module. These would provide detailed instructions for how the module should be implemented. Often, there would be diagrams for the control flow of each subroutine.

And only then, once all the documentation was complete, would the implementation begin. This was seen as a largely mechanical task, akin to compilation. It was generally assigned to the junior members of the team; after all, the hard part was already done.

This approach was a major contributor to the software crisis. Decisions made early in this process would become load-bearing, impossible to change.² This, however, only made the design phase seem more important. Countless methodologies were proposed, all of them design-first.

And so, when Kent Beck began to talk about iterative development, people were ready to listen. In his Extreme Programming (XP) methodology, design and implementation were interleaved. "There will never be a time," he said, "when the system 'is designed.' It will always be subject to change, although there will be parts of the system that remain quiet for a while."³

We can find a number of familiar ideas in the first edition of Extreme Programming Explained. Beck, for instance, also measures complexity in bits:

Simplicity and communication have a wonderful mutually supportive relationship. The more you communicate, the clearer you can see exactly what needs to be done and the more confidence you have about what really doesn't need to be done. The simpler your system is, the less you have to communicate about, which leads to more complete communication, especially if you can simplify the system enough to require fewer programmers.⁴

And for your design to remain simple, you would need "a clear overall metaphor so you were sure future design changes would tend to follow a convergent path."⁵

In fact, metaphors were central to the XP methodology. A metaphor "helps everyone on the project understand the basic elements and their relationships."⁶ And they are especially useful for high-level design:

Architecture is just as important in XP projects as it is in any software project. Part of the architecture is captured by the system metaphor. If you have a good metaphor in place, everyone on the team can tell about how the system as a whole works.⁷

Instead of exhaustive design, Beck wanted just enough design. His system metaphor was something that could be explained in a moment, and was robust to change. It was a pane of frosted glass, a locus.

A year later, Beck's ideas were distilled into the Agile Manifesto.⁸ His notion of lightweight design became a preference for "responding to change over following a plan." What's the point of a plan that solves yesterday's problems?

And then, three years after the Manifesto, Beck released the second edition of Extreme Programming Explained. It had been rewritten from scratch. There was not a single mention of metaphors or system architecture. Nor, really, any discussion of the future. In this iteration of XP, you simply moved from moment to moment.

As readers, our instinct is to treat the second edition as a continuation of the first. After all, they have the same author. And there was a reason Beck called it "extreme" programming:

When I first articulated XP, I had the mental image of knobs on a control board. Each knob was a practice that from experience I knew worked well. I would turn all the knobs up to 10 and see what happened.⁹

The second edition, then, could be Beck simply turning Agile's preference for "responding to change" up to 10. If you're always living in the moment, what's the use of architecture or metaphors?

A broader review of the Agile literature, however, reveals a different story. As Robert Martin explains it:

In the years just before and after the signing of the Agile Manifesto, the Metaphor practice was something of an embarrassment for us because we couldn't describe it. We knew it was important, and we could point to some successful examples. But we were unable to effectively articulate what we meant. In several of our talks, lectures, or classes, we simply bailed out and said things like, "You'll know it when you see it."¹⁰

And this embarrassment, when you look for it, is plain to see. Metaphors are treated like stray parts next to newly constructed furniture; if they're mentioned at all, it's to explain why they probably don't matter. In Domain-Driven Design, for instance, the topic is buried four hundred pages deep:

System metaphor has become a popular approach because it is one of the core practices of extreme programming. Unfortunately, few projects have found really useful metaphors, and people have tried to push the idea into domains where it is counterproductive.¹¹

This was not a refinement; it was a tactical retreat. Despite everyone's best efforts, Beck's ideas about lightweight software design remained stubbornly tacit. And so, they were quietly discarded. With them, the Agile methodology lost any notion of continuity, of describing the future. It was left floating, unmoored, in the eternal now.

---

Decades later, software design has become something of a backwater. Most writing on the subject can only be called "post-design." It is defined by what it refuses to discuss.

"Programmers," Sandi Metz warns us, "are not psychics."

Practical design does not anticipate what will happen to your application. It simply accepts that something will and that, in the present, you cannot know what.¹²

Our only choice, Ron Jeffries tells us, is to focus on the present:

The source code is also the ONLY document in whatever collection you may have that is guaranteed to reflect reality exactly. As such, it is the only design document that is known to be true. The thoughts, dreams, and fantasies of the designer are only real insofar as they are in the code.¹³

This seems like hard-nosed pragmatism, until you realize that software is a living text. Each snapshot is just a stepping stone. And the path they follow is, undeniably, shaped by our "thoughts, dreams, and fantasies."

There is a smallness to this post-design literature. It confines itself to the syntax, offering heuristics for better code. Sometimes, these heuristics are fenced in by warnings about the failures of the past. But more often, the limitations are treated as self-evident; software design is just a collection of heuristics.

This newsletter (and the underlying book) is an attempt to turn back the clock. It imagines a world in which Beck's ideas about software design were more explainable. And it begins, appropriately enough, with a metaphor.

---

This post is an excerpt from my (incomplete) book on software design. For more about the book, see the overview.