This post
is a sequel to the previous post regarding
the lean architect. It’s main topic is the book review of “Lean Architecture – for Agile Software Development” from James O. Coplien & Gertrud Bjørnvig
This is really a follow-up in the sense that I have found most of the ideas
from my previous post expressed in this book, but they are more thoughtfully
presented :). It also goes further than
my previous analysis, which is why I am writing this quick – and incomplete –
book review. James Coplien is both a
prolific author and a serious expert on software architecture, with an
itinerary that is not so different from mine, especially with respect to
object-oriented programming, which use to be my own domain of expertise twenty
years ago. Similarly the
10 years which I have spent working on information systems architecture
from 1997 to 2007, are well mirrored by Coplien track record in system’s
architecture in the 90s and 00s.
The first
key idea of “Lean Architecture” is the reconciliation of agile and
architecture, because of the increase of scale for projects that agile methods
are addressing today. “Extreme
Programming (XP) started out in part by consciously trying to do exactly the
opposite of what conventional wisdom recommended, and in part by limiting
itself to small-scale software development. Over time, we have come in full
circle, and many of the old practices are being restored, even in the halls and
canon of Agiledom”. On page 161, one
finds a nice graph from Boehm and Turner that shows the effect of size on the
need for architecture. As usual with
Boehm, this is a data-derived graph that captures one’s intuition: once a software project becomes large,
anticipation and forethinking is required.
There is no
surprise here. On page 15 we read “Ignoring architecture in the long term
increases long term costs”, which any one with gray hairs knows firsthand. The insight
from this book is that the lean contribution to agile is to bring back the long
term and systemic focus into agile, hence the need for architecture.
Coplien and
Bjørnvig bring a
fresh and simple vision of lean software development, which is summarized in
the book by two formulation of the same principle:
- Lean = All hands on deck + long term planning
- Lean = Everybody, All together, Early on
I would not
say that “The Toyota Way” could be fit into such an equation, but it captures
an important part of what lean management is about, and it’s relevant to
emphasize the long term vision which is indeed a key aspect from lean. It also
helps the authors to introduce a (creative) tension between lean and agile:
- Agile is oriented towards change and organic complexity. For instance it is good to defer decision (this is not new or “agile per se” since Knuth is quoted to have said “premature optimization is the root of all evil”).
- Lean is focused towards large-scale complexity (including “complication”) and long –term resilience. Hence the focus on standardization and bringing decisions forward.
This
distinction is presented in a full table page 12, and, frankly, I disagree with
most of its rows. For instance, team versus individual is not a good
characterization of lean versus agile. Nor is the complicated versus complex
debate … and the “high throughput (lean)” versus “low latency (agile)” debate
is too restrictive. My views are
expressed in
a previous post and I think that there is much more in common between lean
thinking and agile thinking. Still, I would definitely agree that the
management of time, the systemic and holistic view, and the overall long-term
perspective is different between lean and agile. Clearly, the goal of lean
software development is to combine both, and this is where the extra emphasis
on architecture comes from. It also
explains the emphasis on maintainability, as expressed by this quote from
Jeffrey Liker about lean: “a culture
of stopping or slowing down to get quality right the first time to enhance
productivity in the long run”. The tension between lean and agile is helpful to explain why one needs
both the systemic practice of “Five
Whys” in the search of root cause while problem solving, together with the
frequent feedback and adaptive cycles. Indeed the complexity and the rate of
change of our current environment require both: complexity demands the 5 whys,
but the rate of change means that this is not enough, quick feedback loops are
required.
An
interesting development is this book deals with Conway’s law. This
principle states that there is a strong dependency between architecture and
organization. That is, the system architecture (module organization) will be
strongly influenced by the management organization (teams & departments)
that produces the piece of software. Conversely, things work best (from a
management perspective) when people organization follows the system
architecture. Coplien derives two consequences from this law, which strike me
as truly relevant:
- “Manager are über-architects, a responsibility not to be taken lightly” … here goes the myth that one may do a good job at managing software developers without a keen insight about software architecture :)
- “There is a need for modularity in large-scale system” – this is obviously true … and indeed a consequence of Conway’s law. The goal of modularity is to keep changes local, and since it is unfortunately mandatory on the people side as soon as the scale of the projects tips on the “large side”, modularity in the organization must translate into modularity in software, hence the need for architecture :)
This post
will not do justice to this excellent book, which is full of wisdom. I do not
have the time to collect all pearls, such as “Remember that architecture is mainly about people” or “Software development is rarely a matter of
having enough muscle to get the job done, but rather of having the right skill
sets present”. I refer you to my book
for my own two cents of wisdom about information system architecture and
management. Still, it worth repeating that architecture is foremost a
communication tool to manage change, and not a blueprint for a better word. One of my favorite quote from “Lean
Architecture” is the following: “The
customer has a larger stake in the development process than in the service that
your software provides”. This is the
crucial idea that is at the heart of the “Lean
Software Factory”. In a dynamic and changing world, software is not an
object, it is a process. Qualities such as evolvability, modularity and
openness come from the people and the development process, much more than they
apply to a finished product.
This brings me to my second favorite quote, page 131: “The essence of “Lean” in Lean architecture is to take careful, well-considered
analysis and distill it into APIs written in everyday programming language”.
Here we see the long-term/resilience thread of lean thinking brought into software
development in a very practical manner. This quote expresses two key ideas into
one sentence:
- Being “future-proof” is having the right APIs so that the code may be both extended easily as well as reused in different manners from which it was intended in the first place.
- Defining, I would rather say “growing”, the right set of APIs is an art; it requires practice, wisdom and aesthetical judgment. Still it required forethinking and analysis, which is architectural thinking.
This should be enough to reconcile agile programming with architecture, if
there ever was a need. Coplien and Bjornvig spend a few delightful pages debunking
“Agile Myths” such as:
- “You can always refactor your way to a better architecture”. Refactoring is crucial because software is a live object that evolves constantly as its environment changes. Agile methods, like any iterative development processes, are bound to produce “accumulations” that need to be cleaned up.
- “We should
do things at the last moment”.
Here we find the “anticipation versus as-late-as-possible” debate that was mentioned earlier. Many authors, including Mary Poppendieck, consider that lean thinking translates into taking design decision as late as possible. This book does a good job at balancing the arguments. There is no single answer, but there is a strong plea for “thinking ahead” and “preparing oneself” through architectural design. - Agile =
“don’t do documentation”.
The agile tradition of no comprehensive documentation that gets obsolete before it is used still stands. However, as soon as scale grows, and as soon as the life expectancy of the software piece grows, there is a need for serious documentation. There are many tools to automate part of the documentation task, especially that which is closely linked to the code. A large piece of software requires storytelling, and this has nothing to do with software, it’s a consequence of human nature and what is needed to motivate a large group of people (see Daniel Pink).
User stories are an integral part of Agile/SCRUM development methods. There
are a few interesting pages in this book that show the link between user
stories and business processes, which are commonly associated to large-scale
waterfall development process. The last
part of the book deals with DCI
(Data, Context and Interaction), a framework that extends MVC and which is
proposed as the proper foundation for designing modern distributed
object-oriented systems. This topic is out of the scope of this post, although
I find many similarities with the design philosophy of the CLAIRE
programming language. Some of the key insights may be summarized as the
need to recognize and separate the need for complex algorithm from object classes
(“Does this mean that procedural design
is completely dead ? Of course not. There are algorithms that are just
algorithms”), the reification of roles and business processes and the use
of context objects to develop functional polymorphism (hence to share and reuse
more code), a practice that reminds me of my early day at Bouygues Telecom when
we created the PTEs (Processus Techniques
Elementaires) – cf. my book
on Urbanization.
I will conclude with two quotes from page 92:
- “Let the human considerations drive the partitioning [architecture], with software engineering concerns secondary”.
- “A complex system might use several paradigms”.
