The DevOps Handbook: How to Create World-Class Agility, Reliability, & Security in Technology Organizations

Review

The DevOps Handbook promises to be an essential guide for someone looking to start a DevOps initiative. This is done by documenting the theories, principles, and practices of DevOps using the Three Ways, a specific view of the theory introduced in “The Phoenix Project” book.

The book could not be more authoritative: the four authors are very known members of the DevOps community. This work was much needed, this book is a formalization of the DevOps’ body of knowledge, an amalgamation of domains like Agile Software Development, Lean Manufacturing, The Kanban Method, Systems Thinking, ITIL and more. As a cultural movement, DevOps formed from the bottom-up and this book makes a good work to expose the principles that led to it.

And if you have some familiarity with Kanban, you will soon realize that it is at the core of this formalization of DevOps. Just read The Three Ways principles: The Principles of Flow, The Principles of Feedback and The Principles of Continual Learning and Experimentation and relate them to Kanban’s Core Practices. If you do not have an understanding of Kanban, this work will show you the importance of the method. That’s being said, Kanban (David Anderson) and Real-World Kanban (Mattias Skarin) are good follow-up readings.

The only downside of the book is its writing style. The rhythm is too slow and it is too repetitive which turns it a boring reading. This is surprising given that Jez Humble (co-author of the “Continuous Delivery” book) and Patrick Debois have a good writing style. Even so, this is a must-read for any IT manager trying to create an organization with world-class agility, reliability, and security. It provides a blueprint to create the continuous learning environment which is the basis for this while introducing to important practices from the other domains.

Some quotes

DevOps and its resulting technical, architectural, and cultural practices represent a convergence of many philosophical and management movements.

The first phase of work, which includes Design and Development, is akin to Lean Product Development and is highly variable and highly uncertain, often requiring high degrees of creativity and work that may never be performed again, resulting in high variability of process times. In contrast, the second phase of work, which includes Testing and Operations, is akin to Lean Manufacturing. It requires creativity and expertise, and strives to be predictable and mechanistic, with the goal of achieving work outputs with minimized variability (e.g., short and predictable lead times, near zero defects).

The First Way enables fast left-to-right flow of work from Development to Operations to the customer. In order to maximize flow, we need to make work visible, reduce our batch sizes and intervals of work, build in quality by preventing defects from being passed to downstream work centers, and constantly optimize for the global goals.

The Second Way enables the fast and constant flow of feedback from right to left at all stages of our value stream. It requires that we amplify feedback to prevent problems from happening again, or enable faster detection and recovery.

The Third Way enables the creation of a generative, high-trust culture that supports a dynamic, disciplined, and scientific approach to experimentation and risk-taking, facilitating the creation of organizational learning, both from our successes and failures.

However, interrupting technology workers is easy, because the consequences are invisible to almost everyone, even though the negative impact to productivity may be far greater than in manufacturing.

The equivalent to single piece flow in the technology value stream is realized with continuous deployment, where each change committed to version control is integrated, tested, and deployed into production.

We may inadvertently perpetuate unsafe systems of work due to the way we respond to accidents and incidents. In complex systems, adding more inspection steps and approval processes actually increases the likelihood of future failures. The effectiveness of approval processes decreases as we push decision-making further away from where the work is performed. Doing so not only lowers the quality of decisions but also increases our cycle time, thus decreasing the strength of the feedback between cause and effect, and reducing our ability to learn from successes and failures.

In the technology value stream, our goal is to create a high-trust culture, reinforcing that we are all lifelong learners who must take risks in our daily work. By applying a scientific approach to both process improvement and product development, we learn from our successes and failures, identifying which ideas don’t work and reinforcing those that do. Moreover, any local learnings are rapidly turned into global improvements, so that new techniques and practices can be used by the entire organization.

“Even more important than daily work is the improvement of daily work.”

We will actively manage this technical debt by ensuring that we invest at least 20% of all Development and Operations cycles on refactoring, investing in automation work and architecture and non-functional requirements (NFRs, sometimes referred to as the “ilities”), such as maintainability, manageability, scalability, reliability, testability, deployability, and security.

In the field of decision sciences, there are three primary types of organizational structures that inform how we design our DevOps value streams with Conway’s Law in mind: functional, matrix, and market.

Market-oriented organizations optimize for responding quickly to customer needs. These organizations tend to be flat, composed of multiple, cross-functional disciplines (e.g., marketing, engineering, etc.), which often lead to potential redundancies across the organization.

Similarly, functional orientation can also be found with centralized QA and Infosec functions, which may have worked fine (or at least, well enough) when performing less frequent software releases. However, as we increase the number of Development teams and their deployment and release frequencies, most functionally-oriented organizations will have difficulty keeping up and delivering satisfactory outcomes, especially when their work is being performed manually.

We can also achieve our desired DevOps outcomes through functional orientation, as long as everyone in the value stream views customer and organizational outcomes as a shared goal, regardless of where they reside in the organization. (…) Many of the most admired DevOps organizations retain functional orientation of Operations, including Etsy, Google, and GitHub.

Furthermore, integrating the objectives of QA and Operations into everyone’s daily work reduces firefighting, hardship, and toil, while making people more productive and increasing joy in the work we do. We not only improve outcomes, but our organization is better able to win in the marketplace.

Because Operations is part of the product value stream, we should put the Operations work that is relevant to product delivery on the shared kanban board. This enables us to more clearly see all the work required to move our code into production, as well as keep track of all Operations work required to support the product.

Dr. Laurie Williams performed a study in 2001 that showed “paired programmers are 15% slower than two independent individual programmers, while ‘error-free’ code increased from 70% to 85%. Since testing and debugging are often many times more costly than initial programming, this is an impressive result.

To do this, we schedule the post-mortem as soon as possible after the accident occurs and before memories and the links between cause and effect fade or circumstances change. (…) In the meeting, we must reserve enough time for brainstorming and deciding which countermeasures to implement. Once the countermeasures have been identified, they must be prioritized and given an owner and timeline for implementation. Doing this further demonstrates that we value improvement of our daily work more than daily work itself.

We assert that DevOps is transformational to how we perform technology work, just as Lean forever transformed how manufacturing work was performed in the 1980s. Those that adopt DevOps will win in the marketplace, at the expense of those that do not. They will create energized and continually learning organizations that out-perform and out-innovate their competitors.