DevOps, bridging development and operations

Eleven years after the Agile Manifesto and much more years of continuous improvements in the Software Development’s Best Practices had a great impact in our industry. Practices like Test Driven Development (TDD) and Continuous Delivery enabled development teams to develop better software in a more predictably way. But there was still a gap - between operations and developers - and DevOps is the bridge that let us skip the gap.

Enters DevOps

DevOps is, according to Stephen Nelson-Smith, a “movement of people who think it’s time for change in the IT industry - time to stop wasting money, time to start delivering great software, and building systems that scale and last”.

The movement have roots in the Agile Infrastructure and Operations movements that were applying the Agile engineering practices to improve the Continuous Delivery. These movements were a response to a identified problem that was the fact that organizations that were developing software with Agile were fallbacking to a waterfall methodology when it was the time to deploy the developed software.

This boundary with the waterfall process is what Andrew Shafer calls the circle of happiness and wall of confusion. Basically the development team (developers, testers and product owners) sits inside this circle with regular feedback loops between iterations and smooth communication. But communication with people outside this circle is clumsy and this brings a lot of problems that affects the delivery of value.

Stephen also outlines four problems that our software industry faces today that DevOps can help to eliminate or mitigate: fear of change, risky deployments, “it works on my machine!” and siloisation. The fear of change leads to bureaucratic management processes that makes any change to the application (be it the introduction of a new feature or a bug fix) take a long time to be made. Risky deployments specially hurts the development and operations teams since this can lead to deployments at calm times - a lot of times at dawn, which can burn a lot of the teams’ energy - because there is no confidence in the software to be deployed.

“It works on my machine!” is a cliché. Who never experienced a problem that occurs on live and that when it’s reported to the developers, they say: “it works on my machine!”? It is not uncommon to find development teams using different OSes and/or the software’s dependencies environment in different versions from the production environments. This can lead to erroneous assumptions based on the different features available in the different layers of the software stack - which is specially evident when the developers does not know the infrastructure.

And the notorious problem with teams in different silos is the culture of “finger pointing” and the “us and them” mentality. Shafer’s circle of happiness is much about this problem: the different silos are afraid of each other, they collaborate and communicate poorly.

Culture changes

DevOps it is a lot about culture changing since it applies many of the lessons learned in the Agile Software Development practices. One of the lessons is all about communication and collaboration. It reduces the boundaries between development and operations by building a multidisciplinary team with people that, as Stephen points out, “feels comfortable with infrastructure and configuration, but also happy to roll up their sleeves, write tests, debug, and ship features.”

Bringing developers and operations people together means giving opportunities to discover better ways to deliver better software. It propagates the chain of business value that is well known in the circle of happiness to the operations team that starts to be viewed as part of the solution instead of just keeping the servers 24x7. It fills a gap that was missing from the Agile methodologies: it makes operations to be kept in sync with the development process, aligning the development and operations teams to an unified business process.

Surely it’s not easy to go from a black box operations team to a full DevOps team. It’s hard to find people with this multidisciplinary mindset. But DevOps, like Mitchell Hashimoto says, it’s not an absolute, it’s a range. You don’t need to aim having a team where the developers do all the operations, you need to aim your way of doing DevOps. It’s more about identifying people that can be the bridges between development and operations, encouraging and empowering them to do so.

Quoting Stephen again, DevOps “has a tremendous impact on the business. Suddenly the technical team starts trying to pull together as one. An ‘all hands on deck’ mentality emerges, with all technical people feeling empowered, and capable of helping in all areas. The traditionally problematic areas of deployment and maintenance once live become tractable - and the key battlegrounds of developers (‘the sysadmin built an unreliable platform’) versus sysadmins (‘the developers wrote unreliable code’) begins to transform into a cross-disciplinary approach to maximizing reliability in all areas. This, of course, has a positive effect on the bottom line - better reliability and availability, happier clients, faster time to market, and more time to focus the team’s energy on core business rather than wasteful administration and firefighting” (emphasis mine.)

Toolset

Besides the cultural changes, DevOps is loaded with good Agile practices applied in the infrastructure. One big change is the is the concept of Infrastructure in code. In DevOps, everything must be under version control - from network to applications configurations. There is no DevOps without version control since it is the base for all the other practices. Like software, the Infrastructure must be build from source and be equally treated as an application.

Building from source means having a bare metal server to running services with an automated process. The configuration management systems are very helpful here: they put a system into a known state using the configuration that is available in version control and helps achieving consistency in the IT environment. It enables the operations to manage the servers lifecycle, provisioning new boxes by means of their services, in a way similar to applying templates, giving servers responsibilities like “Database server”, “HTTP Cache server” or “Application server”.

There are some open source configuration management system available being Puppet and Chef the most known ones. You can find a lot of interesting use case of both systems. Tumblr (the popular blogging platform), for example, uses Puppet to update the development machines. Travis CI (the hosted continuous integration service) uses Chef to build the CI worker images that are responsible to run the tests for tens of thousands open source projects.

This kind of server management optimizes the operations from manually configuring servers which leads to inconsistency between similar service servers and prevents occurrences of what Shafer calls the “Mistery Machine” - a server that no one knows what runs and that everyone have fear to turn them off. Another important aspect of these systems is idempotence - in this context, the ability to apply the configuration template¹ in the same machine a lot of times without modifying the final result which is the configured machine as described in the template.

Others tools are equally helpful. One of them is Vagrant², which is a tool that eases the creation and configuration of virtualized environments. With Vagrant you can can, for instance, create a virtual machine with an environment as similar as possible from the production environment to be distributed to the team. This is really an important tool for making the “works on my machine” an excuse from the past. Vagrant is also used as a sandbox for testing templates of configuration management systems like Puppet or Chef. You can also just use plain old shell scripts to setup your environment.

The advances of the virtualization tecnologies also plays an important role for achieving better software quality and delivery. Take advantage of the almost ubiquitous cloud computing to quickly provide development, testing, staging and production environments that are as similar as possible for your team.

But also remember that most important than the tools is creating the culture. Create a team with cross-functional skills and involve it in the deployment activities. Make the deploy an activity that both developers and operations can do easily and at any time. Like yours project’s source code, make the infrastructure collectively owned and find your way of doing DevOps in baby steps³.