DevOps: how we got here

Developer toolset

From the developer point of view, there are some tools involved in the source-to-deploy process

  • Source control management tools: Subversion, Git, Mercurial, Perforce,…
  • Build tools: Maven, Ant, Ivy, Buildr, Graddle, Rake,…
  • Continuous Integration tools: Continuum, Jenkins, Hudson, Bamboo,…
  • Repository (Artifact) management tools: Archiva, Nexus, Artifactory,…

The #1 programmer excuse for legitimately slacking off: My code is compiling

When everything is set together, we can have a CI schedule that is building automatically the changes from the SCM as they are committed, deploying to an artifact repository the result of the build or sending a notification if there is any error. Everything fully automated. A change is made to SCM, the CI server kicks in, builds and runs all sort of tests (unit, functional, integration,…) while you go off for a sword fight with your coworkers.

Now what? somebody sends by email the tarball, zipfile,… to the operations team? oh, no that would be too bad. Just send them the url to download it… And even better send some instructions, a changelog, upgrade task list,…

What developers do today to specify deployments and target environments is not enough. 

The simplest solutions are often the cleverest. They are also usually wrong.

Using tools like Maven in the Java world or Bundle in Rubyland you can explicitly list all the dependencies and versions you need. But there are some critical dependencies that are never set.

It is just too simple.

Packages installed, C libraries, databases, all sort of OS and service level configuration,… That’s the next level of dependencies that should be explicitly listed and automated.

For example, think about versions of libc, postgresql, number of connections allowed, ports opened, opened file descriptors limit,…

Operations

Requirements

From the point of view of the operations team the number of requirements is complex: operating system, kernel version, config files, packages installed,…

And then multiply that for several stage configurations that most likely won’t have the exact same configurations.

  • dev
  • QA
  • pre-production
  • production

Deployment

Deployment of the artifacts produced by the development team is always a challenge

  • How do I deploy this?
  • Reading the documentation provided by the development team?
  • Executing some manual steps?

That is obviously prone to errors

Cloud

It’s nothing new but it has increased with the proliferation of Cloud based environments, making it easier and easier to run dozens or hundreds of servers at any point in time. Even knowing how to deploy to one server, how is it deployed to all those servers? what connections need to be established between servers? how is it going to affect the network?

FOSDEM

The slides from my From Dev to DevOps talk at FOSDEM 2012 Brussels are up in Slideshare. The material is also in the Lanyrd page.

The conference was huge, I’ve heard that over 4000 people showed up this year, for this free (as in both software and beer) event, organized on several tracks. Not bad considering that the temperature was between -15 and -6C (5 and 23 Fahrenheit) and got a lot of snow.

I spoke on the main auditorium with capacity for 1400 people, there was plenty of space. Unfortunately (for me, but great for the other speakers and the organization) some other devrooms got filled quickly, and couldn’t get into the Configuration and Systems Management Devroom, which should have been really good as nobody left as I was waiting outside between talks 😦

A lot of care by the organization, including a note with a 24/7 phone number in case I got “lost/confused/arrested” (which makes me think it must have happened before), and a full commitment to promote belgian beer 😉

And thanks to the people that attended my talk and engaged through twitter, got great feedback and hints for improvements on the future of development and infrastructure automation!

Grand Place 360 panorama