Dan Diephouse on Deployment : "I'm continually amazed at how hard of a problem deployment actually is. If you're going to be deploying any reasonably sized application you have an endless list of things to worry about:
* Taking the cluster up and down so there is no downtown * Managing the configuration of individual nodes * Operating system setup * Installation of required libraries/3rd party tools * Managing dev, QA, staging and production deployments * Schema migration/database updates * How to do rollbacks
[...]
There are a few other interesting tools out there."
I think one reason that there are only a few tools for deployment is that it's a general end to end problem, technically and organisationally. When you understand the enormity and complexity of bringing up even middling size systems, never mind big ones where components are constantly failing, it can be an overwhelming thing to bite off. Very possibly it means altering existing build systems, or even how the organisation itself is arranged (since deployment cross cuts standard boundaries such as development, qa and operations). Which could seem like ocean-boiling.
Tools like Puppet and SmartFrog take the problem space head-on and look to be general purpose solutions, so I agree with Dan's pointers to them. As an example Dan links to Steve Loughran';s deck on deploying a Hadoop cluster. They're well ahead of other FOSS tools that I know of and it's remarkable how few people know they exist. But to use those means skilling up and investing in their configuration language, which might seem arduous. Xen images, version control, language and distro packing all add more flavour to the mix - is your deployment unit a tarball, a warfile, a gem, a deb, an image, a git checkout? All of them? Knowing what the container unit is matters.
Hence you see people starting out with point solutions and dealing with either with problem subsets or specific pain points (code rollout but not configuration or health checks), app/framework specific tools like capistrano (deprecated afaik), tail-ending sftp/tomcat tasks onto your Antfile, or in-house shell scripts. None of these scale up as systems get bigger or more layered.
If you won't adopt an external framework, probably the most important thing to do is get past shell scripts to a declarative configuration language so deployment configurations can be managed in their own right. Getting the data structures and component models that represent the state of your running system, right is very important (both puppet and smartfrog have ways to describes and compose systems). Otherwise you're going to being rewriting those scripts forever. This will make your shell scripts more like command tools than one-offs.
"I hope that we start to see more core infrastructure managed by the infamous cloud people. Just write your app, upload, and tell it where to deploy. Then we can focus on building applications, which is what we really want to do anyway"
This offloading reminds me of the early promise of J2EE containers, but it turned into a vendor specific hell. I'd hope the hosted world can do better :) In any case, while good tools matter, deployment automation is as much about improved process quality.
Previous Topic
