Chris Wanstrath (a.k.a. defunkt) just wrote an article on the github blog about how he cut the time to deploy github itself from fifteen minutes to fourteen seconds.
The starting point was the observation that since the standard Capistrano deployment tasks treat the code repository as a black box plugin, they aren't optimized. They treat git repositories pretty much the same as they do subversion repos.
In a search for a better way, Chris takes on a tour of the various Capistrano alternatives, Vlad the Deployer, Heroku's Rush, and finally Fabric a deployment framework written in Python, before coming full circle back to Capistrano and refactoring the deploy recipes, then rewriting the tasks to setup, update and rollback the code on the server using more "gitty" techniques.
Another thing which slowed the old deploy down was having a separate cap task to make each symlink needed on the server. Each cap task has some overhead, which Chris eliminated by making a single task which made all of the symlinks
The last change was moving the task of minimizing JavaScript and CSS from the machine running cap, where it was repeated for each server, to the servers themselves.
This is a great article, with lots of food for thought on how to use Cap and Git.