This isn't your ordinary guide to deployment and tools. As a matter of fact, this is somewhat of an anti-guide -- a guide on how to approach tools and deployment -- rather than a guide to any specific approach. I was inspired for this by a post yesterday to Hacker News by Hynek Schlawack on Python Deployment Anti-Patterns and other tools used in doing Python web development. The article and associated discussion on HN are both excellent and I highly recommend a read.
However, I often find that articles and resources like these can actually be quite counterproductive to my as a developer. I look at these lists and think to myself, "Oh no, I'm not using all of the best practices! I should switch over my development stack to ensure maximum scalability and stability for the long term." Then I spend several hours researching the tools, reading tutorials, reading docs, messing with the server, probably breaking something, fixing it and generally coming back right back to where I started with maybe a few tweaks to my process.
The problem with this philosophy is that it's a bit like putting an air spoiler on my Honda Accord. Certainly the right spoiler, air-filter, tires, etc will make improve the performance of my car, but right now, I need to focus on getting to work and shipping a product. Playing around with a deployment system for a massively distributed system is pointless when you're building out your initial product and trying to nail down the core feature set and build a customer base.
However, sticking with what you know and never moving onward is a surefire recipe for stagnation and boredom as a developer. I got into technology because I didn't want to be bored and I wanted to be pushing the limits of what I can do and I want to have a solid stack and is up to date and using current best practices. There is usually a reason why smart, successful people are developing and implementing such practices.
So here has become my approach to keeping up with best practices and scaling up my applications, while not wasting time with pointless Yak Shaving: one new deployment feature per deployment. Instead of trying to build out the perfect stack from the beginning, I take what I currently have, and know and try one new thing. Maybe I switch to RabbitMQ instead of Redis for message passing. Maybe I use Supervisord instead of upstart for process management. Whatever it is, I optimize for one big improvement to my workflow per big improvement on the user side. This way the stack and architecture grows and improves as the interface does. The additional advantage here is that as I get more comfortable with these technologies, I start to think in those terms and start to build them into my projects from day 1.