Best practices for building a simple, scalable cluster on Amazon EC2 for a Java web app

I want to build a Java web app and deploy it on EC2. It will be written in Java and will use MySQL. I was hoping to get some pointers on the actual deployment process and configuration. In particular I'm interested in the following topics:

• machine images (DIY vs ready made)
• mysql replication and backup to S3
• ways of deploying and redeploying the app to EC2 without interruptions
• firewa开发者_运维百科lls?
• load balancing and auto scaling
• cloudtools (or alternative tools)

I can only speak to a few of your discussion points from experience. I've had to strip out hyperlinks to the various Amazon products because I'm new to Stackoverflow and don't have enough rep to post more than one link.

Machine Images: While you can certainly start with your own machine image and convert it to an AMI with the EC2 AMI Tools, I prefer starting with one of Amazon's ready made images and customizing it to suit my needs. The advantage here is that you already know that the base image will deploy, you're more likely to get help on the forum or from the EC2 staff, and you don't have to go through the trouble of setting up a physical machine or your own VM in order to bundle the image and upload it. If you're using the EC2 API Tools, you can get a list of the available base images with ec2-describe-images -o amazon.

MySQL Replication and Backup: Check out the new(ish) Amazon Relational Database Service. It's designed to work with MySQL, can perform automatic backups, and scales easily.

Firewalls: Handling the firewalls for your instances is easy with the API tools. For example, you can create a group,

ec2-add-group condor –d “Condor Workers”

setup firewall rules for that group (bad example - opens all UDP and TCP ports for a CIDR range),

ec2-authorize condor -P tcp -p 0-65535 -s 129.127.0.0/16
ec2-authorize condor -P udp -p 0-65535 -s 129.127.0.0/16

and then launch your instances as part of the group, so that they inherit the firewall rules.

ec2-run-instances ami-12345678 –g condor –k mykeypair

The tricky part is going the other direction -- allowing your EC2 instances to communicate with your company/school/personal network. Since you don't know what IP your instances will have before they start (Amazon Elastic IP can alleviate this to some extent) you're generally forced to allow some subnet of the EC2 cloud.

You can also setup Iptables or additional firewalls on your instances.

Load Balancing: Consider Amazon Elastic Load Balancing. If that doesn't suit your needs, you can create your own "virtual cluster" and use whatever framework you like.