Amazon EC2: A bullet-proof demo environment that facilitates collaborative and iterative development

Bookmark Blog Post

	del.icio.us
	Digg
	DZone
	Furl
	Reddit

Jesper Joergensen's Blog | March 24, 2008   5:24 PM | Comments (0)

Over the last couple of months, I've been spending some time on Amazon EC2. I started by installing ALBPM Enterprise with an Oracle DB and it went pretty smooth. I moved on to installing ALI, ALIC, ALBPM Enterprise and Oracle on another image and while it took a little longer to get all the parts together I've now also got that one up and running minus a few last items. I took extensive notes but most of my experience will be outdated very soon when we release ALI 6.5 and ALBPM 6.1. These new releases will radically simplify the combined setup and instead of spending the time to clean up and double check my install notes, I am going to wait for this new release and write a post on how to run it on EC2.

I promised in my earlier post that I would discuss the reasons and pros and cons of using Amazon EC2. Here are some of the use cases I see for the service in increasing order of complexity/applicability/uncertainty of value:

1. A bullet-proof demo environment that facilitates collaborative and iterative development of demos
2. A bullet-proof test environment that is highly scalable and can be heavily automated
3. Extra capacity for on-premise systems that are already exposed to external users as well as internal systems (but security implications must be considered)
4. Primary hosting vehicle for certain external and internal solutions configured and managed on premise (for internal solutions, security implications must be considered)
5. Hosting platform for independent service providers (highly dependent on cost structure and value added by the services of the ISV, in some ways this is just a special case of 3 and 4)

The rest of this post describes the first of these use cases.

A bullet-proof demo environment that facilitates collaborative and iterative development of demos

Building and giving demos of enterprise software can be quite a daunting task. A demo often needs to show more than just the basic features. It needs to show how a product works with other products and it can be quite complex to get all this software installed and configured properly. Moreover, often the demo itself will change the state of the system which in best case will require you to "undo" or reset it in some way, in worst case it can make the system unstable and cause future demos to fail. Now, you might say that good quality software shouldn't behave like this. But keep in mind that the software wasn't designed for demos but rather for large-scale, perpetual deployments and requirements are very different for these two scenarios. To be more accurate, software vendors actually do consider demo requirements as part of product planning, but they don't always get high priority and even if they do, there is an important use case for complex demos at the cutting edge that cannot be planned for in advance.

One approach used widely today is to use vmware. With vmware you can build a demo on a vmware image, then "snapshot" it and now you can rerun it as many times as you like without anything breaking.

The problem with the vmware image is that it requires a lot of resources on your local laptop. It can tend to run slower than you'd like and the image often takes up many gigabytes of hard drive space. With an 80-120 GB hard drive, you won't be able to store a whole lot of images, so most people have a separate external hard drive for this purpose. This is annoying in itself, but the thing most people complain about is that it's not easy to share the image with your fellow demoers (pre-sales engineers for example).

Building demos should really be a collaborative and incremental activity. You start with a v1.0, your buddy then adds Gadget X which makes it even cooler. He shares it with the team and other people come up with further improvements and so on. This is really hard if you rely on sneakernet for sharing images. If you're a pre-sales engineer, you're often on the move and it just takes too long to download multi-gigabyte images over some hotel connection. You definitely don't want to rely on it for your demo the next day, and you will rarely have the time to snapshot and upload any improvements you've made. The result is that this just doesn't work. In our SE organization I think only a few people rely on vmware images for demos and they all really wish they could collaborate more.

Amazon EC2 makes this possible. It has all the benefits of a vmware image and on top of that it is stored in the cloud and it runs in the cloud. With a little additional magic you can even build demo configurations consisting of multiple hosts that talk to each other and emulate, say an SOA. Your images are stored on and loaded from S3, Amazon's storage service. So you never have to download to / upload from your laptop. As with vmware, your image can be booted over and over again and you will always start with the exact configuration that you want, so you know your demo will run smoothly.

But the most interesting advantage is that you can share your images with other EC2 users. Say a friend on your team would like to use your demo for a meeting the next day and she wants to add a few bells and whistles to it. Here's how you do it:

1. You add access privileges to your image for your friend's EC2 account.The image is still stored on your S3 account and is read-only, so nothing will happen to it.
2. Your friend can now boot up an instance from your image
3. Once booted your friend adds some new cool features and bundles up the new configuration as a new image stored on her own S3 account.
4. Your friend demos the new stuff to a customer. Everything goes smoothly because the demo is booted from an image that was tested the night before. It can be repeated endlessly without breaking.
5. Now your friend can share this new image with you and everybody else on the team. Everyone can now benefit from these new enhancements with having to shuttle gigabytes of data over the wires or meet in person for some hard drive sharing.

I am not sure if this sounds like modest benefits to existing vmware users. But I am convinced that this model has the potential to dramatically increase creativity and "prototype/demo innovation" which I think is really important to competitive enterprise software sales. I am very interested in feedback from SEs and people in similar roles that may be using this approach or just agree/disagree with this use case.

Costs and limitations

Of course Amazon EC2 comes with some costs and some limitations. The costs are pretty straight forward (copied directly from the EC2 page):

Instances

$0.10 - Small Instance (Default)

1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of instance storage, 32-bit platform

$0.40 - Large Instance

7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each), 850 GB of instance storage, 64-bit platform

$0.80 - Extra Large Instance

15 GB of memory, 8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each), 1690 GB of instance storage, 64-bit platform

Pricing is per instance-hour consumed for each instance type. Partial instance-hours consumed are billed as full hours.

One EC2 Compute Unit provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor. This is also the equivalent to an early-2006 1.7 GHz Xeon processor referenced in our original documentation. See Measuring Compute Resources for a complete description of an EC2 Compute Unit.

See Amazon EC2 Instance Types for details on available instance configurations.

Data Transfer

$0.10 per GB - all data transfer in

$0.18 per GB - first 10 TB / month data transfer out
$0.16 per GB - next 40 TB / month data transfer out
$0.13 per GB - data transfer out / month over 50 TB

Data transfer "in" and "out" refers to transfer into and out of Amazon EC2.

Data transferred within the Amazon EC2 environment, or between Amazon EC2 and Amazon S3 or Amazon SimpleDB, is free of charge (i.e., $0.00 per GB), except data transferred between Amazon EC2 and Amazon S3-Europe, which will be charged at regular rates. All other Amazon S3 storage and request charges will still apply.

Amazon S3 usage is billed separately from Amazon EC2; charges for each service will be billed at the end of the month.

With respect to limitations, here's what comes to my mind:

• EC2 requires network connectivity. This is potentially a severe limitation because when you're on site at a customer you may not be able to get online. But I am suspecting that it's becoming less of a problem as Internet access becomes more pervasive with corporate guest WI-FI networks and 3G mobile network access.
• EC2 is not optimal for demonstrating desktop components: If you want to demo an Eclipse plug-in or a full-fledged desktop app, EC2 might not be the right choice. But it's not impossible. I've seen people putting Windows images on EC2 and you can also get X11 forwarding up and running fairly easily.
• EC2 runs Linux (mostly): Like I said above, I've actually seen descriptions for how to get Windows running, but I am not sure how easy it is. Anything that runs on RedHat/Fedora is pretty easy to put on EC2. If you need Windows or Mac, EC2 may provide less benefits.

Anything I've missed?

Comments