Automating EBS Volume Attach at Boot Time

A few years ago, I found myself attaching volumes to instances with some frequency. The volume often came from a snapshot which contained some test data. Like any lazy programmer, I didn’t want to do this work over and over again! I wrote this little utility which would examine the user data and mount a pre-existing volume, or create a new volume from a snapshot and attach that. Here’s the code;

import java.io.IOException;
import java.util.List;
import java.util.StringTokenizer;

import com.xerox.amazonws.ec2.AttachmentInfo;
import com.xerox.amazonws.ec2.EC2Exception;
import com.xerox.amazonws.ec2.EC2Utils;
import com.xerox.amazonws.ec2.Jec2;
import com.xerox.amazonws.ec2.VolumeInfo;

public class AttachVolume {

	public static void main(String [] args) {
		try {
			String userData = EC2Utils.getInstanceUserdata();
			StringTokenizer st = new StringTokenizer(userData);
			String accessId = st.nextToken();
			String secretKey = st.nextToken();
			String volumeOrSnapId = st.nextToken();

			Jec2 ec2 = new Jec2(accessId, secretKey);
			String volumeId = null;
			if (volumeOrSnapId.startsWith("snap-")) {
				String zone = EC2Utils.getInstanceMetadata("placement/availability-zone");
				// create volume from snapshot and wait
				VolumeInfo vinf = ec2.createVolume(null, volumeOrSnapId, zone);
				volumeId = vinf.getVolumeId();
				List<VolumeInfo> vols = ec2.describeVolumes(new String [] {volumeId});
				while (!vols.get(0).getStatus().equals("available")) {
					System.out.println(vols.get(0).getStatus());
					try { Thread.sleep(2); } catch (InterruptedException ex) {}
					vols = ec2.describeVolumes(new String [] {volumeId});
				}
			}
			if (volumeOrSnapId.startsWith("vol-")) {
				volumeId = volumeOrSnapId;
			}
			// attach volume and wait
			String instanceId = EC2Utils.getInstanceMetadata("instance-id");
			ec2.attachVolume(volumeId, instanceId, "/dev/sdh");
			List<VolumeInfo> vols = ec2.describeVolumes(new String [] {volumeId});
			while (!vols.get(0).getAttachmentInfo().get(0).getStatus().equals("attached")) {
				System.out.println(vols.get(0).getAttachmentInfo().get(0).getStatus());
				try { Thread.sleep(2); } catch (InterruptedException ex) {}
				vols = ec2.describeVolumes(new String [] {volumeId});
			}
		} catch (Exception ex) {
			System.err.println("Couldn't complete the attach : "+ex.getMessage());
			ex.printStackTrace();
			System.exit(-1);
		}
	}
}

Requirements

  • Java Runtime Environment (1.5 or greater)
  • Typica + and it’s dependencies
  • This utility (compiled)

A Few Words About the Code

The first thing you’ll notice is that user data is being parsed. The expectations are that the following items are passed via user data;

  • access id – AWS Access Id
  • secret key – AWS Secret Key
  • volumeOrSnapId – either a volume ID or snapshot ID

The code inspects the last parameter to see if it is a snapshot id. If so, it creates a volume and waits for it to become “available”. One that’s done, it gets the instance ID from meta data and attaches the volume at a hard-coded device.  (obviously, this could be in user data which is an exercise I’ll leave to the reader)

On a linux machines, I’d often call this from the /etc/rc.local script. I should also note that this works just as well with Eucalyptus due to its API fidelity with Amazon EC2

There you have it!

Connecting to the Eucalyptus Community Cloud with typica

Eucalyptus recently announced a public “cloud” sandbox known as Eucalyptus Community Cloud. It is a place where you can kick the tires to some degree and since they support a subset of the Amazon EC2 API, you can generally point EC2 tools at the ECC. This post will deal with using typica to interact with the ECC from within your Java software.

First thing to do is follow the ECC link above and create an account. If you already have an account to get into the Eucalyptus forums, you can login and apply for an ECC account. Once you get a confirmation e-mail and confirm the account, you’ll be able to login and get your access id and secret key. To do that, visit the ECC, login and select “show keys”, which reveal the QueryID (access id) and Secret Key. While you’re hear, you should also download credentials. This gives you a zip that contains something we’ll need later.


Jec2 ec2 = new Jec2(props.getProperty("aws.accessId"), props.getProperty("aws.secretKey"), true, "ecc.eucalyptus.com", 8773);
ec2.setResourcePrefix("/services/Eucalyptus");

Let me explain this code. The first line creates a new Jec2 object, that is configured to talk to the ECC. The “props” variable came from reading a property file containing the access id and secret key. The next parameter specifies SSL. Then, you pass the hostname for the ECC and the port it uses. After that, it would be business as usual. The EC2 sample code demonstrates some normal operations, and the API docs give a more complete picture.

When running the code, there’s a special option you’ll need as compared to using typica to talk to AWS. Since Eucalyptus clouds are generally installed with self signed SSL certs, you’ll need to specify a file that came with that credentials download in your java options. If you don’t do this, you’ll likely see a “SSLPeerUnverifiedException: peer not authenticated” error.


$ java ... -Djavax.net.ssl.trustStore=<path to files from credentials zip>/jssecacerts ... TestJec2

A New Adventure

My professional career has been spent at 2 companies, Eastman Kodak and D.O.Tech/directThought (rebranded 9 years in). At Kodak, I worked on blood analyzers (which they spun off to J&J), Photo CD (which was made obsolete by newer technologies) and Picture Maker, which is still going strong after N generations of hardware/software. At directThought, I had the joy of working with a lot of great people and working on some interesting projects. I worked on a Picture Maker-like kiosk/web-app/desktop app combination at Xerox. They even created a new division for that project called Pixography. We had XML templates that described printed products like greeting cards, calendars, business cards, brochures and photo books (to name a few). Java 2D rendered everything for print and preview. We had tight integration between the 3 different apps but that project died in its original form, but lived on in spirit in a custom production printing installation out on the west coast. After that, I worked on some enterprise apps for Pfizer, a payroll application for Paychex, then back to more custom apps for the services arm of Xerox. At that point, I got involved in Amazon Web Service and started kicking the tires on this new service called EC2. During that time, I started my most successful open source project called typica, which is still has a lot of users. After Xerox, I helped a number of customers run their apps on AWS’s infrastructure. We were fortunate enough to be come an inaugural AWS System Integrator. I was also asked to learn how to write apps for this hot new platform called the iPhone. I’ve had a couple of apps in the app store, and worked on a few more. I also got to go to the only WWDC where Steve didn’t deliver the keynote (because he was getting a new liver). All in all, a pretty great experience with may interesting technologies under my belt.

Now, I feel like it is time for a change. I’ve just accepted a job with Eucalyptus Systems! They build infrastructure that powers clouds. They have a lot of great people working there and I am looking forward to doing my part to help the company grow, if not flourish in this exciting space. Since they just started business last year, I can say I’m now part of a fast growing startup! Very excited!

Persistent Counters in SimpleDB

I’ve already discussed the new consistency features of Amazon SimpleDB. One of the things people have wished for in SimpleDB was a way to manage a universal counter, something similar to an auto-incrementing primary key in MySQL. The consistency features allow clients to implement such a thing very easily. The following is an algorithm;

Read value
Write value+1, but only if the previous value is what we just read
If write failed, increment value and try again
else return new value

To make it easy for Java programmers to do this with typica, I’ve added a Counter class. Usage is very simple as you can see by this example;

SimpleDB sdb = new SimpleDB("AccessId", "SecretKey");
Domain dom = sdb.createDomain("MyDomain");
Counter c = new Counter(dom, "counter1");
for (int i=0; i<20; i++) {
	System.err.println("next val = "+c.nextValue());
}

This code creates a counter and initializes it if there isn’t a current value. It uses a Iterator-like interface, but there is no test for next value because there always is one. The Counter object is stateless, so it relies totally on SimpleDB for its value. This will work very well on multiple app servers, all relying on the same counter for unique values.

To avoid this blog getting out of date, I won’t include the counter code here, rather you can browse it in SVN.

Code has been checked into SVN r311. I’ll update this post once the new version of typica is released which includes this.

For those seeking a more pythonic version, have a look here.

QueryTool now exports data

There are some things I’ve been wanting in the QueryTool, so I just threw them in. Here’s the list;

  • proxy support (add proxy host and port to the command line args)
  • sortable results table (via column header click)
  • export result data to CSV file
  • scrollable query scratchpad

Screen shot 2009-09-18 at 10.00.01 AM

The tool can be downloaded here

To run it, type;

java -jar QueryTool1.2.jar <AccessId> <SecretKey> [ProxyHost] [ProxyPort]

QueryTool 1.1, even more to like!

Screen shot 2009-08-05 at 4.29.39 PM

A number of you have requested some new features, and some of them have made it into this new release. You can now add and delete domains from within the tool. Also, clicking on results lets you copy the cell or row into the clipboard. This later feature can be really handy for putting item names into queries. So, check it out! Once you download the jar, you run it by using this command;

java -jar QueryTool1.1.jar <accessId> <secretKey>

Amazon CloudWatch with Java/typica

Recently, Amazon announced that it’s CloudWatch service went into public beta. I’ve been involved with the private beta of this and the Elastic Load Balancing and Auto Scaling services. I’ve just completed testing of the CloudWatch monitoring service APIs in typica and thought I’d share some of what has been added.

First of all, the Jec2 class has 2 new methods, monitorInstances(..) and unmonitorInstances(..). They do exactly what you’d expect by turning monitoring on or off for one or more instances. What I think more people will use is the new flag on LaunchConfiguration to enable monitoring when you launch an instance. Also, if you describe instances, you’ll get the monitoring status back now also.

The real CloudWatch APIs are in their own package. I did this because it seems like while they are initially released for EC2, they are written to allow monitoring other service also (hence the namespace parameter). The new API has only two methods. The first lets you list the metrics you can query in the second call. To do this, you can use some code like this;

Monitoring mon = new Monitoring(props.getProperty(“aws.accessId”), props.getProperty(“aws.secretKey”));
List<Metric> metrix = mon.listMetrics();
for (Metric m : metrix) {
System.out.println(“name = “+m.getName()+”:”+m.getNamespace());
for (Dimension dim : m.getDimensions()) {
System.out.println(”   “+dim.getName()+”: “+dim.getValue());
}
}
Monitoring mon = new Monitoring(accessId, secretKey);
List<Metric> metrix = mon.listMetrics();
for (Metric m : metrix) {
	System.out.println("name = "+m.getName()+":"+m.getNamespace());
	for (Dimension dim : m.getDimensions()) {
		System.out.println("   "+dim.getName()+": "+dim.getValue());
	}
}
Here is some of the output (trucated because there is a lot more);
     [java] name = NetworkIn:AWS/EC2
     [java] name = NetworkOut:AWS/EC2
     [java]    ImageId: ami-85d037ec
     [java] name = NetworkOut:AWS/EC2
     [java] name = DiskWriteBytes:AWS/EC2
     [java]    InstanceType: m1.small
     [java] name = CPUUtilization:AWS/EC2
     [java]    InstanceType: m1.large
     [java] name = DiskWriteBytes:AWS/EC2
     [java]    InstanceType: m1.large
     [java] name = DiskReadOps:AWS/EC2
     [java]    InstanceId: i-1de3a674
     [java] name = DiskWriteOps:AWS/EC2
     [java]    InstanceType: m1.small
     [java] name = DiskReadOps:AWS/EC2
     [java]    ImageId: ami-24fa86b
     [java] name = DiskReadOps:AWS/EC2
     [java]    InstanceId: i-51423838

Once you have an instance or an image you’d like to monitor, you can use some code like this to fetch the data;

List<Statistics> stats = new ArrayList<Statistics>();
stats.add(Statistics.AVERAGE);

Map<String, String> dimensions = new HashMap<String, String>();
// can be InstanceId, InstanceType, ImageId
dimensions.put("ImageId", "ami-85d037ec");

Date end = new Date();	// that means now
end = new Date(end.getTime() + 3600000*5); // need to adjust for GMT
Date start = new Date(end.getTime() - 3600000*24);	// 1 days ago
MetricStatisticsResult result = mon.getMetricStatistics(
				60,	// must be multiple of 60
				stats,	// see above
				"AWS/EC2",
				dimensions,
				start,	// start of interval
				end,	// end of interval
				// can be NetworkIn, NetworkOut, DiskReadOps,
				// DiskWriteOps, DiskReadBytes, DiskWriteBytes,
				// CPUUtilization
				"CPUUtilization",
				StandardUnit.PERCENT,
				null);
System.out.println("metrics label = "+result.getLabel());
for (Datapoint dp : result.getDatapoints()) {
	System.out.println(dp.getTimestamp().getTime().toString()+
			" samples:"+dp.getSamples()+" "+dp.getAverage()+" "+dp.getUnit());
}
It can be useful monitor by ImageId when you’re running a pool of servers (like with the auto scaling service). I’ve tried to include comments within the code that indicate appropriate values because it can get complicated..
     [java] metrics label = CPUUtilization
     [java] Fri May 22 10:56:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 11:42:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 12:55:00 EDT 2009 samples:1.0 1.54 Percent
     [java] Fri May 22 12:41:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 13:10:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 10:09:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 12:51:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 12:40:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 10:07:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 13:41:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 10:34:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 12:01:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 10:17:00 EDT 2009 samples:1.0 0.39 Percent
     [java] Fri May 22 11:39:00 EDT 2009 samples:1.0 1.15 Percent
     [java] Fri May 22 10:06:00 EDT 2009 samples:1.0 0.38 Percent
     [java] Fri May 22 12:10:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 12:09:00 EDT 2009 samples:1.0 0.76 Percent
     [java] Fri May 22 13:46:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 10:39:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 12:11:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 12:03:00 EDT 2009 samples:1.0 1.15 Percent
     [java] Fri May 22 11:32:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 10:44:00 EDT 2009 samples:1.0 0.0 Percent
     [java] Fri May 22 12:45:00 EDT 2009 samples:1.0 0.0 Percent
This code is available in typica SVN as of r265. Look for typica release 1.6 which will contain CloudWatch, ElasticLoadBalancing and AutoScaling once a little more testing has been completed.

Query Tool for Amazon SimpleDB

Although these projects are never really finished, I can say I’ve completed version 1 of the Amazon SimpleDB Query Tool. This is built on top of a new SimpleDB API that will be part of an upcoming typica release. The code currently resides in a branch, but will hopefully get merges into trunk in the next few weeks.

OK, more about the tool. It was built to provide a convenient way to test queries. That’s it. Towards that end, there is a list of features I included that really met a need for me.

  • flexible query workspace (scratch pad)
  • run query on the line where the cursor is
  • allow domain selection via the UI
  • display domain metadata
  • show results from several queries at once
  • show box usage and other stats

Now, the moment you’ve all been waiting for, a screenshot!

querytool

Right now, the code is still in SVN, so if you’d like to run it, you’ll need to check out the branch and build it.  If you get that far, to run it, you can use this command, “ant test.main -Dclass=QueryTool -Dargs=”<access id> <secret key>”

For an official release, I’ll make an executable jar, so you’d run “java -jar QueryTool.jar <access id> <secret key>”