2011 in review

The WordPress.com stats helper monkeys prepared a 2011 annual report for this blog.

Here’s an excerpt:

The concert hall at the Syndey Opera House holds 2,700 people. This blog was viewed about 32,000 times in 2011. If it were a concert at Sydney Opera House, it would take about 12 sold-out performances for that many people to see it.

Click here to see the complete report.

Automating EBS Volume Attach at Boot Time

A few years ago, I found myself attaching volumes to instances with some frequency. The volume often came from a snapshot which contained some test data. Like any lazy programmer, I didn’t want to do this work over and over again! I wrote this little utility which would examine the user data and mount a pre-existing volume, or create a new volume from a snapshot and attach that. Here’s the code;

import java.io.IOException;
import java.util.List;
import java.util.StringTokenizer;

import com.xerox.amazonws.ec2.AttachmentInfo;
import com.xerox.amazonws.ec2.EC2Exception;
import com.xerox.amazonws.ec2.EC2Utils;
import com.xerox.amazonws.ec2.Jec2;
import com.xerox.amazonws.ec2.VolumeInfo;

public class AttachVolume {

	public static void main(String [] args) {
		try {
			String userData = EC2Utils.getInstanceUserdata();
			StringTokenizer st = new StringTokenizer(userData);
			String accessId = st.nextToken();
			String secretKey = st.nextToken();
			String volumeOrSnapId = st.nextToken();

			Jec2 ec2 = new Jec2(accessId, secretKey);
			String volumeId = null;
			if (volumeOrSnapId.startsWith("snap-")) {
				String zone = EC2Utils.getInstanceMetadata("placement/availability-zone");
				// create volume from snapshot and wait
				VolumeInfo vinf = ec2.createVolume(null, volumeOrSnapId, zone);
				volumeId = vinf.getVolumeId();
				List<VolumeInfo> vols = ec2.describeVolumes(new String [] {volumeId});
				while (!vols.get(0).getStatus().equals("available")) {
					System.out.println(vols.get(0).getStatus());
					try { Thread.sleep(2); } catch (InterruptedException ex) {}
					vols = ec2.describeVolumes(new String [] {volumeId});
				}
			}
			if (volumeOrSnapId.startsWith("vol-")) {
				volumeId = volumeOrSnapId;
			}
			// attach volume and wait
			String instanceId = EC2Utils.getInstanceMetadata("instance-id");
			ec2.attachVolume(volumeId, instanceId, "/dev/sdh");
			List<VolumeInfo> vols = ec2.describeVolumes(new String [] {volumeId});
			while (!vols.get(0).getAttachmentInfo().get(0).getStatus().equals("attached")) {
				System.out.println(vols.get(0).getAttachmentInfo().get(0).getStatus());
				try { Thread.sleep(2); } catch (InterruptedException ex) {}
				vols = ec2.describeVolumes(new String [] {volumeId});
			}
		} catch (Exception ex) {
			System.err.println("Couldn't complete the attach : "+ex.getMessage());
			ex.printStackTrace();
			System.exit(-1);
		}
	}
}

Requirements

  • Java Runtime Environment (1.5 or greater)
  • Typica + and it’s dependencies
  • This utility (compiled)

A Few Words About the Code

The first thing you’ll notice is that user data is being parsed. The expectations are that the following items are passed via user data;

  • access id – AWS Access Id
  • secret key – AWS Secret Key
  • volumeOrSnapId – either a volume ID or snapshot ID

The code inspects the last parameter to see if it is a snapshot id. If so, it creates a volume and waits for it to become “available”. One that’s done, it gets the instance ID from meta data and attaches the volume at a hard-coded device.  (obviously, this could be in user data which is an exercise I’ll leave to the reader)

On a linux machines, I’d often call this from the /etc/rc.local script. I should also note that this works just as well with Eucalyptus due to its API fidelity with Amazon EC2

There you have it!

Using the Ruby Fog library to connect with Eucalyptus

There is a popular Ruby cloud library called Fog. I’ve seen a number of applications that use this for connecting to AWS EC2. I’ve done some testing and it is pretty simple to get Fog talking to Eucalyptus. The first thing you need are your access id and secret key, much like you’d get from EC2. These are available on the credentials tab in Eucalyptus. The other thing you need to specify is the endpoint. In the case of Eucalyptus, that will point to the cloud endpoint for your private cloud (or in this example, the Eucalyptus Community Cloud).

This is an example credentials file, stored in ~/.fog

#######################################################
# Fog Credentials File
#
:default:
  :aws_access_key_id:       IyJWpgObMl2Yp70BlWEP4aNGMfXdhL0FtAx4cQ
  :aws_secret_access_key:   7DeDGG2YMOnOqmWxwnHD5x9Y0PKbwE3xttsew
  :endpoint:                http://ecc.eucalyptus.com:8773/services/Eucalyptus

Notice that the eucalyptus endpoint requires port 8773 and the “/services/Eucalyptus” path.
You can use the Fog interactive tool to test this out. Notice we’re using the AWS compute provider because the Eucalyptus cloud is API compatible with EC2.

# fog
  Welcome to fog interactive!
  :default provides AWS and AWS
>> servers = Compute[:aws].servers
  <Fog::Compute::AWS::Servers
    filters={}
    []
  >
>>

As you can see, there are no servers running. By replacing “servers” with “images”, you can show a list of images available on the ECC.
To start an instance, you can run a command like this;

>> servers = Compute[:aws].servers.create(:image_id => 'emi-9ACB1363', :flavor_id => 'm1.small')
  <Fog::Compute::AWS::Server
    id="i-3D7A079C",
    ami_launch_index=0,
    availability_zone="open",
    block_device_mapping=[],
    client_token=nil,
    dns_name="euca-0-0-0-0.eucalyptus.eucasys.com",
    groups=["default"],
    flavor_id="m1.small",
    image_id="emi-9ACB1363",
    kernel_id="eki-6CBD12F2",
    key_name=nil,
    created_at=Wed Oct 19 15:19:16 UTC 2011,
    monitoring=false,
    placement_group=nil,
    platform=nil,
    product_codes=[],
    private_dns_name="euca-0-0-0-0.eucalyptus.internal",
    private_ip_address=nil,
    public_ip_address=nil,
    ramdisk_id="eri-A97113E4",
    reason="NORMAL:  -- []",
    root_device_name=nil,
    root_device_type=nil,
    state="pending",
    state_reason=nil,
    subnet_id=nil,
    tenancy=nil,
    tags=nil,
    user_data=nil
  >
>> servers = Compute[:aws].servers  <Fog::Compute::AWS::Servers
    filters={}
    [
      <Fog::Compute::AWS::Server
        id="i-3D7A079C",
        ami_launch_index=0,
        availability_zone="open",
        block_device_mapping=[],
        client_token=nil,
        dns_name="euca-0-0-0-0.eucalyptus.eucasys.com",
        groups=["default"],
        flavor_id="m1.small",
        image_id="emi-9ACB1363",
        kernel_id="eki-6CBD12F2",
        key_name=nil,
        created_at=Wed Oct 19 15:19:16 UTC 2011,
        monitoring=false,
        placement_group=nil,
        platform=nil,
        product_codes=[],
        private_dns_name="euca-0-0-0-0.eucalyptus.internal",
        private_ip_address=nil,
        public_ip_address=nil,
        ramdisk_id="eri-A97113E4",
        reason="NORMAL:  -- []",
        root_device_name=nil,
        root_device_type=nil,
        state="pending",
        state_reason={},
        subnet_id=nil,
        tenancy=nil,
        tags={},
        user_data=nil
      >
    ]
  >

You’ll notice that now the instance (which is in “pending” state) appears in the list of servers.
I won’t show more here, but we’ve established basic connectivity to a Eucalyptus cloud. I hope this helps enable many existing and new applications to work with Eucalyptus!

VMWare Fusion Error Recovery

I’m running VMWare Fusion on my MacBook Pro. I run an Ubuntu Maverick VM for some development work. One time, I tried restarting it from a suspended state and kept getting an error that “unexpected signal 10 received”. I was told to look for a log file, send it to VMWare support, etc, etc. I was unable to get the VM back up and had no option to boot the VM from scratch.

The answer turns out to be fairly simple. In my ~/Documents/Virtual Machines/ directory, there’s a directory for my VM. If I browse there in the finder, I can right-click and select “show package contents”. In the files contained in the VM directory, there is one ending in vmem and a directory ending in vmem.lck. I moved both of those to the trash and tried launching the VM again. This time, it said there was an error and would I like to preserve the state or not. I chose not to, and then I was able to boot the VM from scratch!

How I got Amazon’s app store on my AT&T phone

AT&T doesn’t enable side-loading apps. They want you to go through their app marketplace. I’m sure it’s a control thing, but I’m a big boy, I can decide what I want to download and what I don’t. Since Amazon launched their Android App Store today, I thought get my phone (Samsung Captivate) enabled to run other apps. Even as one blog points out that it takes 8 steps, I’ll show you here that it’s worse for some! I found instructions here (or here) that were very helpful (follow the link on the 3rd post). In a nutshell, you root the phone, install busybox, copy the settings.db file to the sd card, move it to your computer and use an SQLite plugin to edit it, move it back to the phone and reboot. It was almost that simple. Rooting worked fine for me. I couldn’t get busybox installed. I even tried pulling it from another source and installing it manually. In any case, all you need from it is the “cp” (or copy) utility. I got around that using cat with redirect.

Once you get to the step where you’re supposed to copy files to the sd card, run these commands instead


cat /dbdata/databases/com.android.providers.settings/settings.db >/sdcard/settings.db
cat /dbdata/databases/com.android.providers.settings/settings.db >/sdcard/settings.db.backup

I found the SQLite plugin for Firefox easy to use. Once you copy the file back to the sd card, you’ll need to move it back to the system location, so use the same “cat” trick I used above.
Once you reboot, you’ll be able to follow the instructions on the Amazon site to download their app store app (and it will work now).
So, enjoy Angry Birds, or whatever other app of the day you grab. Oh, and we (the collective we) should probably buy a few apps to keep new apps coming!

Debugging a Firefox Plugin

This entry will be so brief, it will rival a tweet, but this is pure gold.

If there is any one thing that I found helpful when I needed to debug a Firefox plugin, it was Chromebug. This is the debugger used to debug Firebug, so you know it has what it takes! (yes, I tried Venkman. Fine for some things… just no this) So, get it, follow the instructions and be happy!

Custom Stickers, easily

I thought I’d share an experience I had when trying to get some custom stickers done for the company I work for. In the past, I tried out cafepress.com and found the cost/sticker to be pretty high, though I was ordering small quantities. Last fall I attended a bar camp and one of the sponsors was StickerMule. So, I thought I’d give them a try. I was pleasantly surprised to see the prices listed were all I had to pay. No shipping or tax. They even handled all of artwork and layout for free! On one of my stickers, I wasn’t happy with how the proof looked, and supplied some feedback. They got a new proof turned around in no time, which I was happy with! Here’s a picture of what I received, and I’m very happy with the results!