Configuring the Eucalyptus User Console with a Reverse Proxy

The Eucalyptus User Console can be used standalone, but generally people run Tornado apps (as this is) behind a reverse proxy. There are a few reasons, but most commonly, it is so SSL termination can be handled in one place and several Tornado instances can be managed behind one front end. FriendFeed (who developed Tornado) talked about configuring one Tornado instance per core behind Nginx as the reverse proxy. This is what I’ll talk about in this post.

Eucalyptus Logo

The Eucalyptus User Console is built on top of Tornado. Each time you run the console server, you are getting a Tornado instance. For the 3.2 release, there isn’t a convenient way to set up several instances of the console server on one machine. Thankfully, it isn’t terribly difficult to make some modifications which allow this setup. The problems are really around different config and pid files. Logging is all pushed through syslog, so it ends up in /var/log/messages.

Fixing the config file is simple. We need separate config files because that file specifies the port used. After a package install of the eucalyptus-console (that’s the package name), you will find /etc/eucalyptus-console/console.ini which we need to duplicate for each copy of the server we wish to run. I made separate files in that directory called console-1.ini and console-2.ini. In those files, I set up the uiport value to 8880 and 8881. I also recommend turning off SSL since we’ll set up SSL termination with Nginx. To do that, comment out sslcert and sslkey values in both new config files.

To use the new config files, the startup script needs to be changed. I chose a simple route. In /etc/init.d, copy the eucalyptus-console script to eucalyptus-console-1 and eucalyptus-console-2. In those scripts, you can change the config file name to match the files we created before. For good measure, I also changed the “Provides:” value, SERVICE and LOCKFILE variables. The result is that you’ll now be able to run “service eucalyptus-console-1 start” and “service eucalyptus-console-2 start”.

The other wrinkle is the pid file. That file is specified in the init script, but also in the python code for the server. I’ve committed a change to the euca-console-server file (you’ll find in /usr/bin) and checked that into github. It will be on the “testing” branch and likely in “master” soon. This change allows passing in the pid file location so it is no longer hard-coded. What we can do with that is to specify the PIDFILE variable in the init scripts much like was done for the config file. I’ll attach copies of these files to this post so you can see for yourself.

nginx logo

Once you are able to start 2 (or more) copies of the console server, you can easily test those by pointing your browser to the host and ports 8880 and 8881. Now, we need to install and configure Nginx. On CentOS, you can install using “yum install nginx”.  I’m using the following config file (/etc/nginx/nginx.conf”).


user nginx;
worker_processes 10;

error_log /var/log/nginx/error.log;

pid /var/run/nginx.pid;
events {
 worker_connections 1024;
}

http {
 upstream euca-ui {
 server 127.0.0.1:8880;
 server 127.0.0.1:8881;
 }

server {
 listen 80;
 server_name euca-ui;
 keepalive_requests 500000;
 keepalive_timeout 1000;
 location / {
 proxy_pass http://euca-ui;
 }
 }

log_format main '$remote_addr - $remote_user [$time_local] "$request" '
 '$status $body_bytes_sent "$http_referer" '
 '"$http_user_agent" "$http_x_forwarded_for"';

access_log /var/log/nginx/access.log main;
}

Notice the upstream construct pointing to the 2 servers we have configured. Simply running like this, you’ll be able to get the login screen on port 80. When you try logging into the console, you’ll see a failure after requests get shifted to the “other” console server where you aren’t authenticated. We need Nginx to provide session stickyness. I found the ip_hash directive to be helpful. It may not be optimal, but it does tie requests of a given client IP to a specific server. It isn’t a session stickyness, but it “almost” as good. Simply add the line “ip_hash;” in the upstream block on the line prior to the server list.

Now, I’m able to login and see use the console and it still appears to be at port 80 on the host. There are two other things I’d like to address before calling this done.

1. I can’t tell which of the console servers is logging messages. Need to make messages unique to each instance.

2. Enable ssl termination so that we can interact on port 443 and have some further assurance of security.

I haven’t figured out a simple way to customize the log output via syslog, so let’s talk about ssl first.

Turning on SSL is quite easy. The package start script probably already generated self-signed certs which we can use. Modify the server directive in the nginx.conf file like this;


server {
 listen 443 ssl;
 ssl_certificate /etc/eucalyptus-console/console.crt;
 ssl_certificate_key /etc/eucalyptus-console/console.key;

Now, we’d like to set up forwarding from port 80 to 443, so users don’t have to remember to type “https:”. We can do that by adding another server directive like this;


server {
 listen 80;
 server_name euca-ui;
 rewrite ^ https://$server_name$request_uri? permanent;
 }

That about covers it. We clearly need a better way to manage multiple console servers on a single host, but this should be helpful to get something going. I hope to refine this process in future releases as we iron out the wrinkles. Here’s the final nginx.conf file I used;


user nginx;
worker_processes 10;

error_log /var/log/nginx/error.log;

pid /var/run/nginx.pid;
events {
 worker_connections 1024;
}
http {
 upstream euca-ui {
 ip_hash;
 server 127.0.0.1:8880;
 server 127.0.0.1:8881;
 }

server {
 listen 80;
 server_name euca-ui;
 rewrite ^ https://$server_name$request_uri? permanent;
 }

server {
 listen 443 ssl;
 ssl_certificate /etc/eucalyptus-console/console.crt;
 ssl_certificate_key /etc/eucalyptus-console/console.key;
 server_name euca-ui;
 keepalive_requests 500000;
 keepalive_timeout 1000;
 location / {
 proxy_pass http://euca-ui;
 }
 }

log_format main '$remote_addr - $remote_user [$time_local] "$request" '
 '$status $body_bytes_sent "$http_referer" '
 '"$http_user_agent" "$http_x_forwarded_for"';

access_log /var/log/nginx/access.log main;
}

Advertisements

Installing the Eucalyptus Console from source and packages

I previously posted some information about the new User Console we’ve been working on at Eucalyptus Systems. There has been a lot of activity and we’ve shown it to a lot of users to get feedback. We will be releasing it officially very soon, but till then, you can run it yourself a couple of ways. You can build from source, which is very easy, or install nightly builds which we provide for RHEL 6 and CentOS 5 and 6.

Did You Get the Package?

Packages are available in the nightly directory here:

http://downloads.eucalyptus.com/software/eucalyptus/nightly/3.2/

Configure the repo like this:

rpm -Uvh http://downloads.eucalyptus.com/software/eucalyptus/nightly/3.2/centos/6/x86_64/eucalyptus-release-3.2.0-0.1.el6.noarch.rpm

You’ll also need to have the elrepo configured as well.

rpm -Uvh http://elrepo.org/elrepo-release-6-4.el6.elrepo.noarch.rpm

Next, run

yum install -y eucalytpus-console

Next, you’ll need to configure the cloud location and perhaps some other things. The file is located in /etc/eucalyptus-console/console.ini  The config file has a lot of settings, but there are just a few you need to understand to get going quickly.

clchost – this is the IP or dns name for the eucalyptus cloud you’ll connect to

uiport – defaults to 8888, but you can use a different port if you like

sslcert, sslkey – these values are used to configure SSL, which you don’t need to do for development

usemock – this is important if you don’t have a cloud to talk to. Setting this to true instructs the user console to load mock data and many features operate on the mock data (though many also don’t work as well). In this mode, the console can be run standalone for simple demos or to work on the browser side like when you want to changing branding or other look and feel items.

Now, start the service. You’ll see something like this;


# service eucalyptus-console start
Generating self-signed certificate: [ OK ]
Generating cookie secret: [ OK ]
Starting eucalyptus-console: [ OK ]

By default, SSL is enabled. You can connect to the application using https://localhost:8888/ (assuming you’re on the same host, otherwise substitute the right hostname or IP). Skip down to the “Once You’re Running” section below.

Using the Source

A source install is also very easy, but there are some differences you will need to be aware of. First, the code lives on github here: https://github.com/eucalyptus/eucalyptus/tree/maint/3.2/testing

Notice that I’m showing you the maint/3.2/testing branch. That is because we’re putting the very latest fixes there. If you’re a little more risk-averse, you may want to simply run from maint/3.2/master instead. We push changes from testing to master after the code has passed QA to a reasonable degree.

To get started, you’ll need;

  • a git client
  • python 2.6 (or 2.7)
  • boto
  • m2crypto
  • tornado (2.1 or higher)
On CentOS 6,

rpm -Uvh http://elrepo.org/elrepo-release-6-4.el6.elrepo.noarch.rpm

yum install -y git python python-boto m2crypto python-tornado

or, Ubuntu 12.04,

sudo apt-get install git python-boto m2crypto python-tornado

Now, grab the source.

git clone git://github.com/eucalyptus/eucalyptus.git
cd eucalyptus
git checkout maint/3.2/testing
git pull origin maint/3.2/testing

For better or for worse, the user console is in a subdirectory of the main eucalyptus source tree. This means you pull down a whole lot more than you really need if you're only interested in the console. The good news is that in the console directory, you have a completely independent project that doesn't have any build or run-time dependencies on the rest (aside from connecting to the cloud). To get the console running, you'll first need to configure one or two things.  The console/eucaconsole/console.ini file has many settings, which we cover in the package install section above.

Once you've set things up to your liking, simply run

cd console
./launcher.sh

You should see a message like "2012-11-10 23:56:43 INFO Starting Eucalyptus Console". If not, there may be other issues you need to fix first. If you have problems and need help, visit #eucalytpus-ui or one of the other #eucalyptus channels on freenode.net.

If you saw this message, that's great! It's very likely you're ready to connect with the browser. Assuming you are running the console on the same machine as your browser, use the appropriate localhost URL like "http://localhost:8888/" if you kept the default port and ssl config (off).

Once You're Running

You should see a login page. If you're using the mock (usemock: True), you can put anything in the fields, or nothing and simply login. If you're connecting to a Eucalyptus cloud, you'll need to use the regular web login credentials you'd use to login to the Eucalyptus admin console (read on...)

* Warning: Science Content *

The user console uses temporary session credentials from the STS service to make calls, so your actual access key and secret key are never passed to the console app. As a result, accounts that want to use the console must have a) a password set up and b) access credentials assigned. These things can be done by the cloud admin via the admin console or using euca2ools like this;

euare-accountcreate -a testuser1
euare-useraddloginprofile --delegate testuser1 -u admin -p euca123
euare-useraddkey --delegate testuser1 -u admin

Another way to do this would be using the Eucalyptus Admin UI which you can reach in your web browser here: https://<your-cloud-frontend&gt;:8443/ and there are some helpful docs here.

Assuming you were able to login, you should be ready to explore!