Tag Archives: Ubuntu

VXLAN on Ravello between Google and Amazon EC2

v2Ravello_Logo_largeA week or so ago I did a post around a cross vCenter vMotion lab that I had setup utilizing both Amazon EC2 and Google Cloud through Ravello Systems new beta which allows us to run nested ESXi.  It was a fun project to work on, migrating VMs back and forth through the clouds, but I tried to keep a lot of the technical detail out of the post – focusing more on what Ravello had to offer.  One key aspect of the setup was creating a VXLAN tunnel in order to bridge the two VM networks inside of each cloud – allowing me to complete the vMotion without performing any additional network configuration on my VMs once they had migrated.  Anyways, I thought I’d go into a little more detail on how I accomplished this.

Now, keep in mind here I don’t claim to be any sort of network guy – it’s probably my biggest lack in terms of IT skill-sets, therefore I could be going about this all wrong – any advice, comments, just leave the in the box below – I always appreciate any feedback I get.  I also appreciate any help I get, and had a lot with this project – CTO of Ravello Systems Alex Fishman had a couple of calls with me offering up his experience (very very smart guy).  Also, there’s a blog post on Ravello’s blog which goes over the setup as well – that said, I thought go into a little more detail in the case that someone else at the same level of network knowledge as myself might be looking for help.

So to start let’s have a brief look at the Ravello setup.  Firstly we need a couple of applications, one published to the EC2 cloud and one published to the Google Cloud.  Each application (in this case) contains two VMs – one to act as a client and one to act as the gateway/vxlan bridge.  I’ve used Ubuntu 14.04 server for these but you could use any OS you like, so long as the vxlan module is loaded and supported.  The table below outlines each VM and the networking associated with it in my test setup.

Cloud VM Network IP Notes
Amazon EC2 ec2-vxlan eth0 IP:
GW: None
Network Gateway
eth1 IP:
External network w/ Elastic IP attached
ec2-client eth0 IP:
GW: (in OS) None in Ravello
Google google-vxlan eth0 IP:
GW: None
Internal Network Gateway
eth1 IP:
External Network w/ Elastic IP attached
google-client eth0 IP:
GW: (in OS) None in Ravello

I’ve used a /29 subnet on the external networks as I really only need 3 total IPs available, one for each of the vxlan VMs as well as a third for a gateway – Honestly again you could use whatever you wanted here.  I understand that sometimes a picture is worth a thousand words so here is a side by side of both the Amazon and Google network canvas.

ec2networking googlenet

So a pretty simply setup when looking at the canvas – essentially the vxlan VM will need two NICs, one connected to the internal lan (eth0 in this case) and one connected to an externally routed network (eth1).  Before we finish up within the Ravello canvas and establish a tunnel let’s first look at the EC2 side of things so we can better understand the settings on the vxlan and client VMs that end up making our network canvas look like the above.

ec2-nic1 ec2-nic2

Looking closer at the network configuration of the ec2-vxlan VM (sorry, couldn’t get it all on one screen cap so I put them side-by-side) we can notice a couple of things; eth0, the internal lan (which will act as a gateway for the client VMs) is setup on the network with no gateway.  Also, we have selected public IP for this nic under external access but left the ‘Even without external services’ unchecked.  What this does is ensure that this VM can only be accessed by routing through our vxlan tunnel and cannot be accessed directly from the internet.  The second nic (eth1) is the nic we will use to establish our vxlan bridge.  This nic is subnetted in a way that allows very few IPs within the network, as we only need three anyways (one for ec2, one for google, and one for a gateway).  This nic has an Elastic IP tied to it within the External Access settings.  Since we will need to use this public IP later when we establish the tunnel it’s best that it not change often, and an elastic IP will never change, thus why we used it.


Another note in regards to the vxlan VM is the external services provided.  For this case I’ve simply allowed all external traffic on all ports into eth1 – probably not the greatest feat in terms of security but it sure does ensure I get the communication I need. (might be able to change to just other IP).

Note that you will need to setup the Google side of things inside Ravello exactly the same as shown above, obviously replacing the IP addresses of eth1 with those shown in the table earlier.  Once done with that we have completed our setup within Ravello and it’s now time to setup the tunnel.

Again, let’s start with EC2 and work our way over to google.  Take note of both your EC2 and Google elastic IP’s as we will need them for the configuration below.

First is our network configuration (/etc./network/interfaces).  Below is a shot of mine – the key here is that even though we specified an IP in the Ravello interface for eth0 we are not doing it within Ubuntu – we will be using this IP on our bridge instead, however we still want eth0 to be active so we set it as manual.  So as far as network interfaces your setup should be similar to below – of course with address being on the Google cloud.

# The primary network interface
auto eth0
iface eth0 inet manual
auto eth1
iface eth1 inet static

Once we have our network interfaces set up we can continue with the setup of the vxlan and bridge.  First we will walk through the EC2 side and then move to google.

We begin by adding the vxlan device with “ip link add mtu <MTUSIZE> <VXLAN DEVICE NAME> type vxlan id <VXLAN ID>”.  For example on the EC2 side I ran..

ip link add mtu 65000 vxlan1 type vxlan id 1

Then let’s create a new forwarding database entry on our vxlan device to allow all traffic through using “bridge fdb append <MAC> dev <VXLAN DEVICE NAME> dst <DESTINATION ADDRESS>”.  For example, I used the following on my EC2 instance – ensuring I use the public IP of the Google Cloud.

bridge fdb append 00:00:00:00:00:00 dev vxlan1 dst

From here we can add our bridge using “brctl addbr <BRIDGE NAME>” – again, I ran…

brctl addbr br0

Then add both our vxlan and our internal interface to the newly created bridge with “brctl addif <BRIDGE NAME> <INTERFACE> <INTERFACE>”

brctl addif br0 vxlan1 eth0

Now we can assign our internal IP that we wish to use for the internal lans gateway to our bridge, and then simply bring up our bridge and vxlan interfaces.  Lastly, I ran these commands…

ifconfig br0
ifconfig br0 up
ifconfig vxlan1 up

That does it for the Amazon configuration, the Google configuration is exactly the same, except substituting the Amazon public IP as our destination when adding the forwarding entry.  Once we do this you should be able to ping back and forth between the test client VMs, each located in their separate cloud.  Now think of the possibilities – nested ESXi in each cloud, with a layer 2 VM Network that stretches through the vxlan tunnel – pretty sweet stuff!

One thing that could be a PIA is that this config won’t persist across reboots.  I’ve tried in many ways to get the interfaces added to the persistent rules, but found the quickest way to get this to run on boot is to simply create a small bash script containing the commands; both Google and Amazons are shown below.

ec2script googlescript

Save each bash file in their respective /etc./init.d/ directories and make them executable.  I called the configVXLAN Then run the following on both Amazon and Google to ensure it gets ran on startup

update-rc.d /etc/init.d/configVXLAN defaults 99

And that is it!  A functioning stretched Layer 2 network between Google Cloud and Amazon EC2 using Ravello!  The possibilities are endless…  Again, I’m not a big networking guy, so if you know of any way I can improve this (use NSX?) just let me know…  Thanks for reading!

VMware View on the $99 HP Touchpad

Last month, being the quintessential geek that I am, I spent the greater part of one of my weekends installing the Ubuntu Chroot on the HP Touchpad and blogging about it.  Since then I got thinking, why couldn't I just grab the VMware View Open Client and install that into Ubuntu and essentially run a View desktop on my Touchpad, which brings us to this post.  Although its not running directly on webOS its still kind of cool and thought it deserved a few words…but, since a picture is worth a thousand words I'll show you the proof first.

There you have it, a fully functional Windows 7 desktop from VMware View running on the Ubuntu Chroot on the HP Touchpad.  Now, before getting too excited please remember that obviously none of this is supported with VMware or HP.  I did it just becuase I wanted to see if I could.  Mind you I couldn't get the VMware View open client to compile on the Ubuntu Chroot.  It kept complaining about the Arm processor and not having 'Thumb' support.  So, if anyone knows how to do recompile the open client on an arm processor please leave a comment below and let me know.  The packages that I installed to recompile the open view client were libgtk2.0-0 libgtk2.0-dev libglib2.0-0 libglib2.0-dev libxml2 libxml2-dev libcurl4-openssl-dev intltool libboost-dev boost libboost-all-dev build-essential.  I used the Open View client found here and followed the instructions to recompile here.  

But like I said, I had no luck with the recompile so I had to cheat somewhat in getting the client up and running.  So, if you haven't installed the Ubuntu Chroot, do so by following whatever set of instructions you want to do that, mine are here.  Basically, I just downloaded the HP ThinPro add-on package of their ThinPro software, extracted out the View client Debian package, copied it to the Touchpad and simply ran the following command.

dpkg -i hptc-view-client_4.6.0-1_armel.deb

I can't remember if there were any dependencies that I needed to get, you may have to remove your current rdesktop install first.  A message will be displayed if you need to get and/or remove anything, if so, just use the sudo apt-get install/remove commands to get whatever you need.   Anyways, after installing, simply drop to a terminal from within Ubuntu and run vmware-view and your client should fire right up.

To tell you the truth, the performance is really not that bad.  I thought that it may have some sort of downfalls running on a Chrooted Ubuntu along side with webOS, but it was actually pretty snappy.  

Again, this is definitely not supported, and to tell you the truth, it will drain your Touchpad battery very very fast, so it probably isn't very practical either, but just like the Ubuntu Install, it was fun and I thought I would share….

Installing Ubuntu on your 99 dollar HP Touchpad

About a month ago I was lucky enough to score one of those $99.00 touchpads.  I seen the explosion on the internet the night before and got up nice and early and headed to our local Staples where a line had already formed.  

Essentially I purchased this tablet with the sole intent of porting it over to android honeycomb whenever the instructions and forums came out.  To help pass the time waiting for the android port I’ve decided to throw Ubuntu on the unit, just to see how well it works.  Don’t get me wrong, HP’s webOS is not that bad and I’m really enjoying my experience with it, but I find the built in browser is lacking somewhat.  So I thought I would give Ubuntu port a try and see what Chrome was like.  There is a lot of information out there, below is really just a compilation of information that I found, and the steps that I took to make this work….

For the most part I followed the directions and links off of this page so if you run into any issues you may want to browse the content and links there.
So, Step 1. You need to install Preware.  Basically this is like a homebrew installation, or like jail breaking an ipad.  There are a ton of pages with instructions on how to install Preware and I'm not going to regurgitate them all.  The nicest instructions I have found is this post from Jason Nash.
All that you really need to have installed from that blog post is Preware, but I would recommend pulling down some of the other apps Jason mentioned as well.  One big note is to be sure you are running 3.0.2 of webOS as instructed in Jason's blog as this is what I am going to base our webOS doctor build off of.
After you have Preware installed, grab your linux or osx machine and let the fun begin.  For the purpose of this tutorial I used an Ubuntu based laptop, but I'm sure they wouldn't stray too far from the following on a Mac.
In order to get the packages that we need you will have to add the partners repo to your sources.list using the following command.
sudo echo "deb http://archive.cononical.com/ lucid partner" >> /etc/apt/sources.list
And then install git, git-core, and java if you don't already have them.
sudo apt-get update
sudo apt-get install git git-core
sudo apt-get install sun-java6-jre
Download Palm novacom and install it
32 bit
64 bit
Use the following command to install the respective package
sudo dpkg -i palm-novacom_1.0.76_i386.deb
Now we need to get the meta doctor in order to build our new images for webOS (including our partitioned space for Ubuntu).  For this we will use git.  I did this inside of my users home directory but feel free to do it wherever you like, just remember where you are….
git clone git://git.webos-internals.org/tools/meta-doctor.git
After this is done syncing down the packages go into meta-doctor folder and create a downloads folder
cd meta-doctor
mkdir downloads
cd downloads
Now you need to pull down the appropriate webOS doctor version for your device.  I got this by using the following command
wget http://palm.cdnetworks.net/rom/touchpad/p302r0d08012011/wifip302rod/webosdoctorp302hstnhwifi.jar
Now we need to edit the MakeFile in order to partition off some space for our ext3fs partition.  Hopefully you are familiar with vi, if not, I'll do my best to explain it.
Get back to the root directory of meta-doctor
cd meta-doctor
Open up makefile
sudo vi Makefile
Scroll down through file until you see the line # EXT3FS_PARTITION_SIZE = 2GB.  This is where we will uncomment (remove the #) before this line.  To do this, simply place you cursor over the # and hit the 'x' key so it now reads EXT3FS_PARTITION_SIZE = 2GB
You can increase the size of this as well if you like, I only have a 16GB model, so I just left it at default.
To save and quit vi type the following in the vi window
Now we need to actually build our images for the touchpad.  This is done by typing the following
make DEVICE=touchpad CARRIER=wifi all
At this point I actually received an error stating "Please download the correct version of the webOS doctor .jar file………"  Basically, as long as you downloaded the correct jar file specified in the wget statement above,  you just need to rename your downloaded jar file to the one in the error message by using the following command.
mv downloads/webosdoctorp302hstnhwifi.jar  downloads/webosdoctorp302hstnhwifi-3.0.2.jar
then run the make command again
make DEVICE=touchpad CARRIER=wifi all
After the build is complete, you should be left with a jar file in the build directory.  I attempted to perform the following steps from within Ubuntu but had no luck, so I just copied the build directory back over to my windows box where I had installed the WebOS quick installer (from Jason's blog) and completed the rest of the process there….

Double click the newly created Jar file and set your language, accept the agreement etc.
I ran into a few little issues here trying to get my system to recognize my touchpad.  I had to power on my touchpad in usb mode (holding power button and volumes button in) in order to get it to be recognized by the quick installer.  However you should be able just to plug the unit in and just click 'Close' on the usb prompt when it comes up, but if all else fails, do the boot sequence above.
This process will take a few minutes as it basically re-installs webOS on your touchpad and brings it back to factory default (with our new partition)  As well, it will need to resync all of your applications again from the HP App Catalog (so hopefully you remember your webOS account credentials)  Also none of the tutorials mentioned re-installing Preware, but I had to do that again as well.
Once it's done we need to set our ext3fs partition to mount at boot time.  There are a few ways to do this, but I did it using novaproxy, putty, and python.
Install Python (download)
Get a copy of the novaproxy.py script (novaproxy.py) – Just right click this and save file as.  I put it right in my c:\python\ folder for ease of use.
And you should already have putty  (download)
Now, you need to connect your touchpad to your system again in 'charge only mode' (click close when the usb prompt comes up).
Then drop down to your Python folder and run 'python novaproxy.py' in a command prompt.
It should tell you that it is 'listening on localhost' on a certain port.  Every time I ran it was it was port 8023.  Now we need to fire up putty, connect to localhost on port 8023 using raw as the connection type.  Also, under the terminal options you need to select 'Force off' for all of the Line Discipline options.
You should now be connected to a shell on the touchpad.  Now we will get into actually setting our partition to mount at boot time.  This is done by executing the following commands in the terminal window.
mount -o remount,rw /
mkdir -p /media/ext3fs
mount -o remount,rw /
echo "/dev/mapper/store-ext3fs /media/ext3fs ext3   noatime,data=writeback   0   0" >> /etc/fstab
mount -a
Ok, almost there now.  We need to go back to Preware now and install the following applications.
UbuntuChroot, Xecutah, and XServer
After you have installed these just fire up Xecutah, and tap the Ubuntu Chroot 11.04 option.  You should see a new card pop up and after a few moments you will be sitting at your new Ubuntu installation.  From here, you can install applications and patches using the standard apt-get install options and place whatever you want within your installation.  Some applications however work better than others.  I followed the autoconfig instructions located here in order to get my window manager and everything configured.  Just a note, I used lxde as my window manager and chose to use the onboard keyboard as the default keyboard.  

Just a few other tips…When using the native touchpad keyboard its better to start Xecutah and launch Ubuntu in portrait mode as well, this way you have a bit more real-estate as there is no way that I can find to hide the native keyboard while using it.  If using the other keyboard that comes standard with the auto config it's best to go and grab 'Tweaks' from Preware and hide the onboard keyboard for good.  Also during this whole process, I found that if something didn't work, just reboot the device, it does wonders!  Anyways, Ubuntu clearly doesn't run perfect on the touchpad, but it is nice to have chrome for a browser on my unit, as I thought the original webOS browser seems to be lacking in some ways, plus, being the geek that I am…I just found this kinda fun! :)