Nested SVM allows a you to run a hypervisor like ESXi or KVM inside a Virtual Machine in the cloud.
In this blog I’ll show you how to use this feature to run Red Hat Enterprise Virtualization on Amazon EC2 through the Ravello service. Red Hat Enterprise Virtualization, or RHEV for short, is Red Hat’s server virtualization product. RHEV is very similar in scope and architecture to VMware vSphere. Like vSphere it consists of a central management system and 1 or more hypervisor nodes. However, unlike vSphere which uses ESXi as its hypervisor, RHEV is based on the KVM hypervisor.
Running a virtualization solution like Red Hat Enterprise Virtualization with KVM, or VMware vSphere with ESXi (the topic of a future blog) in the cloud is useful for development and test of your infrastructure, new product trials, customers engagements and partner and end-user training.
Note that in this blog we are running a “full stack” virtualization solution consisting of a centralized management system and multiple hypervisors. It is also possible to run a single hypervisor, be it KVM or ESXi, in an unmanaged mode. Most of the steps in this blog can be skipped in this case.
Step 1: Prerequisites
Before we begin, you need to have access to the following:
- A Ravello account. If you’re not a Ravello user yet, you can sign up for our 14 day free trial.
- A valid RHEV subscription. If you’re not a RHEV customer then there is a 30-day free trial that is available from the Red Hat web site. When you’re on that site, make sure you note the hardware requirements, …and then realize that you can blissfully ignore them. There is no hardware required for this setup!
- You need a computer with the Ravello CLI importer and the Ravello SDK installed. The CLI uploader can be installed from within the product by clicking on “Library” and then “Download VM Imported”. The Ravello SDK can be installed from its Github page.
Step 2: Create Ravello Environment
The next step is to create a Ravello “application”. In Ravello, an “application” is a set of virtual machines, networks and storage that managed together as a unit. For this installation, I went with a simple 4-node setup connected by a single network. However you can make it as complicated as you want. This is the configuration I used:
|Manager||6GB RAM, 2 CPUs, 50+200GB disk||Management node|
|Node1||6GB RAM, 2 CPUs, 50GB disk||Compute node #1|
|Node2||6GB RAM, 2 CPUs, 50GB disk||Compute node #2|
|Node3||6GB RAM, 2 CPUs, 50GB disk||Compute node #3|
The networking configuration of the nodes is as follows:
To create the application in Ravello, use the following steps:
- Log into the Ravello web interface at http://cloud.ravellosystems.com
- Click on “+ New Application” and give the application a name.
- Drag the VM called “Empty” from the library to the canvas 4 times.
- Configure each VM as per the tables above.
Step 3: RHEV Manager installation
The next step is to follow the Red Hat Enterprise Virtualization installation procedure. Follow the steps below:
- Download the RHEL 6.5 DVD from Red Hat Network and upload it into Ravello. The file is called “rhel-server-6.5-x86_64-dvd.iso” and should be uploaded using “ravello import-disk”. Because this is a large file, it makes sense to do this from a computer with a fast Internet connection. If you don’t have a fast internet connection, you can use a VM in the Cloud (e.g. in Ravello or in Amazon EC2). Log on the VM with ssh, download the ISO from RHN via “curl”, and then use the Ravello CLI uploader to upload the ISO to Ravello. Using a cloud VM like this prevents you from having to transfer the file twice over your internet connection (download + upload).
- Configure the Manager VM to boot from CD, and select the RHEL DVD that you uploaded in step #1 as the ISO image.
- Start the manager VM and wait until it is started.
- Open the VNC console. Once the VM has completed booting, this will show the RHEL Anaconda installer.
- Run through the installer. You can install using the “Basic Server” profile. Make sure you set a root password.
- When the installation is done, eject the DVD from the Ravello console. This will prevent the VM from booting from the DVD again. After this you may reboot.
- When the VM has rebooted, log in as root on the VNC console.
- Configure the right network settings by editing the file /etc/sysconfig/network-scripts/ifcfg-eth0. The file should look like this.
- Restart the network by issuing a “service network restart”.
- Subscribe to relevant RHEL and RHEV channels. This is explained in the Installation Guide.
- Update and reboot the system via “yum -y update && reboot”.
- Wait for the system to be rebooted and log back in
- Install rhevm via “yum install rhevm”
- Configure RHEV-M via the command “engine-setup” Make sure you to set up an NFS server for storing the ISO images and VM images, as well as a local database server.
The screenshot below shows when the installation is done at the end of step 5:
Step 4: Log on to the Manager
Now that the manager has been installed, we can log on to it just to make sure that everythign works. Follow the steps below:
- From the Ravello web UI, select the manager VM.
- Right-click and select “Edit”. We need to add 3 extra external services, in addition to the secure shell service. These are: http (port 80), https (port 443) and ws (port 6100). Select the manager IP as the IP to listen on. Then, under “Network” click on the first network interface, and select “External access: Public IP”. Now click “Save” and then “Update”.
- Wait for the update to complete. When complete, select the manger VM again, and locate the “http” service in the bottom right side of the screen. Click on the link named “Go” next to it. The RHEV welcome page will now open.
- Click on the “Administration Portal” link and log in with the password that you configured when you ran “engine-setup”.
Congratulations, we are now half way! In the next steps we will add the hypervisor nodes.
Step 5: Hypervisor Installation
Now it’s time to install the KVM hypervisor. Follow the procedure below. Steps 2-8 should be repeated for every node.
- Download the RHEV hypervisor ISO file from RHN and upload it to Ravello. The file is called “rhev-hypervisor6-6.5-xxx.iso”. Make sure to upload the file using “ravello import-disk” command.
- Make sure the node is powered off.
- Configure the node to boot from CD.
- Add the following external services to the VM: vnc1 (5900), vnc2 (5901), vnc3 (5903) and vnc4 (5904). The, under “Network”, click on the first network interface, and select “External access: Port forwarding”. Port forwarding has a lower latency than the Public IP option and is therefore more suitable for console access.
- Enable SVM for the Node with the command “ravello-set-svm –vm
“. This command is part of the Ravello SDK.
- Set a new UUID for the node with the command “ravello-set-uuid –vm
“. This command is part of the Ravello SDK.
- Start the node and wait until it is started.
- Open the VNC console and run through the installer.
- When the installation is done, eject the CD from the VM in the Ravello console. After this, you may reboot.
There’s 2 small issues that you need to work around:
- Due to an issue with the the user-mode CPU detection in libvirt, this patch needs to be applied to /usr/share/libvirt/cpu_map.xml. This patch forces the CPU type to be an Opteron G2 independent of the CPUID. To apply this patch, log on to the hypervisor in rescue mode, apply the patch, and then issue the command “persist /usr/share/libvirt/cpu_map.xml”. This needs to be done after step 8 above.
- I ran into a small bug where the installation of the bootloader would run into a legacy SYSCALL instruction. The fix will be deployed soon. In the mean time, if the VM crashes when the installer is installing the bootloader, you should delay step #4 until after step #8.
The screen shot below shows that virtualization hardware is detected by the RHEV hypervisor:
Step 6: Hypervisor registration
After you’ve installed all hypervisors, you need to register them to the RHEV manager. Follow the steps below:
- Open a VNC console to the hypervisor node.
- Login with the username “admin” and the password that you chose during installation.
- Configure the static networking from the menu.
- Select “register to manager” and enter the manager IP (“192.168.0.10”). You will be prompted to accept a fingerprint. Click “Accept” to accept it.
- In the RHEV manager web interface, navigate to System -> Data Centers -> Default -> Clusters -> Default -> Hosts. The hypervisor that you registered should appear here.
- Click the “approve” button.
Repeat the steps above for all 3 hypervisors. All 2 should show up in the RHEV manager, as in the screenshot below:
Step 7: Run Virtual Machines
You are now ready to install your first virtual machine! To do so, follow the following procedure.
- Upload an ISO using into the ISO domain. The easiest way to do this is to log on to the manager via SSH, and then download the ISO you want to upload via “curl”. When the download is done, simply copy it to where the installer created the ISO domain.
- Create a new VM and have it boot from CD-ROM.
- To view the VM console, set the console to “Auto”. When you click the console button, your browser will download a file called “console.vv”. This file is a text file and contains the internal IP address and port number of the console for this system, as well as some other data like a certificate. You should edit this file and change the internal IP and port number with the external one that you can see in the Ravello UI. Once you’ve done this, run “remote-viewer console.vv” to get a console.
In this blog post, we’ve installed Red Hat Enterprise Virtualization on Amazon EC2 through the Ravello service. The installation of a virtualization solution inside a public cloud is made possible by Ravello’s support for nested virtualization. Try it out for yourself: register for our 14-day free trial.
In a next post we will talk about running VMware vSphere and ESXi in the cloud in a similar way. Please stay tuned.
About Ravello Systems
Ravello is the industry’s leading nested virtualization and software-defined networking SaaS. It enables enterprises to create cloud-based development, test, UAT, integration and staging environments by automatically cloning their VMware-based applications in AWS. Ravello is built by the same team that developed the KVM hypervisor in Linux.