ESXi Smart Lab technology overview

AWS and Google cloud are amazing platforms. They are ideally suited for a variety of workloads. However, one cannot run hypervisors such as KVM or VMware ESXi™ on AWS or Google. This is unfortunate, because the public cloud is an ideal platform for development/testing, demos, PoCs and training environments. This basically locks out the VMware and OpenStack/KVM ecosystem from being able to effectively use on-demand public cloud resources. However, with Ravello’s nested virtualization technology, it is possible to run ESXi (and KVM) on AWS or Google cloud - with “hardware acceleration”. This opens up a whole host of use-cases for the virtualization (and specifically VMware) ecosystem.

Why one cannot typically run ESXi in the cloud

The reason one cannot typically run a hypervisor in the cloud is that modern hypervisors such as KVM and ESXi need virtualization extensions in silicon (Intel VT/ AMD-V) in order to run. These extensions are not available (or visible) in a cloud VM. Essentially, a cloud VM looks like a pre-2005 era server (albeit faster).

In order to be able to run ESXi natively on AWS or Google cloud, the respective virtualization platforms (Xen in the case of AWS and KVM in the case of Google) need to support a “nested virtualization” feature. You can think of this as the hypervisor passing through VT access to the VM. Both KVM and Xen have rudimentary/preliminary support for this feature. Once this matures (which will take a while), the cloud providers need to upgrade their production versions to support nested virtualization and retool their operational processes to support this feature. All of this for a business/use-case that is not core to their operations. So even if it happens, it will likely take quite a while. In the meantime, the VMware and KVM/OpenStack ecosystems are stuck. Ravello solves this problem today.

HVX: Virtual infrastructure for the cloud - nested virtualization

At Ravello, we have developed an infrastructure technology we call HVX. One element of HVX is a high performance nested hypervisor that is designed to run on already virtualized hardware (so essentially, in a cloud VM). HVX implements binary translation with direct execution, and exposes VMware or KVM devices (in the form of a platform type) to a VM running on top. As a result, enterprises can take their VMware VMs and run them on AWS or Google cloud without making any changes. The VMware VM thinks its running on ESXi but instead its running on HVX.

The main use-case for this deployment mode is for enterprises to be able to take existing VMware workloads (VMware VMs along with their networking and storage) and run them in AWS or Google cloud without making any changes. This is ideal for development, test, demo and training environments for applications such as Microsoft SharePoint, Exchange, Oracle, SAP, and network and security appliances from F5, CheckPoint, Juniper and others.

Nested ^2 virtualization: Running ESXi in AWS or Google cloud

Running VMware VMs in AWS or Google cloud is one thing. Running the ESXi hypervisor is something else. In order to tackle the latter, Ravello implemented Intel VT/AMD-V technology along with Nested Page Tables (NPT) in software in HVX. This makes a cloud VM look like real physical hardware (a modern server). Since virtualization extensions are now visible in the cloud VM, one can run a hypervisor (designed to run on physical servers) such as ESXi in the cloud VM. The diagram below illustrates the 2 levels of nesting (hence nested ^2).

As mentioned earlier, HVX does binary translation for kernel code for the nested guest. This is exactly how its also done for the nested^2 guest - except that it is one additional context switch. With this implementation, the guest instructions are executed directly on the physical CPU.

We believe that this mode of operation has some tremendous implications for the VMware ecosystem. VMware technology alliance partners (TAP), resellers and solutions provider partners, training partners and enterprise customers - can spin up complete ESXi environments on-demand in AWS or Google cloud for development, test, training, demos and PoCs - and pay simply for usage. This capability radically changes their economics as they no longer need to have on-premises lab hardware infrastructure for most use-cases. Spinning up complex ESXi environments in the cloud can cost as little as $0.14 per hour for 2 vCPU/ 4GB RAM chunks.

VMware product names, logos, brands, and other trademarks featured or referred to in the ravellosystems domain are the property of VMware. VMware is not affiliated with Ravello Systems or any of Ravello System's employees or representatives. VMware does not sponsor or endorse the contents, materials, or processes discussed on the site.