Migrating On-premise VMs to Azure

In 2008, the company I worked for at the time finally felt that virtualization was ready to host production workloads.  We stood up a two node VMware ESX 3.5 cluster, and started to migrate a handful of Linux, Windows and Novell Netware (!) servers from bare metal to virtual.  Even with VMware’s migration tooling, it was still a very manual process.  I scripted as much as I could, but my higher ups never felt good about farming the process out to lower level resources.  It was always me who was on the hook for physical to virtual migrations in after hour maintenance windows.

But that was a lifetime ago in terms of technology, and long before today’s DevOps mentality and tooling existed.  I don’t hear as many customers planning P2V (Physical-to-Virtual) migrations these days.  Instead, they’re asking about V2V (Virtual-to-Virtual), or to be more specific, how can they move on-prem workloads to the cloud: V2C (Virtual-to-Cloud).  Quite a few times, I’ve been asked “Can CloudForms help me migrate VMs from my internal virtual infrastructure to the cloud?”

The answer I usually give is, “Not out of the box, but with CloudForms Automation and Red Hat Consulting services, it’s definitely possible.”  No customer ever really pursued this beyond the initial inquiry, however. My own curiosity and interest in Microsoft Azure lead me to try to actually prove this concept out. I submitted a proposal for Red Hat Summit for this year on automating on-prem to Azure migrations using Red Hat CloudForms, which was accepted. I wanted to demonstrate that CloudForms can do just about anything you can think of, with your imagination and knowledge of Ruby being the only real limiting factors.

All of the CloudForms automation methods and Ansible playbooks required to enable the migration of on-premise VMs running on Hyper-V to Azure are available on GitHub. There is also a video available of the process on YouTube.

There are two main challenges when it comes to performing any V2V migration:  dealing with the differences in virtual hardware, and converting between different virtual disk formats.  For the first proof-of-concept, I decided to take a bit of a shortcut by using Microsoft Hyper-V as my on-prem infrastructure source, and Microsoft Azure as my cloud destination.  We are seeing a lot of  interest in Azure as a cloud provider since we added support in CloudForms 4.0.  Since there is a lot common ground between Azure and Hyper-V, it was logical to start with these two platforms.  They both use a similar virtual disk format – only a bit of metadata needs to be removed from a Hyper-V disk before it can be uploaded and used as an image in Azure.  They also use the same virtual hardware, so there is no need to worry about the drivers and kernel modules change.

Here is a workflow of the migration process:

  1. Selecting a VM to migrate
  2. Retrieving Azure information
  3. Preparing the VM
  4. Converting the virtual disk to VHD
  5. Uploading the converted disk
  6. Provisioning the VM in Azure

Selecting a VM to Migrate

The process is initiated by selecting a VM from your on-prem infrastructure.  I used SCVMM/Hyper-V for this test, and I hope to use the same process for VMware & Red Hat Virtualization in the near future. I also tested with Red Hat Enterprise Linux 7 as the guest operating systems, but hope to use other guest operating systems in the future.

Select ‘Migrate to Azure’ from the ‘Migration’ button – a custom button on VMs that leads to the Azure migration dialog.


Retrieving Azure Information

The Azure migration dialog uses several automation methods to retrieve information from the Azure provider in CloudForms.  The basic workflow is:

  1. The Azure credentials and region are derived from the provider information in CloudForms
  2. The resource group list is pulled via the Azure resource manager API, using CloudForms native azure-armrest Ruby gem
  3. Once one of the resource groups is selected, the list of storage accounts, networks, and subnets are refreshed
  4. The user selects the OS type (Windows or Linux), the instance size, enters a password for the “clouduser” account, and a name for the network interface and public IP address resources


Preparing the VM

When the submit button is clicked, CloudForms leverages its Ansible Tower integration to launch a job template that removes VM specific information (e.g. udev rules, SSH host keys, ethernet configuration) and installs the Windows Azure Linux Agent, as required to run on Azure. Similarly, Windows VMs have sysprep run against them to remove machine specific information.


Converting the Virtual Disk to VHD

The VM is shut down as the last task of the Ansible playbook. At this point, the virtual disk can be converted to VHD format to run on Azure.  In the case of Hyper-V, this means CloudForms starts a WinRM remote session against the Hyper-V host the VM was running on.  Using PowerShell, the path to the virtual hard disk is derived, and the disk is converted from VHDX to VHD format – some extra metadata are removed from VHDX to VHD. Upon completion, the file is ready for upload to the selected storage account.

Uploading the Converted Disk

The upload to Azure uses the same WinRM session on the Hyper-V host which requires the installation of Azure Resource Manager powershell commandlets (performed once).  The migration method requires the Azure session credentials to be saved in the file:



Provisioning the VM in Azure

The time required to upload the image depends on the Internet bandwith. Once finished, a new public IP resource is created, along with a new network interface, and the two resources are associated with one another.  With a functioning network interface, a new instance is created from cloning the uploaded virtual disk.

The process takes approximately 2-3 minutes and results in a new instance ready to SSH/RDPe into. This instance is now listed in the Azure inventory in CloudForms.



In this article, we looked at how CloudForms can reduce a multi-step V2C process to a couple of clicks from a dialog.  This allows IT teams to take complex processes that were previously entrusted to the highest level engineers and put them in the hands of lower level administrators. The Ansible Tower integration added since CloudForms 4.1 extends this even further.

A video of the Azure migration process is available, and you can keep up with the development of this automation method on github.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s