Cloud-Init with vRealize Automation and vSphere - No CD Drive!
This article explains how to use Cloud-Init with vRealize Automation and vSphere without requiring a CD drive in every VM.
I've heard it said that you should start with why. I'll get to that, but first I have to set the scene.
vRealize Automation Cloud and vRA 8.x introduced the ability to add Cloud-Init commands / configuration to Cloud Templates and have them executed / applied in the guest operating system after provisioning. Cloud-Init has been around for a number of years and is the defacto standard for customising workloads in most of the public cloud platforms. As vRealize Automation can manage workloads on multiple cloud platforms, it made sense that it support Cloud-Init.
As an example, the following could be sent to Cloud-Init to customise a workload and maybe trigger upgrading the OS or packages installed:
The way that these commands / configurations are passed varies from platform to platform. Cloud-Init supports a number of datasources that are specific to different clouds. These datasources can either be added to a list inside your VM template or explicitly selected or most include a means to test which datasource a cloud supports. For vRealize Automation and vSphere, the supported option is "OVF".
Let's have an example:
The above image is from a simple Cloud Template that I have in the v12n lab to provision a simple CentOS 8 server. The way that Cloud-Init works is that the configuration under cloudConfig, which just sets the guest's hostname and FQDN, is encoded to base64 and placed in an OVF XML wrapper. This XML file is placed in a small ISO file and attached to the VM at provisioning time. When the VM boots and Cloud-Init starts, the OVF datasource finds the XML, decodes the base64 text and executes the instructions. It works.
No though we come to the why...
The Problem with OVF
As I said above, the implemented system works. But, if you haven't realised already, there's a catch. The provisioned VMs must have a CD drive in order for the ISO file to be mounted. One of the customers that I'm working with at present has security controls in place that forbid any VM from having a CD drive. For this reason, and because the ISO file is a little clunky in my opinion, I started thinking about other methods that could be employed.
Finding something that wouldn't require masses of engineering to multiple products (vRA, vSphere, Cloud-Init) was challenging. One of my "back of an envelope" ideas looked like this:
- Cloud-Init starts and is configured to use a new datasource for retrieving its metadata. The service leverages VMtools to request the meta data from the ESXi host that it’s running on.
- The ESXi host consults its cluster configuration and gets the address of vRA or vRA Cloud.
- The ESXi host contacts vRA (maybe via vCenter) and provides something like the name / UUID of the VM to vRA.
- vRA reponds by passing back the user-data and meta-data needed by Cloud-Init through the host and guest tools.
Understandably that would require a fair bit of effort to achieve and I didn't get very far with it. What I eventually came up with is a lot simpler.
The Solution
It turns out that some bright colleagues at VMware had already solved part of the problem for me. Several parts in fact. I just had to join them up.
VMware GuestInfo Cloud-Init Datasource
It's not an official datasource in Cloud-Init, but a small number of contributors have already created a new datasource that uses VM configuration parameters to supply instructions to Cloud-Init. There's a link to the repo below.
vmwareHow it works is that it uses VMtools installed in the guest to access the configuration parameters defined on the VM. These parameters can easily be set through the vCenter UI, API or using another tool such as "govc". Here's one of my test VMs with the necessary parameters added:
Getting the datasource added to my VM templates was fairly easy. There's an install script available in the repo above that sets it up. I just added a couple of lines to my Packer build for CentOS 8 and I was done on that score. I also removed the CD drive from the template.
Specifically, I had to make sure that python3 was installed and I just had to add the following to my script to install and setup Cloud-Init:
curl -sSL https://raw.githubusercontent.com/vmware/cloud-init-vmware-guestinfo/master/install.sh | sh -
Now that my template was ready, I had to find a way to get the Cloud-Init instructions out of vRA and into the VM's configuration parameters.
Extensibility Workflow
I've actually done some work in this area before (not just vRO, but playing with VM Configuration Parameters through vRO) so I was on familiar ground. One of the key challenges this time though was base64 encoding. Last time I checked, there wasn't a native way to do this in vRO. Luckily however, a plugin exists that makes it easy.
Dan Linsley's Crypto Plugin still works with vRA 8.4 and includes a method to encode and decode base64.
vmwareWith that installed in to vRealize Orchestrator, all I needed was a simple workflow to join the pieces of the puzzle. And here it is:
As of vRA 8.4, there's a new extensibility topic available called "Compute initial power on". As the name suggests, it's triggered before a provisioned VM is powered on for the first time. Perfect for my purposes! (Note: "Compute post provision" might have been triggered too late for Cloud-Init.) So the final piece of the puzzle is just to setup that subscription:
Some validation testing shows that it worked!
The Final Touch
You might be thinking that some unscrupulous person could extract and decode the instructions later on and maybe find a password or some other secure piece of information. That would be true if the base64 text was left in place. Luckily the datasource developers thought of that and added a mechanism to clear the parameters once they have been read. I added that as a configurable option in my workflow as troubleshooting Cloud-Init runs can still be a dark art.
If you'd like a copy of the workflow, just follow this link:
v12n-io