Automation of CloudForms appliance setup with Ansible

CloudForms ships as an appliance as a means of greatly minimising the deployment and configuration required. Whilst this deployment method removes a substantial amount of complexity by shipping with all packages and configuration needed to get a working appliance in a very short time, it isn’t entirely without human intervention.

At a minimum you will need to:

  1. Set hostname and network configuration, particularly if you wish to use a static IP address.
  2. Create a new Virtual Management Database (VMDB) and associated Region, or join an existing one.
  3. Configure encryption keys, particularly if you are joining an existing region.
  4. Set up external authentication via IPA, if your deployment method calls for it.
  5. Start the EVM server processes.

These steps can all be performed using the appliance console that ships with the appliance. Unfortunately, this menu-based interface doesn’t lend itself to automation (unless you want to get your hands dirty with expect).

If you’ve got one or two appliances that’s not a big impost. But if you’ve got 5? 10? Then we start to look at Ansible and think “I wonder if I could automate this?”

Turns out, you can!

 

 

Enter appliance_console_cli

appliance_console_cli is a lesser-known tool but, as the name implies, is able to perform some tasks of the appliance console using the command line. It can achieve a good subset of what you can do with the menu-driven console. Note I said “a good subset” – unfortunately the tool can’t achieve full replacement of the menu console (yet!). It’s getting better upstream, so expect those improvements to filter down to CloudForms in the not-too-distant future.

Before we continue, take a look at my sample deployment playbook here on Github, which creates two appliances – a primary database and a generic appliance which connects to the database (there’s also a host configuration for an all-in-one appliance). I’ll be referencing it as we continue.

Running the playbook

Firstly, make sure you’ve amended the hosts in hosts/cfme appropriately. If you’re building an all-in-one appliance that contains its own database, include the same host in both the appliances and primary_db groups (take a look at the cfme-aio example).

Have a look at the variables in group_vars/all/vars. By default these point to equivalents that are expected to be stored in a vault. For testing purposes, you can store them in plaintext if you wish, but best practice is to store them in a vault.

When you’re ready to go, the playbook can be run with the following:

$ ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -i hosts/cfme playbook.yml -u root --ask-vault-pass

We disable host key checking (necessary if you haven’t SSH’d into these new appliances yet) and have ansible ask us for a vault password.  You can drop the --ask-vault-pass if you aren’t using a vault.

The playbook will adjust the SSH password for the root user on the remote host automatically based on the values you provide in cfme_default_root_pw (should be ‘smartvm’ for a default install) and cfme_new_root_pw.

Sample output:

PLAY [all] **********************************************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************
ok: [cfme-db1.vm.home.ajg.id.au]
ok: [cfme-ap1.vm.home.ajg.id.au]

TASK [set root password] ********************************************************************************************************************************************
ok: [cfme-ap1.vm.home.ajg.id.au]
ok: [cfme-db1.vm.home.ajg.id.au]

PLAY [all] **********************************************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************
ok: [cfme-db1.vm.home.ajg.id.au]
ok: [cfme-ap1.vm.home.ajg.id.au]

TASK [set hostname] *************************************************************************************************************************************************
ok: [cfme-db1.vm.home.ajg.id.au]
ok: [cfme-ap1.vm.home.ajg.id.au]

TASK [update /etc/hosts] ********************************************************************************************************************************************
changed: [cfme-db1.vm.home.ajg.id.au]
changed: [cfme-ap1.vm.home.ajg.id.au]

TASK [register red hat subscription] ********************************************************************************************************************************
changed: [cfme-db1.vm.home.ajg.id.au]
changed: [cfme-ap1.vm.home.ajg.id.au]

TASK [update all packages] ******************************************************************************************************************************************
changed: [cfme-db1.vm.home.ajg.id.au]
changed: [cfme-ap1.vm.home.ajg.id.au]

PLAY [primary_db] ***************************************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************
ok: [cfme-db1.vm.home.ajg.id.au]

TASK [configure primary database and region] ************************************************************************************************************************
changed: [cfme-db1.vm.home.ajg.id.au]

PLAY [appliances] ***************************************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************
ok: [cfme-ap1.vm.home.ajg.id.au]

TASK [fetch remote encryption key] **********************************************************************************************************************************
changed: [cfme-ap1.vm.home.ajg.id.au]

TASK [connect to external region in database] ***********************************************************************************************************************
changed: [cfme-ap1.vm.home.ajg.id.au]

TASK [start evmserverd] *********************************************************************************************************************************************
ok: [cfme-ap1.vm.home.ajg.id.au]

PLAY RECAP **********************************************************************************************************************************************************
cfme-ap1.vm.home.ajg.id.au : ok=11   changed=5    unreachable=0    failed=0   
cfme-db1.vm.home.ajg.id.au : ok=9    changed=4    unreachable=0    failed=0

Give it a good few minutes for appliance to come online, seed the VMDB and start all of its workers. After about five minutes or so you should be able to connect to its UI successfully.

Playbook – housekeeping tasks

The playbook does a few housekeeping tasks before getting into the meat of it:

  1. Set the root password. I opt to set the same password on all appliances, using a nice trick I found at the linked blog post. Since password_hash uses a random salt, we’d end up updating the root password on every run through. To avoid that we force password_hash to pick the same salt each time, using the hostname as the  seed to the random number generator.
  2. Set hostname. Set the fully-qualified domain name of the appliance, amending /etc/hosts as well. The script attempts to use the value of cfme_appliance_fqdn first – you can set this as a host-level variable. If that isn’t found, it defaults to using the inventory_hostname.
  3. Red Hat subscription and updates. Connect your CloudForms subscription and update all packages on the system to account for errata and security fixes. You can skip these step using --skip-tags=rhn,updates on the command line to ansible-playbook.

Playbook – appliance configuration

The playbook is divided into multiple plays that are executed sequentially. After performing the housekeeping above, we first configure any host in the primary_db group as a primary database:

- hosts: primary_db
  vars:
    ansible_ssh_pass: "{{ cfme_new_root_pw }}"

  tasks:
    - name: configure primary database and region
      shell: >
        appliance_console_cli
        --internal
        --username={{ cfme_db_user }}
        --password={{ cfme_db_pass }}
        --region={{ cfme_region }}
        {{ ( '--dbdisk=' + cfme_db_disk ) if cfme_db_disk else '' }}
      args:
        chdir: /var/www/miq/vmdb
        creates: config/database.yml

If you’ve specified a separate disk to use (such as /dev/vdb), we’ll include it here on the command line for configuration and use.

Note that there should only be one appliance in primary_db. Running a multi-master database configuration isn’t supported.

Next we move onto the non-database appliances. For those appliances that are connecting to a remote database and region, we configure them too:

- name: configure appliance for remote database
  block:
  - name: fetch remote encryption key
    shell: >
      appliance_console_cli
      --fetch-key={{ cfme_primary_db_ip }}
      --sshlogin=root
      --sshpassword="{{ cfme_default_root_pw }}"

  - name: connect to external region in database
    shell: >
      appliance_console_cli
      --hostname={{ cfme_primary_db_ip }}
      --username={{ cfme_db_user }}
      --password={{ cfme_db_pass }}

    args:
      chdir: /var/www/miq/vmdb
      creates: REGION
  when: not cfme_appliance_allinone is defined

We must fetch the encryption key from the primary database server, to ensure the same key is used on all appliances. Then we connect to the external region, but not if the REGION file already exists – this file is created post appliance configuration.

Finally, we wrap the whole block in a check that prevents this block from executing if the appliance is actually an all-in-one (i.e, the database is on localhost).

Lastly, we start evmserverd on all appliances:

- name: start evmserverd
  service:
    name: evmserverd
    state: started

And that’s it!

If we had an IPA environment established we could join the host to the IPA realm using the --ipa* command line parameters. Hey, I have to leave something as an exercise for the reader! 🙂

Some caveats

  1. VMDB replication. Unfortunately, appliance_console_cli doesn’t (yet) replicate all functionality available from appliance_console. As of CF 4.5 it is not possible to configure database replication – you need to do this manually way via appliance_console. This feature has been added to ManageIQ and will likely make its way into CloudForms at a later date.
  2. Appliance roles. Further, there’s no way to set the roles for an appliance once it has been provisioned via appliance_console_cli. You still need to do this through the Configuration panel of the CloudForms UI. There are possible options to work around this, but they are somewhat hacky, definitely non-standard, and involve more intimacy with the rails runner command than I would like.
  3. Idempotency. This playbook isn’t fully idempotent, given it shells out to appliance_console_cli, but we do try to avoid duplicate runs by checking for key configuration files (REGION, GUID, config/database.yml).

Conclusion

Hopefully this helps demonstrate that you can provision a CFME appliance using Ansible by making use of the appliance_console_cli tool. While you can’t do everything you can do via appliance_console or via the user interface, you can certainly get yourself most of the way there.

Enjoy!

Leave a Reply

Your email address will not be published. Required fields are marked *