Integrating OpenStack Ansible with Let’s Encrypt

Deploying HTTPS is essential for security, and OpenStack Ansible does it by default. However, if no certificates are provided, it will generate self-signed ones, which although are more secure than no SSL at all, it will trigger a warning when accessing the dashboard in the browser. Luckily, the Let’s Encrypt project provides signed SSL certificates for free.

Let’s Encrypt requires your server to be validated before issuing the certificate. This means it will create a temporary file on your server and then try to access it from their servers, to verify that you control the domain you’re trying to get a certificate to.

It can launch a temporary web server to do so, however, this will require to stop your usual web server (e.g. Apache) and lead to a few seconds of downtime. Alternatively, you can provide your web root path. Let’s Encrypt will create the files there, and they will be served directly by your usual web server. This approach does not lead to downtime, but presents some additional challenges when using it with OpenStack Ansible:

  1. OpenStack Ansible does not have a web root path out of the box to be used by Let’s Encrypt.
  2. SSL certificates must be provided to HAProxy, which runs on metal, while the Apache server to be used by Let’s Encrypt runs inside a container.

Installing OpenStack Ansible

Install OpenStack as usual, without providing any certificates. Self-signed ones will be therefore generated, and we will replace them later.

Enable web root

We will not actually create a web root. Since Let’s Encrypt only requires writing on directory, we will create an alias to the .well-known path.

Attach to the horizon container. Replace the container name accordingly with your setup. If you don’t know the name, run lxc-ls | grep horizon to get the container name.

Add the following line to /etc/apache2/sites-enabled/openstack-dashboard.conf, inside the <VirtualHost *:80> tag

Alias /.well-known /var/www/html/.well-known 

Restart the apache2 service:

Now, we can use /var/www/html as our web root, at least from the Let’s Encrypt Certbot point of view.

Getting the certificates

Now install the Let’s Encrypt Certbot. The intention is to only get the certificates files, not configure them in Apache. Use the following commands to do so:

When asked to choose an authentication method, choose 2

When asked for the webroot, input /var/www/html

After this, the certificate files will be placed on /etc/letsencrypt/live/

Allow the container to copy files to the host

Generate an SSH key inside the container:

Print the public key and copy it to the clipboard:

Now append the container’s public key to the authorized_keys file in the host:

This will allow the container to copy the certificates to the host using scp.

Applying the certificates in HAProxy

To use them in HAProxy, we must concatenate some files. Replace accordingly.

Set the permissions properly:

Still inside the horizon container, copy the files we just generated to the host. Replace and HOST_IP_ADDRESS accordingly.

Now exit the container and apply the new certificate:

Renewing the certificates automatically

As Let’s Encrypt certificates are only valid for 90 days, it is highly advisable to schedule automatic renewing. We can do this using crontab inside the horizon container.

Attach to the horizon container. Replace the container name accordingly with your setup. If you don’t know the name, run lxc-ls | grep horizon to get the container name.

Open the crontab editor:

Place this line at the end of the file, replacing and HOST_IP_ADDRESS accordingly.

26 3 * * 5 certbot renew && cat /etc/letsencrypt/live/ > /etc/letsencrypt/live/ && cat /etc/letsencrypt/live/ /etc/letsencrypt/live/ /etc/letsencrypt/live/ > /etc/letsencrypt/live/ && scp /etc/letsencrypt/live/* HOST_IP_ADDRESS:/etc/ssl/private && ssh HOST_IP_ADDRESS service haproxy reload

This will run every week, but it will only actually renew the certificate at most every 60 days, as only certificates that expire in less than 30 days are renewed. Running it more often than every 60 days makes it safer, as even if it fails to run once after the 60 days window, it will still run again before the certificate expire.