When setting up internal services (using Alpine Linux, there's generally a need for certificates for securing internal communication. I brushed over it previously, but a few additional comments about that:
- Encrypting internal communication is a good idea as there are multiple possibilities for eavesdroppers for internal traffic:
- If you have any servers that allow multiple users (particularly with interactive shells), those users could be sniff network traffic.
- Treat systems (whether servers or containers) as though when they'll be compromised, not if. There's ultimately no such thing as a piece of software that has no vulnerabilities, so your infrastructure should have layered defenses.
- I ultimately don't like wildcard certificates across multiple systems, as if one of those systems is compromised, you've given up a large chunk of that asymmetric encryption. Naturally, if you have multiple services on a single system, then sharing a certificate across those services (with, say, a multi-domain certificate) isn't as much of an issue.
From a brief look at various PKI implementations, I settled on OpenXPKI as something that would be fairly easy to use and maintain for the size of what I'm maintaining. EJBCA is, of course, a much more widely-used CA implementation, but it's also more heavyweight than what I'm looking for. As a bonus, OpenXPKI is the primary target for CertNanny, which allows for automated certificate renewal via SCEP. Taking a look at the other open source software that implements a SCEP client, most of it hasn't been updated for considerably longer than CertNanny, other than jSCEP. And, well, I'm trying to avoid running Java where I can, due to how much Java loves chewing through resources (also applies to EJBCA). 😕
I won't be going into installing and setting up OpenXPKI here, although I will mention that needing to install Debian to install the official packages is annoying. I've been reluctant to use a container (due to wanting to treat my CA as core infrastructure), but I may switch over to that at some point in the future.... Just remember to also install openca-tools as well, which isn't listed in the Quickstart documentation, but is needed for SCEP to function.
A few changes/bugfixes are necessary to OpenXPKI so that it works correctly with SCEP:
- If your realm has a profile/I18N_OPENXPKI_PROFILE_TLS_SERVER.yaml (if your OpenXPKI is old enough), update your 05_advanced_style section to add this section beneath subject (this brings it up to the current definition, which allows for specifying SANs on advanced requests, which we need):
.
.
san:
- san_ipv4
- san_dns
.
.
.
- Edit your realm's profile/template/san_dns.yaml, and change the top line, as OpenSSL doesn't like SANs being specified with dns, only DNS:
.
.
.
- After these changes, restart the OpenXPKI daemon so that it reloads the templates.
The service for this example will be PostgreSQL. After starting with the usual Alpine Linux installation and configuration steps is adding PostgreSQL itself:
As per the PostgreSQL documentation, enable SSL in postgresql.conf. Now comes the interesting part: auto-generating server certificates. First off is installing sscep and CertNanny. If your CertNanny installation did not install a JDK, make sure you do so.
A fix is needed for the code here as well:
- Edit your CertNanny's lib/perl/CertNanny/Config.pm and lib/perl/CertNanny/Util.pm so that the openssl calls now use -sha1 instead of -sha:
Then, the CertNanny configuration files need to be set up. The simplest way is to copy the template files and modify them. Key sections to modify include:
Keystore-DEFAULT.cfg
- keystore.DEFAULT.enroll.sscep.URL: set to your OpenXPKI SCEP URL (by default, http://<server>/scep/). sscep enforces using a non-SSL connection, as the connection is (in theory) already encrypted.
Keystore-OpenSSL.cfg
This is the default file we'll be using, since OpenSSL PEM is the de-facto standard on Linux. For PostgreSQL, we're not even concerned about the default entries, since we know where to put the file and that we don't want the server key encrypted. We also want the PostgreSQL server restarted whenever a new key is installed:
keystore.openssl.location = /var/lib/postgresql/12/data/server.crt
keystore.openssl.format = PEM
keystore.openssl.key.file = /var/lib/postgresql/12/data/server.key
keystore.openssl.key.type = OpenSSL
keystore.openssl.key.format = PEM
keystore.openssl.hook.renewal.install.post = /sbin/rc-service postgresql restart
certnanny.cfg
- cmd.sscep: set to the path of your sscep binary.
- Uncomment the include Keystore-OpenSSL.cfg line.
Then, some directory setup:
$ mkdir -p /var/CertNanny/tmp
$ mkdir -p /var/CertNanny/AuthoritativeRootcerts
Put a copy of your root certificate into /var/CertNanny/AuthoritativeRootcerts so that sscep can validate your certificates.
Finally, we can set up automated certificate renewals. For the initial enrollment, use OpenXPKI to create an initial certificate (it uses these as a basis for certificate settings). However, the certificate will need to go through the Extended naming style to perform the following modifications:
- The CN will need to have a suffix of :pkiclient (i.e. CN=<server>:pkiclient), which is the OpenXPKI default for auto-renewing certificates.
- The SAN entries should be all FQDNs the server will listen on, including the hostname used to generate the CN.
The generated certificate and key should then be placed in the locations specified previously in Keystore-OpenSSL.cfg. At this point, you should be clear to enroll the certificate:
You may need to accept the enrollment within OpenXPKI. And then, if you would like to verify, force a renew.
Finally, hooking it up via whichever cron mechanism you're using should finish the job.