How to configure SSL with certificate verification

HTTPS uses SSL to make the HTTP connection secure, but SSL doesn’t actually provide much security at all unless certificate verification is enabled. In the vast majority of deployments, verification is turned off, because it’s difficult to setup properly, and therefore most of the benefit of SSL isn’t realized.

This document describes how to setup SSL certificate verification with the NetApp ONTAP OpenStack Cinder driver.

Create a CA

While it’s possible to obtain certificates from an actual CA, oftentimes that costs money or interacting with a 3rd party can take too long. The simple solution is to create your own CA.

First install easy-rsa:

bswartz@leviathan:~$ sudo apt-get -y install easy-rsa

We will use easy-rsa to manage our certificates. It requires that you run as root, so I assume from here on out that you’ve done:

bswartz@leviathan:~$ sudo -i

Go to the easy-rsa directory and modify the config file. You don’t need to change anything, but the options here will set defaults which you can manually override later on.

root@leviathan:~# cd /usr/share/easy-rsa
root@leviathan:/usr/share/easy-rsa# vi vars

When you’re done, source the file:

root@leviathan:/usr/share/easy-rsa# . vars

Now we’re ready to create our CA cert. This step will create the root CA’s key and certificate. Everything else depends on these files, and the security of every certificate generated relies on keeping the key file’s contents private. This command will ask you a bunch of questions about the names in the certificate. The important one is CN (Common Name) which is how this certificate will be referred to.

root@leviathan:/usr/share/easy-rsa# ./build-ca
Generating a 2048 bit RSA private key
........+++
.......................................+++
writing new private key to 'ca.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) []: US
State or Province Name (full name) []: NC
Locality Name (eg, city) []: RTP
Organization Name (eg, company) []: NetApp
Organizational Unit Name (eg, section) []: DFMG
Common Name (eg, your name or your server's hostname) []: bswartz-ca.rtp.netapp.com
Name []: My CA
Email Address []: bswartz@netapp.com

Next we create the Diffie-Hellman params. Expect this to take a while.

root@leviathan:/usr/share/easy-rsa# ./build-dh
Generating DH parameters, 2048 bit long safe prime, generator 2
This is going to take a long time
..............................................................++*++*

Create a cert for a SVM

Now for each SVM, we need to create a certificate and sign it. It’s important that SVM have DNS names. If you don’t have DNS configured for your SVM management LIFs, now would be a good time to set it up. If you can’t setup DNS, you’ll have to make up hostnames and manually put them on your /etc/hosts file on every OpenStack Cinder node.

For this example, I’m using a SVM called OPSK-01, which has an IP address of 10.63.152.206. I have decided to make up the DNS name opsk-01.rtp.netapp.com because I don’t have working DNS in my lab. I will add an entry to my hosts file now.

root@leviathan:/usr/share/easy-rsa# echo 10.63.152.206 opsk-01.rtp.netapp.com >> /etc/hosts

We will now create the certificate.

root@leviathan:/usr/share/easy-rsa# ./build-key-server opsk-01.rtp.netapp.com
Generating a 2048 bit RSA private key
........................................+++
........................................................................+++
writing new private key to 'opsk-01.rtp.netapp.com.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) []: US
State or Province Name (full name) []: NC
Locality Name (eg, city) []: RTP
Organization Name (eg, company) []: NetApp
Organizational Unit Name (eg, section) []: DFMG
Common Name (eg, your name or your server's hostname) []: opsk-01.rtp.netapp.com.key
Name []: OPSK-01
Email Address []: bswartz@netapp.com

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
Using configuration from /usr/share/easy-rsa/openssl-1.0.0.cnf
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
countryName           :PRINTABLE:'US'
stateOrProvinceName   :PRINTABLE:'NC'
localityName          :PRINTABLE:'RTP'
organizationName      :PRINTABLE:'NetApp'
organizationalUnitName:PRINTABLE:'DFMG'
commonName            :PRINTABLE:'opsk-01.rtp.netapp.com.key'
name                  :PRINTABLE:'OPSK-01'
emailAddress          :IA5STRING:'bswartz@netapp.com'
Certificate is to be certified until Jan 11 20:57:42 2027 GMT (3650 days)
Sign the certificate? [y/n]:y

1 out of 1 certificate requests certified, commit? [y/n] y
Write out database with 1 new entries
Data Base Updated

Now at this point we have what we need to install the certificate on the SVM. There are 3 files we need to copy/paste. I suggest dumping them to the terminal and opening another terminal window to SSH to the SVM.

root@leviathan:/usr/share/easy-rsa# cat keys/opsk-01.rtp.netapp.com.crt
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 4 (0x4)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, ST=NC, L=RTP, O=NetApp, OU=DFMG, CN=bswartz-ca.rtp.netapp.com/name=My CA/emailAddress=bswartz@netapp.com
        Validity
            Not Before: Jan 13 20:21:52 2017 GMT
            Not After : Jan 11 20:21:52 2027 GMT
        Subject: C=US, ST=NC, L=RTP, O=NetApp, OU=DFMG, CN=opsk-01.rtp.netapp.com/name=OPSK-01/emailAddress=bswartz@netapp.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:bc:5a:b1:11:98:65:0f:36:ac:2c:6f:4e:eb:43:
                    94:27:4d:da:d0:32:4a:a4:a7:d4:15:51:83:a1:36:
                    8d:85:91:16:d0:fa:00:47:58:48:50:4d:6e:1b:d1:
                    01:bb:19:c2:fa:ae:70:ec:07:3f:6d:ba:f4:ff:66:
                    90:47:ef:21:5f:ef:9a:b6:ab:cc:e2:84:80:9f:2e:
                    78:32:4b:90:2d:b8:f8:ef:c5:9a:8c:79:76:21:49:
                    9b:38:33:25:12:43:7c:d7:2c:6a:ad:c3:3b:0d:9f:
                    1c:e6:50:f3:48:d3:47:06:5f:98:c1:a2:db:31:36:
                    b6:3e:dd:25:a2:a0:b9:2b:f1:11:73:4a:48:dc:26:
                    57:a9:04:35:c3:42:99:af:95:c9:03:a7:c5:11:59:
                    25:a8:95:7b:a7:18:2a:8d:0d:ee:31:aa:ef:a9:27:
                    35:56:da:7b:8a:01:2c:86:37:62:db:6a:e6:6d:30:
                    f8:1c:87:27:56:1a:b9:56:c6:c0:31:6d:d2:f3:58:
                    2f:cc:c6:30:12:c4:6d:c8:3a:c7:e8:ea:2f:6f:ff:
                    90:33:d0:88:00:25:45:bb:2c:fa:06:bf:2c:37:35:
                    8c:a4:a7:22:cb:25:a9:35:04:7d:ed:e6:54:c2:50:
                    1a:6c:60:3e:d7:14:be:46:b5:b9:8a:6c:28:08:c6:
                    a8:c3
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: 
                CA:FALSE
            Netscape Cert Type: 
                SSL Server
            Netscape Comment: 
                Easy-RSA Generated Server Certificate
            X509v3 Subject Key Identifier: 
                8E:7D:FA:4C:96:A7:E9:21:78:CA:4F:97:2F:C1:AF:4A:44:05:10:6D
            X509v3 Authority Key Identifier: 
                keyid:7A:79:B9:11:19:BA:AC:B1:0F:B1:25:7D:77:0B:18:E8:76:4B:EE:00
                DirName:/C=US/ST=NC/L=RTP/O=NetApp/OU=DFMG/CN=bswartz-ca.rtp.netapp.com/name=My CA/emailAddress=bswartz@netapp.com
                serial:DB:E4:13:7E:F9:33:7B:BF

            X509v3 Extended Key Usage: 
                TLS Web Server Authentication
            X509v3 Key Usage: 
                Digital Signature, Key Encipherment
            X509v3 Subject Alternative Name: 
                DNS:opsk-01.rtp.netapp.com
    Signature Algorithm: sha256WithRSAEncryption
         32:c0:98:43:60:8a:c6:44:59:34:1a:ad:d1:e1:3d:77:d1:b4:
         8d:5b:38:d5:9b:fd:da:14:df:0e:60:97:d2:3e:7a:37:ce:6d:
         dc:59:d9:54:c4:0d:ee:5f:e4:8b:24:01:e1:b0:51:e5:d4:55:
         72:00:5a:ba:52:50:99:b8:4e:44:f7:02:0d:e9:2e:60:7b:a1:
         3c:c9:ea:12:f2:5d:ba:52:17:3f:58:b3:e8:ac:c8:a9:5f:67:
         a4:32:99:87:60:c6:da:98:15:36:86:0d:aa:2c:d8:2e:ba:39:
         56:9c:ee:5f:19:db:47:9c:b0:20:a5:da:4b:2e:72:5a:86:81:
         c6:31:a5:3c:e8:29:9e:37:2d:16:aa:0f:7a:f0:75:41:65:48:
         a2:d0:ac:23:ce:65:42:74:e6:38:eb:43:ee:9a:d1:37:1f:89:
         a5:e3:10:71:c0:44:ef:e0:31:85:b0:55:c2:42:81:06:ea:ff:
         86:e1:67:ff:4a:a7:34:18:6a:c2:74:46:2b:2a:1a:0e:23:02:
         15:1e:39:d6:f9:a8:9b:1c:aa:7c:4e:7c:fa:91:db:b8:96:ea:
         6f:b1:f9:ad:12:9f:30:8e:7f:7b:6e:ad:4c:1f:a2:63:3c:47:
         92:87:c8:0e:01:16:6d:b8:76:bc:1c:bc:9a:95:02:00:02:74:
         17:80:4f:34
-----BEGIN CERTIFICATE-----
MIIFTzCCBDegAwIBAgIBBDANBgkqhkiG9w0BAQsFADCBnzELMAkGA1UEBhMCVVMx
CzAJBgNVBAgTAk5DMQwwCgYDVQQHEwNSVFAxDzANBgNVBAoTBk5ldEFwcDENMAsG
A1UECxMEREZNRzEiMCAGA1UEAxMZYnN3YXJ0ei1jYS5ydHAubmV0YXBwLmNvbTEO
MAwGA1UEKRMFTXkgQ0ExITAfBgkqhkiG9w0BCQEWEmJzd2FydHpAbmV0YXBwLmNv
QWVIotCsI85lQnTmOOtD7prRNx+JpeMQccBE7+AxhbBVwkKBBur/huFn/0qnNBhq
wnRGKyoaDiMCFR451vmomxyqfE58+pHbuJbqb7H5rRKfMI5/e26tTB+iYzxHkofI
DgEWbbh2vBy8mpUCAAJ0F4BPNA==
-----END CERTIFICATE-----

root@leviathan:/usr/share/easy-rsa# cat keys/opsk-01.rtp.netapp.com.key
-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQC8WrERmGUPNqws
b07rQ5QnTdrQMkqkp9QVUYOhNo2FkRbQ+gBHWEhQTW4b0QG7GcL6rnDsBz9tuvT/
pwAAiRWvcpp1qPKcf3R2Uz1dqtzcy8cXZNB4WoKBTUlOdCcn7W6ADiaV6xjuea3y
27imDBFoXJ0yOoQ3AIbUBMd86plOZO4Dj0PMhuRLawKBgFrt2Bl9uWN1+SEpttCl
OYKsDGlLgtJaG2DG9O8tfGPlZEIGUP7phOBokDk2VxX47BrEuOqaIo79IKJmURa0
qGXifA73yzrnvT+wZuCr0ao83pIBK2HsX+767F9t9efDhlz6e1tfal25J95WhAxv
ldXovuA28UwzVE2OI2v5CUSH
-----END PRIVATE KEY-----

root@leviathan:/usr/share/easy-rsa# cat keys/ca.crt
-----BEGIN CERTIFICATE-----
MIIEzTCCA7WgAwIBAgIJANvkE375M3u/MA0GCSqGSIb3DQEBCwUAMIGfMQswCQYD
VQQGEwJVUzELMAkGA1UECBMCTkMxDDAKBgNVBAcTA1JUUDEPMA0GA1UEChMGTmV0
dOVeeJUEbr9qZpW1oiWHQoE/QA4PZ7+XL0wbi1k6Wz+JwGSEAuPwSpiT6PQ1/6kE
1VHz+x+W5wFklAMxZAXpzny5HkHoe4P7KA0G6Z25fa5iRq7zmQ/ZN6AOxxEHNcYD
76awJj5tVMlscCsrM446vYLB1LSHI+5gNyhz4/KULkXW
-----END CERTIFICATE-----

Note that while the two .crt files don’t contain sensitive information, the .key file does. If the contents of this .key file fall into the wrong hands than all SSL communications to the SVM where we install it will be compromised.

Install the certs on the SVM

In a second terminal window, let’s SSH to our SVM. We can use the DNS name that we made up for this purpose.

bswartz@leviathan:~$ ssh admin@opsk-01.rtp.netapp.com

The authenticity of host 'opsk-01.rtp.netapp.com (10.63.152.206)' can't be established.
RSA key fingerprint is SHA256:34IK2mkZgceVzYzdw/+zB56JWBCJD6g30w+plJtYmUU.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'opsk-01.rtp.netapp.com' (RSA) to the list of known hosts.
Password:

First let’s list the installed certificates:

OPSK-01::> security certificate show -vserver OPSK-01
Vserver    Serial Number   Common Name                            Type
---------- --------------- -------------------------------------- ------------
OPSK-01    52AF6048        OPSK-01.cert                           server
Certificate Authority: OPSK-01.cert
Expiration Date: Tue Dec 16 20:19:20 2017

The existing certificate is self-signed and worthless. Let’s delete it:

OPSK-01::> security certificate delete -vserver OPSK-01 *

Warning: Deleting a server certificate will also delete the corresponding server-chain certificate, if one exists.
Do you want to continue? {y|n}: y
1 entry was deleted.

At this point SSL has been disabled, if it was previously enabled.

Next we install the certificate we just created. We will copy paste the SVM cert, and then the SVM key, then it will ask us if we want to continue. We will answer yes the first time and no the second time. Note that you should only copy the text between the -----BEGIN CERTIFICATE----- and -----END CERTIFICATE-----. You don’t need the human-readable part. Also note that you have to enter an empty line after you paste the text each time.

OPSK-01::> security certificate install -vserver OPSK-01 -type server

Please enter Certificate: Press  when done
-----BEGIN CERTIFICATE-----
MIIFTzCCBDegAwIBAgIBBDANBgkqhkiG9w0BAQsFADCBnzELMAkGA1UEBhMCVVMx
CzAJBgNVBAgTAk5DMQwwCgYDVQQHEwNSVFAxDzANBgNVBAoTBk5ldEFwcDENMAsG
h2DG2pgVNoYNqizYLro5VpzuXxnbR5ywIKXaSy5yWoaBxjGlPOgpnjctFqoPevB1
QWVIotCsI85lQnTmOOtD7prRNx+JpeMQccBE7+AxhbBVwkKBBur/huFn/0qnNBhq
wnRGKyoaDiMCFR451vmomxyqfE58+pHbuJbqb7H5rRKfMI5/e26tTB+iYzxHkofI
DgEWbbh2vBy8mpUCAAJ0F4BPNA==
-----END CERTIFICATE-----

Please enter Private Key: Press  when done
-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQC8WrERmGUPNqws
b07rQ5QnTdrQMkqkp9QVUYOhNo2FkRbQ+gBHWEhQTW4b0QG7GcL6rnDsBz9tuvT/
27imDBFoXJ0yOoQ3AIbUBMd86plOZO4Dj0PMhuRLawKBgFrt2Bl9uWN1+SEpttCl
OYKsDGlLgtJaG2DG9O8tfGPlZEIGUP7phOBokDk2VxX47BrEuOqaIo79IKJmURa0
qGXifA73yzrnvT+wZuCr0ao83pIBK2HsX+767F9t9efDhlz6e1tfal25J95WhAxv
ldXovuA28UwzVE2OI2v5CUSH
-----END PRIVATE KEY-----

Please enter certificates of Certification Authorities (CA) which form the certificate chain of the server certificate. This starts with the issuing CA certificate of the server certificate and can range up to the root CA certificate.

Do you want to continue entering root and/or intermediate certificates {y|n}: y

Please enter Intermediate Certificate: Press  when done
-----BEGIN CERTIFICATE-----
MIIEzTCCA7WgAwIBAgIJANvkE375M3u/MA0GCSqGSIb3DQEBCwUAMIGfMQswCQYD
VQQGEwJVUzELMAkGA1UECBMCTkMxDDAKBgNVBAcTA1JUUDEPMA0GA1UEChMGTmV0
dOVeeJUEbr9qZpW1oiWHQoE/QA4PZ7+XL0wbi1k6Wz+JwGSEAuPwSpiT6PQ1/6kE
1VHz+x+W5wFklAMxZAXpzny5HkHoe4P7KA0G6Z25fa5iRq7zmQ/ZN6AOxxEHNcYD
76awJj5tVMlscCsrM446vYLB1LSHI+5gNyhz4/KULkXW
-----END CERTIFICATE-----

Do you want to continue entering root and/or intermediate certificates {y|n}: n

You should keep a copy of the private key and the CA-signed digital certificate for future reference.

Now we can re-enable SSL on this SVM. Note that the serial at the end of the command will vary depending on how many certs your CA has generated. Use tab completion and ONTAP will tell you the correct serial.

OPSK-01::> security ssl modify -vserver OPSK-01 -server-enabled true -common-name opsk-01.rtp.netapp.com -ca bswartz-ca.rtp.netapp.com -serial 04

We can now end our ssh session and go back to finish up the installation on the Cinder node.

Trusting your CA

Because anyone can create a root certificate and perform security attacks on every machine that trusts that certificate, it’s important to be careful about adding trusted certificates. In this case, the root certificate is ours, and we trust ourselves, so we will install it. Just remember that after we do this, anyone who has the ca.key file we created at the beginning can compromise all SSL communications on the client where we install this certificate, so keep your key files safe.

We will copy/paste the contents of that ca.crt file into a file on each Cinder node. I have chosen the filename bswartz-ca.crt to disambiguate it from other cert files.

root@leviathan:/usr/share/easy-rsa# cd /usr/local/share/ca-certificates
root@leviathan:/usr/local/share/ca-certificates# cat > bswartz-ca.crt << EOF
-----BEGIN CERTIFICATE-----
MIIEzTCCA7WgAwIBAgIJANvkE375M3u/MA0GCSqGSIb3DQEBCwUAMIGfMQswCQYD
VQQGEwJVUzELMAkGA1UECBMCTkMxDDAKBgNVBAcTA1JUUDEPMA0GA1UEChMGTmV0
QXBwMQ0wCwYDVQQLEwRERk1HMSIwIAYDVQQDExlic3dhcnR6LWNhLnJ0cC5uZXRh
dOVeeJUEbr9qZpW1oiWHQoE/QA4PZ7+XL0wbi1k6Wz+JwGSEAuPwSpiT6PQ1/6kE
1VHz+x+W5wFklAMxZAXpzny5HkHoe4P7KA0G6Z25fa5iRq7zmQ/ZN6AOxxEHNcYD
76awJj5tVMlscCsrM446vYLB1LSHI+5gNyhz4/KULkXW
-----END CERTIFICATE-----
EOF

Last we have to run the update command which will install the certificate and make it trusted.

root@leviathan:/usr/local/share/ca-certificates# update-ca-certificates
Updating certificates in /etc/ssl/certs...
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...

done.
Updating Mono key store
Linux Cert Store Sync - version 4.2.1.0
Synchronize local certs with certs from local Linux trust store.
Copyright 2002, 2003 Motus Technologies. Copyright 2004-2008 Novell. BSD licensed.

I already trust 173, your new list has 174
Import process completed.
Done
done.

At this point all that’s left to do is update your cinder.conf file to enable https and to set the SVM hostname to the DNS name we created instead of the IP address.

Other SVMs

The process is repeatable for each SVM. The CA certificate can be re-used, and you just need to run the ./build-key-server script for each new SVM, and install the certificate on the SVM. Clients that trust our CA will automatically verify the new certificate.

Other clients

If there are multiple Cinder nodes, each one needs its /etc/hosts file updated for every SVM’s hostname (if you don’t have working DNS). Each Cinder node needs to have the CA certificate installed one time.

You’re done!

Now you can use HTTPS without getting errors or warnings from python libraries which correctly reject self-signed certs as insecure. Configuring SSL this way also protects from MITM (man in the middle) attacks which are way easier to perform than most people believe.

Ben Swartzlander
Ben has been working in the storage industry as a software engineer for more than 15 years and has extensive experience with storage systems, network protocols, virtualization, and open source projects. Ben has been a contributor to the OpenStack project for nearly 5 years.