Hashicorp Vault (Vault) is an open-source tool for managing secrets. This blog post will demonstrate how to use Vault to generate a root CA for trusted TLS communication and how to generate client certificates for mutual TLS communication. Not only does this blog post contain a high-level overview of Vault, it includes working infrastructure-as-code and step-by-step tutorial.

Goals

• Run Hashicorp Vault (Vault) on Docker
• Initialize Vault
• Generate root CA
• Generate intermediate certificate signed by CA
• Import root CA into the Firefox browser
• Generate a client certificate
• Import client certificate into the Firefox browser

DISCLAIMER

This blog post is a proof of concept (POC) for a homelab and does NOT implement best practices for an enterprise environment. Please review the Hashicorp Vault documentation for best practices. In addition, this POC will be setting up a single Vault and Consul instance which does not provide high-availability.

DISCLAIMER

Background

What is Vault

HashiCorp Vault is an open source tool for managing secrets. Application identity management with Vault enables applications and machines to automatically create, change, and rotate secrets needed for communications, services, scripts, etc. Additionally, Vault enables administrators to manage applications and machines by providing access control over different secrets.

• Roles – Roles are created to assign the capabilities a certificate has such as time to live (TTL), extensions, specify users, and CIDR blocks.
• Policies – Policies provide a declarative way to grant or forbid access to certain paths and operations in Vault.
• Auth methods – Auth methods are the components in Vault that perform authentication and are responsible for assigning identity and a set of policies to a user.
• Chain of trust – Certificate hierarchy, a subordinate Intermediate CA certificate will be issued by the Root CA to issue an end-entity SSL certificate. This creates a chain of trust that begins at the Root CA, through the Intermediate CA and ending with the SSL certificate.
  • Root CA – Root CA’s are implicitly trusted by web browsers and applications.
  • Intermediate certificate – An intermediate CA certificate is a subordinate certificate issued by the trusted root specifically to issue end-entity server certificates. The result is a trust-chain that begins at the trusted root CA, through the intermediate and finally ending with a leaf certificate.
  • Leaf certificate – A “leaf certificate” is what is more commonly known as an end-entity certificate. Certificates come in chains, starting with the root CA, each certificate being the CA which issued (signed) the next one. The last certificate is the non-CA certificate which contains the public key you actually want to use.

Trust models

Certificates have a hierarchy (diagram below) which is the root CA, intermediate cert(s), and leaf nodes/certs. Typically, you generate a root CA, generate an intermediate cert based on the root CA, and store the root CA in a super safe location (airgapped). The intermediate certificate generated from the root CA is used for all future actions pertaining to certificate generation and revocation.

Common questions I was asking during this project was “How do you know when to generate intermediate?” or “How does one do segmentation?”. The rule of thumb for generating intermediate certificates is creating them based on your trust model – how you want to do segmentation. In my homelab, I have a production and development network and my trust model/segmentation is targeted at these networks. In an enterprise, you would typically create a trust model/segmentation between different business units like DevOps, Finance, Human resources, security, etc. Another example is how you do segmentation in Active Directory. You don’t place all the users in the root of the domain, you create groups and place users into those groups. You can think of intermediate certificates as those Active Directory groups.

 

 

Create ROOT CA with Vault + Docker

Step 0: Spin up Vault stack

1. git clone https://github.com/CptOfEvilMinions/BlogProjects
2. cd BlogProjects/Docker-vault
3. docker-compose build
4. docker-compose up -d

Step 1: Init Vault

1. Open browser to http://<Docker IP addr>:8200
   1. Enter 1 for Key Shares
      1. NEVER EVER ENTER 1 FOR PRODUCTION – Only enter 1 for testing 
   2. Enter 1 for Key threshold
      1. NEVER EVER ENTER 1 FOR PRODUCTION – Only enter 1 for testing
   3. Select Initialize
2. Select “Download keys”
3. Open terminal
4. cat ~/Downloads/vault-cluster-vault-*
5. Back to browser
6. Select “Continue to Unseal”
7. Enter “<key from downloaded file>” into Master Key Portion
8. Select “Unseal”
9. Select “Token” for method
10. Enter “<root_token from downloaded file>” into sign in
11. Select “Sign In”

Step 3: Install Vault CLI tool on macOS

For other OSes please visit this page

1. brew tap hashicorp/tap
   1. Add the HashiCorp tap, a repository of all our Homebrew packages.
2. brew install hashicorp/tap/vault
   1. Install Vault
3. brew install jq
   1. Install jq which is a command-line tool for interacting with JSON

Step 3: Login into Vault

1. export VAULT_ADDR=https://<Docker IP addr>:8200
2. vault login
   1. Enter root_token from step 1

Step 4: Generate root CA

1. vault secrets enable pki
   1. Enable the PKI engine
2. vault secrets tune -max-lease-ttl=87600h pki
   1. Tune the pki secrets engine to issue certificates with a maximum time-to-live (TTL) of 87600 hours
   2. 87600 hours = 3650 days = 10 years
3. vault write -field=certificate pki/root/generate/internal common_name="<domain>" ttl=87600h > CA_cert.crt
   1.  Generate a self-signed root certificate using the PKI secrets engine and write the public certificate to disk
4. vault write pki/config/urls issuing_certificates="http://127.0.0.1:8200/v1/pki/ca" crl_distribution_points="http://127.0.0.1:8200/v1/pki/crl"

Step 5: Generate intermediate CA

1. vault secrets enable -path=pki_int pki
   1. Enable the pki secrets engine at the pki_int path
2. vault secrets tune -max-lease-ttl=43800h pki_int
   1. Tune the pki_int secrets engine to issue certificates with a maximum time-to-live (TTL) of 43800 hours
   2. 43800 hours = 5 years
3. vault write -format=json pki_int/intermediate/generate/internal common_name="<domain> Intermediate Authority" | jq -r '.data.csr' > pki_intermediate.csr
   1.  Generate an intermediate certificate and save the certificate signing request (CSR) to disk
4. vault write -format=json pki/root/sign-intermediate csr=@pki_intermediate.csr format=pem_bundle ttl="43800h" | jq -r '.data.certificate' > intermediate.crt
   1. Sign the intermediate certificate with the root certificate and write the generated certificate to disk
5. vault write pki_int/intermediate/set-signed [email protected]
   1. Import the signed certificate by the ROOT CA into Vault

Step 6: Create a role

1. vault write pki_int/roles/<domain> allowed_domains="<domain>" allow_subdomains=true max_ttl="8760h"
   1. Create a role that allows certificates for subdomains to be generated

Step 7: Request a leaf certificate

1. vault write -format=json pki_int/issue/<domain> common_name="*.<domain>" ttl="8760h" > wildcard_<domain>.json
   1. Request a new wildcard certificate for the domain specified above
   2. 8760 hours = 1 year
2. cat wildcard_<domain>.json | jq -r '.data.private_key' > wildcard_<domain>.pem
   1. Extract private key from JSON blob
3. cat wildcard_<domain>.json | jq -r '.data.certificate' > wildcard_<domain>.crt
4. cat wildcard_<domain>.json | jq -r '.data.ca_chain[]' >> wildcard_<domain>.crt
   1. Extract public certificate chain from JSON blob
5. rm wildcard_<domain>.json

Step up 8: Write Vault cert data to disk

1. cat CA_cert.crt
   1. Copy root CA certificate
2. cat wildcard_hackinglab_local.pem
   1. Copy private key
3. cat wildcard_hackinglab_local.crt
   1. Copy public certificate
4. Open a new terminal tab
5. cd BlogProjects/Docker-vault
6. Paste root CA output: echo "<CA_cert.crt ouput>" > conf/tls/root_ca/CA_cert.crt
7. Paste wildcard_hackinglab_local.pem output: echo "<wildcard_hackinglab_local.pem ouput>" > conf/tls/signed_cert/wildcard_<domain>.pem
8. Paste wildcard_hackinglab_local.crt output: echo "<wildcard_hackinglab_local.ccrt ouput>" > conf/tls/signed_cert/wildcard_<domain>.crt

Step 9: Install ROOT CA on Firefox

1. Open Firefox
2. Settings > Preferences > Privacy & Security > Certificates
3. Select “View certificates”
   1. Select the “Authorities” tab
   2. Select “Import”
   3. Import BlogProjects/Docker-vault/conf/tls/root_ca/CA_cert.crt
   4. Select “Open”
      1. Select “Trust this CA to identify websites”
         

Step 10: Spin up NGINX stack

1. docker-compose -f docker-compose-signed-cert.yml build
2. docker-compose -f docker-compose-signed-cert.yml up
3. Open a browser to https://<Docker IP addr>:8443

Generating Device/machine/client certificates

Step 1: Generate an intermediate certificate

1. Back to vault terminal
2. vault secrets enable -path=devices_<domain>_pki_int pki
   1. Enable the pki secrets engine at the devices_<domain>_pki_int pki path.
3. vault secrets tune -max-lease-ttl=26280h devices_<domain>_pki_int/
   1. Tune the PKI secrets engine to issue certificates with a maximum time-to-live (TTL) of 26280 hours.
   2. 26280 hours = 3 years
4. vault write -format=json devices_<domain>_pki_int/intermediate/generate/internal common_name="devices.<domain> Intermediate Authority" | jq -r '.data.csr' > devices_<domain>_pki_int.csr
   1. Generate an intermediate certificate and save the certificate signing request (CSR) to disk
5. vault write -format=json pki/root/sign-intermediate csr=@devices_<domain>_pki_int.csr format=pem_bundle ttl="26280h" | jq -r '.data.certificate' > devices_<domain>_int.crt
   1. Sign the device intermediate certificate (devices.hackinglab.local) with the intermediate certificate (hackinglab.local) and write the generated certificate to disk
6. vault write devices_<domain>_pki_int/intermediate/set-signed certificate=@devices_<domain>_int.crt
   1. Import the signed certificate by the intermediate certificate into Vault

Step 2: Create a role 

1. vault write devices_<domain>_pki_int/roles/devices-<domain> allowed_domains="devices.<domain>" allow_subdomains=true max_ttl="26280h"
   1. Create a role that allows client certificates to be generated

Step 3: Generate a client certificate

1. vault write -format=json devices_<domain>_pki_int/issue/devices-<domain> common_name="<hostname>.devices.<domain>" ttl="26280h" > <hostname>_devices_<domain>.json
   1. Request a new wildcard certificate for the domain specified above
   2. 26280 hours = 3 years
2. cat <hostname>.devices.<domain>.json | jq -r '.data.private_key' > <hostname>.devices.<domain>.pem
   1. Extract private key from JSON blob
3. cat <hostname>.devices.<domain>.json | jq -r '.data.certificate' > <hostname>.devices.<domain>.crt
4. cat <hostname>.devices.<domain>.json | jq -r '.data.ca_chain[]' >> <hostname>.devices.<domain>.crt
   1. Extract public certificate from JSON blob
5. rm <hostname>.devices.<domain>.json

Step 4: Write Vault cert data to disk

1. cat devices.<domain>.crt
   1. Copy public certificate
2. Open a new terminal tab
3. cd BlogProjects/Docker-vault
4. Paste devices.<domain>.crt output: echo "devices.<domain>.crt ouput>" > conf/tls/mutual_tls/devices.<domain>.pem

Step 5: Convert PEM format certificate to PKSC12

1. Back to Vault terminal
2. cat <hostname>.devices.<domain>.crt
   1. Copy public certificate
3. cat <hostname>.devices.<domain>.pem
   1. Copy private key
4. Switch back to the other terminal tab
5. echo "< <hostname>.devices.<domain>.crt output >" > conf/tls/client_cert/<hostname>.devices.<domain>.crt
6. echo "< <hostname>.devices.<domain>.pem output >" > conf/tls/client_cert/<hostname>.devices.<domain>.pem
7. openssl pkcs12 -export -out ~/Desktop/<hostname>.devices.<domain>.p12 -inkey conf/tls/client_cert/<hostname>.devices.<domain>.pem -in conf/tls/client_cert/<hostname>.devices.<domain>.crt -certfile conf/tls/root_ca/CA_cert.crt
   1. Enter an export password

Step 6: Spin up NGINX stack with mutual TLS

1. docker-compose -f docker-compose-mutal-tls.yml build
2. docker-compose -f docker-compose-mutal-tls.yml up

Step 7: Browse to the site

1. Open Firefox browser to https://<Docker IP addr>:8443

Step 8: Import PKSC12 client cert into Firefox

1. Open Firefox
2. Settings > Preferences > Privacy & Security > Certificates
3. Select “View certificates”
   1. Select “Your certificates” tab
   2. Select Import
   3. Import <hostname>.devices.<domain>.p12
   4. Select open
   5. Enter the export password from above

Step 9: Browse to the site

1. Open the Firefox browser to https://<Docker IP addr>:8443
2. Firefox will request you to select your identity
   

Lessons learned

I am currently reading a book called “Cracking the Coding Interview” and it is a great book. One interesting part of the book is their matrix to describe projects you worked on and the matrix contains the following sections which are: challenges, mistakes/failures, enjoyed, leadership, conflicts, and what would you do differently. I am going to try and use this model at the end of my blog posts to summarize and reflect on the things I learn. I don’t blog to post things that I know, I blog to learn new things and to share the knowledge of my security research.

New skills/knowledge

• Learned Hashicorp Vault
• Learned how the chain of trust works with certificates
• Learned how mutual TLS works and is implemented

Challenges

• Was not able to find best practices documents and tutorials
• Learning the terminology

What You’d Do Differently

• Not use the root account to perform the activities done above
  • Enable LDAP auth

References

• Build Your Own Certificate Authority (CA)
• Managing Secrets with Vault and Consul
• Vault delete
• Vault auth enable
• DockerHub – containous/whoami
• X.509 Certificate Management with Vault
• Create a client certificate using your CA certificate
• HashiCorp Vault and TLS Certificate Authentication for .NET Applications (Comprehensive guide)
• Create the root pair
• YouTube – Streamline Certificate Management
• SECURING WEBSITES WITH NGINX AND CLIENT-SIDE CERTIFICATE AUTHENTICATION ON LINUX
• How To View A Password Behind The Asterisks In a Browser
• Vault | Free Vectors, Stock Photos & PSD
• Install Vault