How long does it take you to issue a new certificate and apply it where needed? How do you handle your certificate renewal, intermediate and root certificates rotation? How do you handle the CISO request that asks you to reduce the TTL of your certificates? The presented automation will answer these questions and show How to build your internal PKI using Vault How to issue and deploy your certificates where needed How to renew your leaf certificates and rotate your intermediate and root certificates seamlessly How to securely store public certificates in Vault

1. Luxembourg
Automate your
Certificates Lifecycle
with Vault
Online Event | Wed 28th September | 17:00

2. Youtube Channel
Open Source Advocate
Cloud Native Transformation Catalyst
Talent and Organizational Development
Stéphane Este-Gracias
@sestegra
in/sestegra
Luxembourg HUG
#LuxembourgHUG
@hashicorp
HashiCorp User Group
Luxembourg

3. Products News 📰

4. Product News
Infrastructure
▪ Terraform 1.3.0
– Enhance to moved blocks
– Optional attributes for object type constraints (GA)

5. Product News
Infrastructure
▪ Terraform Cloud
– Introducing Sentinel Policies to the
Terraform Registry (Beta)
– Drift Detection is Now
Generally Available
– Terraform Run Tasks Ecosystem
Continues to Grow

6. ▪ Consul-Terraform-Sync (CTS) 0.7
– Adds High Availability and Redundancy (Enterprise Only)
▪ Cloud Development Kit for Terraform (CDKTF) 0.12
– CDK for Terraform Is Now Generally Available
– Support for Terraform Iterators (equivalent of count in HCL)
Product News
Network Infrastructure Automation

7. Product News
Infrastructure
▪ Packer 1.8.1
– New Datasource HTTP
– New Command packer plugins
▪ HCP Packer - Generally Available

8. Product News
Networking
▪ Consul 1.13.0
– Consul on Kubernetes CNI Plugin
(Remove needs of init container and CAP_NET_ADMIN permission)
– Consul on Kubernetes CLI Enhancements for Envoy Troubleshooting
– Terminating Gateways Enhancements
– Cluster Peering (Beta)

9. ▪ Vault 1.11.3
– Kubernetes Secrets Engine
– Integrated Storage Autopilot (for Vault Enterprise)
– Vault agent (add pkiCert option in consul-template)
– Transit secrets engine
– PKI secrets engine (Support for non-disruptive CA certificate rotation)
– Terraform Provider for Vault
– Entropy Augmentation
– Google Cloud auth method
Product News
Security

10. Product News
Security
▪ Vault 1.11.3
- User Interface updates
- Snowflake Database Secrets Engine
- Consul Secrets Engine
- KMIP Secrets Engine (for Vault Enterprise)
- Transform Secrets Engine (for Vault Enterprise)
- Vault Usage Metrics
▪ Vault K8S 1.0.0

11. Product News
Security
▪ Boundary 0.10.0 and Boundary Desktop 1.4.5
– Enhanced IAM Workflows in the Admin UI
– SSH Key Credential Management
▪ HCP Boundary (Public Beta)

12. Product News
Applications
▪ Nomad 1.4.0 (Beta)
– Enhances Nomad Service Discovery support with Health Checks
– Nomad Variables

13. Product News
Applications
▪ Waypoint 0.10.0
- Custom Pipelines (Tech Preview)
- Project Destroy command (waypoint project destroy)
- Nomad/Waypoint Integration Improvements
- Interactive waypoint.hcl Generator (waypoint init)

14. ▪ Vagrant 2.3.0
– Include vagrant-go executable
▪ Journey to Vagrant 3.0
– Vagrant 2.3 - Initial alpha Go-based version
– Vagrant 2.4 - Go-based version as the primary executable
– Vagrant 3.0 - Only Go-based version
Product News
Applications

15. Automate your
Certificates Lifecycle
with Vault

16. ▪ Why Automate your Certificates Lifecycle with Vault?
▪ Build an Internal PKI with Vault
▪ Issue and Deploy Leaf Certificates
▪ Renew Leaf Certificates
▪ Rotate CA Seamlessly
▪ Securely Store and Deploy Public Certificates using Vault
▪ Next Steps
▪ Q&A
⏱ Agenda

17. ▪ To simplify the Demo, the “root” token is used on purpose
to focus only on the PKI secret engine features
▪ Don’t use “root” token on production
▪ See “Next Steps” to enforce auth methods and policies on Vault
⚠ Disclaimer ⚠

18. Why Automate your
Certificates Lifecycle?

19. ▪ Ease the Management of your PKI Hierarchy
▪ Ease the CA Lifecycle (Issue, Rotate)
▪ Ease the Leaf Certificate Renewal and Deployment
▪ Remove the use of Wildcard Certificates (*.example.com
▪ Single Source of Trust and Audit log
▪ Prepare for Post-Quantum Cryptography
▪ (Enterprise Plus) Enhance Security by using an HSM
Why Automate your Certificates Lifecycle
with Vault?

21. Build an Internal PKI with Vault

22. ▪ Three-Tier Hierarchy
▪ Root CA
▪ Intermediate CA
▪ Issuing CA
Three-Tier PKI CA Hierarchy
Root CA
Intermediate
CA
Issuing
CA
Leaf
Cert
Issuing
CA
Issuing
CA
Leaf
Cert
Leaf
Cert
Intermediate
CA
Issuing
CA
Leaf
Cert
Issuing
CA
Issuing
CA
Leaf
Cert
Leaf
Cert

23. Root CA (offline)
Intermediate CA
Issuing CA
Leaf Certificate
▪ Root CA offline
▪ Intermediate CA in Vault
▪ Issuing CA in Vault
▪ A Role for issuing Leaf Certificates
▪ Elliptic Curve
Digital Signature Algorithm (ECDSA)
– Stronger Keys
– Smaller Certificate Size
PKI CA Hierarchy with Vault (Demo)
Leaf Certificate
Leaf Certificate

24. Root CA
Offline root CA
path_len = 2
Steps
● Create a self-signed Root CA using certstrap
Root CA (offline)
path_len = 2

25. TERMINAL
$ certstrap --depot-path root
init
--organization "Example Labs"
--common-name "Example Labs Root CA v1"
--expires "10 years"
--curve P-256
--path-length 2
--passphrase "secret"
Created root/Example_Labs_Root_CA_v1.key (encrypted by passphr
Created root/Example_Labs_Root_CA_v1.crt
Created root/Example_Labs_Root_CA_v1.crl
$ tree root
root
├── Example_Labs_CA_Root_v1.crl
├── Example_Labs_CA_Root_v1.crt
└── Example_Labs_CA_Root_v1.key
Root CA
Create a self-signed root CA

26. Intermediate CA
Intermediate CA managed in Vault
mount = pki_int
path_len = 1
Steps
● Enable PKI secret engine using path=pki_int
● Generate Private Key and CSR
● Sign and Generate Certificate using Root CA
● Store Intermediate Certificate to Vault
Root CA (offline)
path_len = 2
Intermediate CA
mount = pki_int
path_len = 1

27. TERMINAL
$ vault secrets enable -path=pki_int pki
Success! Enabled the pki secrets engine at: pki_int/
$ vault secrets tune -max-lease-ttl=43800h pki_int
Success! Tuned the secrets engine at: pki_int/
$ vault write -format=json
pki_int/intermediate/generate/internal
organization="Example Labs"
common_name="Example Labs Intermediate CA v1"
key_type=ec
key_bits=256
> pki_int.csr.json
$ cat pki_int.csr.json | jq -r '.data.csr'
> pki_int.csr
Intermediate
CA
Enable PKI secret engine
Tune 5 years for Max Lease TTL
Generate Private Key and CSR
(pki_int/intermediate/generate)

28. TERMINAL
$ certstrap --depot-path root
sign
--CA "Example Labs Root CA v1"
--passphrase "secret"
--intermediate
--csr pki_int.csr
--expires "5 years"
--path-length 1
--cert pki_int.crt
"Example Labs Intermediate CA v1"
Building intermediate
$ vault write -format=json
pki_int/intermediate/set-signed
certificate=@pki_int.crt
Intermediate
CA
Sign and Generate Certificate
using Root CA
Store Intermediate
Certificate to Vault
(pki_int/intermediate/set-signed)

29. Issuing CA
Issuing CA managed in Vault
mount = pki_iss
Steps
● Enable PKI secret engine using path=pki_iss
● Generate Private Key and CSR
● Sign and Generate Certificate using Intermediate CA
● Store Issuing + Intermediate Certificates to Vault
Root CA (offline)
path_len = 2
Issuing CA
mount = pki_iss
Intermediate CA
mount = pki_int
path_len = 1

30. TERMINAL
$ vault secrets enable -path=pki_iss pki
Success! Enabled the pki secrets engine at: pki_iss/
$ vault secrets tune -max-lease-ttl=8760h pki_iss
Success! Tuned the secrets engine at: pki_iss/
$ vault write -format=json
pki_iss/intermediate/generate/internal
organization="Example Labs"
common_name="Example Labs Issuing CA v1"
key_type=ec
key_bits=256
> pki_iss.csr.json
$ cat pki_iss.csr.json | jq -r '.data.csr'
> pki_iss.csr
Issuing CA
Enable PKI secret engine
Tune 1 year for Max Lease TTL
Generate Private Key and CSR
(pki_iss/intermediate/generate)

31. TERMINAL
$ vault write -format=json
pki_int/root/sign-intermediate
organization="Example Labs"
csr=@pki_iss.csr
ttl=8760h
format=pem
> pki_iss.crt.json
$ cat pki_iss.crt.json | jq -r '.data.certificate'
> pki_iss.crt
$ cat pki_iss.crt pki_int.crt > pki_iss.chain.crt
$ vault write -format=json
pki_iss/intermediate/set-signed
certificate=@pki_iss.chain.crt
Issuing CA
Sign and Generate Certificate
using Intermediate CA
(pki_int/root/sign-intermediate)
Store Issuing + Intermediate
Certificates to Vault
(pki_iss/intermediate/set-signed)

32. Issuing CA
A Role for issuing Leaf Certificates
name = example
allowed_domains = example.com
allowed_subdomains = true
allow_wildcard_certificates = false
Steps
● Create a Role for issuing Leaf Certificates
Root CA (offline)
path_len = 2
Issuing CA
mount = pki_iss
Role ‘example’
allowed_domains = example.com
allowed_subdomains = true
allowed_wildcard = false
Intermediate CA
mount = pki_int
path_len = 1

33. $ vault write pki_iss/roles/example
organization="Example Labs"
allowed_domains="example.com"
allow_subdomains=true
allow_wildcard_certificates=false
key_type=ec
key_bits=256
generate_lease=true
max_ttl=2160h
$ vault write -format=json
pki_iss/issue/example.com
common_name="test.example.com"
ttl=5m
TERMINAL
Issuing CA
Role for issuing Leaf Certificates
(pki_iss/roles)
name = example
allowed_domains = example.com
allowed_subdomains = true
allow_wildcard_certificates = false
generate_lease = true
max_ttl = 3 months

34. Demo Time

35. Issue and Deploy Leaf Certificates

36. – Consul Template (Demo)
– Vault Agent on VM
– Vault Agent on Kubernetes
– Nomad
– …
Deployment Solutions

37. ▪ Go Template Format and Functions
▪ Additional Functions
▪ Query Consul, Vault and Nomad
Consul Template Templating Language

38. Consul Template for Certificates
CODE EDITOR
{{- with secret "pki_iss/issue/example" "common_name=test.example.com" -}}
{{ .Data.private_key }}
{{- end -}}
{{- with secret "pki_iss/issue/example" "common_name=test.example.com" -}}
{{ .Data.certificate }}
{{- end -}}
{{- with secret "pki_iss/issue/example" "common_name=test.example.com" -}}
{{ .Data.ca_chain }}
{{- end -}}
{{- with secret "pki_iss/cert/ca_chain" -}}
{{ .Data.ca_chain }}
{{- end -}}

39. Vault Agent / Nomad

40. Vault Agent on Kubernetes

41. Demo Time

42. Renew Leaf Certificates

43. ▪ Leaf Certificates are Automatically Renewed
▪ Use exec block in configuration to execute a command
when the template is rendered and the output has changed
Automatic Renewal
CODE EDITOR
exec {
command = [ "systemctl", "reload”, "nginx" ]
timeout = “30s”
}

44. Demo Time

45. Rotate Issuing CA, Intermediate CA
and Root CA Seamlessly

46. ▪ Support of multiple CAs in the
same Secrets Engine mount
▪ For the purpose of CA Rotation
▪ Secret Engine sets a default issuer
▪ Role sets an issuer_ref issuer
Multi-Issuer Capabilities
Vault 1.11.0 or higher
CA
default issuer = v1
v1 Role X
issuer_ref = default
Leaf
Certificate
v1

47. ▪ Create a v2 Issuer
▪ Leaf Certificates
can be issued with
– a Role X using default Issuer
– a Role Y using v1 Issuer
– a Role Z using v2 Issuer
▪ Default Issuer can be set to v2
CA Transition
▪ Create a v2 Issuer
▪ Create a v2 Issuer
▪ Leaf Certificates
can be issued with
– a Role X using default Issuer
– a Role Y using v1 Issuer
– a Role Z using v2 Issuer
CA
default issuer = v2
v1 v2 Role X
issuer_ref = default
Leaf
Certificate
v2
Role Y
issuer_ref = v1
Role Z
issuer_ref = v2
Leaf
Certificate
v1
Leaf
Certificate
v2
CA
default issuer = v1
v1 v2 Role X
issuer_ref = default
Leaf
Certificate
v1
Leaf
Certificate
Role Y
issuer_ref = v1
Role Z
issuer_ref = v2
Leaf
Certificate
v1 v2

48. ▪ Create a v1.1.2 issuer
▪ Set default Issuer to v1.1.2
▪ Leaf Certificate issued by
Role example with Issuer v1.1.2 now
Rotate Issuing CA (Demo)
Steps Root CA (offline)
v1
Issuing CA
default issuer = v1.1.1
Intermediate CA
default issuer = v1.1
v1.1
Issuing CA
default issuer = v1.1.2
v1.1.1 v1.1.2
Leaf
Certificate
v1.1.1
Role ‘example’
issuer_ref = default
▪ Create a v1.1.2 issuer
▪ Create a v1.1.2 issuer
▪ Set default Issuer to v1.1.2
Leaf
Certificate
v1.1.2
v1.1.1 v1.1.2

49. Rotate Intermediate CA (Demo)
Steps Root CA (offline)
v1
Issuing CA
default issuer = v1.1.2
Intermediate CA
default issuer = v1.1
Issuing CA
default issuer = v1.2.1
v1.2.1
v1.2.1
Leaf
Certificate
v1.1.2
Role ‘example’
issuer_ref = default
Leaf
Certificate
v1.2.1
▪ Create a v1.2 issuer
▪ Create a v1.2.1 issuer (using v1.2)
▪ Set default Issuer to v1.2
Set default Issuer to v1.2.1
▪ Leaf Certificate issued by
Role example with Issuer v1.2.1 now
Intermediate CA
default issuer = v1.2
v1.2
v1.2
v1.1
v1.1
v1.1.2
v1.1.2
▪ Create a v1.2 issuer
▪ Create a v1.2.1 issuer (using v1.2)
▪ Set default Issuer to v1.2
Set default Issuer to v1.2.1
▪ Create a v1.2 issuer
▪ Create a v1.2.1 issuer (using v1.2)
▪ Create a v1.2 issuer

50. ▪ Create a v2.1 issuer (using v2)
Rotate Root CA (Demo)
Steps
Root CA
v1
Issuing CA
default issuer = v1.2.1
Intermediate CA
default issuer = v1.2
Issuing CA
default issuer = v2.1.1
v2.1.1
v2.1.1
Leaf
Certificate
v1.2.1
Role ‘example’
issuer_ref = default
Leaf
Certificate
v2.1.1
▪ Create a v2.1 issuer (using v2)
▪ Create a v2.1.1 issuer (using v2.1)
▪ Set default Issuer to v2.1
Set default Issuer to v2.1.1
▪ Leaf Certificate issued by
Role example with Issuer v2.1.1 now
Intermediate CA
default issuer = v2.1
v2.1
v2.1
v1.2
v1.2
v1.2.1
v1.2.1
▪ Create a v2.1 issuer (using v2)
▪ Create a v2.1.1 issuer (using v2.1)
▪ Set default Issuer to v2.1
Set default Issuer to v2.1.1
▪ Create a v2.1 issuer (using v2)
▪ Create a v2.1.1 issuer (using v2.1)
Root CA
v2
▪ Create Root CA v2

51. Demo Time

52. Securely Store and Deploy
Certificates with Vault

53. ▪ ACME.sh - https://acme.sh
A pure Unix shell script implementing
ACME client protocol
▪ ZeroSSL
Free SSL Certificates and SSL Tools
▪ DNS Server
▪ Vault - KV Secret Engine
Automatic Certificate Management
Environment (ACME)
KV
mount=acme
DNS
Server
ACME.sh
Issue DNS
Challenge
Store
Verify

54. Why ZeroSSL over Let’s Encrypt?

55. Consul Template for Public Certificates
CODE EDITOR
{{- with secret "acme/example.com/cert.key" -}}
{{ .Data.value }}
{{- end -}}
{{- with secret "acme/example.com/cert.pem" -}}
{{ .Data.value }}
{{- end -}}
{{- with secret "acme/example.com/chain.pem" -}}
{{ .Data.value }}
{{- end -}}
{{- with secret "acme/example.com/fullchain.pem" -}}
{{ .Data.value }}
{{- end -}}

56. ▪ A pure Unix shell script implementing ACME client protocol
https://acme.sh
▪ Install the script and crontab to renew certificate automatically
curl https://get.acme.sh | sh -s email=me@example.com
▪ Enable KV secret engine on Vault
vault secrets enable -path=acme kv
Automatic Certificate Management
Environment (ACME)

57. Demo Time

58. Next Steps

59. ▪ Deploy Vault Reference Architecture
From POC → MVP → Production

60. ▪ Codify Vault configuration with Vault Terraform provider
– Create PKI, Enable Auth methods, Create Policies (Least Privileges)...
▪
▪
▪
▪
▪ Vault Agent, Kubernetes, Nomad Auth management
▪ Disable old CA certificates
From POC → MVP → Production

61. ▪ Why Automate your Certificates Lifecycle with Vault?
▪ Build an Internal PKI with Vault
▪ Issue, Deploy and Renew Leaf Certificates
▪ Rotate Issuing CA, Intermediate CA and Root CA Seamlessly
▪ Securely Store and Deploy Public Certificates using Vault
Takeaways

62. Q&A

64. Next Events

65. ● Luxembourg HUG Meetup - Next Quarter
● HashiConf Global - October 5 & 6 - Virtual & Los Angeles
https://hashiconf.com/global
● HashiTalks: Deploy - December 7 & 8 - Virtual
https://events.hashicorp.com/hashitalksdeploy
● HashiTalks - February 16 & 17, 2023 - Virtual
https://hashi.co/hashitalks-2023
Next Events

66. learn.hashicorp.com | discuss.hashicorp.com
Luxembourg
Youtube Channel
HashiCorp User Group
Luxembourg

67. Thank You
learn.hashicorp.com | discuss.hashicorp.com
Youtube Channel
HashiCorp User Group
Luxembourg