Creating secrets is easy, but effectively managing them is much more complex. Vault, which is a free and open source tool by HashiCorp, is a popular tool for managing organizational secrets.

1. TAMING THE MODERN DATA CENTER
A Hybrid Talk for a Hybrid World
@sethvargo

2. @sethvargo
Seth Vargo
Director of Technical Advocacy
HashiCorp

3. @sethvargo

4. @sethvargo
DC EVOLUTION
How did we get here?

5. @sethvargo
RISING DATACENTER COMPLEXITY
DC

6. @sethvargo
RISING DATACENTER COMPLEXITY
DC

7. @sethvargo
RISING DATACENTER COMPLEXITY
DC
VM
VMVM
VM VM
VMVM
VM VM
VMVM
VM VM
VMVM
VM

8. @sethvargo
RISING DATACENTER COMPLEXITY
DC
VM
VMVM
VM VM
VMVM
VM VM
VMVM
VM VM
VMVM
VM
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C

9. @sethvargo
RISING DATACENTER COMPLEXITY
DC DNS
Database
CDN

10. @sethvargo
RISING DATACENTER COMPLEXITY
DC-01 DC-02

11. @sethvargo
RISING DATACENTER COMPLEXITY
DC-01 DC-02
VM
VMVM
VM VM
VMVM
VM
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C
C C

12. @sethvargo
RISING DATACENTER COMPLEXITY
IaaS PaaS SaaS

13. @sethvargo
TAMING THE DC
Deployment + Maintenance

14. @sethvargo
PREVIOUSLY
The APUD cycle

15. ACQUIRE PROVISION UPDATE DESTROY

16. ACQUIRE PROVISION UPDATE DESTROY
G’
U
VENDOR

17. ACQUIRE PROVISION UPDATE DESTROY
G U
’
U
’
U
VENDOR DC OPS

18. ACQUIRE PROVISION UPDATE DESTROY
G U
’
U
’
U U
’
U
VENDOR DC OPS SYSADMIN

19. ACQUIRE PROVISION UPDATE DESTROY
G U
’
U
’
U U
’
U U
’
U
VENDOR DC OPS SYSADMIN DC OPS

20. ACQUIRE PROVISION UPDATE DESTROY
VENDOR DC OPS SYSADMIN DC OPS
WEEKS DAYS DAYS DAYS
c c c c

21. @sethvargo
PRESENTLY
The elastic compute and _aaS era

22. ACQUIRE PROVISION UPDATE DESTROY
WEEKS DAYS DAYS DAYS
c c c c
Elastic Compute

23. ACQUIRE PROVISION UPDATE DESTROY
WEEKS DAYS DAYS DAYS
c c c c
Elastic Compute

24. ACQUIRE PROVISION UPDATE DESTROY
MINUTES DAYS DAYS SECONDS
c c c c
Elastic Compute

25. CapEx
#
OpEx
#

26. _aaS

27. ACQUIRE PROVISION UPDATE DESTROY
DAYS DAYS
c c
Configuration Management
MINUTES SECONDS
c c

28. ACQUIRE PROVISION UPDATE DESTROY
DAYS DAYS
c c
Configuration Management
MINUTES SECONDS
c c

29. ACQUIRE PROVISION UPDATE DESTROY
MINUTES SECONDS
c c
Configuration Management
MINUTES SECONDS
c c

30. ACQUIRE PROVISION UPDATE DESTROY
SaaS Proliferation
ACQUIRE PROVISION UPDATE DESTROY
https://specialized.com

31. @sethvargo
RISING DATACENTER COMPLEXITY
DC DNS
Database
CDNVM
VMVM
VM
C C
C C
C C

34. @sethvargo
WHY?
What was our original goal?

35. @sethvargo
EFFECTIVELY DELIVER
AND MAINTAIN
APPLICATIONS

36. @sethvargo
MOVE FAST AND
DON’T BREAK THINGS

37. RUN
Applications, Services, Jobs
SECURE
Applications, Infrastructure
PROVISION
Infrastructure, Code, Images

38. RUN
Applications, Services, Jobs
SECURE
Applications, Infrastructure
PROVISION
Infrastructure, Code, Images

40. @sethvargo
MOTIVATION
Why Terraform?

41. @sethvargo
How do I provision resources?
compute?
storage?
network?

42. @sethvargo
How do I manage resource lifecycles?

43. @sethvargo
How do I balance service providers
providing core technology for my
datacenter?

44. @sethvargo
How do I enforce policy across all these
resources?

45. @sethvargo
How do I automate and share those
configurations?

46. @sethvargo
TERRAFORM'S GOAL

47. @sethvargo
PROVIDE A SINGLE WORKFLOW

48. @sethvargo
WITH A UNIFIED VIEW

49. @sethvargo
USING INFRASTRUCTURE AS CODE

50. @sethvargo
THAT CAN BE ITERATED AND
CHANGED SAFELY

51. @sethvargo
CAPABLE OF COMPLEX N-TIER
APPLICATIONS

52. @sethvargo
resource "digitalocean_droplet" "web" {
name = "tf-web"
size = "512mb"
image = "centos-5-8-x32"
region = "sfo1"
}
resource "dnsimple_record" "hello" {
domain = "example.com"
name = "test"
value = "${digitalocean_droplet.web.ipv4_address}"
type = "A"
}
main.tf

53. @sethvargo
resource "digitalocean_droplet" "web" {
name = "tf-web"
size = "512mb"
image = "centos-5-8-x32"
region = "sfo1"
}
resource "dnsimple_record" "hello" {
domain = "example.com"
name = "test"
value = "${digitalocean_droplet.web.ipv4_address}"
type = "A"
}
main.tf

54. @sethvargo
resource "digitalocean_droplet" "web" {
name = "tf-web"
size = "512mb"
image = "centos-5-8-x32"
region = "sfo1"
}
resource "dnsimple_record" "hello" {
domain = "example.com"
name = "test"
value = "${digitalocean_droplet.web.ipv4_address}"
type = "A"
}
main.tf

55. @sethvargo
resource "digitalocean_droplet" "web" {
name = "tf-web"
size = "512mb"
image = "centos-5-8-x32"
region = "sfo1"
}
resource "dnsimple_record" "hello" {
domain = "example.com"
name = "test"
value = "${digitalocean_droplet.web.ipv4_address}"
type = "A"
}
main.tf

56. @sethvargo
HUMAN-FRIENDLY CONFIGURATION*

57. @sethvargo
VCS-FRIENDLY FORMAT

58. @sethvargo
ENTIRE INFRASTRUCTURE...
CAPTURED TEXT FILES

59. @sethvargo
TERRAFORM PROVIDERS

60. @sethvargo
SINGLE INTEGRATION POINT

61. @sethvargo
EXPOSE ("PROVIDE") A RESOURCE

62. @sethvargo
CRUD API

63. @sethvargo
PLUGGABLE FOR INTEGRATIONS

64. @sethvargo
MANAGE ANYTHING WITH AN API

65. @sethvargo
$ terraform apply

66. @sethvargo
OVER 65 BUILT-IN PROVIDERS
AND COUNTING...

67. @sethvargo
TERRAFORM PLAN

68. @sethvargo
+ digitalocean_droplet.web
backups: "" => "<computed>"
image: "" => "centos-5-8-x32"
ipv4_address: "" => "<computed>"
ipv4_address_private: "" => "<computed>"
name: "" => "tf-web"
private_networking: "" => "<computed>"
region: "" => "sfo1"
size: "" => "512mb"
status: "" => "<computed>"
+ dnsimple_record.hello
domain: "" => "example.com"
Terminal

69. @sethvargo
+ digitalocean_droplet.web
backups: "" => "<computed>"
image: "" => "centos-5-8-x32"
ipv4_address: "" => "<computed>"
ipv4_address_private: "" => "<computed>"
name: "" => "tf-web"
private_networking: "" => "<computed>"
region: "" => "sfo1"
size: "" => "512mb"
status: "" => "<computed>"
+ dnsimple_record.hello
domain: "" => "example.com"
Terminal

70. @sethvargo
+ digitalocean_droplet.web
backups: "" => "<computed>"
image: "" => "centos-5-8-x32"
ipv4_address: "" => "<computed>"
ipv4_address_private: "" => "<computed>"
name: "" => "tf-web"
private_networking: "" => "<computed>"
region: "" => "sfo1"
size: "" => "512mb"
status: "" => "<computed>"
+ dnsimple_record.hello
domain: "" => "example.com"
Terminal

71. @sethvargo
+ digitalocean_droplet.web
backups: "" => "<computed>"
image: "" => "centos-5-8-x32"
ipv4_address: "" => "<computed>"
ipv4_address_private: "" => "<computed>"
name: "" => "tf-web"
private_networking: "" => "<computed>"
region: "" => "sfo1"
size: "" => "512mb"
status: "" => "<computed>"
+ dnsimple_record.hello
domain: "" => "example.com"
Terminal

72. @sethvargo
size: "" => "512mb"
status: "" => "<computed>"
+ dnsimple_record.hello
domain: "" => "example.com"
domain_id: "" => "<computed>"
hostname: "" => "<computed>"
name: "" => "test"
priority: "" => "<computed>"
ttl: "" => "<computed>"
type: "" => "A"
value: "" => "${digitalocean_droplet.web.ipv4_address}"
Terminal

73. @sethvargo
SHOWS YOU WHAT WILL HAPPEN

74. @sethvargo
EXPLAINS CERTAIN ACTIONS

75. @sethvargo
PREVIOUSLY?

76. @sethvargo
STILL UNCERTAINTY…

77. @sethvargo
FUTURE OPS
Managing Tomorrow's Infrastructure

78. @sethvargo
DEPLOY IMMUTABLE
INFRASTRUCTURE

79. @sethvargo
CHANGES
CONFIDENCE Mutable Infrastructure

80. @sethvargo
ITERATIONS
CONSISTENCY Mutable Infrastructure

81. @sethvargo
ITERATIONS
CONSISTENCY Immutable Infrastructure

82. @sethvargo
IMMUTABLE
INFRASTRUCTURE
IS FASTER

83. @sethvargo
IMMUTABLE
INFRASTRUCTURE
ALLOWS FOR
GREATER PARITY

84. @sethvargo
IMMUTABLE
INFRASTRUCTURE
NEEDS AUTOMATION

86. @sethvargo
MACHINE IMAGES

87. @sethvargo
YUCK... IMAGES?

88. @sethvargo
WHY HAVE WE BEEN
GENERALLY AGAINST
MACHINE IMAGES?

89. @sethvargo
GOLDEN IMAGES
USED TO BE THE WAY

90. @sethvargo
QUARTERLY,
UNCHANGED,
AND BLESSED
IMAGES

91. @sethvargo
CHANGES WERE
SLOW AND FRUSTRATING

92. @sethvargo
TOOLING WAS
NOT MATURE
COMPARED TO TODAY

93. @sethvargo
MODERN CONFIG MANAGEMENT
CHANGED THAT

94. @sethvargo
OPS WITHOUT
MACHINE IMAGES IS LIKE
APPLICATIONS WITHOUT BINARIES

95. @sethvargo
APPLICATION LIFECYCLE

96. @sethvargo
APPLICATION LIFECYCLE
Source Code Binary

97. @sethvargo
APPLICATION LIFECYCLE
Source Code Binary
libA 1.0 libB 1.0 libC 1.0

98. @sethvargo
APPLICATION LIFECYCLE
Source Code Binary
libA 1.0 libB 1.0 libC 1.0

99. @sethvargo
APPLICATION LIFECYCLE
Source Code Binary
libA 1.0 libB 1.0 libC 1.0

100. @sethvargo
MUTABLE SERVER LIFECYCLE

101. @sethvargo
APPLICATION LIFECYCLE
Base Server Ready Server

102. @sethvargo
APPLICATION LIFECYCLE
Base Server Ready Server
Packages Network CM

103. @sethvargo
APPLICATION LIFECYCLE
Base Server Ready Server
Packages Network CM

104. @sethvargo
APPLICATION LIFECYCLE
Base Server Ready Server
Packages Network CM

105. @sethvargo
APPLICATION LIFECYCLE
Base Server Ready Server
Packages Network CM

106. @sethvargo
APPLICATION LIFECYCLE
Base Server Ready Server
Packages Network CM

107. @sethvargo
APPLICATION LIFECYCLE
IN THE PATH OF
DOWNTIME

108. @sethvargo
MACHINE IMAGE LIFECYCLE

109. @sethvargo
MACHINE IMAGE LIFECYCLE
Base Server Ready Server

110. @sethvargo
MACHINE IMAGE LIFECYCLE
Base Server Ready Server

111. @sethvargo
PACKER BUILD

112. @sethvargo
EMBRACES CONFIG MANAGEMENT

113. @sethvargo
TRANSITIONS FAILURES FROM
RUNTIME TO BUILD-TIME

114. @sethvargo
ENFORCES PARITY WITH STAGING

115. @sethvargo
… AND EVEN DEVELOPMENT

116. @sethvargo
NEW CHALLENGES

117. @sethvargo
IT DIDN'T BELONG THERE
IN THE FIRST PLACE

118. @sethvargo
LIKE TRYING TO USE LS
TO CREATE A FILE

120. @sethvargo
Consul Features
Service Discovery Health Checking
KV Store Multi Datacenter

121. @sethvargo
Service Discovery

122. @sethvargo
Service Discovery
DNS interface is zero-touch - no application changes are required
HTTP API for modern applications returns rich metadata
Allows discovery of both internal and external services

123. @sethvargo
$ host web.service.consul
10.0.3.83
10.0.1.109
10.0.4.21
Terminal

124. @sethvargo
$ curl $CONSUL_ADDR/v1/health/services/web
[
{
# ...
}
]
Terminal

125. @sethvargo
Health Checking

126. @sethvargo
Health Checking
Integrates with the service discovery layer
DNS does not return results for unhealthy services or nodes
HTTP endpoints can list health and query by health

127. @sethvargo
KV Store

128. @sethvargo
KV Store
Highly available storage for configuration and feature flags
Feature flags without big CM processes
Supports blocking queries for "pushing" changes
Optional ACLs to protect sensitive information at paths

129. @sethvargo
$ consul kv put foo bar
Success! Data written to: foo
Terminal

130. @sethvargo
$ consul kv get foo
bar
Terminal

131. @sethvargo
Multi-Datacenter

132. @sethvargo
Multi-Datacenter
Usually query the local datacenter
Can query other datacenters however you may need to
Can view all datacenters within one OSS UI

133. @sethvargo
$ dig web-frontend.singapore.service.consul. +short
10.3.3.33
10.3.1.18
$ dig web-frontend.germany.service.consul. +short
10.7.3.41
10.7.1.76
Terminal

134. @sethvargo
$ curl http://localhost:8500/v1/kv/foo?raw&dc=asia
true
$ curl http://localhost:8500/v1/kv/foo?raw&dc=eu
false
Terminal

135. @sethvargo
... And More!

136. @sethvargo
Events, Exec, and Watches
Build powerful orchestration tools
Implement client-side leader election
Distributed locking and event system
All approaches proven to scale to thousands of agents

137. @sethvargo
$ consul event deploy 6DF7FE
# ...
$ consul watch -type event -name deploy /usr/bin/deploy.sh
# ...
$ consul exec -service web /usr/bin/deploy.sh
# ...
Terminal

138. @sethvargo
Security
Encrypt gossip traffic with shared key or keyring (UDP)
Encrypt HTTP traffic with TLS (TCP)
Advanced ACLs and token-based system allows for massive scale

139. @sethvargo

140. @sethvargo
Completely Open Source

141. @sethvargo
Completely "Dog Fooded"

142. @sethvargo
Seth Vargo
Director of Technical Advocacy
HashiCorp
Questions?