0
freistil.it
Dynamic
Infrastructure
Orchestration
Jochen Lillich
Jochen Lillich
@geewiz
Sysadmin since 1993
CEO of freistil IT
freistilbox: PaaS for Drupal & WordPress
Config Management
Elimination of manual changes
Consistent configuration
Single source of truth
DRY
Weaknesses
Periodic convergence runs
Central database
Orchestration
Perform changes quickly
Handle failure
Tolerate network partitions
serf
Basics
Cluster communication tool
Developed by Hashicorp
Simple binary,written in Go
Gossip protocol
serf node 1
$ serf agent -node=node1 -bind=srv1
==> Starting Serf agent...
==> Starting Serf agent RPC...
==> Serf agent r...
serf node 2
$ serf agent -node=node2 -bind=srv2
…
$ serf join srv1
Successfully joined cluster by contacting 1
nodes.
$ se...
Membership
Cluster management
Service configuration
Failure detection
Detects failure quickly
Notifies other nodes
Executes handler scripts
Events
member-join
member-leave
member-failed
member-update
member-reap
user
query
User events
Deploy application
Perform convergence run
Query information from the cluster
Example handler
#!/bin/bash
!
echo
echo "New event: ${SERF_EVENT}. Data
follows..."
while read line; do
printf "${line}n"
...
User event
$ serf event deploy-app
!
2014/04/09 17:06:32 [INFO] agent: Received
event: user-event: deploy-app
etcd
Basics
HA key-value store
Developed by CoreOS
Written in Go
Raft consensus algorithm
Pure HTTPAPI
Key-value store
/release = 9607bcfeb48905d26db8f
/cluster
/cluster/node1 = node1.example.com
/cluster/node2 = node2.exampl...
Quorum
Available
Quorum
Available
Quorum
Available
Quorum
Unavailable
Writing
$ etcdctl set /message Hello
Hello
$ curl -L -X PUT 

http://127.0.0.1:4001/v2/keys/message 

-d value=“Test"
{"ac...
Reading
$ etcdctl get /message
Hello
$ curl -L 

http://127.0.0.1:4001/v2/keys/message
{"action":"get","node":{"key":"/
me...
Deleting
$ etcdctl rm /message
$ curl -L -X DELETE 

http://127.0.0.1:4001/v2/keys/message
Waiting
$ curl -L http://127.0.0.1:4001/v2/keys/
release?wait=true








Waiting
$ curl -L http://127.0.0.1:4001/v2/keys/
release?wait=true
{"action":"set","node":{"key":"/
release","value":"9607...
Applications
Locking
Leader election
Database master
Elastic IP
Leader election
etcd
App
Leader election
etcd
App
DB?
Leader election
etcd
App
DB?
Nope.
Leader election
etcd
App
DB?
Nope.
I’ll wait.
Leader election
etcd
App
Leader election
etcd
App DB X
Leader election
etcd
App DB XMaster me!
Leader election
etcd
App DB XMaster me!
Ok!
Leader election
etcd
App DB X
DB X is master.
Leader election
etcd
App DB X
DB X is master.
Leader election
etcd
App DB X
DB X is master.
Leader election
etcd
App DB X
Leader election
etcd
App DB X
DBY
Leader election
etcd
App DB X
DBY
Master me!
Leader election
etcd
App DB X
DBY
Master me!
No,DB X is.
Leader election
etcd
App DB X
DBY
Leader election
etcd
App
DBY
Leader election
etcd
App
DBY
Leader election
etcd
App
DBY
Leader election
etcd
App
DBY
You da master!
Leader election
etcd
App
DBYDB Y is master now.
Leader election
etcd
App
DBY
Summary
Thanks!
@geewiz
jochen@freistil.it
Upcoming SlideShare
Loading in...5
×

OSDC 2014: Jochen Lillich - Dynamic infrastructure orchestration

424

Published on

Getting Configuration Management in place is a big step in the direction of infrastructure automation. Chef, Puppet and Co. replace error-prone manual changes with periodic system convergence runs controlled by a central database. Even with Puppet’s exported resources and Chef’s search capabilities, the weakness of this approach is that it is rather static. In situations where we need to propagate information quickly, handle failure detection, or tolerate network partitions, other tools might offer better solutions.

In this talk, I’m going to present some of these alternatives (e.g. serf, etcd) and how they can be used to allow for more dynamic configuration changes.

0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
424
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
22
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Transcript of "OSDC 2014: Jochen Lillich - Dynamic infrastructure orchestration"

  1. 1. freistil.it Dynamic Infrastructure Orchestration Jochen Lillich
  2. 2. Jochen Lillich @geewiz Sysadmin since 1993 CEO of freistil IT freistilbox: PaaS for Drupal & WordPress
  3. 3. Config Management Elimination of manual changes Consistent configuration Single source of truth DRY
  4. 4. Weaknesses Periodic convergence runs Central database
  5. 5. Orchestration Perform changes quickly Handle failure Tolerate network partitions
  6. 6. serf
  7. 7. Basics Cluster communication tool Developed by Hashicorp Simple binary,written in Go Gossip protocol
  8. 8. serf node 1 $ serf agent -node=node1 -bind=srv1 ==> Starting Serf agent... ==> Starting Serf agent RPC... ==> Serf agent running! Node name: 'srv1' Bind addr: '192.168.0.11:7946' … ==> Log data will now stream in as it occurs: 2014/04/09 16:05:51 [INFO] agent: Serf agent starting 2014/04/09 16:05:51 [INFO] serf: EventMemberJoin: node1 192.168.0.11 2014/04/09 16:05:52 [INFO] agent: Received event: member- join
  9. 9. serf node 2 $ serf agent -node=node2 -bind=srv2 … $ serf join srv1 Successfully joined cluster by contacting 1 nodes. $ serf members node2 192.168.0.12:7946 alive node1 192.168.0.11:7946 alive
  10. 10. Membership Cluster management Service configuration
  11. 11. Failure detection Detects failure quickly Notifies other nodes Executes handler scripts
  12. 12. Events member-join member-leave member-failed member-update member-reap user query
  13. 13. User events Deploy application Perform convergence run Query information from the cluster
  14. 14. Example handler #!/bin/bash ! echo echo "New event: ${SERF_EVENT}. Data follows..." while read line; do printf "${line}n" done
  15. 15. User event $ serf event deploy-app ! 2014/04/09 17:06:32 [INFO] agent: Received event: user-event: deploy-app
  16. 16. etcd
  17. 17. Basics HA key-value store Developed by CoreOS Written in Go Raft consensus algorithm Pure HTTPAPI
  18. 18. Key-value store /release = 9607bcfeb48905d26db8f /cluster /cluster/node1 = node1.example.com /cluster/node2 = node2.example.com
  19. 19. Quorum Available
  20. 20. Quorum Available
  21. 21. Quorum Available
  22. 22. Quorum Unavailable
  23. 23. Writing $ etcdctl set /message Hello Hello $ curl -L -X PUT 
 http://127.0.0.1:4001/v2/keys/message 
 -d value=“Test" {"action":"set","node":{"key":"/ message","value":"Test","modifiedIndex": 15,"createdIndex":15},"prevNode":{"key":"/ message","value":"Hello","modifiedIndex": 2,"createdIndex":2}}
  24. 24. Reading $ etcdctl get /message Hello $ curl -L 
 http://127.0.0.1:4001/v2/keys/message {"action":"get","node":{"key":"/ message","value":"Test","modifiedIndex": 15,"createdIndex":15}
  25. 25. Deleting $ etcdctl rm /message $ curl -L -X DELETE 
 http://127.0.0.1:4001/v2/keys/message
  26. 26. Waiting $ curl -L http://127.0.0.1:4001/v2/keys/ release?wait=true 
 
 
 

  27. 27. Waiting $ curl -L http://127.0.0.1:4001/v2/keys/ release?wait=true {"action":"set","node":{"key":"/ release","value":"9607bcfe","modifiedIndex": 16,"createdIndex":16},"prevNode":{"key":"/ release","value":"18512199","modifiedIndex": 14,"createdIndex":14}}
  28. 28. Applications Locking Leader election Database master Elastic IP
  29. 29. Leader election etcd App
  30. 30. Leader election etcd App DB?
  31. 31. Leader election etcd App DB? Nope.
  32. 32. Leader election etcd App DB? Nope. I’ll wait.
  33. 33. Leader election etcd App
  34. 34. Leader election etcd App DB X
  35. 35. Leader election etcd App DB XMaster me!
  36. 36. Leader election etcd App DB XMaster me! Ok!
  37. 37. Leader election etcd App DB X DB X is master.
  38. 38. Leader election etcd App DB X DB X is master.
  39. 39. Leader election etcd App DB X DB X is master.
  40. 40. Leader election etcd App DB X
  41. 41. Leader election etcd App DB X DBY
  42. 42. Leader election etcd App DB X DBY Master me!
  43. 43. Leader election etcd App DB X DBY Master me! No,DB X is.
  44. 44. Leader election etcd App DB X DBY
  45. 45. Leader election etcd App DBY
  46. 46. Leader election etcd App DBY
  47. 47. Leader election etcd App DBY
  48. 48. Leader election etcd App DBY You da master!
  49. 49. Leader election etcd App DBYDB Y is master now.
  50. 50. Leader election etcd App DBY
  51. 51. Summary
  52. 52. Thanks! @geewiz jochen@freistil.it
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×