OSDC 2014: Jochen Lillich - Dynamic infrastructure orchestration

  • 338 views
Uploaded on

Getting Configuration Management in place is a big step in the direction of infrastructure automation. Chef, Puppet and Co. replace error-prone manual changes with periodic system convergence runs …

Getting Configuration Management in place is a big step in the direction of infrastructure automation. Chef, Puppet and Co. replace error-prone manual changes with periodic system convergence runs controlled by a central database. Even with Puppet’s exported resources and Chef’s search capabilities, the weakness of this approach is that it is rather static. In situations where we need to propagate information quickly, handle failure detection, or tolerate network partitions, other tools might offer better solutions.

In this talk, I’m going to present some of these alternatives (e.g. serf, etcd) and how they can be used to allow for more dynamic configuration changes.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
338
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
17
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. freistil.it Dynamic Infrastructure Orchestration Jochen Lillich
  • 2. Jochen Lillich @geewiz Sysadmin since 1993 CEO of freistil IT freistilbox: PaaS for Drupal & WordPress
  • 3. Config Management Elimination of manual changes Consistent configuration Single source of truth DRY
  • 4. Weaknesses Periodic convergence runs Central database
  • 5. Orchestration Perform changes quickly Handle failure Tolerate network partitions
  • 6. serf
  • 7. Basics Cluster communication tool Developed by Hashicorp Simple binary,written in Go Gossip protocol
  • 8. serf node 1 $ serf agent -node=node1 -bind=srv1 ==> Starting Serf agent... ==> Starting Serf agent RPC... ==> Serf agent running! Node name: 'srv1' Bind addr: '192.168.0.11:7946' … ==> Log data will now stream in as it occurs: 2014/04/09 16:05:51 [INFO] agent: Serf agent starting 2014/04/09 16:05:51 [INFO] serf: EventMemberJoin: node1 192.168.0.11 2014/04/09 16:05:52 [INFO] agent: Received event: member- join
  • 9. serf node 2 $ serf agent -node=node2 -bind=srv2 … $ serf join srv1 Successfully joined cluster by contacting 1 nodes. $ serf members node2 192.168.0.12:7946 alive node1 192.168.0.11:7946 alive
  • 10. Membership Cluster management Service configuration
  • 11. Failure detection Detects failure quickly Notifies other nodes Executes handler scripts
  • 12. Events member-join member-leave member-failed member-update member-reap user query
  • 13. User events Deploy application Perform convergence run Query information from the cluster
  • 14. Example handler #!/bin/bash ! echo echo "New event: ${SERF_EVENT}. Data follows..." while read line; do printf "${line}n" done
  • 15. User event $ serf event deploy-app ! 2014/04/09 17:06:32 [INFO] agent: Received event: user-event: deploy-app
  • 16. etcd
  • 17. Basics HA key-value store Developed by CoreOS Written in Go Raft consensus algorithm Pure HTTPAPI
  • 18. Key-value store /release = 9607bcfeb48905d26db8f /cluster /cluster/node1 = node1.example.com /cluster/node2 = node2.example.com
  • 19. Quorum Available
  • 20. Quorum Available
  • 21. Quorum Available
  • 22. Quorum Unavailable
  • 23. Writing $ etcdctl set /message Hello Hello $ curl -L -X PUT 
 http://127.0.0.1:4001/v2/keys/message 
 -d value=“Test" {"action":"set","node":{"key":"/ message","value":"Test","modifiedIndex": 15,"createdIndex":15},"prevNode":{"key":"/ message","value":"Hello","modifiedIndex": 2,"createdIndex":2}}
  • 24. Reading $ etcdctl get /message Hello $ curl -L 
 http://127.0.0.1:4001/v2/keys/message {"action":"get","node":{"key":"/ message","value":"Test","modifiedIndex": 15,"createdIndex":15}
  • 25. Deleting $ etcdctl rm /message $ curl -L -X DELETE 
 http://127.0.0.1:4001/v2/keys/message
  • 26. Waiting $ curl -L http://127.0.0.1:4001/v2/keys/ release?wait=true 
 
 
 

  • 27. Waiting $ curl -L http://127.0.0.1:4001/v2/keys/ release?wait=true {"action":"set","node":{"key":"/ release","value":"9607bcfe","modifiedIndex": 16,"createdIndex":16},"prevNode":{"key":"/ release","value":"18512199","modifiedIndex": 14,"createdIndex":14}}
  • 28. Applications Locking Leader election Database master Elastic IP
  • 29. Leader election etcd App
  • 30. Leader election etcd App DB?
  • 31. Leader election etcd App DB? Nope.
  • 32. Leader election etcd App DB? Nope. I’ll wait.
  • 33. Leader election etcd App
  • 34. Leader election etcd App DB X
  • 35. Leader election etcd App DB XMaster me!
  • 36. Leader election etcd App DB XMaster me! Ok!
  • 37. Leader election etcd App DB X DB X is master.
  • 38. Leader election etcd App DB X DB X is master.
  • 39. Leader election etcd App DB X DB X is master.
  • 40. Leader election etcd App DB X
  • 41. Leader election etcd App DB X DBY
  • 42. Leader election etcd App DB X DBY Master me!
  • 43. Leader election etcd App DB X DBY Master me! No,DB X is.
  • 44. Leader election etcd App DB X DBY
  • 45. Leader election etcd App DBY
  • 46. Leader election etcd App DBY
  • 47. Leader election etcd App DBY
  • 48. Leader election etcd App DBY You da master!
  • 49. Leader election etcd App DBYDB Y is master now.
  • 50. Leader election etcd App DBY
  • 51. Summary
  • 52. Thanks! @geewiz jochen@freistil.it