Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Beyond the Node Advanced 'Arkestration' with Noah PuppetConf 2011
.plan <ul><li>What's the problem?
Orchestration
Overview of Noah
Use Cases
Q&A </li></ul>
 
A Brief History of Time (with apologies to Hawking)
Configuration Management <ul><li>Manual </li></ul>
Configuration Management
Configuration Management
Configuration Management <ul><li>Manual
Scripted </li></ul>
Configuration Management
Configuration Management <ul><li>Manual
Scripted
Intelligent </li></ul>
CFEngine
Chef
Puppet
Configuration Management <ul><li>Manual
Scripted
Intelligent </li></ul>
So we've solved this, right?
“ I think..and I don't know if anyone would agree, that configuration management is a solved problem at this point, right?”
“ I think..and I don't know if anyone would agree, that configuration management is a solved problem at this point, right?...
&quot;The point I want to make..Configuration management is  not  a solved problem...and it's dangerous to make the mistak...
“ what I was attempting to say ... is that the current crop of configuration management tools have reached a usable point ...
“ what I was attempting to say (epic fail, I might add) is that the current crop of configuration management tools have re...
Orchestration and Noah Noah attempts to do two things: <ul><li>Bridge an impedance mismatch  between operational, developm...
Provide mechanisms for coordination  between applications, nodes, services, configuration management and other infrastruct...
Example Use Case 1) Capacity Reached. Tell Noah 2) Noah tells provisioning system 3) Capacity allocated. Tell Noah 4) Noah...
“ Inspired” by
&quot;ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchron...
Features <ul><li>Hierarchical name space of data registers  similar to a Unix filesystem ( znodes )
Leader election, locking, configuration, sequences
Distributed and highly available
Watches </li></ul>Downsides <ul><li>Persistent connections required
Official bindings are C and Java
Somewhat &quot;complex&quot; to setup and run </li></ul>
Noah and ZooKeeper Noah is essentially a re-imagining of ZK, taking the things I liked: <ul><li>Znode concept
Watches </li></ul>and making it work over  HTTP  using  JSON
Noah Basics
The Primitives <ul><li>Hosts
Has status  ( up ,  down ,  pending )
Can have   Services
Services </li></ul><ul>Has status  ( up ,  down ,  pending ) <li>Must belong to a   Host </li></ul><ul><li>Applications
Can have   Configurations
Configurations
Can belong to many   Applications </li></ul>
The Not So Primitives <ul><li>Ephemerals
Identified by a  “path”
Tags
Can be applied to any  Primitive  or  Ephemeral
Links </li></ul><ul>Can be applied to any  Primitive, Ephemeral  or  Tag </ul><ul><li>Watches
Can be applied at any hierarchy or specific object </li></ul>
URL Patterns <ul><li>Hosts </li><ul><li>/ hosts / app01 / services / httpd </li></ul><li>Services </li><ul><li>/ services ...
GET ting Data
{ &quot;foo&quot;: { &quot;created_at&quot; : &quot;2011-04-20 04:50:41 UTC&quot;, &quot;updated_at&quot; : &quot;2011-04-...
GET ting Data <ul><li>Hosts </li></ul>
{ &quot;foo&quot; : { &quot;created_at&quot;: &quot;2011-04-20 04:50:41 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-2...
{ &quot;foo&quot;: { &quot;created_at&quot;: &quot;2011-04-20 04:50:41 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-20...
GET ting Data <ul><li>Hosts
Services </li></ul>
{ &quot;foo_svc&quot; : { &quot;foo&quot; : { &quot;created_at&quot;: &quot;2011-04-20 05:09:44 UTC&quot;, &quot;updated_a...
{ &quot;foo_svc&quot; : { &quot;foo&quot; : { &quot;created_at&quot;: &quot;2011-04-20 05:09:44 UTC&quot;, &quot;updated_a...
GET ting Data <ul><li>Hosts
Services
Applications </li></ul>
{ &quot;myrailsapp1&quot; : { &quot;created_at&quot;: &quot;2011-04-21 06:57:42 UTC&quot;, &quot;updated_at&quot;: &quot;2...
{ &quot;myrailsapp1&quot;: { &quot;created_at&quot;: &quot;2011-04-21 06:57:42 UTC&quot;, &quot;updated_at&quot;: &quot;20...
GET ting Data <ul><li>Hosts
Services
Applications
Configurations  * </li></ul>
{ &quot;barconf1&quot; : { &quot;created_at&quot;: &quot;2011-04-27 01:20:37 UTC&quot;, &quot;format&quot;: &quot;string&q...
GET ting Data <ul><li>Hosts
Services
Applications
Configurations *
Upcoming SlideShare
Loading in …5
×

Beyond the Node: Arkestration with Noah

6,247 views

Published on

Slides from my presentation at PuppetConf 2011.

Published in: Technology
  • Be the first to comment

Beyond the Node: Arkestration with Noah

  1. 1. Beyond the Node Advanced 'Arkestration' with Noah PuppetConf 2011
  2. 2. .plan <ul><li>What's the problem?
  3. 3. Orchestration
  4. 4. Overview of Noah
  5. 5. Use Cases
  6. 6. Q&A </li></ul>
  7. 8. A Brief History of Time (with apologies to Hawking)
  8. 9. Configuration Management <ul><li>Manual </li></ul>
  9. 10. Configuration Management
  10. 11. Configuration Management
  11. 12. Configuration Management <ul><li>Manual
  12. 13. Scripted </li></ul>
  13. 14. Configuration Management
  14. 15. Configuration Management <ul><li>Manual
  15. 16. Scripted
  16. 17. Intelligent </li></ul>
  17. 18. CFEngine
  18. 19. Chef
  19. 20. Puppet
  20. 21. Configuration Management <ul><li>Manual
  21. 22. Scripted
  22. 23. Intelligent </li></ul>
  23. 24. So we've solved this, right?
  24. 25. “ I think..and I don't know if anyone would agree, that configuration management is a solved problem at this point, right?”
  25. 26. “ I think..and I don't know if anyone would agree, that configuration management is a solved problem at this point, right?” WTF?
  26. 27. &quot;The point I want to make..Configuration management is not a solved problem...and it's dangerous to make the mistake to think that the way we do things now is the best way to do them...&quot; - Andrew Clay Shafer
  27. 28. “ what I was attempting to say ... is that the current crop of configuration management tools have reached a usable point where they do enough (for now). What we’re seeing as questions now are 'How do I think beyond the single node where this tool is running?'”
  28. 29. “ what I was attempting to say (epic fail, I might add) is that the current crop of configuration management tools have reached a usable point where they do enough (for now). What we’re seeing as questions now are ' How do I think beyond the single node where this tool is running? '”
  29. 30. Orchestration and Noah Noah attempts to do two things: <ul><li>Bridge an impedance mismatch between operational, developmental and application configuration
  30. 31. Provide mechanisms for coordination between applications, nodes, services, configuration management and other infrastructure aspects </li></ul>
  31. 32. Example Use Case 1) Capacity Reached. Tell Noah 2) Noah tells provisioning system 3) Capacity allocated. Tell Noah 4) Noah triggers Puppet on LB 5) Noah triggers Puppet on Nagios Noah Nagios Puppet (app) Load Balancer
  32. 33. “ Inspired” by
  33. 34. &quot;ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications.&quot;
  34. 35. Features <ul><li>Hierarchical name space of data registers similar to a Unix filesystem ( znodes )
  35. 36. Leader election, locking, configuration, sequences
  36. 37. Distributed and highly available
  37. 38. Watches </li></ul>Downsides <ul><li>Persistent connections required
  38. 39. Official bindings are C and Java
  39. 40. Somewhat &quot;complex&quot; to setup and run </li></ul>
  40. 41. Noah and ZooKeeper Noah is essentially a re-imagining of ZK, taking the things I liked: <ul><li>Znode concept
  41. 42. Watches </li></ul>and making it work over HTTP using JSON
  42. 43. Noah Basics
  43. 44. The Primitives <ul><li>Hosts
  44. 45. Has status ( up , down , pending )
  45. 46. Can have Services
  46. 47. Services </li></ul><ul>Has status ( up , down , pending ) <li>Must belong to a Host </li></ul><ul><li>Applications
  47. 48. Can have Configurations
  48. 49. Configurations
  49. 50. Can belong to many Applications </li></ul>
  50. 51. The Not So Primitives <ul><li>Ephemerals
  51. 52. Identified by a “path”
  52. 53. Tags
  53. 54. Can be applied to any Primitive or Ephemeral
  54. 55. Links </li></ul><ul>Can be applied to any Primitive, Ephemeral or Tag </ul><ul><li>Watches
  55. 56. Can be applied at any hierarchy or specific object </li></ul>
  56. 57. URL Patterns <ul><li>Hosts </li><ul><li>/ hosts / app01 / services / httpd </li></ul><li>Services </li><ul><li>/ services / httpd / app02 </li></ul><li>Applications </li><ul><li>/ applications / my_app / configurations / log_level </li></ul><li>Configurations </li><ul><li>/ configurations / log_level </li></ul></ul>
  57. 58. GET ting Data
  58. 59. { &quot;foo&quot;: { &quot;created_at&quot; : &quot;2011-04-20 04:50:41 UTC&quot;, &quot;updated_at&quot; : &quot;2011-04-20 04:50:41 UTC&quot;, &quot;tags&quot; : [ ], &quot;id&quot; : &quot;4af53f8d-36d2-2158-c4a1-514ff3578048&quot;, &quot;links&quot; : [ ], &quot;services&quot;: { }, &quot;status&quot;: &quot;up&quot; }, &quot;bar&quot;: { } }
  59. 60. GET ting Data <ul><li>Hosts </li></ul>
  60. 61. { &quot;foo&quot; : { &quot;created_at&quot;: &quot;2011-04-20 04:50:41 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-20 04:50:41 UTC&quot;, &quot;tags&quot;: [ ], &quot;id&quot;: &quot;4af53f8d-36d2-2158-c4a1-514ff3578048&quot;, &quot;links&quot;: [ ], &quot;services&quot;: { “ mysql”:”pending”, “ httpd”:”up” }, &quot;status&quot;: &quot;up&quot; }, &quot;bar&quot;: { } } Host Name
  61. 62. { &quot;foo&quot;: { &quot;created_at&quot;: &quot;2011-04-20 04:50:41 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-20 04:50:41 UTC&quot;, &quot;tags&quot;: [ ], &quot;id&quot;: &quot;4af53f8d-36d2-2158-c4a1-514ff3578048&quot;, &quot;links&quot;: [ ], &quot;services&quot; : { “ mysql”:”pending”, “ httpd”:”up” }, &quot;status&quot;: &quot;up&quot; }, &quot;bar&quot;: { } } Services
  62. 63. GET ting Data <ul><li>Hosts
  63. 64. Services </li></ul>
  64. 65. { &quot;foo_svc&quot; : { &quot;foo&quot; : { &quot;created_at&quot;: &quot;2011-04-20 05:09:44 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-20 05:09:44 UTC&quot;, &quot;tags&quot;: [ ], &quot;id&quot;: &quot;53504526-a5d7-a604-5fe8-1f8fd2e1aef0&quot;, &quot;links&quot;: [ ], &quot;status&quot;: &quot;down&quot; } }, &quot;both&quot;: { } } Service name
  65. 66. { &quot;foo_svc&quot; : { &quot;foo&quot; : { &quot;created_at&quot;: &quot;2011-04-20 05:09:44 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-20 05:09:44 UTC&quot;, &quot;tags&quot;: [ ], &quot;id&quot;: &quot;53504526-a5d7-a604-5fe8-1f8fd2e1aef0&quot;, &quot;links&quot;: [ ], &quot;status&quot;: &quot;down&quot; } }, &quot;both&quot;: { } } Host Name
  66. 67. GET ting Data <ul><li>Hosts
  67. 68. Services
  68. 69. Applications </li></ul>
  69. 70. { &quot;myrailsapp1&quot; : { &quot;created_at&quot;: &quot;2011-04-21 06:57:42 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-21 06:57:42 UTC&quot;, &quot;tags&quot;: [ &quot;sample_data&quot;, &quot;production&quot; ], &quot;id&quot;: &quot;8774fead-e63a-8e5c-0453-1b5330bbd04c&quot;, &quot;links&quot;: [ &quot;/my_sample_organization&quot; ], &quot;configurations&quot;: { &quot;database.yml&quot;: { &quot;format&quot;: &quot;yaml&quot;, &quot;body&quot;: &quot;development:n database: development_databasen adapter: mysqln username: dev_usern password: dev_passwordn&quot; } } }, &quot;myrestapp1&quot;: { } } App Name
  70. 71. { &quot;myrailsapp1&quot;: { &quot;created_at&quot;: &quot;2011-04-21 06:57:42 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-21 06:57:42 UTC&quot;, &quot;tags&quot;: [ &quot;sample_data&quot;, &quot;production&quot; ], &quot;id&quot;: &quot;8774fead-e63a-8e5c-0453-1b5330bbd04c&quot;, &quot;links&quot;: [ &quot;/my_sample_organization&quot; ], &quot;configurations&quot; : { &quot;database.yml&quot;: { &quot;format&quot;: &quot;yaml&quot;, &quot;body&quot;: &quot;development:n database: development_databasen adapter: mysqln username: dev_usern password: dev_passwordn&quot; } } }, &quot;myrestapp1&quot;: { } } Configs
  71. 72. GET ting Data <ul><li>Hosts
  72. 73. Services
  73. 74. Applications
  74. 75. Configurations * </li></ul>
  75. 76. { &quot;barconf1&quot; : { &quot;created_at&quot;: &quot;2011-04-27 01:20:37 UTC&quot;, &quot;format&quot;: &quot;string&quot;, &quot;body&quot;: &quot;barbody1&quot;, &quot;updated_at&quot;: &quot;2011-04-27 01:20:37 UTC&quot;, &quot;tags&quot;: [ ], &quot;id&quot;: &quot;aa183dce-30ec-2d7e-c6f9-f3385161a7e6&quot;, &quot;links&quot;: [ ] }, &quot;barconf2&quot;: { } }
  76. 77. GET ting Data <ul><li>Hosts
  77. 78. Services
  78. 79. Applications
  79. 80. Configurations *
  80. 81. Ephemerals * </li></ul>
  81. 82. GET ting Data <ul><li>Hosts
  82. 83. Services
  83. 84. Applications
  84. 85. Configurations *
  85. 86. Ephemerals *
  86. 87. /ephemerals/my/config/setting returns the data stored there. </li></ul>
  87. 88. PUT ting Data <ul><li>/ hosts /host_name
  88. 89. / services /service_name/host_name
  89. 90. / applications /application_name
  90. 91. / configurations /configuration_name
  91. 92. / ephemerals /arbitrary/path/to/whatever </li></ul>
  92. 93. PUT ting Data <ul><li>/ hosts / host_name
  93. 94. / services / service_name / host_name
  94. 95. / applications / application_name
  95. 96. / configurations / configuration_name
  96. 97. / ephemerals / arbitrary / path / to / whatever </li></ul>
  97. 98. PUT ting Data <ul><li>Host {“status”:”up”}
  98. 99. Service {“status”:”up”,”host_status”:”up”}
  99. 100. Application {“name”:”app_name”}
  100. 101. Configuration {“format”:”format_type”,”body”:”data”}
  101. 102. Ephemeral raw data </li></ul>
  102. 103. DELETE ing Data <ul><li>/hosts/ host_name
  103. 104. /services/ service_name / host_name
  104. 105. /applications/ application_name
  105. 106. /configurations/ configuration_name
  106. 107. /ephemerals/ arbitrary / path / to / whatever </li></ul>
  107. 108. Tagging <ul><li>Append “tag” to any PUT url with PUT method
  108. 109. e.g. ' /applications/some_application/tag'
  109. 110. {“tags”:”sometag”} or {“tags”:[“tag1”,”tag2”]}
  110. 111. Call DELETE instead of PUT to remove tags
  111. 112. GET ' /tags ' for all tags or ' /tag/tagname ' for specific tag </li></ul>
  112. 113. Linking <ul><li>Append “link” to any PUT url with PUT method
  113. 114. e.g. ' /applications/some_application/link'
  114. 115. {“link_name”:”my_organization”}
  115. 116. Call DELETE instead of PUT to remove links *
  116. 117. GET ' /link_name ' for all linked objects *
  117. 118. Designed to allow custom namespaces </li></ul>
  118. 119. Watches
  119. 120. What are Watches? <ul><li>A bit different than ZK watches
  120. 121. Async
  121. 122. Persistent *
  122. 123. Pluggable (URI pattern based)
  123. 124. Hierarchical (superset and subset)
  124. 125. Append ' /watch ' to almost any url path
  125. 126. {“endpoint”:”http://some_host:port/ping_me”}
  126. 127. ' /watch ' root endpoint as well
  127. 128. {
  128. 129. “ endpoint”:”http://some_host/ping”,
  129. 130. ” pattern”:”//noah/applications/app_name”
  130. 131. } </li></ul>
  131. 132. Create a Watch $ curl -XPUT -d '{“pattern”:”//noah/applications”,&quot;endpoint&quot;:&quot; http://host/ping &quot;}' http://noahserver/watches
  132. 133. Perform some operation $ curl -XPUT -d '{&quot;name&quot;:&quot;fooapp2&quot;}' http://noahserver/applications/fooapp2
  133. 134. Message sent to Endpoint { &quot;id&quot;:&quot;be9f6a45-0d49-3f00-59a8-c6ad61da24e1&quot;, &quot;tags&quot;:[], &quot;links&quot;:[], &quot;name&quot;:&quot;fooapp2&quot; , &quot;created_at&quot;:&quot;2011-09-15 03:07:39 UTC&quot;, &quot;updated_at&quot;:&quot;2011-09-15 03:07:39 UTC&quot;, &quot;configurations&quot;:{}, &quot;action&quot;:&quot;create&quot; , &quot;pubcategory&quot;:&quot;//noah/applications/fooapp2&quot; }
  134. 135. Perform another operation $ curl -XPUT -d '{&quot;tags&quot;:&quot;footag2&quot;}' http://noahserver/applications/fooapp2/tag
  135. 136. Message sent to Endpoint { &quot;id&quot;:&quot;be9f6a45-0d49-3f00-59a8-c6ad61da24e1&quot;, &quot;tags&quot;:[&quot;footag&quot;] , &quot;links&quot;:[], &quot;name&quot;:&quot;fooapp2&quot; , &quot;created_at&quot;:&quot;2011-09-15 03:07:39 UTC&quot;, &quot;updated_at&quot;:&quot;2011-09-15 03:15:03 UTC&quot;, &quot;configurations&quot;:{}, &quot;action&quot;:&quot;update&quot; , &quot;pubcategory&quot;:&quot;//noah/applications/fooapp2&quot; }
  136. 137. Watches are “plugin” based You can write your own plugins with Ruby. <ul><li>URI Pattern *
  137. 138. amqp://user:pass@host:port/exchange
  138. 139. rundeck: //token@host :port/job_id
  139. 140. puppet://hostname/
  140. 141. statsd://host:port/metric.path.name.action
  141. 142. Currently based on Event Machine
  142. 143. Future based on Actors and worker pools </li></ul>
  143. 144. Watch Notes <ul><li>Creator of the watch is unrelated to receiver
  144. 145. Don't have to use the “official” watch daemon
  145. 146. You can watch the watches (PubSub) </li></ul>
  146. 147. Basic Puppet Integration <ul><li>http://github.com/jamtur01/puppet-noah
  147. 148. ENC, Functions and Facts
  148. 149. Hosts -> Nodes
  149. 150. Host's Services -> Classes
  150. 151. Configurations -> Facts (string types only) </li></ul>
  151. 152. So where does it fit? Noah Puppet Apps Scripts Monitoring Deploy CI
  152. 153. Noah + Puppet + Applications Noah Puppet Applications
  153. 154. Common P+A Use Case <ul><li>Puppet knows Nodes and Classes
  154. 155. App has configuration file
  155. 156. Configuration file controlled by Puppet
  156. 157. Puppet client run required to update file </li></ul>
  157. 158. Common P+A Issues <ul><li>Puppet repo is not app repo
  158. 159. Devs must learn ERB
  159. 160. Devs might even have to learn Puppet
  160. 161. Introducing new settings and communication
  161. 162. Daemon-mode is exception rather than rule </li></ul>
  162. 163. N+P+A Version <ul><li>Noah knows Nodes and Classes
  163. 164. Move volatile settings into Noah
  164. 165. Configuration file controlled by Puppet
  165. 166. App gets volatile settings from Noah *
  166. 167. Future Puppet runs update Noah
  167. 168. Noah tells App *
  168. 169. Truth vs. Reality
  169. 170. My DB master is foo
  170. 171. My DB master right NOW is bar </li></ul>
  171. 172. Volatile Settings? <ul><li>Not all configuration is created equal
  172. 173. “ Elasticity ” - Pool of worker node EC2 ips
  173. 174. Ad-hoc configuration – Logging levels
  174. 175. Environment aware – Nodes of class “foo” </li></ul>
  175. 176. N+P+A Benefits <ul><li>Devs don't HAVE to learn ERB
  176. 177. Devs don't HAVE to learn Puppet
  177. 178. Ops and Dev friendly bridge point
  178. 179. No unintended changes released </li></ul>
  179. 180. Integration with CI Noah Puppet CI Deploy
  180. 181. Integration with CI <ul><li>Jenkins asks Noah for nodes of class “foo”
  181. 182. Jenkins triggers Puppet (or deploy) on appropriate nodes
  182. 183. Jenkins is always “current” </li></ul>
  183. 184. Integration with Monitoring Noah Puppet Monitoring App
  184. 185. Integration with Monitoring <ul><li>Puppet manages Nagios configs
  185. 186. Nagios updates Noah status (event handler)
  186. 187. Noah optionally fires watches based on status change
  187. 188. Endpoint is application. App “reconfigures” </li></ul>
  188. 189. Integration with Scripts Noah Scripts (node 1) Scripts (node 2) Scripts (node 3)
  189. 190. Integration with Scripts <ul><li>Provide “locking” and “coordination” of cron jobs across hosts
  190. 191. Stop dealing with lock files on host
  191. 192. Use ephemerals and curl </li></ul>
  192. 193. Gotchas <ul><li>Noah itself is NOT distributed *
  193. 194. Data redundancy based on Redis
  194. 195. No ACLs *
  195. 196. Async </li></ul>
  196. 197. Other options <ul><li>ZooKeeper
  197. 198. Doozer
  198. 199. NoSQL stores (Redis + Webdis, Riak) *
  199. 200. Nesoi *
  200. 201. Write your own! </li></ul>
  201. 202. Questions?
  202. 203. Noah <ul><li>Github: http://github.com/lusis /Noah
  203. 204. IRC: #noah
  204. 205. Blog(s): http://blog.lusis.org/blog/categories/noah </li></ul><ul><ul><ul><ul><li>http://lusislog.blogspot.com/search/label/noah </li></ul></ul></ul></ul>
  205. 206. John E. Vincent <ul><li>Twitter: @lusis
  206. 207. Github: http://github.com/lusis
  207. 208. IRC: lusis
  208. 209. LinkedIn: http://linkedin.com/in/lusis
  209. 210. Blog(s): http://blog.lusis.org </li></ul><ul><ul><ul><ul><li>http://lusislog.blogspot.com </li></ul></ul></ul></ul>
  210. 211. Thank You!

×