Beyond the Node Advanced 'Arkestration' with Noah PuppetConf 2011
.plan <ul><li>What's the problem?
Orchestration
Overview of Noah
Use Cases
Q&A </li></ul>
 
A Brief History of Time (with apologies to Hawking)
Configuration Management <ul><li>Manual </li></ul>
Configuration Management
Configuration Management
Configuration Management <ul><li>Manual
Scripted </li></ul>
Configuration Management
Configuration Management <ul><li>Manual
Scripted
Intelligent </li></ul>
CFEngine
Chef
Puppet
Configuration Management <ul><li>Manual
Scripted
Intelligent </li></ul>
So we've solved this, right?
“ I think..and I don't know if anyone would agree, that configuration management is a solved problem at this point, right?”
“ I think..and I don't know if anyone would agree, that configuration management is a solved problem at this point, right?...
&quot;The point I want to make..Configuration management is  not  a solved problem...and it's dangerous to make the mistak...
“ what I was attempting to say ... is that the current crop of configuration management tools have reached a usable point ...
“ what I was attempting to say (epic fail, I might add) is that the current crop of configuration management tools have re...
Orchestration and Noah Noah attempts to do two things: <ul><li>Bridge an impedance mismatch  between operational, developm...
Provide mechanisms for coordination  between applications, nodes, services, configuration management and other infrastruct...
Example Use Case 1) Capacity Reached. Tell Noah 2) Noah tells provisioning system 3) Capacity allocated. Tell Noah 4) Noah...
“ Inspired” by
&quot;ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchron...
Features <ul><li>Hierarchical name space of data registers  similar to a Unix filesystem ( znodes )
Leader election, locking, configuration, sequences
Distributed and highly available
Watches </li></ul>Downsides <ul><li>Persistent connections required
Official bindings are C and Java
Somewhat &quot;complex&quot; to setup and run </li></ul>
Noah and ZooKeeper Noah is essentially a re-imagining of ZK, taking the things I liked: <ul><li>Znode concept
Watches </li></ul>and making it work over  HTTP  using  JSON
Noah Basics
The Primitives <ul><li>Hosts
Has status  ( up ,  down ,  pending )
Can have   Services
Services </li></ul><ul>Has status  ( up ,  down ,  pending ) <li>Must belong to a   Host </li></ul><ul><li>Applications
Can have   Configurations
Configurations
Can belong to many   Applications </li></ul>
The Not So Primitives <ul><li>Ephemerals
Identified by a  “path”
Tags
Can be applied to any  Primitive  or  Ephemeral
Links </li></ul><ul>Can be applied to any  Primitive, Ephemeral  or  Tag </ul><ul><li>Watches
Can be applied at any hierarchy or specific object </li></ul>
URL Patterns <ul><li>Hosts </li><ul><li>/ hosts / app01 / services / httpd </li></ul><li>Services </li><ul><li>/ services ...
GET ting Data
{ &quot;foo&quot;: { &quot;created_at&quot; : &quot;2011-04-20 04:50:41 UTC&quot;, &quot;updated_at&quot; : &quot;2011-04-...
GET ting Data <ul><li>Hosts </li></ul>
{ &quot;foo&quot; : { &quot;created_at&quot;: &quot;2011-04-20 04:50:41 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-2...
{ &quot;foo&quot;: { &quot;created_at&quot;: &quot;2011-04-20 04:50:41 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-20...
GET ting Data <ul><li>Hosts
Services </li></ul>
{ &quot;foo_svc&quot; : { &quot;foo&quot; : { &quot;created_at&quot;: &quot;2011-04-20 05:09:44 UTC&quot;, &quot;updated_a...
{ &quot;foo_svc&quot; : { &quot;foo&quot; : { &quot;created_at&quot;: &quot;2011-04-20 05:09:44 UTC&quot;, &quot;updated_a...
GET ting Data <ul><li>Hosts
Services
Applications </li></ul>
{ &quot;myrailsapp1&quot; : { &quot;created_at&quot;: &quot;2011-04-21 06:57:42 UTC&quot;, &quot;updated_at&quot;: &quot;2...
{ &quot;myrailsapp1&quot;: { &quot;created_at&quot;: &quot;2011-04-21 06:57:42 UTC&quot;, &quot;updated_at&quot;: &quot;20...
GET ting Data <ul><li>Hosts
Services
Applications
Configurations  * </li></ul>
{ &quot;barconf1&quot; : { &quot;created_at&quot;: &quot;2011-04-27 01:20:37 UTC&quot;, &quot;format&quot;: &quot;string&q...
GET ting Data <ul><li>Hosts
Services
Applications
Configurations *
Upcoming SlideShare
Loading in...5
×

Beyond the Node: Arkestration with Noah

5,911
-1

Published on

Slides from my presentation at PuppetConf 2011.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,911
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
34
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Speaker Notes Next slide - plan
  • What problem is noah trying to help solve?
  • In the beginning we did things manually. This was generally regarded as a very bad idea.
  • We took notes! (Yes this is on an actual server right now as we speak)
  • We wrote handbooks!
  • Then we started writing Shell scripts!
  • This script is actually broken.
  • Then we got smart!
  • In the general scheme of things, we&apos;ve evolved. We treat our infrastructure as code. We&apos;ve adopted behaviors and practices that developers have been doing for years. We&apos;re green, ship it!
  • At DevOpsDay Mt. View, a revelation was made.
  • The response was generally overwhelmingly supportive....
  • Ideas were “refined”
  • This is the heart of “orchestration”. This is the problem space that needs addressing right now.
  • Orchestraton means many things however. Luke mentioned “Command and Control” for instance. Impedence mismatch – mention Capistrano and boatload of “deploy” tools. Something is missing. Mechanisms – traditional orchestration
  • Just a quick example use case to demonstrate “orchestration”
  • Znodes are easily grokable. We get “paths”. Programatically friendly. Watches are cool. Downsides:
  • Have always needed something LIKE ZK. Haven&apos;t always had the “luxury” of language bindings. Haven&apos;t always needed the full feature set. Another unofficial goal is that it work in a “disconnected” world.
  • Going over the basics of interacting with Noah. All of this is on the wiki.
  • Host – node, server Service – http/https/ftp OSI L3 analogy. An open port. Host/Service inspired by Nagios Application – tomcat, apache, rails app. OSI L7 analogy Configuration – httpd.conf, server.xml, database.yml
  • Ephemerals – Arbitrary blob of information Tag – tags...not much to say here Links – similar to symlinks. Create a custom namespace Watches – Async pluggable callbacks
  • Explain the general idea behind programatic friendly paths. Operate with standard HTTP verbs – GET, PUT, DELETE
  • Explain the general idea behind programmatic friendly paths.
  • Common fields in all GETs
  • This is a GET to /hosts/
  • Remember that services must be bound to hosts.
  • Note the asterix here
  • JSON version is default in most cases: Accept: application/json to ensure this representation application/octet will ensure that you get a proper mime typed version.
  • Reminder of the paths
  • Note that services has a requirement on hostname in the path.
  • This is what you PUT to each path in previous slide Maybe have to jump back and forth.
  • Same paths as put. No payload required.
  • Explain a bit how ZK watches work Mention race condition.
  • Made up of endpoints and patterns. Persistence will eventually be tunable Supersets If you declare a watch for endpoint on: /hosts You can&apos;t declare a watch for endpoint on: /hosts/foo_host Subsets are the reverse
  • This creates a watch for anything under /applications. When something “changes” under /applications, the change will be send to the endpoint.
  • Again, watches have the same common fields
  • Here we&apos;re going to add a tag to that new application
  • Note that every example listed does not actually exist YET
  • Creator unrelated to the reciever is important. Allows proxy management You can write your own at the cost of fragility
  • James wrote these. Need to be updated to newest Noah API. Mention that this was supposed to be done before this presentation but there was some “confusion” about the schedule - Functions pull data from/put in Noah - noah_data fact pulls configurations from Noah Cover the mappings
  • Taking Noah out of the mix for a moment, what&apos;s a traditional workflow for managing applications with Puppet?
  • Problems I&apos;ve seen Different repos Dev interaction What happens when we add a new setting? How many people run puppet in daemon mode or via cron as opposed to on demand? Rampination.
  • Adding Noah in the mix Looks a bit more complicated. Noah becomes your System of Record Talk about Volatile settings
  • This is where I disagree with Luke a little bit. There are valid use cases for “dynamic” configuration. JMX exists.
  • Explain CI
  • We currently do this for deploys at VA using a curl script in a bash step You can either trigger the deploy itself or wrap it around a puppet run. Both can be done by CI. Upshot is that you get more templated CI jobs.
  • This is similar to the original example use case.
  • The nagios event handler, again, can be implemented as a curl script. If there&apos;s a watch attached, you can trigger any number of actions. You can have endpoints in your applications that make it dynamically respond.
  • This is an immediate win
  • Ghetto job dependencies Easier than dealing with mysql command lines and escaping.
  • ZK and Doozer are a bit more complex. Require persistent connections. NoSQL and Nesoi are nice because they&apos;re http and json!
  • Beyond the Node: Arkestration with Noah

    1. 1. Beyond the Node Advanced 'Arkestration' with Noah PuppetConf 2011
    2. 2. .plan <ul><li>What's the problem?
    3. 3. Orchestration
    4. 4. Overview of Noah
    5. 5. Use Cases
    6. 6. Q&A </li></ul>
    7. 8. A Brief History of Time (with apologies to Hawking)
    8. 9. Configuration Management <ul><li>Manual </li></ul>
    9. 10. Configuration Management
    10. 11. Configuration Management
    11. 12. Configuration Management <ul><li>Manual
    12. 13. Scripted </li></ul>
    13. 14. Configuration Management
    14. 15. Configuration Management <ul><li>Manual
    15. 16. Scripted
    16. 17. Intelligent </li></ul>
    17. 18. CFEngine
    18. 19. Chef
    19. 20. Puppet
    20. 21. Configuration Management <ul><li>Manual
    21. 22. Scripted
    22. 23. Intelligent </li></ul>
    23. 24. So we've solved this, right?
    24. 25. “ I think..and I don't know if anyone would agree, that configuration management is a solved problem at this point, right?”
    25. 26. “ I think..and I don't know if anyone would agree, that configuration management is a solved problem at this point, right?” WTF?
    26. 27. &quot;The point I want to make..Configuration management is not a solved problem...and it's dangerous to make the mistake to think that the way we do things now is the best way to do them...&quot; - Andrew Clay Shafer
    27. 28. “ what I was attempting to say ... is that the current crop of configuration management tools have reached a usable point where they do enough (for now). What we’re seeing as questions now are 'How do I think beyond the single node where this tool is running?'”
    28. 29. “ what I was attempting to say (epic fail, I might add) is that the current crop of configuration management tools have reached a usable point where they do enough (for now). What we’re seeing as questions now are ' How do I think beyond the single node where this tool is running? '”
    29. 30. Orchestration and Noah Noah attempts to do two things: <ul><li>Bridge an impedance mismatch between operational, developmental and application configuration
    30. 31. Provide mechanisms for coordination between applications, nodes, services, configuration management and other infrastructure aspects </li></ul>
    31. 32. Example Use Case 1) Capacity Reached. Tell Noah 2) Noah tells provisioning system 3) Capacity allocated. Tell Noah 4) Noah triggers Puppet on LB 5) Noah triggers Puppet on Nagios Noah Nagios Puppet (app) Load Balancer
    32. 33. “ Inspired” by
    33. 34. &quot;ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications.&quot;
    34. 35. Features <ul><li>Hierarchical name space of data registers similar to a Unix filesystem ( znodes )
    35. 36. Leader election, locking, configuration, sequences
    36. 37. Distributed and highly available
    37. 38. Watches </li></ul>Downsides <ul><li>Persistent connections required
    38. 39. Official bindings are C and Java
    39. 40. Somewhat &quot;complex&quot; to setup and run </li></ul>
    40. 41. Noah and ZooKeeper Noah is essentially a re-imagining of ZK, taking the things I liked: <ul><li>Znode concept
    41. 42. Watches </li></ul>and making it work over HTTP using JSON
    42. 43. Noah Basics
    43. 44. The Primitives <ul><li>Hosts
    44. 45. Has status ( up , down , pending )
    45. 46. Can have Services
    46. 47. Services </li></ul><ul>Has status ( up , down , pending ) <li>Must belong to a Host </li></ul><ul><li>Applications
    47. 48. Can have Configurations
    48. 49. Configurations
    49. 50. Can belong to many Applications </li></ul>
    50. 51. The Not So Primitives <ul><li>Ephemerals
    51. 52. Identified by a “path”
    52. 53. Tags
    53. 54. Can be applied to any Primitive or Ephemeral
    54. 55. Links </li></ul><ul>Can be applied to any Primitive, Ephemeral or Tag </ul><ul><li>Watches
    55. 56. Can be applied at any hierarchy or specific object </li></ul>
    56. 57. URL Patterns <ul><li>Hosts </li><ul><li>/ hosts / app01 / services / httpd </li></ul><li>Services </li><ul><li>/ services / httpd / app02 </li></ul><li>Applications </li><ul><li>/ applications / my_app / configurations / log_level </li></ul><li>Configurations </li><ul><li>/ configurations / log_level </li></ul></ul>
    57. 58. GET ting Data
    58. 59. { &quot;foo&quot;: { &quot;created_at&quot; : &quot;2011-04-20 04:50:41 UTC&quot;, &quot;updated_at&quot; : &quot;2011-04-20 04:50:41 UTC&quot;, &quot;tags&quot; : [ ], &quot;id&quot; : &quot;4af53f8d-36d2-2158-c4a1-514ff3578048&quot;, &quot;links&quot; : [ ], &quot;services&quot;: { }, &quot;status&quot;: &quot;up&quot; }, &quot;bar&quot;: { } }
    59. 60. GET ting Data <ul><li>Hosts </li></ul>
    60. 61. { &quot;foo&quot; : { &quot;created_at&quot;: &quot;2011-04-20 04:50:41 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-20 04:50:41 UTC&quot;, &quot;tags&quot;: [ ], &quot;id&quot;: &quot;4af53f8d-36d2-2158-c4a1-514ff3578048&quot;, &quot;links&quot;: [ ], &quot;services&quot;: { “ mysql”:”pending”, “ httpd”:”up” }, &quot;status&quot;: &quot;up&quot; }, &quot;bar&quot;: { } } Host Name
    61. 62. { &quot;foo&quot;: { &quot;created_at&quot;: &quot;2011-04-20 04:50:41 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-20 04:50:41 UTC&quot;, &quot;tags&quot;: [ ], &quot;id&quot;: &quot;4af53f8d-36d2-2158-c4a1-514ff3578048&quot;, &quot;links&quot;: [ ], &quot;services&quot; : { “ mysql”:”pending”, “ httpd”:”up” }, &quot;status&quot;: &quot;up&quot; }, &quot;bar&quot;: { } } Services
    62. 63. GET ting Data <ul><li>Hosts
    63. 64. Services </li></ul>
    64. 65. { &quot;foo_svc&quot; : { &quot;foo&quot; : { &quot;created_at&quot;: &quot;2011-04-20 05:09:44 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-20 05:09:44 UTC&quot;, &quot;tags&quot;: [ ], &quot;id&quot;: &quot;53504526-a5d7-a604-5fe8-1f8fd2e1aef0&quot;, &quot;links&quot;: [ ], &quot;status&quot;: &quot;down&quot; } }, &quot;both&quot;: { } } Service name
    65. 66. { &quot;foo_svc&quot; : { &quot;foo&quot; : { &quot;created_at&quot;: &quot;2011-04-20 05:09:44 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-20 05:09:44 UTC&quot;, &quot;tags&quot;: [ ], &quot;id&quot;: &quot;53504526-a5d7-a604-5fe8-1f8fd2e1aef0&quot;, &quot;links&quot;: [ ], &quot;status&quot;: &quot;down&quot; } }, &quot;both&quot;: { } } Host Name
    66. 67. GET ting Data <ul><li>Hosts
    67. 68. Services
    68. 69. Applications </li></ul>
    69. 70. { &quot;myrailsapp1&quot; : { &quot;created_at&quot;: &quot;2011-04-21 06:57:42 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-21 06:57:42 UTC&quot;, &quot;tags&quot;: [ &quot;sample_data&quot;, &quot;production&quot; ], &quot;id&quot;: &quot;8774fead-e63a-8e5c-0453-1b5330bbd04c&quot;, &quot;links&quot;: [ &quot;/my_sample_organization&quot; ], &quot;configurations&quot;: { &quot;database.yml&quot;: { &quot;format&quot;: &quot;yaml&quot;, &quot;body&quot;: &quot;development:n database: development_databasen adapter: mysqln username: dev_usern password: dev_passwordn&quot; } } }, &quot;myrestapp1&quot;: { } } App Name
    70. 71. { &quot;myrailsapp1&quot;: { &quot;created_at&quot;: &quot;2011-04-21 06:57:42 UTC&quot;, &quot;updated_at&quot;: &quot;2011-04-21 06:57:42 UTC&quot;, &quot;tags&quot;: [ &quot;sample_data&quot;, &quot;production&quot; ], &quot;id&quot;: &quot;8774fead-e63a-8e5c-0453-1b5330bbd04c&quot;, &quot;links&quot;: [ &quot;/my_sample_organization&quot; ], &quot;configurations&quot; : { &quot;database.yml&quot;: { &quot;format&quot;: &quot;yaml&quot;, &quot;body&quot;: &quot;development:n database: development_databasen adapter: mysqln username: dev_usern password: dev_passwordn&quot; } } }, &quot;myrestapp1&quot;: { } } Configs
    71. 72. GET ting Data <ul><li>Hosts
    72. 73. Services
    73. 74. Applications
    74. 75. Configurations * </li></ul>
    75. 76. { &quot;barconf1&quot; : { &quot;created_at&quot;: &quot;2011-04-27 01:20:37 UTC&quot;, &quot;format&quot;: &quot;string&quot;, &quot;body&quot;: &quot;barbody1&quot;, &quot;updated_at&quot;: &quot;2011-04-27 01:20:37 UTC&quot;, &quot;tags&quot;: [ ], &quot;id&quot;: &quot;aa183dce-30ec-2d7e-c6f9-f3385161a7e6&quot;, &quot;links&quot;: [ ] }, &quot;barconf2&quot;: { } }
    76. 77. GET ting Data <ul><li>Hosts
    77. 78. Services
    78. 79. Applications
    79. 80. Configurations *
    80. 81. Ephemerals * </li></ul>
    81. 82. GET ting Data <ul><li>Hosts
    82. 83. Services
    83. 84. Applications
    84. 85. Configurations *
    85. 86. Ephemerals *
    86. 87. /ephemerals/my/config/setting returns the data stored there. </li></ul>
    87. 88. PUT ting Data <ul><li>/ hosts /host_name
    88. 89. / services /service_name/host_name
    89. 90. / applications /application_name
    90. 91. / configurations /configuration_name
    91. 92. / ephemerals /arbitrary/path/to/whatever </li></ul>
    92. 93. PUT ting Data <ul><li>/ hosts / host_name
    93. 94. / services / service_name / host_name
    94. 95. / applications / application_name
    95. 96. / configurations / configuration_name
    96. 97. / ephemerals / arbitrary / path / to / whatever </li></ul>
    97. 98. PUT ting Data <ul><li>Host {“status”:”up”}
    98. 99. Service {“status”:”up”,”host_status”:”up”}
    99. 100. Application {“name”:”app_name”}
    100. 101. Configuration {“format”:”format_type”,”body”:”data”}
    101. 102. Ephemeral raw data </li></ul>
    102. 103. DELETE ing Data <ul><li>/hosts/ host_name
    103. 104. /services/ service_name / host_name
    104. 105. /applications/ application_name
    105. 106. /configurations/ configuration_name
    106. 107. /ephemerals/ arbitrary / path / to / whatever </li></ul>
    107. 108. Tagging <ul><li>Append “tag” to any PUT url with PUT method
    108. 109. e.g. ' /applications/some_application/tag'
    109. 110. {“tags”:”sometag”} or {“tags”:[“tag1”,”tag2”]}
    110. 111. Call DELETE instead of PUT to remove tags
    111. 112. GET ' /tags ' for all tags or ' /tag/tagname ' for specific tag </li></ul>
    112. 113. Linking <ul><li>Append “link” to any PUT url with PUT method
    113. 114. e.g. ' /applications/some_application/link'
    114. 115. {“link_name”:”my_organization”}
    115. 116. Call DELETE instead of PUT to remove links *
    116. 117. GET ' /link_name ' for all linked objects *
    117. 118. Designed to allow custom namespaces </li></ul>
    118. 119. Watches
    119. 120. What are Watches? <ul><li>A bit different than ZK watches
    120. 121. Async
    121. 122. Persistent *
    122. 123. Pluggable (URI pattern based)
    123. 124. Hierarchical (superset and subset)
    124. 125. Append ' /watch ' to almost any url path
    125. 126. {“endpoint”:”http://some_host:port/ping_me”}
    126. 127. ' /watch ' root endpoint as well
    127. 128. {
    128. 129. “ endpoint”:”http://some_host/ping”,
    129. 130. ” pattern”:”//noah/applications/app_name”
    130. 131. } </li></ul>
    131. 132. Create a Watch $ curl -XPUT -d '{“pattern”:”//noah/applications”,&quot;endpoint&quot;:&quot; http://host/ping &quot;}' http://noahserver/watches
    132. 133. Perform some operation $ curl -XPUT -d '{&quot;name&quot;:&quot;fooapp2&quot;}' http://noahserver/applications/fooapp2
    133. 134. Message sent to Endpoint { &quot;id&quot;:&quot;be9f6a45-0d49-3f00-59a8-c6ad61da24e1&quot;, &quot;tags&quot;:[], &quot;links&quot;:[], &quot;name&quot;:&quot;fooapp2&quot; , &quot;created_at&quot;:&quot;2011-09-15 03:07:39 UTC&quot;, &quot;updated_at&quot;:&quot;2011-09-15 03:07:39 UTC&quot;, &quot;configurations&quot;:{}, &quot;action&quot;:&quot;create&quot; , &quot;pubcategory&quot;:&quot;//noah/applications/fooapp2&quot; }
    134. 135. Perform another operation $ curl -XPUT -d '{&quot;tags&quot;:&quot;footag2&quot;}' http://noahserver/applications/fooapp2/tag
    135. 136. Message sent to Endpoint { &quot;id&quot;:&quot;be9f6a45-0d49-3f00-59a8-c6ad61da24e1&quot;, &quot;tags&quot;:[&quot;footag&quot;] , &quot;links&quot;:[], &quot;name&quot;:&quot;fooapp2&quot; , &quot;created_at&quot;:&quot;2011-09-15 03:07:39 UTC&quot;, &quot;updated_at&quot;:&quot;2011-09-15 03:15:03 UTC&quot;, &quot;configurations&quot;:{}, &quot;action&quot;:&quot;update&quot; , &quot;pubcategory&quot;:&quot;//noah/applications/fooapp2&quot; }
    136. 137. Watches are “plugin” based You can write your own plugins with Ruby. <ul><li>URI Pattern *
    137. 138. amqp://user:pass@host:port/exchange
    138. 139. rundeck: //token@host :port/job_id
    139. 140. puppet://hostname/
    140. 141. statsd://host:port/metric.path.name.action
    141. 142. Currently based on Event Machine
    142. 143. Future based on Actors and worker pools </li></ul>
    143. 144. Watch Notes <ul><li>Creator of the watch is unrelated to receiver
    144. 145. Don't have to use the “official” watch daemon
    145. 146. You can watch the watches (PubSub) </li></ul>
    146. 147. Basic Puppet Integration <ul><li>http://github.com/jamtur01/puppet-noah
    147. 148. ENC, Functions and Facts
    148. 149. Hosts -> Nodes
    149. 150. Host's Services -> Classes
    150. 151. Configurations -> Facts (string types only) </li></ul>
    151. 152. So where does it fit? Noah Puppet Apps Scripts Monitoring Deploy CI
    152. 153. Noah + Puppet + Applications Noah Puppet Applications
    153. 154. Common P+A Use Case <ul><li>Puppet knows Nodes and Classes
    154. 155. App has configuration file
    155. 156. Configuration file controlled by Puppet
    156. 157. Puppet client run required to update file </li></ul>
    157. 158. Common P+A Issues <ul><li>Puppet repo is not app repo
    158. 159. Devs must learn ERB
    159. 160. Devs might even have to learn Puppet
    160. 161. Introducing new settings and communication
    161. 162. Daemon-mode is exception rather than rule </li></ul>
    162. 163. N+P+A Version <ul><li>Noah knows Nodes and Classes
    163. 164. Move volatile settings into Noah
    164. 165. Configuration file controlled by Puppet
    165. 166. App gets volatile settings from Noah *
    166. 167. Future Puppet runs update Noah
    167. 168. Noah tells App *
    168. 169. Truth vs. Reality
    169. 170. My DB master is foo
    170. 171. My DB master right NOW is bar </li></ul>
    171. 172. Volatile Settings? <ul><li>Not all configuration is created equal
    172. 173. “ Elasticity ” - Pool of worker node EC2 ips
    173. 174. Ad-hoc configuration – Logging levels
    174. 175. Environment aware – Nodes of class “foo” </li></ul>
    175. 176. N+P+A Benefits <ul><li>Devs don't HAVE to learn ERB
    176. 177. Devs don't HAVE to learn Puppet
    177. 178. Ops and Dev friendly bridge point
    178. 179. No unintended changes released </li></ul>
    179. 180. Integration with CI Noah Puppet CI Deploy
    180. 181. Integration with CI <ul><li>Jenkins asks Noah for nodes of class “foo”
    181. 182. Jenkins triggers Puppet (or deploy) on appropriate nodes
    182. 183. Jenkins is always “current” </li></ul>
    183. 184. Integration with Monitoring Noah Puppet Monitoring App
    184. 185. Integration with Monitoring <ul><li>Puppet manages Nagios configs
    185. 186. Nagios updates Noah status (event handler)
    186. 187. Noah optionally fires watches based on status change
    187. 188. Endpoint is application. App “reconfigures” </li></ul>
    188. 189. Integration with Scripts Noah Scripts (node 1) Scripts (node 2) Scripts (node 3)
    189. 190. Integration with Scripts <ul><li>Provide “locking” and “coordination” of cron jobs across hosts
    190. 191. Stop dealing with lock files on host
    191. 192. Use ephemerals and curl </li></ul>
    192. 193. Gotchas <ul><li>Noah itself is NOT distributed *
    193. 194. Data redundancy based on Redis
    194. 195. No ACLs *
    195. 196. Async </li></ul>
    196. 197. Other options <ul><li>ZooKeeper
    197. 198. Doozer
    198. 199. NoSQL stores (Redis + Webdis, Riak) *
    199. 200. Nesoi *
    200. 201. Write your own! </li></ul>
    201. 202. Questions?
    202. 203. Noah <ul><li>Github: http://github.com/lusis /Noah
    203. 204. IRC: #noah
    204. 205. Blog(s): http://blog.lusis.org/blog/categories/noah </li></ul><ul><ul><ul><ul><li>http://lusislog.blogspot.com/search/label/noah </li></ul></ul></ul></ul>
    205. 206. John E. Vincent <ul><li>Twitter: @lusis
    206. 207. Github: http://github.com/lusis
    207. 208. IRC: lusis
    208. 209. LinkedIn: http://linkedin.com/in/lusis
    209. 210. Blog(s): http://blog.lusis.org </li></ul><ul><ul><ul><ul><li>http://lusislog.blogspot.com </li></ul></ul></ul></ul>
    210. 211. Thank You!
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×