Your SlideShare is downloading. ×
0
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Data Driven Infrastructure
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Data Driven Infrastructure

768

Published on

"Data Driven Infrastructure" by James Fryman of GitHub at Puppet Camp London 2013. Find the video here: http://puppetlabs.com/community/puppet-camp

"Data Driven Infrastructure" by James Fryman of GitHub at Puppet Camp London 2013. Find the video here: http://puppetlabs.com/community/puppet-camp

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
768
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
53
Comments
0
Likes
4
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Friday, August 23, 13
  • 2. Data Driven Infrastructure Friday, August 23, 13
  • 3. Friday, August 23, 13
  • 4. ? Friday, August 23, 13
  • 5. ? http://git.io/UmUdKA Friday, August 23, 13
  • 6. Let me Tell you a story Friday, August 23, 13
  • 7. What is a Cloud Friday, August 23, 13
  • 8. With the Cloud... I don’t have to think about IT Friday, August 23, 13
  • 9. Friday, August 23, 13
  • 10. Software Platform Infrastructure Friday, August 23, 13 aaS
  • 11. Friday, August 23, 13
  • 12. Friday, August 23, 13
  • 13. Where is our cloud? Friday, August 23, 13
  • 14. Friday, August 23, 13
  • 15. Utopia Friday, August 23, 13
  • 16. James Fryman Friday, August 23, 13
  • 17. Friday, August 23, 13
  • 18. Auto matio ns Junk ie Friday, August 23, 13
  • 19. Are you Crazy? Friday, August 23, 13
  • 20. What Data? Goals? Friday, August 23, 13
  • 21. What Data? Goals? Friday, August 23, 13
  • 22. What Data? Goals? Friday, August 23, 13
  • 23. ne hi ac M le ab rs Pa Friday, August 23, 13
  • 24. There is Friday, August 23, 13 One System
  • 25. gPanel PuppetDB Provisioning Configuring Destroying Feedback Friday, August 23, 13
  • 26. Parts of the System Friday, August 23, 13
  • 27. provisioner Friday, August 23, 13
  • 28. Home Grown Friday, August 23, 13
  • 29. Home Grown Friday, August 23, 13
  • 30. controller Friday, August 23, 13
  • 31. Lots O’ Models Friday, August 23, 13
  • 32. file { '/etc/facter/facts.d/quagga_manage_service': ensure => present, replace => false, }   if $::quagga_manage_service { $service_state[ensure] = running $service_state[enable] = true } else { $service_state[ensure] = undef $service_state[enable] = undef }   service { 'quagga': ensure => $service_state[ensure], enable => $service_state[enable], } Friday, August 23, 13
  • 33. orchestrator Friday, August 23, 13
  • 34. Chat Ops Friday, August 23, 13
  • 35. Chat Ops Friday, August 23, 13
  • 36. Feedback of the System Friday, August 23, 13
  • 37. & Metrics Monitoring Friday, August 23, 13
  • 38. begin JSON.parse(RestClient.get(url)).each do |cache| data["#{cache['target']}"] = 0 count = 0 cache["datapoints"].each do |point| unless (point[0].nil?) data["#{cache['target']}"] += point[0] count += 1 end end data["#{cache['target']}"] /= count data["total"] += data["#{cache['target']}"] end rescue ZeroDivisionError => e puts "UNKNOWN metric not in graphite!" exit EXIT_UNKNOWN rescue => e puts e.message exit EXIT_CRITICAL end Friday, August 23, 13
  • 39. begin JSON.parse(RestClient.get(url)).each do |cache| data["#{cache['target']}"] = 0 count = 0 cache["datapoints"].each do |point| unless (point[0].nil?) data["#{cache['target']}"] += point[0] count += 1 end end data["#{cache['target']}"] /= count data["total"] += data["#{cache['target']}"] end rescue ZeroDivisionError => e puts "UNKNOWN metric not in graphite!" exit EXIT_UNKNOWN rescue => e puts e.message exit EXIT_CRITICAL end Friday, August 23, 13
  • 40. event Friday, August 23, 13
  • 41. event log Friday, August 23, 13
  • 42. event log Friday, August 23, 13 alert
  • 43. event log Friday, August 23, 13 alert error
  • 44. Must self-correct Friday, August 23, 13
  • 45. gPanel PuppetDB Provisioning Configuring Destroying Feedback Friday, August 23, 13
  • 46. Friday, August 23, 13
  • 47. Deployable using text files Friday, August 23, 13
  • 48. larity odu M Friday, August 23, 13
  • 49. haproxy::proxy proxy => mode => ip => port => config => balance => }, } Friday, August 23, 13 { $es_proxy_name: 'listen', 'http', $::ipaddress_lo, '9200', { 'roundrobin',
  • 50. haproxy::proxy proxy => mode => ip => port => config => balance => }, } Friday, August 23, 13 { $es_proxy_name: 'listen', 'http', $::ipaddress_lo, '9200', { 'roundrobin',
  • 51. haproxy::proxy::member { $es_proxy_name: hostname => $::ec2_local_ipv4, port => '9200', param => [ 'weight 1', 'maxconn 1000', 'check', ], } Friday, August 23, 13
  • 52. collectd::plugin { [ 'cpu', 'load', 'memory', 'swap', 'irq', 'exec', 'entropy', ]: }   Friday, August 23, 13 collectd::plugin { [ 'df', 'interface', 'protocols', 'disk', ]: config => true }
  • 53. Level 4: Templates Friday, August 23, 13 Level 5: Data Driven
  • 54. Self Authoritative Friday, August 23, 13
  • 55. def self.collect_resources(resource) query = [ "and", ["=", "type", "File"], ["=", "exported", true], ["=", "tag", "nagios::object::#{resource}"], ["=", ["node", "active"], true] ] self.search('/v2/resources', query.to_json) end Friday, August 23, 13
  • 56. def self.collect_resources(resource) query = [ "and", ["=", "type", "File"], ["=", "exported", true], ["=", "tag", "nagios::object::#{resource}"], ["=", ["node", "active"], true] ] self.search('/v2/resources', query.to_json) end Friday, August 23, 13
  • 57. # Seed the initial file with false, and enable # on next run with gh-nagios enable_pager <hostname> file { '/etc/facter/facts.d/enable_pager.txt': ensure => file, owner => 'root', group => 'root', mode => '0644', content => 'enable_pager=false', replace => false, } Friday, August 23, 13
  • 58. } @@file { content mode group notify Friday, August 23, 13 "/etc/nagios/objects/${type}.d/${filename}.cfg": => template('nagios/etc/nagios/object.erb'), => '0444', => 'nagios', => Service['nagios'],
  • 59. Tips Friday, August 23, 13
  • 60. ng ri to ac ef R Friday, August 23, 13
  • 61. ng ri to ac ef R Friday, August 23, 13
  • 62. least to most specific Friday, August 23, 13
  • 63. frymanet.com nginx common rails ruby mysql admin package repos Friday, August 23, 13
  • 64. graduate to params Friday, August 23, 13
  • 65. class ntp::params {   $defaults = { package => { version => ‘latest’, }, config => { servers => [‘pool.ntp.org’], }, } } Friday, August 23, 13
  • 66. externalize Friday, August 23, 13
  • 67. class ntp::params {   $defaults = { package => { version => hiera(‘ntp_package_version’), }, config => { servers => hiera(‘ntp_servers’), }, } } Friday, August 23, 13
  • 68. Cloud Forma tion Friday, August 23, 13
  • 69. Cloud Forma tion Friday, August 23, 13
  • 70. "Resources": { "RendererServerGroup": { "Type": "AWS::AutoScaling::AutoScalingGroup", "Properties": { "AvailabilityZones": { "Fn::GetAZs": "" }, "LaunchConfigurationName": { "Ref": "LaunchConfig" }, "MinSize": "2", "MaxSize": "16", "Tags": [ { "Key": "Environment", "Value": "Production", "PropagateAtLaunch": "true" }, { "Key": "Role", "Value": "renderer", "PropagateAtLaunch": "true" } ] } }, Friday, August 23, 13
  • 71. ding toloa Au Friday, August 23, 13
  • 72. # autoloader.pp class nagios::autoload_helpers { $helpers = get_nagios_helpers() case $::puppetversion { /^3/: { include $helpers } default: { nagios::autoload_helpers::import_shim { $helpers: } } } } Friday, August 23, 13
  • 73. # autoloader.pp class nagios::autoload_helpers { $helpers = get_nagios_helpers() case $::puppetversion { /^3/: { include $helpers } default: { nagios::autoload_helpers::import_shim { $helpers: } } } } Friday, August 23, 13
  • 74. module Puppet::Parser::Functions newfunction(:get_nagios_helpers, :type => :rvalue, :doc => "Grab all modules that have nagios helpers for import") do module_path = File.expand_path('..', Puppet::Module.find('nagios',compiler.environment.to_s).path) helpers = Dir["#{module_path}/**/nagios/helpers.pp"].map do |d| "#{d.split('/')[-4]}::nagios::helpers" done helpers end end Friday, August 23, 13
  • 75. Modeling Friday, August 23, 13
  • 76. Be Dynamic Friday, August 23, 13
  • 77. Be Dynamic Friday, August 23, 13
  • 78. <%- if @comment -%> ### <%= @comment %> <%- end -%> <%= @type %> <% if @label %><%= @label %><% end %> { <%- @config.sort.reverse.each do | key,value| -%> <%- if value.class == Array -%> <%- value.each do |element| %> <%= key %>(<%= element %>); <%- end -%> <%- else -%> <%= key %>(<%= value %>); <%- end -%> <%- end -%> }; Friday, August 23, 13
  • 79. ing nc e F Friday, August 23, 13 so Re es rc u
  • 80. # Seed the initial file with false, and enable # on next run with gh-nagios enable_pager <hostname> file { '/etc/facter/facts.d/enable_pager.txt': ensure => file, owner => 'root', group => 'root', mode => '0644', content => 'enable_pager=false', replace => false, } Friday, August 23, 13
  • 81. if $::enable_pager { @@file { "/etc/nagios/objects/${type}.d/${filename}.cfg": content => template('nagios/etc/nagios/object.erb'), mode => '0444', group => 'nagios', notify => Service['nagios'], } } Friday, August 23, 13
  • 82. Buy it? Friday, August 23, 13
  • 83. Buy it? Friday, August 23, 13
  • 84. It’s About Friday, August 23, 13
  • 85. What’s Missing? Friday, August 23, 13
  • 86. Language Friday, August 23, 13
  • 87. Predictive Analysis Friday, August 23, 13
  • 88. g in m o e C m o H Friday, August 23, 13
  • 89. What Data? Goals? Friday, August 23, 13
  • 90. p a c e R Systems Thinking Friday, August 23, 13
  • 91. There is ne hi ac M le ab rs Pa One System Must be a Self Authoritative Known Quantity Must self-correct Friday, August 23, 13
  • 92. Friday, August 23, 13
  • 93. jfryman fryman@github.com Friday, August 23, 13
  • 94. jfryman fryman@github.com Friday, August 23, 13
  • 95. what’s next? Friday, August 23, 13
  • 96. what’s next? Friday, August 23, 13

×