Downloaded 34 times
![class apache2 {
package { "apache2":
ensure => installed
}
service { "apache2":
enable => true,
ensure => running,
require => [ Package["apache2"] ]
}
}](https://image.slidesharecdn.com/burn-down-the-silos-110130055415-phpapp02/85/Burn-down-the-silos-Helping-dev-and-ops-gel-on-high-availability-websites-27-320.jpg)
![class capistrano::user {
group { "deploy":
gid => 499
}
user { "deploy":
uid => 499,
gid => 499,
home => "/home/deploy",
shell => "/bin/bash",
require => Group["deploy"]
}
file { "/home/deploy/.ssh/authorized_keys":
source => "puppet:///modules/capistrano/authorized_keys",
mode => 644,
owner => "deploy",
group => "deploy",
require => [ User["deploy"] ]
}
}](https://image.slidesharecdn.com/burn-down-the-silos-110130055415-phpapp02/85/Burn-down-the-silos-Helping-dev-and-ops-gel-on-high-availability-websites-63-320.jpg)
![define capistrano::site {
include capistrano::user
file { "/srv/$name":
ensure => directory,
owner => deploy,
group => deploy,
require => [ User["deploy"] ]
}
file { "/srv/$name/releases":
ensure => directory,
owner => deploy,
group => deploy,
require => [ File["/srv/$name"] ]
}
...](https://image.slidesharecdn.com/burn-down-the-silos-110130055415-phpapp02/85/Burn-down-the-silos-Helping-dev-and-ops-gel-on-high-availability-websites-64-320.jpg)
![sudo mmm_control show # blocks under high IO
echo -en “shownquitn” | nc 127.1 9988 # instant
socket = ::TCPSocket.new("127.0.0.1", 9988)
socket.print("shownquitn")
output = socket.read.split("n")
hosts = output.map do |line|
parts = line.scan(/nasty regex/).flatten
{ "hostname" => parts[0],
"address" => parts[1],
"mode" => parts[2],
"state" => parts[3],
"role" => parts[5],
"role_address" => parts[6] }
end](https://image.slidesharecdn.com/burn-down-the-silos-110130055415-phpapp02/85/Burn-down-the-silos-Helping-dev-and-ops-gel-on-high-availability-websites-108-320.jpg)
Uploaded byLindsay Holmwood
Burn down the silos! Helping dev and ops gel on high availability websites
The document discusses a DevOps case study focused on applying technologies for a fundraising website, emphasizing the need for consistency, repeatability, and visibility in deployment and configuration management. It highlights tools like Puppet and Capistrano for automating deployments and managing complex application environments while detailing the challenges and solutions around configuration drift and metrics collection. The case study illustrates the importance of collaboration between development and operations teams and incorporates real-world examples of deployment strategies and monitoring practices.
More Related Content
Similar to Burn down the silos! Helping dev and ops gel on high availability websites
Burn down the silos! Helping dev and ops gel on high availability websites
- 1. Burn Down TheSilos Lindsay Holmwood
- 2.
- 3.
- 4.
- 5.
- 6. Strong siloisation Devs +Ops in di"erent companies
- 7.
- 9.
- 10.
- 11.
- 12.
- 14.
- 15.
- 16.
- 17. or across multiple environments
- 19.
- 20.
- 22. Puppet language library client/server
- 23. package { "apache2": ensure => installed } service { "apache2": enable => true, ensure => running }
- 24. class apache2 { package { "apache2": ensure => installed } service { "apache2": enable => true, ensure => running } }
- 25. Puppet work#ow write apply debug
- 26. class apache2 { package { "apache2": ensure => installed } service { "apache2": enable => true, ensure => running } }
- 27. class apache2 { package { "apache2": ensure => installed } service { "apache2": enable => true, ensure => running, require => [ Package["apache2"] ] } }
- 28.
- 29.
- 30.
- 31.
- 33.
- 34.
- 35.
- 36. define app_server($collectd_destination, $logging_destination){ server { $fqdn: logging_destination => $logging_destination } include apache2 include mysql::client collectd::client { $fqdn: collection_destination => $collectd_destination } if $environment == "production" { include production::only::module } }
- 37. node "app-01.stage.charity.com" { app_server { $fqdn: collectd_destination => "stats-01.stage.charity.com", logging_destination => "log-01.stage.charity.com" } } node "app-01.production.charity.com" { app_server { $fqdn: collectd_destination => "stats-01.production.charity.com", logging_destination => "log-01.production.charity.com" } }
- 38. node /app-d+.stage.charity.com/ { app_server { $fqdn: collectd_destination => "stats-01.stage.charity.com", logging_destination => "log-01.stage.charity.com" } } node /app-d+.production.charity.com/ { app_server { $fqdn: collectd_destination => "stats-01.production.charity.com", logging_destination => "log-01.production.charity.com" } }
- 39.
- 40.
- 42.
- 43.
- 44.
- 45. increase speed byshortening feedback loops
- 47.
- 48.
- 50.
- 51.
- 52. Simple, powerful, can blowyour legs o"
- 53. Not a substitutefor con!guration management
- 54. railsless-deploy http://bit.ly/i56ra9
- 55.
- 56.
- 57. capistrano-multistage (part of capistrano-ext) http://bit.ly/i2moIp
- 58. # config/deploy.rb set :user, "deploy" set :application, "charity.com" set :keep_releases, 10 set :deploy_to, "/srv/#{application}" set :stages, %w(uat staging production)
- 59. # config/deploy/stage.rb role :app, "app-01.stage.charity.com", "app-02.stage.charity.com" role :static, "static-01.stage.charity.com”
- 60. # config/deploy/production.rb role :app, "app-01.prod.charity.com", "app-02.prod.charity.com", "app-03.prod.charity.com" role :static, "static-01.prod.charity.com”
- 61. cap staging deploy # deploy to staging # test cap production deploy # deploy to production
- 62.
- 63. class capistrano::user { group { "deploy": gid => 499 } user { "deploy": uid => 499, gid => 499, home => "/home/deploy", shell => "/bin/bash", require => Group["deploy"] } file { "/home/deploy/.ssh/authorized_keys": source => "puppet:///modules/capistrano/authorized_keys", mode => 644, owner => "deploy", group => "deploy", require => [ User["deploy"] ] } }
- 64. define capistrano::site { include capistrano::user file { "/srv/$name": ensure => directory, owner => deploy, group => deploy, require => [ User["deploy"] ] } file { "/srv/$name/releases": ensure => directory, owner => deploy, group => deploy, require => [ File["/srv/$name"] ] } ...
- 65. ... file { "/srv/$name/shared/log": ensure => directory, owner => www-data, group => www-data, } file { "/etc/$name": source => "puppet:///modules/capistrano/etc/$name", recurse => true, mode => "644", owner => root, group => root } }
- 66. define app_server($collectd_destination, $logging_destination){ include apache2 include mysql::client capistrano::site { “charity.com”: } }
- 67. Deploying to anew app server is as easy as:
- 68. # config/deploy/production.rb role :app, "app-01.prod.charity.com", "app-02.prod.charity.com", "app-03.prod.charity.com" role :static, "static-01.prod.charity.com”
- 69. # config/deploy/production.rb role :app, "app-01.prod.charity.com", "app-02.prod.charity.com", "app-03.prod.charity.com", "app-04.prod.charity.com" role :static, "static-01.prod.charity.com”
- 70.
- 71. # deploy/production.rb role :app, "app-01.prod.charity.com", "app-02.prod.charity.com", "app-03.prod.charity.com", "app-04.prod.charity.com" role :static, "static-01.prod.charity.com”
- 72. # deploy/production.rb role :app,*(1..4).map do |n| "app-%.2d.prod.charity.com" % n end role :static, "static-01.prod.charity.com”
- 74.
- 75. 182MB * 20== PAIN
- 76.
- 77. bad with svntags
- 78.
- 79. fast clones commitaccess 21st century tech
- 80.
- 82.
- 83. one eye onthe past
- 84. one eye onthe future
- 86.
- 87.
- 88.
- 89.
- 91.
- 92.
- 93. lightweight statistics collection daemon
- 94. platform for collecting time series data
- 95.
- 96.
- 97.
- 98.
- 99. <Plugin curl_json> <URL"http://localhost:5984/_stats"> Instance "httpd" <Key "httpd/requests/count"> Type "http_requests" </Key> <Key "httpd_status_codes/*/count"> Type "http_response_codes" </Key> </URL> </Plugin>
- 100.
- 102.
- 103. application & con!g management
- 105. Your new bestfriend
- 106.
- 107. sudo mmm_control show # blocks under high IO echo -en “shownquitn” | nc 127.1 9988 # instant
- 108. sudo mmm_control show # blocks under high IO echo -en “shownquitn” | nc 127.1 9988 # instant socket = ::TCPSocket.new("127.0.0.1", 9988) socket.print("shownquitn") output = socket.read.split("n") hosts = output.map do |line| parts = line.scan(/nasty regex/).flatten { "hostname" => parts[0], "address" => parts[1], "mode" => parts[2], "state" => parts[3], "role" => parts[5], "role_address" => parts[6] } end
- 110.
- 111. mk-query-digest & logrotate
- 112. # prerotate SLOWLOG_FILENAME="/var/log/mysql/mysql-slow.log" OPTIONS="--report --group-bydistill --order-by Query_time:max --timeline --report-format query_report,profile" DATE="$(date +%Y-%m-%dT%H:%M:%S%z)" REPORT_FILENAME="/tmp/$(hostname)-mysql-slow-query-report-$DATE" mk-query-digest $SLOWLOG_FILENAME $OPTIONS > $REPORT_FILENAME SUBJECT="MySQL Slow Queries Report for $(hostname) as of $DATE" RECIPIENTS="developers@charity.com,ops@bulletproof.net" cat $REPORT_FILENAME | nail -n -E -s "$SUBJECT" "$RECIPIENTS"
- 114.
- 115.
- 116.
- 117.
- 118.
- 119.
- 120. REPLICATION_FAIL on one slave
- 121. Down to 3nodes
- 122.
- 123. REPLICATION_DELAY on another slave
- 124. Down to 2nodes
- 125.
- 126. Swapping like mad Halfthe memory allocated
- 127. Shutdown, upgrade, boot
- 128.
- 129.
- 130.
- 132.
- 133.
- 134.
- 135. Con!g parses +loads
- 136.
- 137.
- 138.
- 139.
- 140.
- 141.
- 142. 2 reviewers of con!gmanagement
- 143. both in opsteam
- 144.
- 145.
- 146. Your new bestfriend
- 147. reviewer diversity devs shouldhave visibility of ops changes
- 149.
- 150.
- 151.
- 152.
- 153.
- 154. uat ! stage production
- 155. uat ! stage ! production
- 156. uat ! stage ! production "
- 157. uat ! stage ! production "
- 158. CREATE TABLE foo should have been CREATE TABLE IF NOT EXIST foo
- 159.
- 160.
- 161.
- 162.
- 163.
- 165.
- 166.
- 167.
- 168. or across multiple environments
- 169.
- 170.
- 171.
- 172. increase speed byshortening feedback loops
- 173.
- 174. one eye onthe past
- 175. one eye onthe future
- 176.
- 177. Thank you! Credits: http://www.#ickr.com/photos/48722974@N07/4682302824/ http://www.#ickr.com/photos/lyza/4144764381/ http://www.#ickr.com/photos/acediscovery/3030548744/ http://www.#ickr.com/photos/matchew/424026531/ http://www.#ickr.com/photos/andrew_wertheimer/5268407700/ http://www.#ickr.com/photos/mrwoodnz/4289893182/ http://www.#ickr.com/photos/azrasta/4528604334/ http://www.#ickr.com/photos/myprofe/4396178084/ http://www.#ickr.com/photos/boliston/2351083198/ http://www.#ickr.com/photos/nnova/4834954885/ http://www.#ickr.com/photos/brianwestcott/1497708345/ http://www.#ickr.com/photos/pjern/2150874047/ http://www.#ickr.com/photos/brunogirin/73014722/ http://www.#ickr.com/photos/rubodewig/5161937181/ http://www.#ickr.com/photos/eole/4500783172/ http://www.#ickr.com/photos/rutty/460520720/ http://www.#ickr.com/photos/jacockshaw/1811056252/ http://www.#ickr.com/photos/sarah_lincoln/4740037328/ http://www.#ickr.com/photos/jenny-pics/2719309611/ http://www.#ickr.com/photos/shindotv/3835365695/ http://www.#ickr.com/photos/ldsykora/2414497811/ http://www.#ickr.com/photos/thalamus/306881919/ http://www.#ickr.com/photos/listed_crime/1342164481/ http://www.#ickr.com/photos/traviscrawford/323366600/ http://www.#ickr.com/photos/localsurfer/369116556/ http://www.#ickr.com/photos/webtreatsetc/5303216304/