Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Effective Devops - Collaboration and Tools - Velocity Santa Clara 2015

1,439 views

Published on

DevOps is a professional and cultural movement designed to improve the processes involved with developing and running computer software and systems. It does this by focusing on communication and collaboration. DevOps has the potential to improve the products we create as an industry, and the health and well-being of the people creating them.

In this one-day introductory training, you will identify actionable strategies and tools that can be used to leverage DevOps to implement noticeable, long-lasting improvements to your culture and development processes in your environment, regardless of your level and role.

Published in: Technology
  • Be the first to comment

Effective Devops - Collaboration and Tools - Velocity Santa Clara 2015

  1. 1. Effective Devops: Collaboration and Tools Jennifer Davis & Katherine Daniels 1
  2. 2. Jennifer Davis • Automation Engineer, Chef • Co-author of "Effective Devops" • DevOpsDays SV Organizer • Sparkly Devops Princess • CoffeeOps Organizer 2
  3. 3. Katherine Daniels • Web Operations Engineer, Etsy • Co-author of "Effective Devops" • DevopsDays NYC Organizer • Ladies Who Linux NYC Organizer • Ship Show Podcast Co-host 3
  4. 4. Communication • Jennifer Davis Twitter: @sigje Email: iennae@gmail.com • Katherine Daniels Twitter: @beerops Email: sparklyyakshaver@gmail.com #EffectiveDevops 4
  5. 5. Feedback • Constructive feedback • What did you find helpful? • What would you like to see more/less of? • Was there anything you found unclear? 5
  6. 6. Schedule • Introduction to teams, devops principles • 10:30-11am Morning Break • Visualization of work, Git, Infrastructure automation • 12:30-1:30pm Lunch 6
  7. 7. Schedule • Testing infrastucture automation and other changes • 3-3:30pm Afternoon Break • Measuring, monitoring, and wrap-up 7
  8. 8. Network Connectivity Network Name: Conference_Private Access Code: velocity2015 8
  9. 9. Expectations • Safe space to share experiences, learn from each other • Code of Conduct • Learn effective workflows for using and testing source control and configuration management 9
  10. 10. Team Introductions • Meet your team! • Identify your team's... • Speaker • Gatekeeper • Notetaker Time: 10 minutes 10
  11. 11. 11
  12. 12. What is Devops • Technical cultural weave that shapes how we work, and why 12
  13. 13. Folk Models • general popularly understood meaning particular to a socio-cultural grouping but which has not been formally defined or standardized. 13
  14. 14. Why Devops? 14
  15. 15. High Performing Devops Teams are more agile 30X more frequent deployments 8000X faster lead times than peers 2014 PuppetLabs State of DevOps Survey 15
  16. 16. High Performing Devops Teams are more reliable 2X change success rate 12X faster mean time to recovery (MTTR) 2014 PuppetLabs State of DevOps Survey 16
  17. 17. Five Pillars of Effective Devops • Collaboration • Hiring • Affinity • Tools • Scaling 17
  18. 18. Collaboration • Individuals Working Together 18
  19. 19. Hiring • Choosing Individuals 19
  20. 20. Affinity • From Individuals to Teams 20
  21. 21. Tools • Choosing and Using them 21
  22. 22. Scaling • Growing and Decreasing 22
  23. 23. Collaboration and Tools 23
  24. 24. Recognizing your Devops Narrative 24
  25. 25. The Devops Compact • shared mutual understanding • established boundaries 25
  26. 26. Stormtrooper Syndrome • Agency • Adaptability: Role adherence 26
  27. 27. Learned Helplessness 27
  28. 28. A life becomes meaningful when one sees himself or herself as an actor within the context of a story. -- George Howard 28
  29. 29. Swift Trust • Skillful self disclosure • What you share, how you share it • Collective tasks vs personal agendas 29
  30. 30. Team • Common purpose • Defined beliefs • Empowered 30
  31. 31. Small vs Large teams • Large teams - roles may be highly segregated • Small teams - one person may be responsible for many roles 31
  32. 32. Cultivating Empathy • Collect stories • Listen • Circle back 32
  33. 33. Smarter Teams Build Better Value1 • Lots of Communication • Contribute equally to team's discussions • Theory of Mind • Increased diversity 1 Engel, David et al. 'Reading The Mind In The Eyes Or Reading Between The Lines? Theory Of Mind Predicts Collective Intelligence Equally Well Online And Face-To-Face'. PLoS ONE 9.12 (2014): e115212. Web. 26 May 2015. 33
  34. 34. Critical Habits for Teams • Code Review • Pairing 34
  35. 35. Code Review • Max 90 minutes in one setting 35
  36. 36. Pairing • Agile software development • 2 people work together on 1 workstation • Driver - writes code • Observer - reviews each line • Roles switch frequently 36
  37. 37. Types of Pairing • Expert-expert • Expert-novice • Novice-novice 37
  38. 38. Discussion: Team Intro • What are motivations? • What are current beliefs? • What are current skills? Gaps in skills? Time: 15 minutes 38
  39. 39. Share: Team Intro • Team Name: • Common Motivations? • Skills? Gaps? 39
  40. 40. Fluxx • Game about change. • Rules change as you play. • Goal changes frequently. 40
  41. 41. Fluxx • Starts out with 1 Rule - Draw 1, Play 1 • Find this card (white backing) • Place card in the middle of table 41
  42. 42. Fluxx • Shuffle deck • Deal 3 cards to each player 42
  43. 43. Fluxx • First player is the person to left of Dealer. • Start out following the Draw 1, Play 1 rule until overridden by new rules. 43
  44. 44. Fluxx 4 types of cards • Goal - pink • Keeper - green played in front of you • Action - blue used once and discarded • Rule - yellow 44
  45. 45. Fluxx - On your turn • Draw the number of cards currently required. • Play the number of cards currently required. • Discard down to hand limit (if any hand limit). • Discard down to keeper limit (if any keeper limit). 45
  46. 46. Fluxx • Winner is the first to meet the current Goal condition. Time: 15 minutes 46
  47. 47. Retrospective Change is inevitable. 47
  48. 48. Importance of Games • Build resilience • Build collaboration (or competition) • Roles change 48
  49. 49. Specific to Fluxx • Frustration of constant changing goals • Completed "work", lost value • Visualization of goals, everyone in alignment 49
  50. 50. Visualization of Work • Bug/issue queue • Kanban 50
  51. 51. Kanban • Start with what you do now • Agree to incremental, evolutionary change • Respect • Everyone is a leader 51
  52. 52. Kanban Practices • Visualize • Limit WIP • Manage flow • Make policies explicit • Implement feedback loops 52
  53. 53. Visualize • Intent • Alignment • Coherence 53
  54. 54. Limit WIP • Pull (don't push) 54
  55. 55. Manage Flow • Monitor/measure/report • Incremental change 55
  56. 56. Make Policies Explicit • Document processes • Group signoff 56
  57. 57. Implement Feedback Loops • Collaboration • Retrospectives 57
  58. 58. Intro JoeNGo 58
  59. 59. Application Deployment Planning Deming Cycle • Plan: Identify and Analyze problem • Do: Develop and Test potential solution • Check: Measuring effectiveness • Act: Implement solution 59
  60. 60. Application Deployment Planning • (L)inux • (A)pache • (M)ySQL • (P)HP, Perl, or Python 60
  61. 61. Group Discussion Time: 15 minutes 61
  62. 62. Devops Tools • Establish local development environment • Version control • Manual -> Automation -> Continuous • Artifacts • Infrastructure • Sandbox 62
  63. 63. Local Development Environment (LDE) • Consistent set of tools across the team • Ability to quickly onboard new engineers 63
  64. 64. Provisioned Node - LDE • AWS instance node • Chef DK • Test Kitchen • Ruby • ChefSpec, ServerSpec • Git 64
  65. 65. Nodes Countoff http://bit.ly/1Fg9rxs 65
  66. 66. Configuration Management • Process of identifying, managing, monitoring, and auditing a product through its entire life including the processes, documentation, people, tools, software, and systems. 66
  67. 67. Version Control • Records changes to files or sets of files stored within the system • Enable revisions • Integrity checking • Collaboration 67
  68. 68. Artifact Repository • Secure • Trusted • Stable • Accessible • Versioned 68
  69. 69. Introduction to Git and Workflows 69
  70. 70. Isolated Development Environments • Not automatically updated • Manually pull upstream commits 70
  71. 71. Workspace Environment $ git init $ git add . adds all files and directories to version control 71
  72. 72. Commit $ git commit -m "message about commit" -a 72
  73. 73. All these changes are local! How do you collaborate? 73
  74. 74. Show all remotes $ git remote -v 74
  75. 75. Add remote server $ git remote add NAME URL 75
  76. 76. Update changes to remote directory $ git push REPONAME BRANCH 76
  77. 77. View the branches $ git branch -a 77
  78. 78. Creating the branches $ git checkout -b BRANCHNAME 78
  79. 79. Merging branches $ git merge BRANCH 79
  80. 80. git fetch • import commits, from remote to local git checkout master git log origin/master git merge origin/master 80
  81. 81. Different workflows • feature branch • gitflow • forking • centralized 81
  82. 82. Pull Requests • collaboration prior to integration • discussion • follow up commits • selfie gifs3 3 https://github.com/thieman/github-selfies 82
  83. 83. git pull git pull REMOTE 83
  84. 84. git pull --rebase • ensure linear history by preventing unnecessary merge commits 84
  85. 85. git push remote branch • transfer commits from a local repo to a remote repo. • counterpart to git fetch 85
  86. 86. 86
  87. 87. Assignment 1 Effective Devops http://bit.ly/1JVuLh4 Time: 15 minutes 87
  88. 88. 88
  89. 89. Infrastructure • Aggregate of applications, configurations, access control, data, compute nodes, network, storage, processes, and people. 89
  90. 90. Infrastructure Automation • Systems that reduce the burden on people to manage services and increase the quality, accuracy and precision of a service to the consumers of a service 90
  91. 91. Infrastructure Automation Tools • Chef • Puppet • Ansible • Salt • CFEngine 91
  92. 92. Introduction to Chef 92
  93. 93. Resources • Ingredients of infrastructure • Basic building blocks 93
  94. 94. Resource Declaration RESOURCETYPE "RESOURCE_NAME" do PARAMETER PARAMETER_VALUE end 94
  95. 95. Example Resource Type - package A package to be installed: package "httpd" do action :install end 95
  96. 96. Example Resource Type - service A service that should be started: service "httpd" do supports :restart => :true action [:enable, :start] end 96
  97. 97. Resources A resource is a statement of policy that: • Describes the desired state for an element • Specifies a resource type---such as package, template, or service • Lists additional details (also known as parameters), as necessary • Are grouped into recipes 97
  98. 98. Recipes • Collection of ordered resources • Combination of ruby and Chef DSL 98
  99. 99. Cookbooks • Thematic • Collection of recipes and other supporting files 99
  100. 100. Roles • Abstraction describing function of system • Name • Description • Run list (ordered list of recipes and roles) 100
  101. 101. Run List • Ordered list of recipes and roles • Specific to a node 101
  102. 102. Nodes • Machine (virtual, physical, cloud server, or other device) that is managed by Chef 102
  103. 103. Environments • Abstraction models workflow • Name • Description • Cookbook version pinning 103
  104. 104. Supermarket • Community site with a number of cookbooks • Read before using in your environment 104
  105. 105. Chef DK • Chef development kit • Includes a number of utilities and software to facilitate cookbook creation • Free download off of the website 105
  106. 106. Berkshelf • Dependency management • Included with Chef DK 106
  107. 107. Test Kitchen • Included with Chef DK • Sandbox automation • Test harness 107
  108. 108. Test Kitchen • Execute code on one or more platforms • Driver plugins supporting various cloud and virtualization providers 108
  109. 109. .kitchen.yml • driver • provisioner • platforms • suites 109
  110. 110. .kitchen.yml driver • virtualization or cloud provider Example: vagrant, docker 110
  111. 111. .kitchen.yml provisioner • application to configure the node Example: chef_zero 111
  112. 112. .kitchen.yml platforms • target operating systems Example: centos-6.5 112
  113. 113. .kitchen.yml suites • target configurations Example: name:default run_list: - recipe[apache::default] attributes: 113
  114. 114. Kitchen commands (1/2) • kitchen init • kitchen list • kitchen create • kitchen converge 114
  115. 115. Kitchen commands (2/2) • kitchen verify • kitchen destroy • kitchen test 115
  116. 116. Docker • Images • Registries • Containers 116
  117. 117. 117
  118. 118. Assignment 2 Time: 20 minutes 118
  119. 119. 119
  120. 120. Managing Risk • Test • Small frequent releases 120
  121. 121. Linting • Ensure code adheres to styles and conventions • Weave expectations into development • Encourages collaboration 121
  122. 122. Testing • Documenting objectives and intent • Measuring "done" 122
  123. 123. Code Correctness • foodcritic • rubocop 123
  124. 124. Integration Tests • ServerSpec 124
  125. 125. Rubocop • Ruby linter • Ruby style guide • Included with ChefDK 125
  126. 126. Rubocop Example $ rubocop cookbooks/COOKBOOK1 cookbooks/COOKBOOK2 cookbooks/COOKBOOK4 126
  127. 127. Reading Rubocop Output Inspecting 8 files CWCWCCCC • . means that the file contains no issues • C means a issue with convention • W means a warning • E means an error • F means an fatal error 127
  128. 128. Disabling Rubocop cops Any configuration in .rubocop.yml is disabled. To disable string literals: StringLiterals: Enabled: false 128
  129. 129. Foodcritic • Chef linter • Chef style guide • Included with ChefDK 129
  130. 130. Foodcritic Example $ foodcritic cookbooks/setup 130
  131. 131. Reading Foodcritic Output FC008: Generated cookbook metadata needs updating: ./metadata.rb:2 131
  132. 132. ServerSpec • Tests to verify servers functionality • Resource types • Package, service, user, and many others • Integrates with Test Kitchen • http://serverspec.org 132
  133. 133. ServerSpec Generic Form describe "<subject>" do it "<description>" do expect(thing).to eq result end end 133
  134. 134. ServerSpec Potential Tests • Is the service running? • Is the port accessible? • Is the expected content being served? 134
  135. 135. ServerSpec Example describe 'apache' do it "is installed" do expect(package 'httpd').to be_installed end it "is running" do expect(service 'httpd').to be_running end end 135
  136. 136. Reading ServerSpec Output app::default httpd service is running Finished in 0.26429 seconds (files took 0.7166 seconds to load) 1 example, 0 failures 136
  137. 137. 137
  138. 138. Assignment 3 Time: 10 minutes 138
  139. 139. 139
  140. 140. Retrospective on Assignment 3 Time: 15 minutes 140
  141. 141. 141
  142. 142. Assignment 4 Time: 20 minutes 142
  143. 143. 143
  144. 144. Automated Test and Build 144
  145. 145. Test, Monitor, or Diagnostic2 1. Where is it going to run? 2. When is it going to run? 3. How often will it run? 4. Who is going to consume the result? 5. What is the entity going to do with it? 2 Lam, Yvonne. 'Sysadvent: Day 5 - How To Talk About Monitors, Tests, And Diagnostics'. Sysadvent.blogspot.com. N.p., 2014. Web. 26 May 2015. 145
  146. 146. Measuring Impact and Value of Change 146
  147. 147. Impact of Change 147
  148. 148. Impact on Availability • Overall site/app availability • Individual service availability 148
  149. 149. Availability Monitoring • Uptime: • Pingdom, Monitis, Uptrends, etc • Vertical Line Technology: • Availability after deploys/changes 149
  150. 150. 150
  151. 151. Eventinator 151
  152. 152. 152
  153. 153. Service Availability • Nagios: Service-level monitoring and alerting • Nagios-herald: Alert context • OpsWeekly: Historical alert data 153
  154. 154. Nagios 154
  155. 155. define command { command_name check_mongodb_query command_line $USER1$/nagios-plugin-mongodb/check_mongodb.py -H $HOSTADDRESS$ -A $ARG1$ -P $ARG2$ -W $ARG3$ -C $ARG4$ -q $ARG5$ } define service { use generic-service hostgroup_name Mongo Servers service_description Mongo Connect Check check_command check_mongodb!connect!27017!2!4 } 155
  156. 156. define servicedependency{ host_name WWW1 service_description Apache Web Server dependent_host_name WWW1 dependent_service_description Main Web Site execution_failure_criteria n notification_failure_criteria w,u,c } 156
  157. 157. Nagios-herald 157
  158. 158. 158
  159. 159. OpsWeekly 159
  160. 160. 160
  161. 161. 161
  162. 162. 162
  163. 163. 163
  164. 164. Impact on Quality • Service quality (SLAs) • Visibility of quality 164
  165. 165. Statsd 165
  166. 166. >>> import statsd >>> >>> timer = statsd.Timer('MyApplication') >>> >>> timer.start() >>> # do something here >>> timer.stop('SomeTimer') 166
  167. 167. >>> import statsd >>> >>> counter = statsd.Counter('MyApplication') >>> # do something here >>> counter += 1 167
  168. 168. >>> import statsd >>> >>> average = statsd.Average('MyApplication', connection) >>> # do something here >>> average.send('SomeName', 'somekey:%d'.format(value)) 168
  169. 169. Graphite 169
  170. 170. Value of Change 170
  171. 171. Value of Availability • Better for customers • Better for employees (internal services) • Fewer pages 171
  172. 172. Value of Quality • Deploys take less time • Also better for customers • More visibility into issues 172
  173. 173. 173
  174. 174. Assignment 5 Time: 20 minutes 174
  175. 175. 175
  176. 176. Retrospective ! 176
  177. 177. Review • Recognizing your Devops Narrative • Application Deployment Planning • Infrastructure as code • Introducing repeatable, testable change • Measuring impact and value of change 177
  178. 178. Next Steps • Manual, Automation to Continuous "X" • Be the storylistener and storyteller in your org • Effective Devops available in Early Release 178
  179. 179. Slides 179
  180. 180. Thank you! ❤ @sigje @beerops 180
  181. 181. 181

×