Successfully reported this slideshow.

Operations: a developer's guide

5

Share

Loading in …3
×
1 of 185
1 of 185

Operations: a developer's guide

5

Share

Download to read offline

Slides from my http://2015.ffconf.org/ talk.

Links on the last page:

Configuration management
https://www.scriptrock.com/articles/the-7-configuration-management-tools-you-need-to-know
http://gettingstartedwithchef.com/first-steps-with-chef.html
https://docs.vagrantup.com/v2/getting-started/provisioning.html

Virtualisation
http://searchvirtualdatacentre.techtarget.co.uk/definition/Virtualisation
http://searchservervirtualization.techtarget.com/definition/server-virtualization
http://www.infoworld.com/article/2621446/server-virtualization/server-virtualization-top-10-benefits-of-server-virtualization.html

Using Vagrant
https://www.vagrantup.com/
http://blog.bennycornelissen.nl/otto-a-modern-developers-new-best-friend/
https://github.com/patrickdlee/vagrant-examples (Useful getting started examples)

Docker
http://patg.net/containers,virtualization,docker/2014/06/05/docker-intro/
https://zeltser.com/security-risks-and-benefits-of-docker-application/

Containerisation vs Virtualisation
http://www.slideshare.net/bcantrill/docker-and-the-future-of-containers-in-production
https://www.scriptrock.com/articles/docker-vs-vagrant

Make instead of Grunt/Gulp
http://blog.keithcirkel.co.uk/why-we-should-stop-using-grunt/
https://www.youtube.com/watch?v=0RYETb9YVrk (Talk on using NPM as a build tool)
https://blog.jcoglan.com/2014/02/05/building-javascript-projects-with-make/

Tools for better dev
http://www.leancrew.com/all-this/2011/12/more-shell-less-egg/ (More detail on the 6-line Unix program)

Slides from my http://2015.ffconf.org/ talk.

Links on the last page:

Configuration management
https://www.scriptrock.com/articles/the-7-configuration-management-tools-you-need-to-know
http://gettingstartedwithchef.com/first-steps-with-chef.html
https://docs.vagrantup.com/v2/getting-started/provisioning.html

Virtualisation
http://searchvirtualdatacentre.techtarget.co.uk/definition/Virtualisation
http://searchservervirtualization.techtarget.com/definition/server-virtualization
http://www.infoworld.com/article/2621446/server-virtualization/server-virtualization-top-10-benefits-of-server-virtualization.html

Using Vagrant
https://www.vagrantup.com/
http://blog.bennycornelissen.nl/otto-a-modern-developers-new-best-friend/
https://github.com/patrickdlee/vagrant-examples (Useful getting started examples)

Docker
http://patg.net/containers,virtualization,docker/2014/06/05/docker-intro/
https://zeltser.com/security-risks-and-benefits-of-docker-application/

Containerisation vs Virtualisation
http://www.slideshare.net/bcantrill/docker-and-the-future-of-containers-in-production
https://www.scriptrock.com/articles/docker-vs-vagrant

Make instead of Grunt/Gulp
http://blog.keithcirkel.co.uk/why-we-should-stop-using-grunt/
https://www.youtube.com/watch?v=0RYETb9YVrk (Talk on using NPM as a build tool)
https://blog.jcoglan.com/2014/02/05/building-javascript-projects-with-make/

Tools for better dev
http://www.leancrew.com/all-this/2011/12/more-shell-less-egg/ (More detail on the 6-line Unix program)

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Operations: a developer's guide

  1. 1. Operations: a developer’s guide Anna Shipman @annashipman
  2. 2. @annashipman Technical architect at the Government Digital Service
  3. 3. @annashipman My background
  4. 4. @annashipman Previous career in publishing
  5. 5. @annashipman Self taught programmer: HTML, CSS, terrible JS
  6. 6. @annashipman First employed gig, 2005: backend, Java
  7. 7. @annashipman First job at GDS, 2012: redirecting URLs
  8. 8. @annashipman
  9. 9. @annashipman Wrote perl to generate nginx config
  10. 10. @annashipman Didn’t understand the emails from the infrastructure team
  11. 11. @annashipman So joined the infrastructure team
  12. 12. @annashipman Which led to where I am now…
  13. 13. @annashipman Technical architect on a large infrastructure project
  14. 14. @annashipman What I learned has made me a better developer
  15. 15. @annashipman Knowing these things might help you
  16. 16. @annashipman Wrangling servers Virtualisation Containerisation Some tools to make you a better developer
  17. 17. @annashipman Each section will have a take-home
  18. 18. @annashipman 1. Wrangling servers
  19. 19. @annashipman How the internet works Wrangling servers
  20. 20. @annashipman “It’s a series of tubes” —Sen. Ted Stevens Wrangling servers
  21. 21. @annashipman Servers Wrangling servers
  22. 22. @annashipman Where are your servers? Wrangling servers
  23. 23. @annashipman Where are your servers? You own them (e.g. in the office) Wrangling servers
  24. 24. @annashipman Where are your servers? You own them (e.g. in the office) Shared hosting (e.g. Dreamhost) Wrangling servers
  25. 25. @annashipman Where are your servers? You own them (e.g. in the office) Shared hosting (e.g. Dreamhost) The cloud (e.g. AWS) Wrangling servers
  26. 26. @annashipman Where are your servers? You own them (e.g. in the office) Shared hosting (e.g. Dreamhost) The cloud (e.g. AWS) PaaS/application hosting (e.g. Heroku) Wrangling servers
  27. 27. @annashipman Where are your servers? You own them (e.g. in the office) Shared hosting (e.g. Dreamhost) The cloud (e.g. AWS) PaaS/application hosting (e.g. Heroku) Something else/don’t know Wrangling servers
  28. 28. @annashipman You need to make sure the server has the software you need Wrangling servers
  29. 29. @annashipman Handcrafting servers Wrangling servers
  30. 30. @annashipman But what happens if your server dies? Wrangling servers
  31. 31. @annashipman It’s also easy to make a mistake Wrangling servers
  32. 32. @annashipman Configuration management Wrangling servers
  33. 33. @annashipman Tools that use configuration you’ve written to build servers Wrangling servers
  34. 34. https://www.getfilecloud.com/blog/2014/08/top-8-configuration-management-tools-for-sys-admins/
  35. 35. @annashipman GOV.UK uses Puppet, my current project uses Ansible Wrangling servers
  36. 36. @annashipman Config management tools automate building your servers Wrangling servers
  37. 37. @annashipman So it is reliable and repeatable Wrangling servers
  38. 38. @annashipman Some getting started guides at the end Wrangling servers
  39. 39. @annashipman If nothing else, just write a script Wrangling servers
  40. 40. @annashipman “Cattle not pets” Wrangling servers
  41. 41. @annashipman You should not be afraid to lose your servers Wrangling servers
  42. 42. @annashipman 2. Virtualisation
  43. 43. @annashipman Creating logical computing resources from available physical resources Virtualisation
  44. 44. @annashipman Virtual machines Virtualisation
  45. 45. @annashipman What’s a hypervisor? Virtualisation
  46. 46. @annashipman Hypervisor is the software that runs the VMs Virtualisation
  47. 47. @annashipman
  48. 48. @annashipman Slight digression into cloud computing Virtualisation
  49. 49. @annashipman “Someone else’s computers” Virtualisation
  50. 50. @annashipman Your servers are in a data centre Virtualisation
  51. 51. @annashipman A layer of abstraction that pools the resources Virtualisation
  52. 52. @annashipman
  53. 53. @annashipman Some advantages of cloud computing Virtualisation
  54. 54. @annashipman Some advantages of cloud computing Increased uptime & disaster recovery Virtualisation
  55. 55. @annashipman Some advantages of cloud computing Increased uptime & disaster recovery Can get a VM straight away Virtualisation
  56. 56. @annashipman Some advantages of cloud computing Increased uptime & disaster recovery Can get a VM straight away Charged for what you use Virtualisation
  57. 57. @annashipman That’s the cloud – now your computer Virtualisation
  58. 58. @annashipman How is virtualisation useful to you? Virtualisation
  59. 59. @annashipman Used to be software = slow, resource-intensive Virtualisation
  60. 60. @annashipman From 2005 Intel & AMD started doing hardware- accelerated virtualisation Virtualisation
  61. 61. @annashipman So can run lots of VMs on your computer Virtualisation
  62. 62. @annashipman Vagrant is a lightweight way to create VMs Virtualisation
  63. 63. @annashipman Vagrant.configure(2) do |config| config.vm.box = "puppetlabs/centos-7.0-64-puppet" config.vm.provider :virtualbox |vb| vb.customize ["modifyvm", :id, "--memory", "2048"] end end Virtualisation
  64. 64. @annashipman Vagrant.configure(2) do |config| config.vm.box = "ubuntu/trusty64" end Virtualisation
  65. 65. @annashipman $ vagrant up Virtualisation
  66. 66. @annashipman $ vagrant ssh Virtualisation
  67. 67. @annashipman $ cd /vagrant Virtualisation
  68. 68. @annashipman Links to documentation and simple examples at the end Virtualisation
  69. 69. @annashipman You can develop locally using the same software as is deployed remotely Virtualisation
  70. 70. @annashipman You can use configuration management to build your VM Virtualisation
  71. 71. @annashipman It makes it easy to collaborate Virtualisation
  72. 72. @annashipman Vagrant makes it easy to get started/pick up where you left off Virtualisation
  73. 73. @annashipman Currently Otto uses Vagrant Virtualisation
  74. 74. @annashipman Vagrant can improve your project tomorrow Virtualisation
  75. 75. @annashipman 3. Containerisation
  76. 76. @annashipman (Docker is not the only containerisation tech) Containerisation
  77. 77. @annashipman Docker is based on Linux Containers 1. Put netting on garlic and put back outside Containerisation
  78. 78. @annashipman 2008: LXC 2004: Solaris zones 2000: BSD jails 1982: chroot (tech based on) Containerisation
  79. 79. @annashipman You can run multiple containers Containerisation
  80. 80. @annashipman Containers are isolated from each other Containerisation
  81. 81. @annashipman Why would you use containers? Containerisation
  82. 82. @annashipman Configuring a VM can be slow Containerisation
  83. 83. @annashipman You might instead decide to take a snapshot Containerisation
  84. 84. @annashipman And that is what you deploy to production Containerisation
  85. 85. @annashipman Containers are immutable Containerisation
  86. 86. @annashipman So you can be sure what you’ve tested is what is running in production Containerisation
  87. 87. @annashipman With all the same dependencies 1. Put netting on garlic and put back outside Containerisation
  88. 88. @annashipman However… Containerisation
  89. 89. @annashipman The immutability can make developing locally difficult Containerisation
  90. 90. @annashipman You can’t run them everywhere Containerisation
  91. 91. @annashipman Possibly a harder concept to grasp Containerisation
  92. 92. @annashipman Right now, it’s not essential that you understand Docker unless you are interested Containerisation
  93. 93. @annashipman 2 + 3: Virtualisation + Containerisation Turtles
  94. 94. @annashipman Locally, it’s Vagrant or Docker Turtles
  95. 95. @annashipman But if you deploy to the cloud… Turtles
  96. 96. @annashipman
  97. 97. @annashipman How do you know where the problem is? Turtles
  98. 98. @annashipman It’s turtles all the way down Turtles
  99. 99. @annashipman BREAK
  100. 100. @annashipman Wrangling servers “Cattle not pets”
  101. 101. @annashipman Virtualisation Use Vagrant
  102. 102. @annashipman Containerisation You don’t need to learn Docker right now
  103. 103. @annashipman 4. Some tools to make you a better developer
  104. 104. @annashipman “Should all my tools be written in JavaScript (warning: if the answer is ‘no’, there might be a revolution!!!)” —Remy Tools
  105. 105. @annashipman Build systems – Grunt, Gulp… Tools
  106. 106. @annashipman Don’t just work out of the box Tools
  107. 107. @annashipman Dependencies and plug-ins Tools
  108. 108. @annashipman You could use NPM Tools
  109. 109. @annashipman Make Tools
  110. 110. @annashipman Can create tasks to do anything: clean, compile, build… Tools
  111. 111. @annashipman Advantages of Make Tools
  112. 112. @annashipman Advantages of Make Dependency tracking and resolution Tools
  113. 113. @annashipman Advantages of Make Dependency tracking and resolution Only does what it needs to Tools
  114. 114. @annashipman all: styles.css print.css styles.css: styles.scss sass styles.scss:styles.css print.css: print.scss sass print.scss:print.css Tools
  115. 115. @annashipman all: styles.css print.css styles.css: styles.scss sass styles.scss:styles.css print.css: print.scss sass print.scss:print.css Compiles your SCSS to CSS Tools
  116. 116. @annashipman all: styles.css print.css styles.css: styles.scss sass styles.scss:styles.css print.css: print.scss sass print.scss:print.css This is what Grunt, Gulp etc are doing Tools
  117. 117. @annashipman all: styles.css print.css styles.css: styles.scss sass styles.scss:styles.css print.css: print.scss sass print.scss:print.css $ make styles.css Tools
  118. 118. @annashipman all: styles.css print.css styles.css: styles.scss sass styles.scss:styles.css print.css: print.scss sass print.scss:print.css Only runs if SCSS is newer than CSS Tools
  119. 119. @annashipman all: styles.css print.css styles.css: styles.scss sass styles.scss:styles.css print.css: print.scss sass print.scss:print.css $ make print.css Tools
  120. 120. @annashipman all: styles.css print.css styles.css: styles.scss sass styles.scss:styles.css print.css: print.scss sass print.scss:print.css $ make all Tools
  121. 121. @annashipman all: styles.css print.css styles.css: styles.scss sass styles.scss:styles.css print.css: print.scss sass print.scss:print.css $ make all Tools
  122. 122. @annashipman all: styles.css print.css styles.css: styles.scss sass styles.scss:styles.css print.css: print.scss sass print.scss:print.css $ make Tools
  123. 123. @annashipman all: styles.css print.css styles.css: styles.scss sass styles.scss:styles.css print.css: print.scss sass print.scss:print.css If print.css doesn’t need updating, only runs styles.css Tools
  124. 124. @annashipman Advantages of Make Dependency tracking and resolution Only does what it needs to Tools
  125. 125. @annashipman Advantages of Make Dependency tracking and resolution Only does what it needs to Included in your OS* *Linux, Unix, MacOS Tools
  126. 126. @annashipman Has everything you need Tools
  127. 127. @annashipman It just isn’t written in JS Tools
  128. 128. @annashipman Unix tools Tools
  129. 129. @annashipman grep Tools
  130. 130. @annashipman grep Search given input files for patterns Tools
  131. 131. @annashipman cat Tools
  132. 132. @annashipman cat Concatenate and print files Tools
  133. 133. @annashipman awk Tools
  134. 134. @annashipman awk Scan input files for patterns; perform action Tools
  135. 135. @annashipman And the most useful: man Tools
  136. 136. @annashipman man format and display manual pages Tools
  137. 137. @annashipman $ man grep Tools
  138. 138. @annashipman $ man man Tools
  139. 139. @annashipman Being unixy Tools
  140. 140. @annashipman Each tool does one thing very well Tools
  141. 141. @annashipman Composable Tools
  142. 142. @annashipman | Tools
  143. 143. @annashipman | (Pipe) Tools
  144. 144. @annashipman Read a file of text, determine the n most frequently used words, and print out a sorted list of those words along with their frequencies. Tools
  145. 145. @annashipman 1 tr -cs A-Za-z 'n' | 2 tr A-Z a-z | 3 sort | 4 uniq -c | 5 sort -rn | 6 sed ${1}q Tools
  146. 146. @annashipman 1 tr -cs A-Za-z 'n' | 2 tr A-Z a-z | 3 sort | 4 uniq -c | 5 sort -rn | 6 sed ${1}q Tools
  147. 147. @annashipman 1 tr -cs A-Za-z 'n' | 2 tr A-Z a-z | 3 sort | 4 uniq -c | 5 sort -rn | 6 sed ${1}q Tools
  148. 148. @annashipman 1 tr -cs A-Za-z 'n' | 2 tr A-Z a-z | 3 sort | 4 uniq -c | 5 sort -rn | 6 sed ${1}q Tools
  149. 149. @annashipman 1 tr -cs A-Za-z 'n' | 2 tr A-Z a-z | 3 sort | 4 uniq -c | 5 sort -rn | 6 sed ${1}q Tools
  150. 150. @annashipman 1 tr -cs A-Za-z 'n' | 2 tr A-Z a-z | 3 sort | 4 uniq -c | 5 sort -rn | 6 sed ${1}q Tools
  151. 151. @annashipman 1 tr -cs A-Za-z 'n' | 2 tr A-Z a-z | 3 sort | 4 uniq -c | 5 sort -rn | 6 sed ${1}q Tools
  152. 152. @annashipman 1 tr -cs A-Za-z 'n' | 2 tr A-Z a-z | 3 sort | 4 uniq -c | 5 sort -rn | 6 sed ${1}q Tools
  153. 153. @annashipman 1 tr -cs A-Za-z 'n' | 2 tr A-Z a-z | 3 sort | 4 uniq -c | 5 sort -rn | 6 sed ${1}q Tools
  154. 154. @annashipman 1 tr -cs A-Za-z 'n' | 2 tr A-Z a-z | 3 sort | 4 uniq -c | 5 sort -rn | 6 sed ${1}q Tools
  155. 155. @annashipman 1 tr -cs A-Za-z 'n' | 2 tr A-Z a-z | 3 sort | 4 uniq -c | 5 sort -rn | 6 sed ${1}q Tools
  156. 156. @annashipman Read a file of text, determine the n most frequently used words, and print out a sorted list of those words along with their frequencies. Tools
  157. 157. @annashipman You might not want to write all your programs in Unix Tools
  158. 158. @annashipman It can be really useful for doing complex tasks Tools
  159. 159. @annashipman It’s worth getting familiar with Unix tools locally Tools
  160. 160. @annashipman So that the first time isn’t when you’re debugging on the server Tools
  161. 161. @annashipman $ rake -T | grep "some detail" Tools
  162. 162. @annashipman Read the output Tools
  163. 163. @annashipman Git instructions Tools
  164. 164. @annashipman $ git add accidental_new_file.txt $ git status On branch master Your branch is up-to-date with 'origin/master'. Changes to be committed: (use "git reset HEAD <file>..." to unstage) new file: accidental_new_file.txt Tools
  165. 165. @annashipman $ git add accidental_new_file.txt $ git status On branch master Your branch is up-to-date with 'origin/master'. Changes to be committed: (use "git reset HEAD <file>..." to unstage) new file: accidental_new_file.txt Tools
  166. 166. @annashipman Unix error messages Tools
  167. 167. @annashipman $ grep -r anna Tools
  168. 168. @annashipman $ grep -r anna grep: warning: recursive search of stdin Tools
  169. 169. @annashipman $ grep -r anna . Tools
  170. 170. @annashipman Unix tools can help you Tools
  171. 171. @annashipman The main three things you should take home:
  172. 172. @annashipman Use Vagrant
  173. 173. @annashipman Unix tools are your friends
  174. 174. @annashipman Read the output
  175. 175. Thank you! Anna Shipman @annashipman
  176. 176. Configuration management https://www.scriptrock.com/articles/the-7-configuration-management-tools-you-need-to-know http://gettingstartedwithchef.com/first-steps-with-chef.html https://docs.vagrantup.com/v2/getting-started/provisioning.html Virtualisation http://searchvirtualdatacentre.techtarget.co.uk/definition/Virtualisation http://searchservervirtualization.techtarget.com/definition/server-virtualization http://www.infoworld.com/article/2621446/server-virtualization/server-virtualization-top-10-benefits-of-server- virtualization.html Using Vagrant https://www.vagrantup.com/ http://blog.bennycornelissen.nl/otto-a-modern-developers-new-best-friend/ https://github.com/patrickdlee/vagrant-examples (Useful getting started examples) Docker http://patg.net/containers,virtualization,docker/2014/06/05/docker-intro/ https://zeltser.com/security-risks-and-benefits-of-docker-application/ Containerisation vs Virtualisation http://www.slideshare.net/bcantrill/docker-and-the-future-of-containers-in-production https://www.scriptrock.com/articles/docker-vs-vagrant Make instead of Grunt/Gulp http://blog.keithcirkel.co.uk/why-we-should-stop-using-grunt/ https://www.youtube.com/watch?v=0RYETb9YVrk (Talk on using NPM as a build tool) https://blog.jcoglan.com/2014/02/05/building-javascript-projects-with-make/ Tools for better dev http://www.leancrew.com/all-this/2011/12/more-shell-less-egg/ (More detail on the 6-line Unix program)

×