Living system or build factory - Chris Maxwell

1,238 views
1,066 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,238
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
86
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Living system or build factory - Chris Maxwell

  1. 1. Living System or Build Factory? ! Chris Maxwell ! @WrathOfChris Cloud guy @ Ubiquiti Networks
  2. 2. Who is this talk for? • We build systems • We turn design into implementation • We operate systems forever**not forever
  3. 3. We keep systems from failing
  4. 4. Holding up the system?
  5. 5. We do it wrong
  6. 6. Not helping
  7. 7. What is a Living System?
  8. 8. The business has a need
  9. 9. Instructions are unclear
  10. 10. What about THIS space?
  11. 11. Instructions are not followed
  12. 12. ...except a VCR, CD’s, and a remote?
  13. 13. Production loads change unexpectedly
  14. 14. Additional capacity added
  15. 15. Wrong tools at hand
  16. 16. 12ga crimp on 24ga wire doesn’t work
  17. 17. Patches are put into production
  18. 18. Patches become production
  19. 19. Production is unexpected
  20. 20. Unexpected Scotsman
  21. 21. Parts not available
  22. 22. No ventilation ducting available
  23. 23. Inconsistent Origins
  24. 24. One of these things is not like the other http://www.flickr.com/photos/msittig/4801606785/
  25. 25. Living System • Has a long life • Mutates over time • Each system slightly different • The system is the state http://www.flickr.com/photos/anijdam/2362660336/
  26. 26. Problems ! Can you rebuild it … fast? … accurately? http://www.flickr.com/photos/adactio/2856955322/
  27. 27. Problems ! Do you keep: • HA failover? • Standby host?
  28. 28. Problems ! Have you tested? ! When both fail?
  29. 29. What is a Build Factory?
  30. 30. For business…:
  31. 31. http://www.flickr.com/photos/pasukaru76/6893926948/ Run the line
  32. 32. For devs…:
  33. 33. make clean && make
  34. 34. For ops…:
  35. 35. http://www.flickr.com/photos/comedynose/3549464622/ Kill it with fire
  36. 36. Looking back on Build Factory
  37. 37. Copying systems
  38. 38. Cloning systems
  39. 39. Early build factories took time
  40. 40. ...hours on 10baseT
  41. 41. ...tumbleweeds...
  42. 42. It was a manual process
  43. 43. How to restore a VM image... ...in 7 easy pages of screenshots Sorry, couldn’t release these pics :( It was just lots of screenshots + do this, then that instructions
  44. 44. It was a manual process
  45. 45. http://spalinux.com/wp-content/uploads/2008/10/ windows_join_domain.png ...and still manual work after
  46. 46. Scripting is a start
  47. 47. I will replace you with a short script
  48. 48. Makefiles track dependencies
  49. 49. or a Makefile it builds your kernel, and probably your OS
  50. 50. Today’s tools
  51. 51. Not a complete list
  52. 52. Describe
  53. 53. http://www.flickr.com/photos/spree2010/4960430323/ Write down all of the steps
  54. 54. Automate
  55. 55. http://www.flickr.com/photos/sepblog/3679949333/ “too complex for any one man’s memory”
  56. 56. Replicate
  57. 57. Build 3 the same, tie them together
  58. 58. Repeat
  59. 59. Training makes elements replaceable
  60. 60. Build Factory • Has a short life • Replaced for each change • Each system the same • The procedure is the state http://upload.wikimedia.org/wikipedia/commons/5/50/Krispy_Kreme_Doughnuts.jpg
  61. 61. When something jams ! deep in the system
  62. 62. a scrap of paper caught in the drive gear
  63. 63. Replace It! •fix it later •send it for repair •get production working!
  64. 64. When production patches take over
  65. 65. Production maintenance changes the system
  66. 66. Rebuild it!
  67. 67. Sometimes you get to rebuild them
  68. 68. In the cloud? ! live replace might cost up to 1 hour of billing
  69. 69. In the datacentre? ! live replace can be done with spare capacity
  70. 70. Takeaway • Describe it • Automate it • Produce it • Repeat it
  71. 71. Questions? ! @WrathOfChris ! chris@wrathofchris.com ! Like the Loch Ness Monster,
 Answers are over there...
  72. 72. • Forgotten attributions: • ghost-1: http://www.all-internet-security.com/images/pc-copy.jpg • ghost-2: http://www2.expta.com/uploaded_images/Ghost-741732.png • ghost-3: http://internship.wesmilliron.com/wp-content/uploads/2012/08/vmware-norton-ghost.jpg • laplink: http://thumbs1.ebaystatic.com/d/l225/m/mkhsfLBMjkjjMw9skfQUNag.jpg • lap link: http://www.old-games.com/screenshot/5736-1-lap-link-3-for-dos.jpg

×