Mistakes were made

1,809 views
1,711 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,809
On SlideShare
0
From Embeds
0
Number of Embeds
37
Actions
Shares
0
Downloads
15
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Mistakes were made

  1. 1. Mistakes were made Selena Deckelmann @selenamarieSo O mS eCC O on N fer 20 en 11 e c
  2. 2. This goes out to all the sysadmins.So O mS eCC O on N fer 20 en 11 e c
  3. 3. You can never think about failure too much.So O mS eCC O on N fer 20 en 11 e c
  4. 4. c e 11 en 20 fer O on N eCC O mSSo
  5. 5. Some goals around pessimism.So O mS eCC O on N fer 20 en 11 e c
  6. 6. Plan for the worst. c e 11 en 20 fer O on N eCC O mS So
  7. 7. Minimize risk. c e 11 en 20 fer O on N eCC O mS So
  8. 8. Recover, gracefully. c e 11 en 20 fer O on N eCC O mS So
  9. 9. Plan for the worst. Minimize risk. Recover, gracefully.So O mS eCC O on N fer 20 en 11 e c
  10. 10. c e 11 en 20 fer O on N eCC O mSSo
  11. 11. c e 11 en 20 fer O on N eCC O mSSo
  12. 12. Tales of failure to... Document Test Verify Imagine ImplementSo O mS eCC O on N fer 20 en 11 e c
  13. 13. Failure to document. c e 11 en 20 fer O on N eCC O mS So
  14. 14. Moving Day Thanks, David Prior!So O mS eCC O on N fer 20 en 11 e c
  15. 15. Prevent documentation failures. • Write documentation. • Update documentation. • Make documenting a step in your written process. • Assign a fixed amount of time to that step.So O mS eCC O on N fer 20 en 11 e c
  16. 16. Documentation tools • Graphic designers. (Pretty wikis. Pretty docs. (Sphinx?) Diagrams.) • Timelines. • Bug tracking. • Ordered todo lists.So O mS eCC O on N fer 20 en 11 e c
  17. 17. Failure to test. c e 11 en 20 fer O on N eCC O mS So
  18. 18. “My first day posing as a sysadmin (~1990, no previous training....) I deleted all zero length files on a Sun workstation.”So O mS eCC O on N fer 20 en 11 e c
  19. 19. Prevent testing failures. • Verify success criteria. • Write tests. • Test with a buddy. • Have a plan.So O mS eCC O on N fer 20 en 11 e c
  20. 20. Testing tools • Your favorite test framework • Repeatable shell scripts • Staging environmentsSo O mS eCC O on N fer 20 en 11 e c
  21. 21. Failure to verify. c e 11 en 20 fer O on N eCC O mS So
  22. 22. “What does ‘-d’ actually do?”So O mS eCC O on N fer 20 en 11 e c
  23. 23. Prevent verification failures. • Have a plan for things going wrong. • Have a staging environment. • Test your rollback plan, not just your implementation plan.So O mS eCC O on N fer 20 en 11 e c
  24. 24. Verification tools • Staging environments • Your buddySo O mS eCC O on N fer 20 en 11 e c
  25. 25. Failure to imagine. c e 11 en 20 fer O on N eCC O mS So
  26. 26. For my group the bottom line was "dont trust anyone".So Thanks, Maggie! O mS eCC O on N fer 20 en 11 e c
  27. 27. Recover from failures to imagine. • Share your stories of failure. • Talk with people who are different from you. • Act out implementation scenarios.So O mS eCC O on N fer 20 en 11 e c
  28. 28. Failure to implement. c e 11 en 20 fer O on N eCC O mS So
  29. 29. Re-implement. • Learn from mistakes.So O mS eCC O on N fer 20 en 11 e c
  30. 30. Reflection. (or, the Post-Mortem)So O mS eCC O on N fer 20 en 11 e c
  31. 31. Before • Document the plan with numbered steps and a timeline. • Test the plan and the rollback plan. • Identify a “point of no return”.So O mS eCC O on N fer 20 en 11 e c
  32. 32. During • Screen sharing: UNIX screen,VNC, etc. • Chatroom: AIM, Campfire (scrollback!) • Voice: Campfire, Skype,VOIP, POTS call line • Headsets! • Designated time-keeper.So O mS eCC O on N fer 20 en 11 e c
  33. 33. After • Documentation updates • Post-mortems to identify areas of success and areas for improvement. • Limit improvements to 1-2 things.So O mS eCC O on N fer 20 en 11 e c
  34. 34. Plan for the worst. Minimize risk. Recover, gracefully.So O mS eCC O on N fer 20 en 11 e c
  35. 35. Thanks! 20 fer 11 en c e O on N eCC O mS So
  36. 36. Photo credits • Flickr: sheepguardingllamaSo O mS eCC O on N fer 20 en 11 e c

×