Software Entomology or Where Do Bugs Come From?

6,106 views

Published on

An internal training talk that Michelle D'Netto and I periodically give for Customer Support representatives at Etsy. Introduces advanced Software Quality concepts such as the halting problem, the impossibility of complete testing and the extreme difficulty of discovering all of the significant bugs in one's own software. Winds up by encouraging anyone responsible for online customer experience, to envision themselves as a participant observer embedded in the rapidly-evolving culture of the Web.

Published in: Technology
2 Comments
5 Likes
Statistics
Notes
No Downloads
Views
Total views
6,106
On SlideShare
0
From Embeds
0
Number of Embeds
1,256
Actions
Shares
0
Downloads
47
Comments
2
Likes
5
Embeds 0
No embeds

No notes for slide

Software Entomology or Where Do Bugs Come From?

  1. 1. Software Entomology or, Where Do Bugs Come From? Noah Sussman and Michelle D’Netto etsy
  2. 2. There will always be bugs in production
  3. 3. Most Bugs Are Non-Obvious
  4. 4. Debugging is hard
  5. 5. I can remember the exactinstant when I realized that alarge part of my life... wasgoing to be spent in findingmistakes in my own programs ~Maurice Wilkes, 1949
  6. 6. Admiral Grace Hopper likedto tell a story in which atechnician solved a glitchin the Harvard Mark IImainframe by pulling anactual insect out frombetween the contacts ofone of its relays ~ The Jargon File
  7. 7. Complete testing is impossible.
  8. 8. Even the simplest Website has more edge casesthan you could test, evenif you ran one test persecond until the heatdeath of the universe.
  9. 9. assure that the probability of failuredue to hibernating bugsis sufficiently lowto be acceptable~ Boris Beizer, 1983
  10. 10. What constitutes a “sufficiently low”number of problems?
  11. 11. Declaring aprogram“bug free,”is anentirelysubjectiveexcercise
  12. 12. Engineers suck at findingbugs in their own code
  13. 13. Engineers + designers aretoo close to the software
  14. 14. Computer programs cannot detectbugs in other computer programs
  15. 15. Debugging is twice ashard as writing the codein the first place. ~ Brian Kernighan
  16. 16. Getting frustrated by software isnot the same as reporting a bug
  17. 17. Non-source-aware userstend to report only surfacesymptoms; they take theirenvironment for granted... ~ ESR, “How Many Eyeballs Tame Complexity”, 1997
  18. 18. Users never read manualsUsers would save time in the longterm by learning more about thesystem. But thats not how peoplebehave in the real world ~Jakob Nielsen
  19. 19. On two occasionsI have been asked...if you put into themachine wrongfigures, will the rightanswers come Babbage, 1864 ~Charles out?
  20. 20. When you report a bugyou have done a hard thing
  21. 21. A bug report is a story about a problem anda good story has a beginning, middle and end
  22. 22. Description of problemSteps to reproduceExpected resolution
  23. 23. Provide context, and lots of it
  24. 24. Be Quantitative Engineers respond well to hard evidence
  25. 25. Graphmonitorit didn’t happenAt Etsy we or over 100,000 real-time metrics
  26. 26. A good editor ismeticulously attentive to detail,a stickler for technical correctness,aware of the work as a whole andsensitive to the expectations of the audience
  27. 27. You are doing scienceKeep a journalTake screenshots (use Skitch)Log IRC, Skype and other chatsMake multiple redundant backups (use DropBox)Save everything— you won’t know you need it until you need it
  28. 28. How many times has this happened?How many users were affected?How much time has been lost?What was the overall cost?
  29. 29. Users
  30. 30. The SupportTeam spendthe most timeengaging andempathizingwith users
  31. 31. You are a participant observer
  32. 32. When you talk to your users, andyou write down what you learn,and you share what you wrotewith others, you’re doing science
  33. 33. digital computer: less than 70 years old,personal computer: less than 40,the Web: less than 20,mobile Web: less than 15
  34. 34. The Web is going to keep on growing and changing
  35. 35. 80% of us aren’teven on the Web yet
  36. 36. Most users of a Web site are doing something that is entirely new to them And you are teaching them how to do it

×