Your SlideShare is downloading. ×
A Technique for Enabling and Supporting Debugging of Field Failures (ICSE 2007)
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

A Technique for Enabling and Supporting Debugging of Field Failures (ICSE 2007)

52
views

Published on

Published in: Technology, Business

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
52
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. A Technique for Enabling andSupporting Debugging ofField FailuresJames Clause and Alessandro OrsoGeorgia Institute of TechnologyThis work was supported in part by NSF awardsCCF-0541080 and CCR-0205422 to Georgia Tech.1
  • 2. 3
  • 3. 3Field failures:Anomalous behavior (or crashes) ofdeployed software that occur on user machines
  • 4. 4
  • 5. Crash logs4
  • 6. Crash logsUser-providedinformation4
  • 7. Our solution5
  • 8. Our solution5Record
  • 9. Our solution5RecordReplay
  • 10. Our solution5RecordReplayMinimize
  • 11. Our solution5RecordReplayMinimizeDebug
  • 12. Usage Scenario6In house In the fieldReplay / Minimize(off line)Record (on line)DevelopReplay / DebugExecutionrepository✔/✘
  • 13. Existing record / replay approaches7Regression testing(e.g. Elbaum et al. 06, Orso et al. 06, Orsoand Kennedy 05, Saff et al. 05, MercuryWinRunner)• Replay only a portion of anexecution by recording eventsfor specific subsystemsBoth types of technique are not amenable tominimization and may cause unacceptable overheadDeterministic debugging(e.g. Chen et al. 01, King et al. 05,Narayanasamy et al. 05, Netzer and Weaver94, Srinivasan et al. 04,VMWare)• Replay an entire execution byrecording every component ofan application
  • 14. Outline✓Motivation & background• Our technique• record• replay• minimization• Empirical evaluation• Conclusion & future work8
  • 15. Record & Replay• Goal: develop an approach that has low overhead and isamenable to minimization• Key insight: avoid focusing on low-level (internal) events• expensive (large number of events)• not amenable to minimization (high interdependence)9
  • 16. Record & Replay• Goal: develop an approach that has low overhead and isamenable to minimization• Key insight: avoid focusing on low-level (internal) events• expensive (large number of events)• not amenable to minimization (high interdependence)➡Focus on high-level (external) interactions with theenvironment• efficient (fewer, more “expensive” interactions)• amenable to minimization (low interdependence)10
  • 17. Environment interactions11
  • 18. Environment interactionsStreams11
  • 19. Environment interactionsStreams Files11
  • 20. Environment interactionsStreams Files11
  • 21. Environment interactionsStreams Files11Interaction events:FILE — interaction with a filePOLL — checks for availability of data on a streamPULL — read data from a stream
  • 22. Event log:Environment data (files):12Environment data (streams):
  • 23. Event log:Environment data (files):12Environment data (streams):FILE foo.1foo.1
  • 24. Event log:Environment data (files):12Environment data (streams):FILE foo.1foo.1
  • 25. Event log:Environment data (files):12Environment data (streams):FILE foo.1foo.1POLL KEYBOARD NOK
  • 26. Event log:Environment data (files):12Environment data (streams):FILE foo.1foo.1POLL KEYBOARD NOK
  • 27. Event log:Environment data (files):12Environment data (streams):FILE foo.1foo.1KEYBOARD: {5680}POLL KEYBOARD OKPOLL KEYBOARD NOK
  • 28. Event log:Environment data (files):12Environment data (streams):FILE foo.1foo.1KEYBOARD: {5680}POLL KEYBOARD OKPOLL KEYBOARD NOK
  • 29. Event log:Environment data (files):12Environment data (streams):FILE foo.1foo.1KEYBOARD: {5680} helloPOLL KEYBOARD OKPULL KEYBOARD 5POLL KEYBOARD NOK
  • 30. Event log:Environment data (files):12Environment data (streams):FILE foo.1foo.1KEYBOARD: {5680} helloPOLL KEYBOARD OKPULL KEYBOARD 5POLL KEYBOARD NOK
  • 31. Event log:Environment data (files):12Environment data (streams):FILE foo.1foo.1KEYBOARD: {5680} helloPOLL KEYBOARD OKPULL KEYBOARD 5POLL KEYBOARD NOKPOLL NETWORK OKNETWORK: {3405}
  • 32. Event log:Environment data (files):12Environment data (streams):FILE foo.1foo.1KEYBOARD: {5680} helloPOLL KEYBOARD OKPULL KEYBOARD 5POLL KEYBOARD NOKPOLL NETWORK OKNETWORK: {3405}❙
  • 33. Event log:Environment data (files):12Environment data (streams):FILE foo.1foo.1KEYBOARD: {5680} helloPOLL KEYBOARD OKPULL KEYBOARD 5POLL KEYBOARD NOKPOLL NETWORK OKNETWORK: {3405}❙
  • 34. Environment data (files):13Event log:Environment data (streams):KEYBOARD: {5680}hello ❙ {4056}c ❙ {300}...NETWORK: {3405}<html><body>... ❙ {202}...FILE foo.1POLL KEYBOARD NOKPOLL KEYBOARD OKPULL KEYBOARD 1POLL NETWORK OKPULL NETWORK 1024FILE bar.1POLL NETWORK NOKPOLL NETWORK OKFILE foo.2...PULL NETWORK 1024FILE foo.2POLL KEYBOARD NOK...foo.1 foo.2 bar.1
  • 35. Environment data (files):14Event log:Environment data (streams):KEYBOARD: {5680}hello ❙ {4056}c ❙ {300}...NETWORK: {3405}<html><body>... ❙ {202}...FILE foo.1POLL KEYBOARD NOKPOLL KEYBOARD OKPULL KEYBOARD 1POLL NETWORK OKPULL NETWORK 1024FILE bar.1POLL NETWORK NOKPOLL NETWORK OKFILE foo.2...PULL NETWORK 1024FILE foo.2POLL KEYBOARD NOK...foo.1 foo.2 bar.1
  • 36. Environment data (files):14Event log:Environment data (streams):KEYBOARD: {5680}hello ❙ {4056}c ❙ {300}...NETWORK: {3405}<html><body>... ❙ {202}...FILE foo.1POLL KEYBOARD NOKPOLL KEYBOARD OKPULL KEYBOARD 1POLL NETWORK OKPULL NETWORK 1024FILE bar.1POLL NETWORK NOKPOLL NETWORK OKFILE foo.2...PULL NETWORK 1024FILE foo.2POLL KEYBOARD NOK...foo.1 foo.2 bar.1
  • 37. Environment data (files):14Event log:Environment data (streams):KEYBOARD: {5680}hello ❙ {4056}c ❙ {300}...NETWORK: {3405}<html><body>... ❙ {202}...FILE foo.1POLL KEYBOARD NOKPOLL KEYBOARD OKPULL KEYBOARD 1POLL NETWORK OKPULL NETWORK 1024FILE bar.1POLL NETWORK NOKPOLL NETWORK OKFILE foo.2...PULL NETWORK 1024FILE foo.2POLL KEYBOARD NOK...foo.1 foo.2 bar.1✔
  • 38. Environment data (files):14Event log:Environment data (streams):KEYBOARD: {5680}hello ❙ {4056}c ❙ {300}...NETWORK: {3405}<html><body>... ❙ {202}...FILE foo.1POLL KEYBOARD NOKPOLL KEYBOARD OKPULL KEYBOARD 1POLL NETWORK OKPULL NETWORK 1024FILE bar.1POLL NETWORK NOKPOLL NETWORK OKFILE foo.2...PULL NETWORK 1024FILE foo.2POLL KEYBOARD NOK...foo.1 foo.2 bar.1✔
  • 39. Environment data (files):14Event log:Environment data (streams):KEYBOARD: {5680}hello ❙ {4056}c ❙ {300}...NETWORK: {3405}<html><body>... ❙ {202}...FILE foo.1POLL KEYBOARD NOKPOLL KEYBOARD OKPULL KEYBOARD 1POLL NETWORK OKPULL NETWORK 1024FILE bar.1POLL NETWORK NOKPOLL NETWORK OKFILE foo.2...PULL NETWORK 1024FILE foo.2POLL KEYBOARD NOK...foo.1 foo.2 bar.1✔
  • 40. Environment data (files):14Event log:Environment data (streams):KEYBOARD: {5680}hello ❙ {4056}c ❙ {300}...NETWORK: {3405}<html><body>... ❙ {202}...FILE foo.1POLL KEYBOARD NOKPOLL KEYBOARD OKPULL KEYBOARD 1POLL NETWORK OKPULL NETWORK 1024FILE bar.1POLL NETWORK NOKPOLL NETWORK OKFILE foo.2...PULL NETWORK 1024FILE foo.2POLL KEYBOARD NOK...foo.1 foo.2 bar.1✔✔✔✔✔✔✔✔✔✔✔✔✔
  • 41. Minimize15Goal: focus debugging effort
  • 42. Minimize15Goal: focus debugging effortExecutionrecording
  • 43. Minimize15Goal: focus debugging effortExecutionrecording↺Timeminimization
  • 44. Minimize15Goal: focus debugging effortExecutionrecordingExecutionrecording↺Timeminimization
  • 45. Minimize15Goal: focus debugging effortExecutionrecordingExecutionrecording↺Timeminimization✂Dataminimization
  • 46. Minimize15Goal: focus debugging effortExecutionrecordingExecutionrecordingExecutionrecording↺Timeminimization✂Dataminimization
  • 47. Minimize: timeEnvironment data (files):16Event log:Environment data (streams):KEYBOARD: {5680}hello ❙ {4056}c ❙ {300}...NETWORK: {3405}<html><body>... ❙ {202}...FILE foo.1POLL KEYBOARD NOKPOLL KEYBOARD OKPULL KEYBOARD 1POLL NETWORK OKPULL NETWORK 1024FILE bar.1POLL NETWORK NOKPOLL NETWORK OKFILE foo.2PULL NETWORK 1024FILE foo.2POLL KEYBOARD NOK
  • 48. Minimize: timeEnvironment data (files):16Event log:Environment data (streams):KEYBOARD: {5680}hello ❙ {4056}c ❙ {300}...NETWORK: {3405}<html><body>... ❙ {202}...FILE foo.1POLL KEYBOARD NOKPOLL KEYBOARD OKPULL KEYBOARD 1POLL NETWORK OKPULL NETWORK 1024FILE bar.1POLL NETWORK NOKPOLL NETWORK OKFILE foo.2PULL NETWORK 1024FILE foo.2POLL KEYBOARD NOK
  • 49. Minimize: timeEnvironment data (files):17Event log:Environment data (streams):FILE foo.1POLL KEYBOARD OKPULL KEYBOARD 1POLL NETWORK OKPULL NETWORK 1024FILE bar.1POLL NETWORK OKFILE foo.2PULL NETWORK 1024FILE foo.2KEYBOARD: {5680}hello ❙ {4056}c ❙ {300}...NETWORK: {3405}<html><body>... ❙ {202}...
  • 50. Minimize: timeEnvironment data (files):17Event log:Environment data (streams):FILE foo.1POLL KEYBOARD OKPULL KEYBOARD 1POLL NETWORK OKPULL NETWORK 1024FILE bar.1POLL NETWORK OKFILE foo.2PULL NETWORK 1024FILE foo.2KEYBOARD: {5680}hello ❙ {4056}c ❙ {300}...NETWORK: {3405}<html><body>... ❙ {202}...
  • 51. Minimize: data18AtomsChunksWholeentitiesData minimization Environment
  • 52. Minimize: data18AtomsChunksWholeentitiesData minimization Environment
  • 53. Minimize: data18✔AtomsChunksWholeentitiesData minimization Environment
  • 54. Minimize: data18AtomsChunksWholeentitiesData minimization Environment
  • 55. Minimize: data18AtomsChunksWholeentitiesData minimization Environment
  • 56. Minimize: data18AtomsChunksWholeentitiesData minimization Environment✘
  • 57. Minimize: data18AtomsChunksWholeentitiesData minimization Environment
  • 58. Minimize: data18AtomsChunksWholeentitiesData minimization Environment
  • 59. Minimize: data18AtomsChunksWholeentitiesData minimization Environment
  • 60. Minimize: data18AtomsChunksWholeentitiesData minimization Environment
  • 61. Minimize: data18AtomsChunksWholeentitiesData minimization Environment
  • 62. Minimize: data18AtomsChunksWholeentitiesData minimization Environment
  • 63. Minimize: data18AtomsChunksWholeentitiesData minimization Environment
  • 64. Minimize: data18AtomsChunksWholeentitiesData minimization Environment
  • 65. The tool:ADDAAssisting the Debuggingof Deployed Applications• Record and Replay:• Works on x86 (c-lib based) binaries• Based on dynamic instrumentation (Pin)• Maps c-library calls to interaction events• Minimization:• Set of extensible scripts19
  • 66. LimitationsTwo main limitations:• Technique:May not replay non-deterministic failures• Implementation:Does not handle window system events (yet)20
  • 67. Empirical evaluation• Research questions• Can ADDA produce minimized executions that can be usedto debug the original failure?• How much overhead does ADDA impose?• Subject:• Pine — widely-used email / news client• Data:• Two real field failures from Pine’s history• Set of 20 failing executions, 10 per failure21
  • 68. Empirical evaluation• Research questions• Can ADDA produce minimized executions that can be usedto debug the original failure?• How much overhead does ADDA impose?• Subject:• Pine — widely-used email / news client• Data:• Two real field failures from Pine’s history• Set of 20 failing executions, 10 per failure22
  • 69. Minimization results0%10%20%30%40%50%60%70%80%90%100%# entities streams size files size23AveragevalueafterminimizationHeader-color fault Address book fault
  • 70. Minimization results0%10%20%30%40%50%60%70%80%90%100%# entities streams size files size23AveragevalueafterminimizationHeader-color fault Address book faultMoreover, these results are conservative: recorded executions only contain the minimalamount of data needed to perform an action.
  • 71. Minimization results0%10%20%30%40%50%60%70%80%90%100%# entities streams size files size23AveragevalueafterminimizationHeader-color fault Address book faultOverhead• Offline: less than 75 minutes for minimization• Online: negligible overhead while recordingMoreover, these results are conservative: recorded executions only contain the minimalamount of data needed to perform an action.
  • 72. Specific Example: Address Book Failure• Complete execution• 34 entities (files and streams)• ≈800kb• Minimized execution• 5 partial entities (4 files,1 stream)• ≈72kb
  • 73. Future work• More studies: additional applications and real users• Extend technique / implementation• Support windowing system• Investigate ad-hoc minimization algorithms• Include non-deterministic events (if needed)25
  • 74. Conclusions• Novel approach that supports debuggingfield failures• Prototype implementation for x86 binaries• Preliminary empirical evaluation: for the casesconsidered, our technique can1. minimize failing executions,2. preserve their failing behavior, and3. impose low overhead on users26
  • 75. Questions?27