Your SlideShare is downloading. ×
0
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Penumbra: Automatically Identifying Failure-Relevant Inputs (ISSTA 2009)

224

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
224
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  1. Penumbra:Automatically IdentifyingFailure-Relevant InputsJames Clause and Alessandro OrsoCollege of ComputingGeorgia Institute of TechnologySupported in part by:NSF awards CCF-0725202 and CCF-0541080to Georgia Tech
  2. Automated Debugging• Gupta and colleagues ’05• Jones and colleagues ’02• Korel and Laski ’88• Liblit and colleagues ’05• Nainar and colleagues ’07• Renieris and Reiss ’03• Seward and Nethercote ’05• Tucek and colleagues ’07• Weiser ’81• Zhang and colleagues ’05• Zhang and colleagues ’06• ...
  3. Automated DebuggingCode-centric• Gupta and colleagues ’05• Jones and colleagues ’02• Korel and Laski ’88• Liblit and colleagues ’05• Nainar and colleagues ’07• Renieris and Reiss ’03• Seward and Nethercote ’05• Tucek and colleagues ’07• Weiser ’81• Zhang and colleagues ’05• Zhang and colleagues ’06• ...
  4. Automated DebuggingCode-centric• Gupta and colleagues ’05• Jones and colleagues ’02• Korel and Laski ’88• Liblit and colleagues ’05• Nainar and colleagues ’07• Renieris and Reiss ’03• Seward and Nethercote ’05• Tucek and colleagues ’07• Weiser ’81• Zhang and colleagues ’05• Zhang and colleagues ’06• ...What about inputs which cause the failure?
  5. • Chan and Lakhotia ’98• Zeller and Hildebrandt ’02• Misherghi and Su ’06Data-centric Techniques
  6. • Chan and Lakhotia ’98• Zeller and Hildebrandt ’02• Misherghi and Su ’06Delta DebuggingData-centric Techniques
  7. • Chan and Lakhotia ’98• Zeller and Hildebrandt ’02• Misherghi and Su ’06Delta DebuggingData-centric TechniquesRequires:1. Multiple executions2. Large amounts of manualeffort (oracle creation, setup)
  8. • Chan and Lakhotia ’98• Zeller and Hildebrandt ’02• Misherghi and Su ’06Delta DebuggingData-centric TechniquesRequires:1. Multiple executions2. Large amounts of manualeffort (oracle creation, setup)Penumbra
  9. • Chan and Lakhotia ’98• Zeller and Hildebrandt ’02• Misherghi and Su ’06Delta DebuggingData-centric TechniquesRequires:1. Multiple executions2. Large amounts of manualeffort (oracle creation, setup)PenumbraComparableperformance
  10. • Chan and Lakhotia ’98• Zeller and Hildebrandt ’02• Misherghi and Su ’06Delta DebuggingData-centric TechniquesRequires:1. Multiple executions2. Large amounts of manualeffort (oracle creation, setup)Requires:1. Single execution2. Reduced manual effortPenumbraComparableperformance
  11. Intuition and TerminologyFailure-revealing input vector
  12. Intuition and TerminologyFailure-revealing input vectorFailure-relevant subset(inputs which are useful for investigating the failure)
  13. Intuition and TerminologyFailure-revealing input vectorFailure-relevant subset(inputs which are useful for investigating the failure)Approximate failure-relevant subsets byidentifying inputs that reach the failure alongprogram dependencies.
  14. Motivating Exampleint main(int argc, char **argv) {1. int verbose, i, total_size = 0;2. struct stat buf;3. verbose = atoi(argv[1]);4. for(i = 2; i < argc; i++) {5. int fd = open(argv[i], O_RDONLY);6. fstat(fd, &buf);7. char *out = malloc(60);8. sprintf(out, "%d", buf.st_size);9. if(verbose) {10. char *pview = malloc(51);11. read(fd, pview, 50);12. pview[50] = 0;13. strcat(out, pview);14. }15. printf("%s: %sn", argv[i], out);16. total_size += buf.st_size;17. }18. printf("total: %dn", total_size);}fileinfo
  15. Motivating Exampleint main(int argc, char **argv) {1. int verbose, i, total_size = 0;2. struct stat buf;3. verbose = atoi(argv[1]);4. for(i = 2; i < argc; i++) {5. int fd = open(argv[i], O_RDONLY);6. fstat(fd, &buf);7. char *out = malloc(60);8. sprintf(out, "%d", buf.st_size);9. if(verbose) {10. char *pview = malloc(51);11. read(fd, pview, 50);12. pview[50] = 0;13. strcat(out, pview);14. }15. printf("%s: %sn", argv[i], out);16. total_size += buf.st_size;17. }18. printf("total: %dn", total_size);}fileinfoCommand line arguments(flag, list of file names)
  16. Motivating Exampleint main(int argc, char **argv) {1. int verbose, i, total_size = 0;2. struct stat buf;3. verbose = atoi(argv[1]);4. for(i = 2; i < argc; i++) {5. int fd = open(argv[i], O_RDONLY);6. fstat(fd, &buf);7. char *out = malloc(60);8. sprintf(out, "%d", buf.st_size);9. if(verbose) {10. char *pview = malloc(51);11. read(fd, pview, 50);12. pview[50] = 0;13. strcat(out, pview);14. }15. printf("%s: %sn", argv[i], out);16. total_size += buf.st_size;17. }18. printf("total: %dn", total_size);}fileinfoFile statistics (for each file)(size, last modified date, ...)Command line arguments(flag, list of file names)
  17. Motivating Exampleint main(int argc, char **argv) {1. int verbose, i, total_size = 0;2. struct stat buf;3. verbose = atoi(argv[1]);4. for(i = 2; i < argc; i++) {5. int fd = open(argv[i], O_RDONLY);6. fstat(fd, &buf);7. char *out = malloc(60);8. sprintf(out, "%d", buf.st_size);9. if(verbose) {10. char *pview = malloc(51);11. read(fd, pview, 50);12. pview[50] = 0;13. strcat(out, pview);14. }15. printf("%s: %sn", argv[i], out);16. total_size += buf.st_size;17. }18. printf("total: %dn", total_size);}fileinfoFile statistics (for each file)(size, last modified date, ...)File contents (for each file)(first 50 characters)Command line arguments(flag, list of file names)
  18. Motivating Exampleint main(int argc, char **argv) {1. int verbose, i, total_size = 0;2. struct stat buf;3. verbose = atoi(argv[1]);4. for(i = 2; i < argc; i++) {5. int fd = open(argv[i], O_RDONLY);6. fstat(fd, &buf);7. char *out = malloc(60);8. sprintf(out, "%d", buf.st_size);9. if(verbose) {10. char *pview = malloc(51);11. read(fd, pview, 50);12. pview[50] = 0;13. strcat(out, pview);14. }15. printf("%s: %sn", argv[i], out);16. total_size += buf.st_size;17. }18. printf("total: %dn", total_size);}fileinfoFile statistics (for each file)(size, last modified date, ...)File contents (for each file)(first 50 characters)Command line arguments(flag, list of file names)Inputvector
  19. Motivating Exampleint main(int argc, char **argv) {1. int verbose, i, total_size = 0;2. struct stat buf;3. verbose = atoi(argv[1]);4. for(i = 2; i < argc; i++) {5. int fd = open(argv[i], O_RDONLY);6. fstat(fd, &buf);7. char *out = malloc(60);8. sprintf(out, "%d", buf.st_size);9. if(verbose) {10. char *pview = malloc(51);11. read(fd, pview, 50);12. pview[50] = 0;13. strcat(out, pview);14. }15. printf("%s: %sn", argv[i], out);16. total_size += buf.st_size;17. }18. printf("total: %dn", total_size);}fileinfo
  20. Motivating Exampleint main(int argc, char **argv) {1. int verbose, i, total_size = 0;2. struct stat buf;3. verbose = atoi(argv[1]);4. for(i = 2; i < argc; i++) {5. int fd = open(argv[i], O_RDONLY);6. fstat(fd, &buf);7. char *out = malloc(60);8. sprintf(out, "%d", buf.st_size);9. if(verbose) {10. char *pview = malloc(51);11. read(fd, pview, 50);12. pview[50] = 0;13. strcat(out, pview);14. }15. printf("%s: %sn", argv[i], out);16. total_size += buf.st_size;17. }18. printf("total: %dn", total_size);}fileinfoOverflow out
  21. Motivating Exampleint main(int argc, char **argv) {1. int verbose, i, total_size = 0;2. struct stat buf;3. verbose = atoi(argv[1]);4. for(i = 2; i < argc; i++) {5. int fd = open(argv[i], O_RDONLY);6. fstat(fd, &buf);7. char *out = malloc(60);8. sprintf(out, "%d", buf.st_size);9. if(verbose) {10. char *pview = malloc(51);11. read(fd, pview, 50);12. pview[50] = 0;13. strcat(out, pview);14. }15. printf("%s: %sn", argv[i], out);16. total_size += buf.st_size;17. }18. printf("total: %dn", total_size);}fileinfobuf.st_size ≥ 1GBOverflow out
  22. Motivating Exampleint main(int argc, char **argv) {1. int verbose, i, total_size = 0;2. struct stat buf;3. verbose = atoi(argv[1]);4. for(i = 2; i < argc; i++) {5. int fd = open(argv[i], O_RDONLY);6. fstat(fd, &buf);7. char *out = malloc(60);8. sprintf(out, "%d", buf.st_size);9. if(verbose) {10. char *pview = malloc(51);11. read(fd, pview, 50);12. pview[50] = 0;13. strcat(out, pview);14. }15. printf("%s: %sn", argv[i], out);16. total_size += buf.st_size;17. }18. printf("total: %dn", total_size);}fileinfobuf.st_size ≥ 1GBverbose is trueOverflow out
  23. Motivating Exampleint main(int argc, char **argv) {1. int verbose, i, total_size = 0;2. struct stat buf;3. verbose = atoi(argv[1]);4. for(i = 2; i < argc; i++) {5. int fd = open(argv[i], O_RDONLY);6. fstat(fd, &buf);7. char *out = malloc(60);8. sprintf(out, "%d", buf.st_size);9. if(verbose) {10. char *pview = malloc(51);11. read(fd, pview, 50);12. pview[50] = 0;13. strcat(out, pview);14. }15. printf("%s: %sn", argv[i], out);16. total_size += buf.st_size;17. }18. printf("total: %dn", total_size);}fileinfobuf.st_size ≥ 1GBverbose is trueOverflow outread 50 characters
  24. Motivating Exampleint main(int argc, char **argv) {1. int verbose, i, total_size = 0;2. struct stat buf;3. verbose = atoi(argv[1]);4. for(i = 2; i < argc; i++) {5. int fd = open(argv[i], O_RDONLY);6. fstat(fd, &buf);7. char *out = malloc(60);8. sprintf(out, "%d", buf.st_size);9. if(verbose) {10. char *pview = malloc(51);11. read(fd, pview, 50);12. pview[50] = 0;13. strcat(out, pview);14. }15. printf("%s: %sn", argv[i], out);16. total_size += buf.st_size;17. }18. printf("total: %dn", total_size);}fileinfo
  25. 1. Many more inputs than linesof code.Motivating Exampleint main(int argc, char **argv) {1. int verbose, i, total_size = 0;2. struct stat buf;3. verbose = atoi(argv[1]);4. for(i = 2; i < argc; i++) {5. int fd = open(argv[i], O_RDONLY);6. fstat(fd, &buf);7. char *out = malloc(60);8. sprintf(out, "%d", buf.st_size);9. if(verbose) {10. char *pview = malloc(51);11. read(fd, pview, 50);12. pview[50] = 0;13. strcat(out, pview);14. }15. printf("%s: %sn", argv[i], out);16. total_size += buf.st_size;17. }18. printf("total: %dn", total_size);}fileinfo
  26. 1. Many more inputs than linesof code.2. Understanding the failurerequires tracing interactionsbetween inputs from multiplesources.Motivating Exampleint main(int argc, char **argv) {1. int verbose, i, total_size = 0;2. struct stat buf;3. verbose = atoi(argv[1]);4. for(i = 2; i < argc; i++) {5. int fd = open(argv[i], O_RDONLY);6. fstat(fd, &buf);7. char *out = malloc(60);8. sprintf(out, "%d", buf.st_size);9. if(verbose) {10. char *pview = malloc(51);11. read(fd, pview, 50);12. pview[50] = 0;13. strcat(out, pview);14. }15. printf("%s: %sn", argv[i], out);16. total_size += buf.st_size;17. }18. printf("total: %dn", total_size);}fileinfo
  27. 1. Many more inputs than linesof code.2. Understanding the failurerequires tracing interactionsbetween inputs from multiplesources.3. Only a small percentage of allinputs are relevant for thefailure.Motivating Exampleint main(int argc, char **argv) {1. int verbose, i, total_size = 0;2. struct stat buf;3. verbose = atoi(argv[1]);4. for(i = 2; i < argc; i++) {5. int fd = open(argv[i], O_RDONLY);6. fstat(fd, &buf);7. char *out = malloc(60);8. sprintf(out, "%d", buf.st_size);9. if(verbose) {10. char *pview = malloc(51);11. read(fd, pview, 50);12. pview[50] = 0;13. strcat(out, pview);14. }15. printf("%s: %sn", argv[i], out);16. total_size += buf.st_size;17. }18. printf("total: %dn", total_size);}fileinfo
  28. fileinfoPenumbra Overviewfoo: 512 ... bar: 1024 ... baz: 150... total: 150...Foo512BBar1KBBaz1.5GB
  29. fileinfoPenumbra Overviewfoo: 512 ... bar: 1024 ... baz: 150... total: 150...Foo512BBar1KBBaz1.5GBRelevant context:1. When the failure occurs.2. Which data are involved inthe failure.
  30. fileinfoPenumbra Overviewfoo: 512 ... bar: 1024 ... baz: 150... total: 150...Foo512BBar1KBBaz1.5GB13. strcat(out, pview);In general, it is chosen usingtraditional debugging methods.
  31. fileinfoPenumbra Overviewfoo: 512 ... bar: 1024 ... baz: 150... total: 150...Foo512BBar1KBBaz1.5GB
  32. fileinfoPenumbra Overviewfoo: 512 ... bar: 1024 ... baz: 150... total: 150...Foo512BBar1KBBaz1.5GB1 Taint inputs
  33. fileinfoPenumbra Overviewfoo: 512 ... bar: 1024 ... baz: 150... total: 150...Foo512BBar1KBBaz1.5GB1 Taint inputs1234567890
  34. fileinfoPenumbra Overviewfoo: 512 ... bar: 1024 ... baz: 150... total: 150...Foo512BBar1KBBaz1.5GB1 Taint inputs2 Propagatetaint marks1234567890
  35. fileinfoPenumbra Overviewfoo: 512 ... bar: 1024 ... baz: 150... total: 150...Foo512BBar1KBBaz1.5GB1 Taint inputs2 Propagatetaint marks
  36. fileinfoPenumbra Overviewfoo: 512 ... bar: 1024 ... baz: 150... total: 150...Foo512BBar1KBBaz1.5GB1 Taint inputs2 Propagatetaint marks3 Identifyrelevant inputs
  37. fileinfoPenumbra Overviewfoo: 512 ... bar: 1024 ... baz: 150... total: 150...Foo512BBar1KBBaz1.5GB1 Taint inputs2 Propagatetaint marks3 Identifyrelevant inputs0 8 9
  38. fileinfoPenumbra Overviewfoo: 512 ... bar: 1024 ... baz: 150... total: 150...Foo512BBar1KBBaz1.5GB1 Taint inputs2 Propagatetaint marks3 Identifyrelevant inputs0 8 9
  39. fileinfoPenumbra Overviewfoo: 512 ... bar: 1024 ... baz: 150... total: 150...Foo512BBar1KBBaz1.5GB1 Taint inputs2 Propagatetaint marks3 Identifyrelevant inputs0 8 9verbose is trueread 50 charactersbuf.st_size ≥ 1GB
  40. Outline• Penumbra approach1. Tainting inputs2. Propagating taint marks3. Identifying relevant inputs• Evaluation• Conclusions and future work
  41. 1: Tainting InputsAssign a taint mark to each input as it enters the application.
  42. 1: Tainting InputsAssign a taint mark to each input as it enters the application.Per-byte Per-entity Domain specific
  43. 1: Tainting InputsAssign a uniquetaint mark to eachbyte.(read from files)Assign the sametaint mark torelated bytes.(argv, argc, fstat, ...)Assign taint marksbased on user-providedinformation.Assign a taint mark to each input as it enters the application.Per-byte Per-entity Domain specific
  44. 1: Tainting InputsAssign a uniquetaint mark to eachbyte.(read from files)Assign the sametaint mark torelated bytes.(argv, argc, fstat, ...)Assign taint marksbased on user-providedinformation.Assign a taint mark to each input as it enters the application.PreciseidentificationPer-byte Per-entity Domain specific
  45. 1: Tainting InputsAssign a uniquetaint mark to eachbyte.(read from files)Assign the sametaint mark torelated bytes.(argv, argc, fstat, ...)Assign taint marksbased on user-providedinformation.Assign a taint mark to each input as it enters the application.PreciseidentificationUnnecessarilyexpensivePer-byte Per-entity Domain specific
  46. 1: Tainting InputsAssign a uniquetaint mark to eachbyte.(read from files)Assign the sametaint mark torelated bytes.(argv, argc, fstat, ...)Assign taint marksbased on user-providedinformation.Assign a taint mark to each input as it enters the application.PreciseidentificationUnnecessarilyexpensivePer-byte Per-entity Domain specific
  47. 1: Tainting InputsAssign a uniquetaint mark to eachbyte.(read from files)Assign the sametaint mark torelated bytes.(argv, argc, fstat, ...)Assign taint marksbased on user-providedinformation.Assign a taint mark to each input as it enters the application.PreciseidentificationUnnecessarilyexpensiveMaintains per -byte precisionPer-byte Per-entity Domain specific
  48. 1: Tainting InputsAssign a uniquetaint mark to eachbyte.(read from files)Assign the sametaint mark torelated bytes.(argv, argc, fstat, ...)Assign taint marksbased on user-providedinformation.Assign a taint mark to each input as it enters the application.PreciseidentificationUnnecessarilyexpensiveMaintains per -byte precisionIncreasesscalabilityPer-byte Per-entity Domain specific
  49. 1: Tainting InputsAssign a uniquetaint mark to eachbyte.(read from files)Assign the sametaint mark torelated bytes.(argv, argc, fstat, ...)Assign taint marksbased on user-providedinformation.Assign a taint mark to each input as it enters the application.PreciseidentificationUnnecessarilyexpensiveMaintains per -byte precisionIncreasesscalabilityPer-byte Per-entity Domain specific
  50. 1: Tainting InputsAssign a uniquetaint mark to eachbyte.(read from files)Assign the sametaint mark torelated bytes.(argv, argc, fstat, ...)Assign taint marksbased on user-providedinformation.Assign a taint mark to each input as it enters the application.PreciseidentificationUnnecessarilyexpensiveMaintains per -byte precisionIncreasesscalabilityPer-byte Per-entity Domain specificMaintains per -byte precision
  51. 1: Tainting InputsAssign a uniquetaint mark to eachbyte.(read from files)Assign the sametaint mark torelated bytes.(argv, argc, fstat, ...)Assign taint marksbased on user-providedinformation.Assign a taint mark to each input as it enters the application.PreciseidentificationUnnecessarilyexpensiveMaintains per -byte precisionIncreasesscalabilityPer-byte Per-entity Domain specificMaintains per -byte precisionFurther increasesscalability
  52. 1: Tainting InputsAssign a uniquetaint mark to eachbyte.(read from files)Assign the sametaint mark torelated bytes.(argv, argc, fstat, ...)Assign taint marksbased on user-providedinformation.Assign a taint mark to each input as it enters the application.When a taint mark is assigned to an input, log theinput’s value and where the input was read from.PreciseidentificationUnnecessarilyexpensiveMaintains per -byte precisionIncreasesscalabilityPer-byte Per-entity Domain specificMaintains per -byte precisionFurther increasesscalability
  53. 2: Propagating Taint Marks
  54. 2: Propagating Taint MarksData-flowPropagation (DF)Data- and control-flowPropagation (DF + CF)
  55. 2: Propagating Taint MarksTaint marks flow along onlydata dependencies.Taint marks flow along data andcontrol dependencies.Data-flowPropagation (DF)Data- and control-flowPropagation (DF + CF)
  56. 2: Propagating Taint MarksTaint marks flow along onlydata dependencies.Taint marks flow along data andcontrol dependencies.C = A + B;Data-flowPropagation (DF)Data- and control-flowPropagation (DF + CF)
  57. 2: Propagating Taint MarksTaint marks flow along onlydata dependencies.Taint marks flow along data andcontrol dependencies.C = A + B;1 2Data-flowPropagation (DF)Data- and control-flowPropagation (DF + CF)
  58. 2: Propagating Taint MarksTaint marks flow along onlydata dependencies.Taint marks flow along data andcontrol dependencies.C = A + B;1 21 2Data-flowPropagation (DF)Data- and control-flowPropagation (DF + CF)
  59. 2: Propagating Taint MarksTaint marks flow along onlydata dependencies.Taint marks flow along data andcontrol dependencies.C = A + B;1 21 2Data-flowPropagation (DF)Data- and control-flowPropagation (DF + CF)
  60. 2: Propagating Taint MarksTaint marks flow along onlydata dependencies.Taint marks flow along data andcontrol dependencies.C = A + B;if(X) {C = A + B;}1 21 2Data-flowPropagation (DF)Data- and control-flowPropagation (DF + CF)
  61. 2: Propagating Taint MarksTaint marks flow along onlydata dependencies.Taint marks flow along data andcontrol dependencies.C = A + B;if(X) {C = A + B;}1 21 21 23Data-flowPropagation (DF)Data- and control-flowPropagation (DF + CF)
  62. 2: Propagating Taint MarksTaint marks flow along onlydata dependencies.Taint marks flow along data andcontrol dependencies.C = A + B;if(X) {C = A + B;}1 21 21 231 2 3Data-flowPropagation (DF)Data- and control-flowPropagation (DF + CF)
  63. 2: Propagating Taint MarksTaint marks flow along onlydata dependencies.Taint marks flow along data andcontrol dependencies.C = A + B;if(X) {C = A + B;}1 21 21 231 2 3The effectiveness of each option depends on the particular failure.Data-flowPropagation (DF)Data- and control-flowPropagation (DF + CF)
  64. 3: Identifying Relevant-inputs1. Relevant context indicateswhich data is involved in theconsidered failure.2. Identify which taint marks asassociated with the dataindicated by the relevantcontext.3. Use recorded logs toreconstruct inputs that areidentified by the taint marks.Baz1.5GB
  65. Prototype ImplementationTraceProcessorTracegenerator
  66. input vectorexecutabletracerelevant contextPrototype ImplementationTraceProcessorTracegenerator
  67. input vectorexecutabletracerelevant contextPrototype ImplementationTraceProcessorTracegeneratorImplemented using Dytan, ageneric x86 tainting frameworkdeveloped in previous work[Clause and Orso 2007].
  68. input vectorexecutabletracerelevant contextPrototype ImplementationTraceProcessorTracegenerator
  69. input vectorexecutabletracerelevant contextPrototype ImplementationTraceProcessorTracegeneratorinput subset(DF)input subset(DF+CF)
  70. EvaluationStudy 1: Effectiveness for debugging real failuresStudy 2: Comparison with Delta Debugging
  71. EvaluationStudy 1: Effectiveness for debugging real failuresStudy 2: Comparison with Delta DebuggingApplication KLoC Fault locationbc 1.06 10.5 more_arrays : 177gzip 1.24 6.3 get_istat : 828ncompress 4.24 1.4 comprexx : 896pine 4.44 239.1 rfc822_cat : 260squid 2.3 69.9 ftpBuildTitleUrl : 1024Subjects:
  72. EvaluationStudy 1: Effectiveness for debugging real failuresStudy 2: Comparison with Delta DebuggingApplication KLoC Fault locationbc 1.06 10.5 more_arrays : 177gzip 1.24 6.3 get_istat : 828ncompress 4.24 1.4 comprexx : 896pine 4.44 239.1 rfc822_cat : 260squid 2.3 69.9 ftpBuildTitleUrl : 1024Subjects:We selected a failure-revealing input vector for each subject.
  73. Data GenerationPenumbra Delta DebuggingSetup(manual)Execution(automated)Choose a relevantcontextCreate an automatedoracleUse prototype tool toidentify failure-relevantinputs (DF and DF +CF)Use the standard DeltaDebuggingimplementation tominimize inputs.
  74. Data GenerationPenumbra Delta DebuggingSetup(manual)Execution(automated)Choose a relevantcontextCreate an automatedoracleUse prototype tool toidentify failure-relevantinputs (DF and DF +CF)Use the standard DeltaDebuggingimplementation tominimize inputs.
  75. Data GenerationPenumbra Delta DebuggingSetup(manual)Execution(automated)Choose a relevantcontextCreate an automatedoracleUse prototype tool toidentify failure-relevantinputs (DF and DF +CF)Use the standard DeltaDebuggingimplementation tominimize inputs.• Location: statement wherethe failure occurs.• Data: any data read by suchstatement
  76. Data GenerationPenumbra Delta DebuggingSetup(manual)Execution(automated)Choose a relevantcontextCreate an automatedoracleUse prototype tool toidentify failure-relevantinputs (DF and DF +CF)Use the standard DeltaDebuggingimplementation tominimize inputs.
  77. Data GenerationPenumbra Delta DebuggingSetup(manual)Execution(automated)Choose a relevantcontextCreate an automatedoracleUse prototype tool toidentify failure-relevantinputs (DF and DF +CF)Use the standard DeltaDebuggingimplementation tominimize inputs.
  78. Data GenerationPenumbra Delta DebuggingSetup(manual)Execution(automated)Choose a relevantcontextCreate an automatedoracleUse prototype tool toidentify failure-relevantinputs (DF and DF +CF)Use the standard DeltaDebuggingimplementation tominimize inputs.• Use gdb to inspect stacktrace and program data.• One second timeout toprevent incorrect results.
  79. Data GenerationPenumbra Delta DebuggingSetup(manual)Execution(automated)Choose a relevantcontextCreate an automatedoracleUse prototype tool toidentify failure-relevantinputs (DF and DF +CF)Use the standard DeltaDebuggingimplementation tominimize inputs.
  80. Study 1: EffectivenessIs the information thatPenumbra provides helpful fordebugging real failures?
  81. Study 1 Results: gzip & ncompressCrash when a file name is longer than 1,024 characters.
  82. Study 1 Results: gzip & ncompressContents&AttributesContents&AttributesbarContents&Attributesfoo./gzipCrash when a file name is longer than 1,024 characters.# Inputs: 10,000,056longfilename[ ]
  83. Study 1 Results: gzip & ncompressContents&AttributesContents&AttributesbarContents&Attributesfoo./gzipCrash when a file name is longer than 1,024 characters.# Inputs: 10,000,056 # Relevant (DF): 1longfilename[ ]
  84. Study 1 Results: gzip & ncompressContents&AttributesContents&AttributesbarContents&Attributesfoo./gzipCrash when a file name is longer than 1,024 characters.# Relevant (DF + CF): 3# Inputs: 10,000,056 # Relevant (DF): 1longfilename[ ]
  85. Study 1 Results: pineCrash when a “from” field contains 22 or more double quote characters.
  86. Study 1 Results: pine# Inputs: 15,103,766...From clause@boar Tue Feb 20 11:49:53 2007Return-Path: <clause@boar>X-Original-To: clauseDelivered-To: clause@boarReceived: by boar (Postfix, from userid 1000)id 88EDD1724523; Tue, 20 Feb 2007 11:49:53 -0500 (EST)To: clause@boarSubject: testMessage-Id: <20070220164953.88EDD1724523@boar>Date: Tue, 20 Feb 2007 11:49:53 -0500 (EST)From: """"""""""""""""""""""""""""""""@host.fubarX-IMAPbase: 1172160370 390Status: OX-Status:X-Keywords:X-UID: 5...Crash when a “from” field contains 22 or more double quote characters.
  87. Study 1 Results: pine# Inputs: 15,103,766...From clause@boar Tue Feb 20 11:49:53 2007Return-Path: <clause@boar>X-Original-To: clauseDelivered-To: clause@boarReceived: by boar (Postfix, from userid 1000)id 88EDD1724523; Tue, 20 Feb 2007 11:49:53 -0500 (EST)To: clause@boarSubject: testMessage-Id: <20070220164953.88EDD1724523@boar>Date: Tue, 20 Feb 2007 11:49:53 -0500 (EST)From: """"""""""""""""""""""""""""""""@host.fubarX-IMAPbase: 1172160370 390Status: OX-Status:X-Keywords:X-UID: 5...… …" " " " " " " " " " " "Crash when a “from” field contains 22 or more double quote characters.
  88. Study 1 Results: pine# Inputs: 15,103,766 # Relevant (DF): 26...From clause@boar Tue Feb 20 11:49:53 2007Return-Path: <clause@boar>X-Original-To: clauseDelivered-To: clause@boarReceived: by boar (Postfix, from userid 1000)id 88EDD1724523; Tue, 20 Feb 2007 11:49:53 -0500 (EST)To: clause@boarSubject: testMessage-Id: <20070220164953.88EDD1724523@boar>Date: Tue, 20 Feb 2007 11:49:53 -0500 (EST)From: """"""""""""""""""""""""""""""""@host.fubarX-IMAPbase: 1172160370 390Status: OX-Status:X-Keywords:X-UID: 5...… …" " " " " " " " " " " "Crash when a “from” field contains 22 or more double quote characters.
  89. Study 1 Results: pine# Relevant (DF + CF):15,100,344# Inputs: 15,103,766 # Relevant (DF): 26...From clause@boar Tue Feb 20 11:49:53 2007Return-Path: <clause@boar>X-Original-To: clauseDelivered-To: clause@boarReceived: by boar (Postfix, from userid 1000)id 88EDD1724523; Tue, 20 Feb 2007 11:49:53 -0500 (EST)To: clause@boarSubject: testMessage-Id: <20070220164953.88EDD1724523@boar>Date: Tue, 20 Feb 2007 11:49:53 -0500 (EST)From: """"""""""""""""""""""""""""""""@host.fubarX-IMAPbase: 1172160370 390Status: OX-Status:X-Keywords:X-UID: 5...… …" " " " " " " " " " " "Crash when a “from” field contains 22 or more double quote characters.
  90. Study 1: Conclusions
  91. Study 1: Conclusions1. Data-flow propagation is always effective,data- and control-flow propagation is sometimeseffective.➡ Use data-flow first then, if necessary, use control-flow.
  92. Study 1: Conclusions1. Data-flow propagation is always effective,data- and control-flow propagation is sometimeseffective.➡ Use data-flow first then, if necessary, use control-flow.2. Inputs identified by Penumbra correspond to thefailure conditions.➡ Our technique is effective in assisting the debugging ofreal failures.
  93. Study 2: Comparison with Delta DebuggingRQ1: How much manual effortdoes each technique require?RQ2: How long does it take tofix a considered failure giventhe information provided byeach technique?
  94. RQ1: Manual effortUse setup-time as a proxy for manual (developer) effort.
  95. RQ1: Manual effortUse setup-time as a proxy for manual (developer) effort.5,40012,6001,8001,8001259731470163ncompress bc pineSetup-time(s)gzipPenumbraDelta Debuggingsquid
  96. RQ1: Manual effortUse setup-time as a proxy for manual (developer) effort.5,40012,6001,8001,8001259731470163ncompress bc pineSetup-time(s)gzipPenumbraDelta Debuggingsquid
  97. RQ1: Manual effortUse setup-time as a proxy for manual (developer) effort.5,40012,6001,8001,8001259731470163ncompress bc pineSetup-time(s)gzipPenumbraDelta Debuggingsquid
  98. RQ1: Manual effortUse setup-time as a proxy for manual (developer) effort.5,40012,6001,8001,8001259731470163ncompress bc pineSetup-time(s)gzipPenumbraDelta DebuggingsquidPenumbra requires considerably less setup time than Delta Debugging(although more time time overall for gzip and ncompress).
  99. RQ2: Debugging EffortUse number of relevant inputs as a proxy for debugging effort.
  100. RQ2: Debugging EffortSubject PenumbraPenumbra Delta DebuggingDF DF + CFbc 209 743 285gzip 1 3 1ncompress 1 3 1pine 26 15,100,344 90squid 89 2,056 —Use number of relevant inputs as a proxy for debugging effort.
  101. RQ2: Debugging EffortSubject PenumbraPenumbra Delta DebuggingDF DF + CFbc 209 743 285gzip 1 3 1ncompress 1 3 1pine 26 15,100,344 90squid 89 2,056 —Use number of relevant inputs as a proxy for debugging effort.• Penumbra (DF) is comparable to (slightly better than) Delta Debugging.
  102. RQ2: Debugging EffortSubject PenumbraPenumbra Delta DebuggingDF DF + CFbc 209 743 285gzip 1 3 1ncompress 1 3 1pine 26 15,100,344 90squid 89 2,056 —Use number of relevant inputs as a proxy for debugging effort.• Penumbra (DF) is comparable to (slightly better than) Delta Debugging.• Penumbra (DF + CF) is likely less effective for bc, pine, and squid
  103. Conclusions & Future Work• Novel technique for identifying failure-relevantinputs.• Overcomes limitations of existing approaches• Single execution• Minimal manual effort• Comparable effectiveness• Combine Penumbra with existing code-centrictechniques.

×