Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Parser-Directed Fuzzing

9 views

Published on

Slides for Parser-Directed Fuzzing presented at PLDI 2019 in Phoenix.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Parser-Directed Fuzzing

  1. 1. PARSER-DIRECTED FUZZING BJÖRN MATHIS | RAHUL GOPINATH | MICHAËL MERA | ALEXANDER KAMPMANN | 
 MATTHIAS HÖSCHELE | ANDREAS ZELLER
  2. 2. WHY FUZZING? 2
  3. 3. WHY FUZZING? 2 if (nextChar() == '{') { if (nextChar() == '"') { while (nextchar() != '"'); if (nextChar() == ':') { run(); } else { syntaxError(); } } else { syntaxError(); } } else { syntaxError(); }
  4. 4. WHY FUZZING? 2 func error { " ¬" " : ¬{ ¬" ¬:
  5. 5. WHY FUZZING? 2 HOW TO TEST THIS? func error { " ¬" " : ¬{ ¬" ¬:
  6. 6. RANDOM FUZZING 3 func error { " ¬" " : ¬{ ¬" ¬: RANDOM FUZZER
  7. 7. RANDOM FUZZING 3 func error { " ¬" " : ¬{ ¬" ¬: RANDOM FUZZER 83HRF[IN
  8. 8. RANDOM FUZZING 3 func error { " ¬" " : ¬{ ¬" ¬: RANDOM FUZZER 83HRF[IN249Y8VAFG
  9. 9. RANDOM FUZZING 3 func error { " ¬" " : ¬{ ¬" ¬: RANDOM FUZZER 83HRF[IN249Y8VAFG{AP083NGRV
  10. 10. SYMBOLIC FUZZING 4 SYMBOLIC FUZZER func error { " ¬" " : ¬{ ¬" ¬:
  11. 11. SYMBOLIC FUZZING 4 SYMBOLIC FUZZER func error { " ¬" " : ¬{ ¬" ¬:
  12. 12. SYMBOLIC FUZZING 4 SYMBOLIC FUZZER func error { " ¬" " : ¬{ ¬" ¬: {#
  13. 13. SYMBOLIC FUZZING 4 SYMBOLIC FUZZER func error { " ¬" " : ¬{ ¬" ¬: {#
  14. 14. SYMBOLIC FUZZING 4 SYMBOLIC FUZZER func error { " ¬" " : ¬{ ¬" ¬: {#
  15. 15. SYMBOLIC FUZZING 4 SYMBOLIC FUZZER func error { " ¬" " : ¬{ ¬" ¬: {#{"A"=
  16. 16. SYMBOLIC FUZZING 4 SYMBOLIC FUZZER func error { " ¬" " : ¬{ ¬" ¬: {#{"A"=
  17. 17. func error { " ¬" " : ¬{ ¬" ¬: SYMBOLIC FUZZING 5 SYMBOLIC FUZZER
  18. 18. func error { " ¬" " : ¬{ ¬" ¬: SYMBOLIC FUZZING 5 SYMBOLIC FUZZER
  19. 19. 6
  20. 20. 6 PARSER- DIRECTED FUZZING OUR ALTERNATIVE
  21. 21. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER
  22. 22. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A
  23. 23. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A
  24. 24. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A
  25. 25. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A {
  26. 26. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A {
  27. 27. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A {
  28. 28. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A {
  29. 29. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A { {"
  30. 30. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A { {"
  31. 31. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A { {"
  32. 32. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A { {"
  33. 33. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A { {" {""
  34. 34. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A { {" {""
  35. 35. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A { {" {""
  36. 36. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A { {" {""
  37. 37. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A { {" {"" {"":
  38. 38. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A { {" {"" {"":
  39. 39. PARSER-DIRECTED FUZZING 7 func error { " ¬" " : ¬{ ¬" ¬: PFUZZER A { {" {"" {"":
  40. 40. PARSER-DIRECTED FUZZING - GENERAL STEPS 8
  41. 41. PARSER-DIRECTED FUZZING - GENERAL STEPS 8 1. Start with a random input
  42. 42. PARSER-DIRECTED FUZZING - GENERAL STEPS 8 1. Start with a random input 2. Execute input on program under tainting
  43. 43. PARSER-DIRECTED FUZZING - GENERAL STEPS 8 1. Start with a random input 2. Execute input on program under tainting 3. Extract character comparisons
  44. 44. PARSER-DIRECTED FUZZING - GENERAL STEPS 8 1. Start with a random input 2. Execute input on program under tainting 3. Extract character comparisons 4. Replace the last compared character with one of the values it was compared to
  45. 45. PARSER-DIRECTED FUZZING - GENERAL STEPS 8 1. Start with a random input 2. Execute input on program under tainting 3. Extract character comparisons 4. Replace the last compared character with one of the values it was compared to 5. Repeat at 2. with the newly generated input
  46. 46. PARSER-DIRECTED FUZZING - GENERAL STEPS 8 1. Start with a random input 2. Execute input on program under tainting 3. Extract character comparisons 4. Replace the last compared character with one of the values it was compared to 5. Repeat at 2. with the newly generated input GENERATING INPUTS FOR COMPLEX INPUT LANGUAGES
  47. 47. EVALUATION - SETUP 9
  48. 48. EVALUATION - SETUP 9 AFL KLEE PFUZZER TOOLS
  49. 49. EVALUATION - SETUP 9 AFL KLEE PFUZZER TOOLS 48 HOURS | 3 RUNS | BEST RUN
  50. 50. EVALUATION - SETUP 9 AFL KLEE PFUZZER TOOLS INI CSV JSON TINYC MJS SUBJECTS 48 HOURS | 3 RUNS | BEST RUN
  51. 51. EVALUATION - COVERAGE ACHIEVED WITH VALID INPUTS 10
  52. 52. EVALUATION - COVERAGE ACHIEVED WITH VALID INPUTS 10 BranchCoverage 25 % 50 % 75 % 100 % Subject ini csv Json tinyC mjs 21 81 14 63 56 10 68 18 60 50 34 80 20 68 73 AFL Klee pFuzzer
  53. 53. EVALUATION - TOKENS >4 USED IN VALID MJS INPUTS 11
  54. 54. EVALUATION - TOKENS >4 USED IN VALID MJS INPUTS 11 AFL KLEE pFuzzer break ✘ ✘ ✔ catch ✘ ✘ ✔ false ✘ ✘ ✔ throw ✘ ✘ ✔ typeof ✘ ✘ ✔ delete ✘ ✘ ✔ switch ✘ ✘ ✔ return ✘ ✘ ✔ continue ✘ ✘ ✔ undefined ✘ ✘ ✔ instanceof ✘ ✘ ✔
  55. 55. EVALUATION - LENGTH OF TOKENS USED 12
  56. 56. EVALUATION - LENGTH OF TOKENS USED 12 Token Length 1 2 3 4 5 6 7 8 9 10 AFL Klee pFuzzer
  57. 57. EVALUATION - LENGTH OF TOKENS USED 12 Token Length 1 2 3 4 5 6 7 8 9 10 AFL Klee pFuzzer 53 24 9 2
  58. 58. EVALUATION - LENGTH OF TOKENS USED 12 Token Length 1 2 3 4 5 6 7 8 9 10 AFL Klee pFuzzer 25 2 2 1 53 24 9 2
  59. 59. EVALUATION - LENGTH OF TOKENS USED 12 Token Length 1 2 3 4 5 6 7 8 9 10 AFL Klee pFuzzer 52 19 6 8 6 4 1 1 1 25 2 2 1 53 24 9 2
  60. 60. EVALUATION - LENGTH OF TOKENS USED 12 Token Length 1 2 3 4 5 6 7 8 9 10 AFL Klee pFuzzer 52 19 6 8 6 4 1 1 1 25 2 2 1 53 24 9 2 if(2)(2); let eeeeeeee … 9===9===1= …
  61. 61. EVALUATION - LENGTH OF TOKENS USED 12 Token Length 1 2 3 4 5 6 7 8 9 10 AFL Klee pFuzzer 52 19 6 8 6 4 1 1 1 25 2 2 1 53 24 9 2 if(2)(2); let eeeeeeee … 9===9===1= … 000<(((n(nd … /*nÿ {"":0}ÿ
  62. 62. EVALUATION - LENGTH OF TOKENS USED 12 Token Length 1 2 3 4 5 6 7 8 9 10 AFL Klee pFuzzer 52 19 6 8 6 4 1 1 1 25 2 2 1 53 24 9 2 if(2)(2); let eeeeeeee … 9===9===1= … 000<(((n(nd … /*nÿ {"":0}ÿ {;;while (9);} 5typeof' continue
  63. 63. CURRENT WORK 13
  64. 64. CURRENT WORK 13 • Apply pFuzzer on multi-pass input processing
  65. 65. CURRENT WORK 13 • Apply pFuzzer on multi-pass input processing • apply to scanner, learning lexical tokens
  66. 66. CURRENT WORK 13 • Apply pFuzzer on multi-pass input processing • apply to scanner, learning lexical tokens • apply to parser using tokens learned
  67. 67. CURRENT WORK 13 • Apply pFuzzer on multi-pass input processing • apply to scanner, learning lexical tokens • apply to parser using tokens learned • Combine pFuzzer with grammar learning
  68. 68. CURRENT WORK 13 • Apply pFuzzer on multi-pass input processing • apply to scanner, learning lexical tokens • apply to parser using tokens learned • Combine pFuzzer with grammar learning • to learn lexical + syntactical descriptions
 of input formats
  69. 69. CURRENT WORK 13 • Apply pFuzzer on multi-pass input processing • apply to scanner, learning lexical tokens • apply to parser using tokens learned • Combine pFuzzer with grammar learning • to learn lexical + syntactical descriptions
 of input formats • to use resulting grammars
 for highly efficient test generation
  70. 70. CURRENT WORK 13 • Apply pFuzzer on multi-pass input processing • apply to scanner, learning lexical tokens • apply to parser using tokens learned • Combine pFuzzer with grammar learning • to learn lexical + syntactical descriptions
 of input formats • to use resulting grammars
 for highly efficient test generation • Guide test generation towards desired semantics!
  71. 71. PARSER-DIRECTED FUZZING BJÖRN MATHIS | RAHUL GOPINATH | MICHAËL MERA | ALEXANDER KAMPMANN | 
 MATTHIAS HÖSCHELE | ANDREAS ZELLER
  72. 72. PARSER-DIRECTED FUZZING BJÖRN MATHIS | RAHUL GOPINATH | MICHAËL MERA | ALEXANDER KAMPMANN | 
 MATTHIAS HÖSCHELE | ANDREAS ZELLER
  73. 73. PARSER-DIRECTED FUZZING BJÖRN MATHIS | RAHUL GOPINATH | MICHAËL MERA | ALEXANDER KAMPMANN | 
 MATTHIAS HÖSCHELE | ANDREAS ZELLER
  74. 74. PARSER-DIRECTED FUZZING BJÖRN MATHIS | RAHUL GOPINATH | MICHAËL MERA | ALEXANDER KAMPMANN | 
 MATTHIAS HÖSCHELE | ANDREAS ZELLER
  75. 75. PARSER-DIRECTED FUZZING BJÖRN MATHIS | RAHUL GOPINATH | MICHAËL MERA | ALEXANDER KAMPMANN | 
 MATTHIAS HÖSCHELE | ANDREAS ZELLER
  76. 76. PARSER-DIRECTED FUZZING BJÖRN MATHIS | RAHUL GOPINATH | MICHAËL MERA | ALEXANDER KAMPMANN | 
 MATTHIAS HÖSCHELE | ANDREAS ZELLER GITHUB.COM/UDS-SE/PFUZZER

×