Fuzz Testing

Atul S. Khot (atul.khot@gmail.com)
VodQA ThoughtWorks Pune - 2013
Random behavior aka Insanity


Testing the “drink maker”



lemon juice + milk + tea leaves + (black?) salt



Rather a...
Of talking gibberish


Try throwing senseless data at your system



And see what is uncovered







Hangs/infinite ...
Is tommath right?




How do I test tommath gets its arithmetic right?
Generate random numbers – next generate artihmeti...
Uncovering performance
bottlenecks










A campaign manager – customer needs to send a text sms to
16 million cel...
The surprise is revealed


Our algorithms were right



No big deadlocks



For this huge run – profiler indicated log4...
Ideas galore








Needed to test a complex tree manipulation algorithm written
in TCL
I coded the algorithm – to te...
Bugs surface...


Revealed a bug - we needed to make some regex greedier



Was a corner case



Hard to see how we cou...
Platypus – (http://platypus.pz.org/)


It is just (?) simplified Latex



Elaborate parser



Fuzz unleashed



Produc...
Platypus – (http://platypus.pz.org/)


It is just (?) simplified Latex



Elaborate parser



Fuzz unleashed



Produc...
Upcoming SlideShare
Loading in …5
×

Fuzz Testing-Atul Khot

472 views
360 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
472
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Fuzz Testing-Atul Khot

  1. 1. Fuzz Testing Atul S. Khot (atul.khot@gmail.com) VodQA ThoughtWorks Pune - 2013
  2. 2. Random behavior aka Insanity  Testing the “drink maker”  lemon juice + milk + tea leaves + (black?) salt  Rather a fuzzy drink ;-)   We human beings are somewhat “conditioned” - computers aren't And that is good!!!
  3. 3. Of talking gibberish  Try throwing senseless data at your system  And see what is uncovered    Hangs/infinite loops/exceptions/Deadlocks/race conditions whatever ;-) Better let the computer go insane (it is all raring to go...) And no call to recall your initial C days... Pointers going haywire? Etc...
  4. 4. Is tommath right?   How do I test tommath gets its arithmetic right? Generate random numbers – next generate artihmetic expressions (*,/,+,-)  Run the expressions throught tommath  Run the expressions through gnu bc   Compare – 30 million – different expressions – over 4 days You get a fair good idea All gory details in my Linux For You article
  5. 5. Uncovering performance bottlenecks      A campaign manager – customer needs to send a text sms to 16 million cell numbers Cannot test – as one run would cost $35000/Decouple (very handy techique) – instead of sending to real webservice – send it to a mock Shell scripts run in parallel – you can spawn many thousand parallel processes easily... Each process is a simple socket client – sending a mobile number – and the message
  6. 6. The surprise is revealed  Our algorithms were right  No big deadlocks  For this huge run – profiler indicated log4j as the culprit  Log4j's writing to a log file – was a bottleneck   Solution - use an Async appender – Events are logged asynchronously Nobody thought log4j as a possible suspect ;-)
  7. 7. Ideas galore     Needed to test a complex tree manipulation algorithm written in TCL I coded the algorithm – to test I needed very big trees Directories – Perl slicing and dicing – C++ boost library (open source) – Files correspond to leaves in the tree Directories are essentially random trees –
  8. 8. Bugs surface...  Revealed a bug - we needed to make some regex greedier  Was a corner case  Hard to see how we could have come upon it with manual testing  A TCL expert from Norway carefully reviewed  Okayed – big moment ;-)
  9. 9. Platypus – (http://platypus.pz.org/)  It is just (?) simplified Latex  Elaborate parser  Fuzz unleashed  Produced a hang  Deemed low priority –  Will eventually get addressed
  10. 10. Platypus – (http://platypus.pz.org/)  It is just (?) simplified Latex  Elaborate parser  Fuzz unleashed  Produced a hang  Deemed low priority –  Will eventually get addressed

×