Adam Doupé, Marco Cova and Giovanni Vigna          University of California, Santa Barbara                             DIM...
   Introduction to black box web vulnerability    scanners   Design of custom vulnerable website –    WackoPicko   Resu...
   Describe the design of a testing web    application   Identify a number of challenges that scanners    need to overco...
ServerCrawler      Attack   Analysis
   Authentication   Upload Pictures   Comment on Pictures   “Purchase” Pictures   Tag Search   Guestbook   Admin Area
   XSS    ◦ Reflected, Stored, and Reflected behind JavaScript   Session ID   Weak Password   Reflected SQL Injection...
   Reflected XSS behind Flash   Stored SQL Injection   Directory Traversal   Multi-step Stored XSS   Forceful Browsin...
   HTML Parsing   Multi-Step Process / State   Infinite Website   Authentication   Client-side Code    ◦ Web Input Ve...
Name           PriceAcunetix       $4,995 - $6,350AppScan        $12,550 - $32,500Burp           £125 ($190.82)Grendel-Sca...
   Each scanner run four times:    ◦ WackoPicko      Initial – No configuration (point and click)      Config – Given v...
Name         Reflected   Stored     SQL         Command     File        File        XSS via      XSS via             XSS  ...
   Missed by all scanners    ◦   Session ID    ◦   Weak Password    ◦   Parameter Manipulation    ◦   Forceful Browsing  ...
   Ranged from 0 to 200+    ◦ Average was ~25   Why?    ◦ Server Path Disclosure   “Actual” False Positives    ◦ Hailst...
   Strictly Dominates
MoreDominantLessDominant
   Default values   XSS attacks   Command-line Injection   SQL Injection   File Exposure   Remote Code Execution
   Number of Accesses    ◦ Range from ~50 per page to ~3,000 per page    ◦ Hailstorm accessed vulnerable pages that requi...
   Uploading a Picture    ◦ 2 Scanners uploaded without help    ◦ 3 Scanners unable to upload one!
   WIVET    ◦ 3 Scanners couldn’t complete      Paros and Burp - <base>      N-Stalker – Frame?    ◦ Dynamic JavaScript...
   Created an account successfully    ◦ 4 Scanners        Hailstorm        N-Stalker        NTOSpider        WebInspect
   Incorporate lots of logging in the application   Two versions of the site    ◦ No vulnerabilities    ◦ All vulnerabil...
   Ability to crawl as important as detection   Many vulnerabilities cannot be detected   Cost not directly proportiona...
Why Johnny Can't Pentest: An Analysis of Black-box Web Vulnerability Scanners
Why Johnny Can't Pentest: An Analysis of Black-box Web Vulnerability Scanners
Why Johnny Can't Pentest: An Analysis of Black-box Web Vulnerability Scanners
Upcoming SlideShare
Loading in …5
×

Why Johnny Can't Pentest: An Analysis of Black-box Web Vulnerability Scanners

4,900 views

Published on

Presentation at DIMVA 2010 of the paper "Why Johnny Can't Pentest: An Analysis of Black-box Web Vulnerability Scanners"

Full paper:
http://cs.ucsb.edu/~adoupe/static/black-box-scanners-dimva2010.pdf

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,900
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
49
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Why Johnny Can't Pentest: An Analysis of Black-box Web Vulnerability Scanners

  1. 1. Adam Doupé, Marco Cova and Giovanni Vigna University of California, Santa Barbara DIMVA 2010 - 7/8/10
  2. 2.  Introduction to black box web vulnerability scanners Design of custom vulnerable website – WackoPicko Results Analysis
  3. 3.  Describe the design of a testing web application Identify a number of challenges that scanners need to overcome when testing modern web applications Test the performance of eleven real-world scanners and identify areas that need further work
  4. 4. ServerCrawler Attack Analysis
  5. 5.  Authentication Upload Pictures Comment on Pictures “Purchase” Pictures Tag Search Guestbook Admin Area
  6. 6.  XSS ◦ Reflected, Stored, and Reflected behind JavaScript Session ID Weak Password Reflected SQL Injection Command Line Injection File Inclusion File Exposure Parameter Manipulation
  7. 7.  Reflected XSS behind Flash Stored SQL Injection Directory Traversal Multi-step Stored XSS Forceful Browsing Logic Flaw
  8. 8.  HTML Parsing Multi-Step Process / State Infinite Website Authentication Client-side Code ◦ Web Input Vector Extractor Teaser (WIVET)
  9. 9. Name PriceAcunetix $4,995 - $6,350AppScan $12,550 - $32,500Burp £125 ($190.82)Grendel-Scan Open sourceHailstorm $10,000Milescan $495 - $1,495N-Stalker $899 - $6,299NTOSpider $10,000Paros Open sourcew3af Open sourceWebinspect $6,000 - $30,000
  10. 10.  Each scanner run four times: ◦ WackoPicko  Initial – No configuration (point and click)  Config – Given valid Username/Password  Manual – Used proxy to thoroughly browse site. ◦ WIVET – Testing JavaScript capabilities Limitations
  11. 11. Name Reflected Stored SQL Command File File XSS via XSS via XSS XSS Injection line Inclusion Exposure JavaScript Flash Injection Acunetix Initial Initial Initial Initial Initial Initial AppScan Initial Initial Initial Initial Initial Burp Initial Manual Initial Initial Initial Manual Grendel- Manual Config ScanHailstorm Initial Config Config Manual Milescan Initial Manual ConfigN-Stalker Initial Manual Manual Initial Initial ManualNTOSpider Initial Initial Initial Paros Initial Initial Config Manual w3af Initial Manual Initial Initial ManualWebinspect Initial Initial Initial Initial Initial Manual
  12. 12.  Missed by all scanners ◦ Session ID ◦ Weak Password ◦ Parameter Manipulation ◦ Forceful Browsing ◦ Logic Flaw Will discuss later ◦ Stored SQL Injection ◦ Directory Traversal ◦ Stored XSS Behind Login
  13. 13.  Ranged from 0 to 200+ ◦ Average was ~25 Why? ◦ Server Path Disclosure “Actual” False Positives ◦ Hailstorm  XSS, 2 Code Injection ◦ NTOSpider  3 XSS ◦ w3af  PHP eval() Injection
  14. 14.  Strictly Dominates
  15. 15. MoreDominantLessDominant
  16. 16.  Default values XSS attacks Command-line Injection SQL Injection File Exposure Remote Code Execution
  17. 17.  Number of Accesses ◦ Range from ~50 per page to ~3,000 per page ◦ Hailstorm accessed vulnerable pages that required an account on INITIAL scan! HTML ◦ Burp and N-Stalker  <TEXTAREA> ◦ Milescan and Grendel-Scan  POST ◦ Hailstorm  No-Injection ◦ w3af  No Default
  18. 18.  Uploading a Picture ◦ 2 Scanners uploaded without help ◦ 3 Scanners unable to upload one!
  19. 19.  WIVET ◦ 3 Scanners couldn’t complete  Paros and Burp - <base>  N-Stalker – Frame? ◦ Dynamic JavaScript  Webinspect, Acunetix, NTOSpider, Hailstorm ◦ JavaScript library ◦ No Flash
  20. 20.  Created an account successfully ◦ 4 Scanners  Hailstorm  N-Stalker  NTOSpider  WebInspect
  21. 21.  Incorporate lots of logging in the application Two versions of the site ◦ No vulnerabilities ◦ All vulnerabilities Script running the tests Include: ◦ File upload forms ◦ AJAX ◦ Several JavaScript UI Libraries
  22. 22.  Ability to crawl as important as detection Many vulnerabilities cannot be detected Cost not directly proportional to functionality

×