SlideShare a Scribd company logo
Adam Doupé, Marco Cova and Giovanni Vigna
          University of California, Santa Barbara
                             DIMVA 2010 - 7/8/10
   Introduction to black box web vulnerability
    scanners
   Design of custom vulnerable website –
    WackoPicko
   Results
   Analysis
   Describe the design of a testing web
    application

   Identify a number of challenges that scanners
    need to overcome when testing modern web
    applications

   Test the performance of eleven real-world
    scanners and identify areas that need further
    work
Server




Crawler      Attack   Analysis
   Authentication
   Upload Pictures
   Comment on Pictures
   “Purchase” Pictures
   Tag Search
   Guestbook
   Admin Area
   XSS
    ◦ Reflected, Stored, and Reflected behind JavaScript
   Session ID
   Weak Password
   Reflected SQL Injection
   Command Line Injection
   File Inclusion
   File Exposure
   Parameter Manipulation
   Reflected XSS behind Flash
   Stored SQL Injection
   Directory Traversal
   Multi-step Stored XSS
   Forceful Browsing
   Logic Flaw
   HTML Parsing
   Multi-Step Process / State
   Infinite Website
   Authentication
   Client-side Code
    ◦ Web Input Vector Extractor Teaser (WIVET)
Name           Price
Acunetix       $4,995 - $6,350
AppScan        $12,550 - $32,500
Burp           £125 ($190.82)
Grendel-Scan   Open source
Hailstorm      $10,000
Milescan       $495 - $1,495
N-Stalker      $899 - $6,299
NTOSpider      $10,000
Paros          Open source
w3af           Open source
Webinspect     $6,000 - $30,000
   Each scanner run four times:
    ◦ WackoPicko
      Initial – No configuration (point and click)
      Config – Given valid Username/Password
      Manual – Used proxy to thoroughly browse site.

    ◦ WIVET – Testing JavaScript capabilities

   Limitations
Name         Reflected   Stored     SQL         Command     File        File        XSS via      XSS via
             XSS         XSS        Injection   line        Inclusion   Exposure    JavaScript   Flash
                                                Injection

 Acunetix      Initial    Initial     Initial                 Initial     Initial     Initial

 AppScan       Initial    Initial     Initial                 Initial     Initial

  Burp         Initial   Manual       Initial     Initial                 Initial                Manual

 Grendel-
              Manual                 Config
   Scan

Hailstorm      Initial   Config      Config                                                      Manual

 Milescan      Initial   Manual      Config

N-Stalker      Initial   Manual      Manual                               Initial     Initial    Manual

NTOSpider      Initial    Initial     Initial

  Paros        Initial    Initial    Config                                                      Manual

  w3af         Initial   Manual       Initial                 Initial                            Manual

Webinspect     Initial    Initial     Initial                 Initial                 Initial    Manual
   Missed by all scanners
    ◦   Session ID
    ◦   Weak Password
    ◦   Parameter Manipulation
    ◦   Forceful Browsing
    ◦   Logic Flaw
   Will discuss later
    ◦ Stored SQL Injection
    ◦ Directory Traversal
    ◦ Stored XSS Behind Login
   Ranged from 0 to 200+
    ◦ Average was ~25
   Why?
    ◦ Server Path Disclosure
   “Actual” False Positives
    ◦ Hailstorm
      XSS, 2 Code Injection
    ◦ NTOSpider
      3 XSS
    ◦ w3af
      PHP eval() Injection
   Strictly Dominates
More
Dominant




Less
Dominant
   Default values
   XSS attacks
   Command-line Injection
   SQL Injection
   File Exposure
   Remote Code Execution
   Number of Accesses
    ◦ Range from ~50 per page to ~3,000 per page
    ◦ Hailstorm accessed vulnerable pages that required an
      account on INITIAL scan!

   HTML
    ◦ Burp and N-Stalker
      <TEXTAREA>
    ◦ Milescan and Grendel-Scan
      POST
    ◦ Hailstorm
      No-Injection
    ◦ w3af
      No Default
   Uploading a Picture
    ◦ 2 Scanners uploaded without help

    ◦ 3 Scanners unable to upload one!
   WIVET
    ◦ 3 Scanners couldn’t complete
      Paros and Burp - <base>
      N-Stalker – Frame?
    ◦ Dynamic JavaScript
      Webinspect, Acunetix, NTOSpider, Hailstorm
    ◦ JavaScript library
    ◦ No Flash
   Created an account successfully
    ◦ 4 Scanners
        Hailstorm
        N-Stalker
        NTOSpider
        WebInspect
   Incorporate lots of logging in the application
   Two versions of the site
    ◦ No vulnerabilities
    ◦ All vulnerabilities
   Script running the tests
   Include:
    ◦ File upload forms
    ◦ AJAX
    ◦ Several JavaScript UI Libraries
   Ability to crawl as important as detection

   Many vulnerabilities cannot be detected

   Cost not directly proportional to functionality
Why Johnny Can't Pentest: An Analysis of Black-box Web Vulnerability Scanners
Why Johnny Can't Pentest: An Analysis of Black-box Web Vulnerability Scanners

More Related Content

Similar to Why Johnny Can't Pentest: An Analysis of Black-box Web Vulnerability Scanners

Pangolin Datasheet
Pangolin DatasheetPangolin Datasheet
Pangolin Datasheet
mattotamhe
 
Kraken
KrakenKraken
Kraken
PayPal
 
Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...
Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...
Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...
Christian Schneider
 
Romulus OWASP
Romulus OWASPRomulus OWASP
Romulus OWASP
Grupo Gesfor I+D+i
 
Effective DevSecOps
Effective DevSecOpsEffective DevSecOps
Effective DevSecOps
Pawel Krawczyk
 
[OWASP Poland Day] Application frameworks' vulnerabilities
[OWASP Poland Day] Application frameworks' vulnerabilities[OWASP Poland Day] Application frameworks' vulnerabilities
[OWASP Poland Day] Application frameworks' vulnerabilities
OWASP
 
Problems with parameters b sides-msp
Problems with parameters b sides-mspProblems with parameters b sides-msp
Problems with parameters b sides-msp
Mike Saunders
 
Web Based Security
Web Based SecurityWeb Based Security
Web Based Security
John Wiley
 
Hacking WebApps for fun and profit : how to approach a target?
Hacking WebApps for fun and profit : how to approach a target?Hacking WebApps for fun and profit : how to approach a target?
Hacking WebApps for fun and profit : how to approach a target?
Yassine Aboukir
 
OWASP App Sec US - 2010
OWASP App Sec US - 2010OWASP App Sec US - 2010
OWASP App Sec US - 2010
Aditya K Sood
 
Oleh Shpyrna "Security Testing Basics: Check your Webapp for gaps before l_unch"
Oleh Shpyrna "Security Testing Basics: Check your Webapp for gaps before l_unch"Oleh Shpyrna "Security Testing Basics: Check your Webapp for gaps before l_unch"
Oleh Shpyrna "Security Testing Basics: Check your Webapp for gaps before l_unch"
Dakiry
 
Hacking Client Side Insecurities
Hacking Client Side InsecuritiesHacking Client Side Insecurities
Hacking Client Side Insecurities
amiable_indian
 
Technical Architecture of RASP Technology
Technical Architecture of RASP TechnologyTechnical Architecture of RASP Technology
Technical Architecture of RASP Technology
Priyanka Aash
 
Web Application Defences
Web Application DefencesWeb Application Defences
Web Application Defences
Damilola Longe, CISSP, CCSP, MSc
 
Identifying Web Servers: A First-look Into the Future of Web Server Fingerpri...
Identifying Web Servers: A First-look Into the Future of Web Server Fingerpri...Identifying Web Servers: A First-look Into the Future of Web Server Fingerpri...
Identifying Web Servers: A First-look Into the Future of Web Server Fingerpri...
Jeremiah Grossman
 
Static code analysis with sonar qube
Static code analysis with sonar qubeStatic code analysis with sonar qube
Static code analysis with sonar qube
Hayi Nukman
 
Cyber ppt
Cyber pptCyber ppt
Cyber ppt
karthik menon
 
Jon Stace Web Cryptography API
Jon Stace Web Cryptography APIJon Stace Web Cryptography API
Jon Stace Web Cryptography API
Jon Stace
 
Introduction to Mod security session April 2016
Introduction to Mod security session April 2016Introduction to Mod security session April 2016
Introduction to Mod security session April 2016
Rahul
 
Web Application Security
Web Application SecurityWeb Application Security
Web Application Security
Siarhei Barysiuk
 

Similar to Why Johnny Can't Pentest: An Analysis of Black-box Web Vulnerability Scanners (20)

Pangolin Datasheet
Pangolin DatasheetPangolin Datasheet
Pangolin Datasheet
 
Kraken
KrakenKraken
Kraken
 
Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...
Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...
Security DevOps - Wie Sie in agilen Projekten trotzdem sicher bleiben // DevO...
 
Romulus OWASP
Romulus OWASPRomulus OWASP
Romulus OWASP
 
Effective DevSecOps
Effective DevSecOpsEffective DevSecOps
Effective DevSecOps
 
[OWASP Poland Day] Application frameworks' vulnerabilities
[OWASP Poland Day] Application frameworks' vulnerabilities[OWASP Poland Day] Application frameworks' vulnerabilities
[OWASP Poland Day] Application frameworks' vulnerabilities
 
Problems with parameters b sides-msp
Problems with parameters b sides-mspProblems with parameters b sides-msp
Problems with parameters b sides-msp
 
Web Based Security
Web Based SecurityWeb Based Security
Web Based Security
 
Hacking WebApps for fun and profit : how to approach a target?
Hacking WebApps for fun and profit : how to approach a target?Hacking WebApps for fun and profit : how to approach a target?
Hacking WebApps for fun and profit : how to approach a target?
 
OWASP App Sec US - 2010
OWASP App Sec US - 2010OWASP App Sec US - 2010
OWASP App Sec US - 2010
 
Oleh Shpyrna "Security Testing Basics: Check your Webapp for gaps before l_unch"
Oleh Shpyrna "Security Testing Basics: Check your Webapp for gaps before l_unch"Oleh Shpyrna "Security Testing Basics: Check your Webapp for gaps before l_unch"
Oleh Shpyrna "Security Testing Basics: Check your Webapp for gaps before l_unch"
 
Hacking Client Side Insecurities
Hacking Client Side InsecuritiesHacking Client Side Insecurities
Hacking Client Side Insecurities
 
Technical Architecture of RASP Technology
Technical Architecture of RASP TechnologyTechnical Architecture of RASP Technology
Technical Architecture of RASP Technology
 
Web Application Defences
Web Application DefencesWeb Application Defences
Web Application Defences
 
Identifying Web Servers: A First-look Into the Future of Web Server Fingerpri...
Identifying Web Servers: A First-look Into the Future of Web Server Fingerpri...Identifying Web Servers: A First-look Into the Future of Web Server Fingerpri...
Identifying Web Servers: A First-look Into the Future of Web Server Fingerpri...
 
Static code analysis with sonar qube
Static code analysis with sonar qubeStatic code analysis with sonar qube
Static code analysis with sonar qube
 
Cyber ppt
Cyber pptCyber ppt
Cyber ppt
 
Jon Stace Web Cryptography API
Jon Stace Web Cryptography APIJon Stace Web Cryptography API
Jon Stace Web Cryptography API
 
Introduction to Mod security session April 2016
Introduction to Mod security session April 2016Introduction to Mod security session April 2016
Introduction to Mod security session April 2016
 
Web Application Security
Web Application SecurityWeb Application Security
Web Application Security
 

Recently uploaded

How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
David Brossard
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
Zilliz
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 

Recently uploaded (20)

How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 

Why Johnny Can't Pentest: An Analysis of Black-box Web Vulnerability Scanners

  • 1. Adam Doupé, Marco Cova and Giovanni Vigna University of California, Santa Barbara DIMVA 2010 - 7/8/10
  • 2. Introduction to black box web vulnerability scanners  Design of custom vulnerable website – WackoPicko  Results  Analysis
  • 3. Describe the design of a testing web application  Identify a number of challenges that scanners need to overcome when testing modern web applications  Test the performance of eleven real-world scanners and identify areas that need further work
  • 4. Server Crawler Attack Analysis
  • 5. Authentication  Upload Pictures  Comment on Pictures  “Purchase” Pictures  Tag Search  Guestbook  Admin Area
  • 6. XSS ◦ Reflected, Stored, and Reflected behind JavaScript  Session ID  Weak Password  Reflected SQL Injection  Command Line Injection  File Inclusion  File Exposure  Parameter Manipulation
  • 7. Reflected XSS behind Flash  Stored SQL Injection  Directory Traversal  Multi-step Stored XSS  Forceful Browsing  Logic Flaw
  • 8. HTML Parsing  Multi-Step Process / State  Infinite Website  Authentication  Client-side Code ◦ Web Input Vector Extractor Teaser (WIVET)
  • 9. Name Price Acunetix $4,995 - $6,350 AppScan $12,550 - $32,500 Burp £125 ($190.82) Grendel-Scan Open source Hailstorm $10,000 Milescan $495 - $1,495 N-Stalker $899 - $6,299 NTOSpider $10,000 Paros Open source w3af Open source Webinspect $6,000 - $30,000
  • 10. Each scanner run four times: ◦ WackoPicko  Initial – No configuration (point and click)  Config – Given valid Username/Password  Manual – Used proxy to thoroughly browse site. ◦ WIVET – Testing JavaScript capabilities  Limitations
  • 11. Name Reflected Stored SQL Command File File XSS via XSS via XSS XSS Injection line Inclusion Exposure JavaScript Flash Injection Acunetix Initial Initial Initial Initial Initial Initial AppScan Initial Initial Initial Initial Initial Burp Initial Manual Initial Initial Initial Manual Grendel- Manual Config Scan Hailstorm Initial Config Config Manual Milescan Initial Manual Config N-Stalker Initial Manual Manual Initial Initial Manual NTOSpider Initial Initial Initial Paros Initial Initial Config Manual w3af Initial Manual Initial Initial Manual Webinspect Initial Initial Initial Initial Initial Manual
  • 12.
  • 13. Missed by all scanners ◦ Session ID ◦ Weak Password ◦ Parameter Manipulation ◦ Forceful Browsing ◦ Logic Flaw  Will discuss later ◦ Stored SQL Injection ◦ Directory Traversal ◦ Stored XSS Behind Login
  • 14. Ranged from 0 to 200+ ◦ Average was ~25  Why? ◦ Server Path Disclosure  “Actual” False Positives ◦ Hailstorm  XSS, 2 Code Injection ◦ NTOSpider  3 XSS ◦ w3af  PHP eval() Injection
  • 15. Strictly Dominates
  • 17. Default values  XSS attacks  Command-line Injection  SQL Injection  File Exposure  Remote Code Execution
  • 18. Number of Accesses ◦ Range from ~50 per page to ~3,000 per page ◦ Hailstorm accessed vulnerable pages that required an account on INITIAL scan!  HTML ◦ Burp and N-Stalker  <TEXTAREA> ◦ Milescan and Grendel-Scan  POST ◦ Hailstorm  No-Injection ◦ w3af  No Default
  • 19. Uploading a Picture ◦ 2 Scanners uploaded without help ◦ 3 Scanners unable to upload one!
  • 20. WIVET ◦ 3 Scanners couldn’t complete  Paros and Burp - <base>  N-Stalker – Frame? ◦ Dynamic JavaScript  Webinspect, Acunetix, NTOSpider, Hailstorm ◦ JavaScript library ◦ No Flash
  • 21. Created an account successfully ◦ 4 Scanners  Hailstorm  N-Stalker  NTOSpider  WebInspect
  • 22. Incorporate lots of logging in the application  Two versions of the site ◦ No vulnerabilities ◦ All vulnerabilities  Script running the tests  Include: ◦ File upload forms ◦ AJAX ◦ Several JavaScript UI Libraries
  • 23. Ability to crawl as important as detection  Many vulnerabilities cannot be detected  Cost not directly proportional to functionality