SlideShare a Scribd company logo
1 of 38
Download to read offline
Improving Web Vulnerability Scanning   1


Daniel Zulla
Introduction
Hey!                                           2



  ■ Hi there!
  ■ I’m Dan. This is my first year at DEFCON.
  ■ I do programming and security start-ups.
  ■ I do some penetration testing as well
More Introduction                                           3



 ■ Today I’m going to talk about vulnerability scanning
 ■ Primary on the web
 ■ “The cloud” is involved as well
 ■ Network security too
 ■ I’ll show some things, so there is plenty of demo time
 ■ Have fun, thanks for being here!
Some Facts                                                                  4


 ■ There are a lot of web vulnerability scanners, fuzzers and penetration
    testing tools out there already

 ■ Some of them work, some of them do not
 ■ But basically all of them have one thing in common:
    They actually don’t attack web applications on the application layer

 ■ They mostly fuzz HTTP and sometimes perform injection attacks
Some more facts                                                       5




 ■ The fundamental design of web scanners has not changed in over a
    decade

 ■ But: The web has changed.
 ■ So there seems to be a problem.
Software Architecture
What web vulnerability scanners and fuzzers look like                            6




                                                               Plugins
                      A HTTP Library

                                                        RXSS   BSQLI     EVAL
                        The Core

                                                        PXSS    LFI      OSC
                                Multithreading /
             Output Engine
                                    Forking
                                                        SQL     RFI      [...]
A pentesters point of view                          7




■ Javascript/Ajax rich applications are still not
    supported

■ Authenticated scanning is still incredibly
    challenging / not reliable

■ Exploitation techniques are mostly poor

■ “I don’t know which scanner will work for
    foo.com and which one for bar.com, so I
    use toolchains”
A developers point of view                                                                                 8



                                                    ■ HTTP Libraries don’t support JS -
■ Javascript/Ajax rich applications are still not       Scanners are based on an HTTP
    supported                                           Libraries

■ Authenticated scanning is still incredibly        ■ Web Logins are not standarized -
    challenging / not reliable                          So how should they be detected

■ Exploitation techniques are mostly poor           ■ No time for exploits
                                                        (Already spent 100000 lines [and nights] of code
                                                        making the crawler immune to encoding issues,
■ “I don’t know which scanner will work for             malformed HTML, redirects and binary content!)

    foo.com and which one for bar.com, so I
    use toolchains”
                                                    ■ A false positive is better than a
                                                        false negative
How I see it                                                                                    9



■ Both of them are right.

■ The web is a mess. Nobody cares about RFCs anymore. (Especially these SEO guys!)

■ 10 years ago, you would have expected a Query String at the end of a URL like
   https://foo.com/xxx/yyy?foo=bar

■ Nowadays, https://foo.com/something.ext/foo/bar is good practice

■ The result: It’s incredibly hard for scanner developers to figure out the dynamic components
   of an HTTP request. Because of that, we feel overhelmed and fuzz nearly everything.

■ Header Keys, Header Values, VHost, Cookie, Method, Path, Version, ...
How I see it                                                                                     10



■ Fuzzing HTTP is incredibly important. You never know if you are talking to an apache2, nginx
    or some hidden application server upstream

■ But it has nothing to-do with web vulnerability scanning

■ So - developers are struggling with websites because they use HTTP to crawl and attack
    them. Things like flash, images, javascript seems to be an unsolveable problem

■ Redirects are hard to handle sometimes (wait there is more)

■ Javascript redirects (after 10 seconds!) and of course: onmouseover, onclick, onfocus, ...

■ Flash isn’t helpful either
Web 2.0                                                                                         11




■ But - WE DO SECURITY

■ Is it really our job to make sure that our software executed all the JS and grabbed all the
    links?

■ When we spend 100 hours on the crawler, and 5 hours on the actual payloads (that’s how it
    looks right now) something, somewhere, went terribly wrong

■ So - Is there a (open source?) piece of software that we could use instead of the HTTP
    library? Something that has prooven its mastery in handling unpredictably broken web
    content already? There is.
Webkit!   12
Webkit knows                                            13


■   Javascript          ■   CSS Rendering

■   Javascript events   ■   Binary Downloads

■   Redirects           ■   Broken HTML

■   Flash               ■   Broken CSS

■   Images              ■   Performance

■   Websockets          ■   Forking / Multiprocessing

■   WebGL               ■   [...]
Software Architecture
What it should look like                                             14



                               The Front-End


          A HTTP                            RXSS     BSQLI   EVAL
                       The Core
          Library
                                            PXSS      LFI    OSC

            Reporting Engine                 SQL      RFI    [...]



                           The Exploitation Engine
Changes? Improvments?                                                15



 ■ Replacing the HTTP library by a Webkit Engine
 ■ Less code (A lot less code)
 ■ 100% support for JS/Ajax/Broken HTML/JS Events/Crazy Redirects
    and all kinds of things

 ■ The ability to simulate human user behaviour
 ■ CSS Renderings (Two text fields beside each other: 10px - one of
    them is a input[type=password]) - May be a login!
Making it scale (heavily)                                                 16




 ■ Webkit is slow (Website rendering, Executing JS, ... - compared to -
    Speaking Plaintext HTTP)

 ■ Downloading Images is slow
 ■ Waiting for delayed JS events is slow
 ■ Flash is even slower
Making it scale (heavily)
Bad news: Qt / PyQt / PySide                                             17




  ■ QtWebkit does not support multithreading
  ■ It tends to SEGFAULT from time to time :(
  ■ Multiple QApplication instances are almost impossible to handle in
     one Python namespace
Making it scale (heavily)
Good news: Building a preforking TCP Server                                  18




  ■ Spawning a pool of processes works quite well (one QApplication
     +one Browser instance per Process)

  ■ Simultaneous downloads
  ■ Better accessibility inside the scanner (multiprocessing insides loops
     to increase performance)
Missing pieces                                            19




 ■ Mastering Authentication
 ■ Exploitation & Privilege Escalation
 ■ Geographically distributed scanning: Using the cloud
 ■ Reporting
Mastering Authentication                                                      20



 ■ There is no such thing as a standarized web login
 ■ Basically, everybody develops access control on the web slightly
    differently

 ■ You can try to detect them by the name/id of the attributes, but that is
    not reliable

 ■ But in the end, Web logins generally have a few things in common
    that makes them easily detectable. At least, for our browser engine
Mastering Authentication
Not more than 2 visible (!) text fields          21




                            has_login_texts()
Mastering Authentication
Man-Behind-You Protection                       22



                            is_input_hidden()
Mastering Authentication
   Geometry! Usually, the two visible text fields are under(), next_to() or at least
   near(radius=10px) each other                                                       23




X1 = X2
                                                         X1 = X2
                                                    !




                                                                 Y1 = Y2
Mastering Authentication                                                  24



 ■ That was easy!
 ■ The common way to solve that problem, is to iterate through a
    wordlist (login, auth, signin, [...]) while checking the input[id],
    input[name] attributes

 ■ That’s not necessarily wrong or bad practice
 ■ After putting the pieces together:
 ■ .login(“username”, “password”)
Mastering Authentication
Demo Time                                                              25




 ■ Proof Of Concept 1: Twitter (Some Javascript)
 ■ Proof Of Concept 2: Facebook (More Javascript)
 ■ Proof Of Concept 3: Google Plus (Most Javascript + Browser Hacks)
Mastering Authentication
When we are signed in                                                26



  ■ New problems occur: How can we let the scanner check if we are
     indeed signed in?

  ■ Common practive: Looking for a /logout/i String
  ■ The problem: Inefficient. Likely to cause false positives
  ■ There has to be a better way:
  ■ Introduction “Strategies”
Strategy.Authentication
Step 1: Identification                                                 27


  ■ Identifying a login form (3-way approach, input[type=password],
     geometry, [...])
Strategy.Authentication
Step 2: Error messages (Why a browser engines rocks)                        28


  ■ Verifying wrong credentials - Random strings - Failed login




                                                        #BA.... -> #E4...
Strategy.Authentication
Step 3: Going in: .login(“..”, “..”)                                        29


  ■ Verifying valid credentials - Behaviour should not be similiar to the
      behaviour of a invalid login
Strategy.Authentication
Step 4: Going out. .logout()                                             30


  ■ Doing similiar work again for .logout() function seems obsolote
  ■ But it really isn’t.
  ■ It is the basis to a .is_still_loggedin() function
  ■ Which is really important to stay logged in during crawling
  ■ And if the scanner logged itself out, it can simply .login() again
  ■ That’s cool. :-)
Exploitation and Privilege Escalation                                           31



■ There is a whole universe besides injection vulnerabilities
■ Usually, scanners don’t detect them
■ But they should
■ And now they can: .login(“user1”, “...”); .logout(); .login(“user2”, “...”)
■ => Demo Time: Privilege Escalation, Multi-User Systems
Geographically distributed scanning:
Using the cloud                                                               32



 ■ When (injection) vulnerabilities are getting complicated:
 ■ Scenario 1: The backend of a website creates a log entry for every
    new IP address. It logs the USERAGENT. The log entries are kept in a
    SQL database. The function that creates the log entries, is vulnerable.
    The User-Agent is injectable. The problem is:

 ■ It only works once. As soon as the IP is in the database, the function
    won’t be executed anymore :-(

 ■ ==> SQLMap (and every other tool) will fail.
Geographically distributed scanning:
Using the cloud                                                33




 ■ But they shouldn’t!
 ■ The limitation is totally detectable
 ■ And a new IP is just as far away as a single EC2 API call
Geographically distributed scanning:
Using the cloud                                             34

      ■ Indeed! The cloud is a good thing for security :)
      ■ Demo Time: Introducing:
         sqlmap and w3af (on steroids)
Combining “Strategies” and the
distributed scanning                                    35




 ■ Introducing next generation vulnerability scanning
 ■ Exploiting a really amazingly hard SQL Injection




 ■ Demo Time
Further Research & Additional Ideas                                      36




 ■ Country specific restrictions can be by-passed in a fully automatic
    manner

 ■ (Error) messages can be parsed and interpreted: Wolfram Alpha
 ■ Bloomfilters should be integrated
 ■ Other “Strategies” should be implemented (the limitations are gone)
More Live Demos                                                       37



 ■ Demonstrating a logical layer beyond Authentication:
    .pay(“0000111122223333”, CVV=121, type=VISA)
    .search(“search query”)
    .sort(“DESC UNION SELECT [...]”)

 ■ Interpreting error messages
 ■ Pivoting on penetrated hosts - Spawning another scanner instance
 ■ And finally: Reporting!
Thanks!   38

More Related Content

What's hot

An Abusive Relationship with AngularJS
An Abusive Relationship with AngularJSAn Abusive Relationship with AngularJS
An Abusive Relationship with AngularJSMario Heiderich
 
Copy & Pest - A case-study on the clipboard, blind trust and invisible cross-...
Copy & Pest - A case-study on the clipboard, blind trust and invisible cross-...Copy & Pest - A case-study on the clipboard, blind trust and invisible cross-...
Copy & Pest - A case-study on the clipboard, blind trust and invisible cross-...Mario Heiderich
 
Locking the Throne Room - How ES5+ might change views on XSS and Client Side ...
Locking the Throne Room - How ES5+ might change views on XSS and Client Side ...Locking the Throne Room - How ES5+ might change views on XSS and Client Side ...
Locking the Throne Room - How ES5+ might change views on XSS and Client Side ...Mario Heiderich
 
Owasp web application security trends
Owasp web application security trendsOwasp web application security trends
Owasp web application security trendsbeched
 
How to choose a web framework and be surprised
How to choose a web framework and be surprisedHow to choose a web framework and be surprised
How to choose a web framework and be surprisedJose María Arranz
 
Html5 security
Html5 securityHtml5 security
Html5 securityKrishna T
 
The Image that called me - Active Content Injection with SVG Files
The Image that called me - Active Content Injection with SVG FilesThe Image that called me - Active Content Injection with SVG Files
The Image that called me - Active Content Injection with SVG FilesMario Heiderich
 
Scriptless Attacks - Stealing the Pie without touching the Sill
Scriptless Attacks - Stealing the Pie without touching the SillScriptless Attacks - Stealing the Pie without touching the Sill
Scriptless Attacks - Stealing the Pie without touching the SillMario Heiderich
 
In the DOM, no one will hear you scream
In the DOM, no one will hear you screamIn the DOM, no one will hear you scream
In the DOM, no one will hear you screamMario Heiderich
 
NullCon 2012 - Ra.2: blackbox DOM-based XSS scanner
NullCon 2012 - Ra.2: blackbox DOM-based XSS scannerNullCon 2012 - Ra.2: blackbox DOM-based XSS scanner
NullCon 2012 - Ra.2: blackbox DOM-based XSS scannerNishant Das Patnaik
 
Breaking AngularJS Javascript sandbox
Breaking AngularJS Javascript sandboxBreaking AngularJS Javascript sandbox
Breaking AngularJS Javascript sandboxMathias Karlsson
 
Google chrome presentation
Google chrome presentationGoogle chrome presentation
Google chrome presentationreza jalaluddin
 
Secure web messaging in HTML5
Secure web messaging in HTML5Secure web messaging in HTML5
Secure web messaging in HTML5Krishna T
 
Find maximum bugs in limited time
Find maximum bugs in limited timeFind maximum bugs in limited time
Find maximum bugs in limited timebeched
 
Polyglot payloads in practice by avlidienbrunn at HackPra
Polyglot payloads in practice by avlidienbrunn at HackPraPolyglot payloads in practice by avlidienbrunn at HackPra
Polyglot payloads in practice by avlidienbrunn at HackPraMathias Karlsson
 
JSFoo Chennai 2012
JSFoo Chennai 2012JSFoo Chennai 2012
JSFoo Chennai 2012Krishna T
 
Browser Internals-Same Origin Policy
Browser Internals-Same Origin PolicyBrowser Internals-Same Origin Policy
Browser Internals-Same Origin PolicyKrishna T
 
Ajax Tutorial
Ajax TutorialAjax Tutorial
Ajax Tutorialoscon2007
 

What's hot (19)

An Abusive Relationship with AngularJS
An Abusive Relationship with AngularJSAn Abusive Relationship with AngularJS
An Abusive Relationship with AngularJS
 
Copy & Pest - A case-study on the clipboard, blind trust and invisible cross-...
Copy & Pest - A case-study on the clipboard, blind trust and invisible cross-...Copy & Pest - A case-study on the clipboard, blind trust and invisible cross-...
Copy & Pest - A case-study on the clipboard, blind trust and invisible cross-...
 
Locking the Throne Room - How ES5+ might change views on XSS and Client Side ...
Locking the Throne Room - How ES5+ might change views on XSS and Client Side ...Locking the Throne Room - How ES5+ might change views on XSS and Client Side ...
Locking the Throne Room - How ES5+ might change views on XSS and Client Side ...
 
Owasp web application security trends
Owasp web application security trendsOwasp web application security trends
Owasp web application security trends
 
How to choose a web framework and be surprised
How to choose a web framework and be surprisedHow to choose a web framework and be surprised
How to choose a web framework and be surprised
 
Html5 security
Html5 securityHtml5 security
Html5 security
 
The Image that called me - Active Content Injection with SVG Files
The Image that called me - Active Content Injection with SVG FilesThe Image that called me - Active Content Injection with SVG Files
The Image that called me - Active Content Injection with SVG Files
 
Scriptless Attacks - Stealing the Pie without touching the Sill
Scriptless Attacks - Stealing the Pie without touching the SillScriptless Attacks - Stealing the Pie without touching the Sill
Scriptless Attacks - Stealing the Pie without touching the Sill
 
In the DOM, no one will hear you scream
In the DOM, no one will hear you screamIn the DOM, no one will hear you scream
In the DOM, no one will hear you scream
 
NullCon 2012 - Ra.2: blackbox DOM-based XSS scanner
NullCon 2012 - Ra.2: blackbox DOM-based XSS scannerNullCon 2012 - Ra.2: blackbox DOM-based XSS scanner
NullCon 2012 - Ra.2: blackbox DOM-based XSS scanner
 
JavaScript-Core
JavaScript-CoreJavaScript-Core
JavaScript-Core
 
Breaking AngularJS Javascript sandbox
Breaking AngularJS Javascript sandboxBreaking AngularJS Javascript sandbox
Breaking AngularJS Javascript sandbox
 
Google chrome presentation
Google chrome presentationGoogle chrome presentation
Google chrome presentation
 
Secure web messaging in HTML5
Secure web messaging in HTML5Secure web messaging in HTML5
Secure web messaging in HTML5
 
Find maximum bugs in limited time
Find maximum bugs in limited timeFind maximum bugs in limited time
Find maximum bugs in limited time
 
Polyglot payloads in practice by avlidienbrunn at HackPra
Polyglot payloads in practice by avlidienbrunn at HackPraPolyglot payloads in practice by avlidienbrunn at HackPra
Polyglot payloads in practice by avlidienbrunn at HackPra
 
JSFoo Chennai 2012
JSFoo Chennai 2012JSFoo Chennai 2012
JSFoo Chennai 2012
 
Browser Internals-Same Origin Policy
Browser Internals-Same Origin PolicyBrowser Internals-Same Origin Policy
Browser Internals-Same Origin Policy
 
Ajax Tutorial
Ajax TutorialAjax Tutorial
Ajax Tutorial
 

Similar to Defcon 20-zulla-improving-web-vulnerability-scanning

Hacking Vulnerable Websites to Bypass Firewalls
Hacking Vulnerable Websites to Bypass FirewallsHacking Vulnerable Websites to Bypass Firewalls
Hacking Vulnerable Websites to Bypass FirewallsNetsparker
 
External JavaScript Widget Development Best Practices (updated) (v.1.1)
External JavaScript Widget Development Best Practices (updated) (v.1.1) External JavaScript Widget Development Best Practices (updated) (v.1.1)
External JavaScript Widget Development Best Practices (updated) (v.1.1) Volkan Özçelik
 
External JavaScript Widget Development Best Practices
External JavaScript Widget Development Best PracticesExternal JavaScript Widget Development Best Practices
External JavaScript Widget Development Best PracticesVolkan Özçelik
 
Java scriptwidgetdevelopmentjstanbul2012
Java scriptwidgetdevelopmentjstanbul2012Java scriptwidgetdevelopmentjstanbul2012
Java scriptwidgetdevelopmentjstanbul2012Volkan Özçelik
 
Abusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec gloryAbusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec gloryPriyanka Aash
 
PrairieDevCon 2014 - Web Doesn't Mean Slow
PrairieDevCon 2014 -  Web Doesn't Mean SlowPrairieDevCon 2014 -  Web Doesn't Mean Slow
PrairieDevCon 2014 - Web Doesn't Mean Slowdmethvin
 
Influx/Days 2017 San Francisco | Emily Nakashima
Influx/Days 2017 San Francisco | Emily NakashimaInflux/Days 2017 San Francisco | Emily Nakashima
Influx/Days 2017 San Francisco | Emily NakashimaInfluxData
 
XSS Without Browser
XSS Without BrowserXSS Without Browser
XSS Without Browserkosborn
 
Security research over Windows #defcon china
Security research over Windows #defcon chinaSecurity research over Windows #defcon china
Security research over Windows #defcon chinaPeter Hlavaty
 
Week01 jan19 introductionto_php
Week01 jan19 introductionto_phpWeek01 jan19 introductionto_php
Week01 jan19 introductionto_phpJeanho Chu
 
TSC Summit #4 - Howto get browser persitence and remote execution (JS)
TSC Summit #4 - Howto get browser persitence and remote execution (JS)TSC Summit #4 - Howto get browser persitence and remote execution (JS)
TSC Summit #4 - Howto get browser persitence and remote execution (JS)Mikal Villa
 
(In)Security Implication in the JS Universe
(In)Security Implication in the JS Universe(In)Security Implication in the JS Universe
(In)Security Implication in the JS UniverseStefano Di Paola
 
Quo vadis, JavaScript? Devday.pl keynote
Quo vadis, JavaScript? Devday.pl keynoteQuo vadis, JavaScript? Devday.pl keynote
Quo vadis, JavaScript? Devday.pl keynoteChristian Heilmann
 
Why and How You Should Move from PHP to Node.js
Why and How You Should Move from PHP to Node.jsWhy and How You Should Move from PHP to Node.js
Why and How You Should Move from PHP to Node.jsBrainhub
 

Similar to Defcon 20-zulla-improving-web-vulnerability-scanning (20)

Hacking Vulnerable Websites to Bypass Firewalls
Hacking Vulnerable Websites to Bypass FirewallsHacking Vulnerable Websites to Bypass Firewalls
Hacking Vulnerable Websites to Bypass Firewalls
 
External JavaScript Widget Development Best Practices (updated) (v.1.1)
External JavaScript Widget Development Best Practices (updated) (v.1.1) External JavaScript Widget Development Best Practices (updated) (v.1.1)
External JavaScript Widget Development Best Practices (updated) (v.1.1)
 
External JavaScript Widget Development Best Practices
External JavaScript Widget Development Best PracticesExternal JavaScript Widget Development Best Practices
External JavaScript Widget Development Best Practices
 
Java scriptwidgetdevelopmentjstanbul2012
Java scriptwidgetdevelopmentjstanbul2012Java scriptwidgetdevelopmentjstanbul2012
Java scriptwidgetdevelopmentjstanbul2012
 
Taming 3rd party content
Taming 3rd party contentTaming 3rd party content
Taming 3rd party content
 
Introduction to Node.js
Introduction to Node.jsIntroduction to Node.js
Introduction to Node.js
 
orcreatehappyusers
orcreatehappyusersorcreatehappyusers
orcreatehappyusers
 
orcreatehappyusers
orcreatehappyusersorcreatehappyusers
orcreatehappyusers
 
Abusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec gloryAbusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec glory
 
PrairieDevCon 2014 - Web Doesn't Mean Slow
PrairieDevCon 2014 -  Web Doesn't Mean SlowPrairieDevCon 2014 -  Web Doesn't Mean Slow
PrairieDevCon 2014 - Web Doesn't Mean Slow
 
Influx/Days 2017 San Francisco | Emily Nakashima
Influx/Days 2017 San Francisco | Emily NakashimaInflux/Days 2017 San Francisco | Emily Nakashima
Influx/Days 2017 San Francisco | Emily Nakashima
 
XSS Without Browser
XSS Without BrowserXSS Without Browser
XSS Without Browser
 
Security research over Windows #defcon china
Security research over Windows #defcon chinaSecurity research over Windows #defcon china
Security research over Windows #defcon china
 
Week01 jan19 introductionto_php
Week01 jan19 introductionto_phpWeek01 jan19 introductionto_php
Week01 jan19 introductionto_php
 
Real time web
Real time webReal time web
Real time web
 
TSC Summit #4 - Howto get browser persitence and remote execution (JS)
TSC Summit #4 - Howto get browser persitence and remote execution (JS)TSC Summit #4 - Howto get browser persitence and remote execution (JS)
TSC Summit #4 - Howto get browser persitence and remote execution (JS)
 
Be faster then rabbits
Be faster then rabbitsBe faster then rabbits
Be faster then rabbits
 
(In)Security Implication in the JS Universe
(In)Security Implication in the JS Universe(In)Security Implication in the JS Universe
(In)Security Implication in the JS Universe
 
Quo vadis, JavaScript? Devday.pl keynote
Quo vadis, JavaScript? Devday.pl keynoteQuo vadis, JavaScript? Devday.pl keynote
Quo vadis, JavaScript? Devday.pl keynote
 
Why and How You Should Move from PHP to Node.js
Why and How You Should Move from PHP to Node.jsWhy and How You Should Move from PHP to Node.js
Why and How You Should Move from PHP to Node.js
 

Recently uploaded

Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncObject Automation
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?SANGHEE SHIN
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfAnna Loughnan Colquhoun
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServicePicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServiceRenan Moreira de Oliveira
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceMartin Humpolec
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 

Recently uploaded (20)

Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdf
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer ServicePicPay - GenAI Finance Assistant - ChatGPT for Customer Service
PicPay - GenAI Finance Assistant - ChatGPT for Customer Service
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your Salesforce
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 

Defcon 20-zulla-improving-web-vulnerability-scanning

  • 1. Improving Web Vulnerability Scanning 1 Daniel Zulla
  • 2. Introduction Hey! 2 ■ Hi there! ■ I’m Dan. This is my first year at DEFCON. ■ I do programming and security start-ups. ■ I do some penetration testing as well
  • 3. More Introduction 3 ■ Today I’m going to talk about vulnerability scanning ■ Primary on the web ■ “The cloud” is involved as well ■ Network security too ■ I’ll show some things, so there is plenty of demo time ■ Have fun, thanks for being here!
  • 4. Some Facts 4 ■ There are a lot of web vulnerability scanners, fuzzers and penetration testing tools out there already ■ Some of them work, some of them do not ■ But basically all of them have one thing in common: They actually don’t attack web applications on the application layer ■ They mostly fuzz HTTP and sometimes perform injection attacks
  • 5. Some more facts 5 ■ The fundamental design of web scanners has not changed in over a decade ■ But: The web has changed. ■ So there seems to be a problem.
  • 6. Software Architecture What web vulnerability scanners and fuzzers look like 6 Plugins A HTTP Library RXSS BSQLI EVAL The Core PXSS LFI OSC Multithreading / Output Engine Forking SQL RFI [...]
  • 7. A pentesters point of view 7 ■ Javascript/Ajax rich applications are still not supported ■ Authenticated scanning is still incredibly challenging / not reliable ■ Exploitation techniques are mostly poor ■ “I don’t know which scanner will work for foo.com and which one for bar.com, so I use toolchains”
  • 8. A developers point of view 8 ■ HTTP Libraries don’t support JS - ■ Javascript/Ajax rich applications are still not Scanners are based on an HTTP supported Libraries ■ Authenticated scanning is still incredibly ■ Web Logins are not standarized - challenging / not reliable So how should they be detected ■ Exploitation techniques are mostly poor ■ No time for exploits (Already spent 100000 lines [and nights] of code making the crawler immune to encoding issues, ■ “I don’t know which scanner will work for malformed HTML, redirects and binary content!) foo.com and which one for bar.com, so I use toolchains” ■ A false positive is better than a false negative
  • 9. How I see it 9 ■ Both of them are right. ■ The web is a mess. Nobody cares about RFCs anymore. (Especially these SEO guys!) ■ 10 years ago, you would have expected a Query String at the end of a URL like https://foo.com/xxx/yyy?foo=bar ■ Nowadays, https://foo.com/something.ext/foo/bar is good practice ■ The result: It’s incredibly hard for scanner developers to figure out the dynamic components of an HTTP request. Because of that, we feel overhelmed and fuzz nearly everything. ■ Header Keys, Header Values, VHost, Cookie, Method, Path, Version, ...
  • 10. How I see it 10 ■ Fuzzing HTTP is incredibly important. You never know if you are talking to an apache2, nginx or some hidden application server upstream ■ But it has nothing to-do with web vulnerability scanning ■ So - developers are struggling with websites because they use HTTP to crawl and attack them. Things like flash, images, javascript seems to be an unsolveable problem ■ Redirects are hard to handle sometimes (wait there is more) ■ Javascript redirects (after 10 seconds!) and of course: onmouseover, onclick, onfocus, ... ■ Flash isn’t helpful either
  • 11. Web 2.0 11 ■ But - WE DO SECURITY ■ Is it really our job to make sure that our software executed all the JS and grabbed all the links? ■ When we spend 100 hours on the crawler, and 5 hours on the actual payloads (that’s how it looks right now) something, somewhere, went terribly wrong ■ So - Is there a (open source?) piece of software that we could use instead of the HTTP library? Something that has prooven its mastery in handling unpredictably broken web content already? There is.
  • 12. Webkit! 12
  • 13. Webkit knows 13 ■ Javascript ■ CSS Rendering ■ Javascript events ■ Binary Downloads ■ Redirects ■ Broken HTML ■ Flash ■ Broken CSS ■ Images ■ Performance ■ Websockets ■ Forking / Multiprocessing ■ WebGL ■ [...]
  • 14. Software Architecture What it should look like 14 The Front-End A HTTP RXSS BSQLI EVAL The Core Library PXSS LFI OSC Reporting Engine SQL RFI [...] The Exploitation Engine
  • 15. Changes? Improvments? 15 ■ Replacing the HTTP library by a Webkit Engine ■ Less code (A lot less code) ■ 100% support for JS/Ajax/Broken HTML/JS Events/Crazy Redirects and all kinds of things ■ The ability to simulate human user behaviour ■ CSS Renderings (Two text fields beside each other: 10px - one of them is a input[type=password]) - May be a login!
  • 16. Making it scale (heavily) 16 ■ Webkit is slow (Website rendering, Executing JS, ... - compared to - Speaking Plaintext HTTP) ■ Downloading Images is slow ■ Waiting for delayed JS events is slow ■ Flash is even slower
  • 17. Making it scale (heavily) Bad news: Qt / PyQt / PySide 17 ■ QtWebkit does not support multithreading ■ It tends to SEGFAULT from time to time :( ■ Multiple QApplication instances are almost impossible to handle in one Python namespace
  • 18. Making it scale (heavily) Good news: Building a preforking TCP Server 18 ■ Spawning a pool of processes works quite well (one QApplication +one Browser instance per Process) ■ Simultaneous downloads ■ Better accessibility inside the scanner (multiprocessing insides loops to increase performance)
  • 19. Missing pieces 19 ■ Mastering Authentication ■ Exploitation & Privilege Escalation ■ Geographically distributed scanning: Using the cloud ■ Reporting
  • 20. Mastering Authentication 20 ■ There is no such thing as a standarized web login ■ Basically, everybody develops access control on the web slightly differently ■ You can try to detect them by the name/id of the attributes, but that is not reliable ■ But in the end, Web logins generally have a few things in common that makes them easily detectable. At least, for our browser engine
  • 21. Mastering Authentication Not more than 2 visible (!) text fields 21 has_login_texts()
  • 23. Mastering Authentication Geometry! Usually, the two visible text fields are under(), next_to() or at least near(radius=10px) each other 23 X1 = X2 X1 = X2 ! Y1 = Y2
  • 24. Mastering Authentication 24 ■ That was easy! ■ The common way to solve that problem, is to iterate through a wordlist (login, auth, signin, [...]) while checking the input[id], input[name] attributes ■ That’s not necessarily wrong or bad practice ■ After putting the pieces together: ■ .login(“username”, “password”)
  • 25. Mastering Authentication Demo Time 25 ■ Proof Of Concept 1: Twitter (Some Javascript) ■ Proof Of Concept 2: Facebook (More Javascript) ■ Proof Of Concept 3: Google Plus (Most Javascript + Browser Hacks)
  • 26. Mastering Authentication When we are signed in 26 ■ New problems occur: How can we let the scanner check if we are indeed signed in? ■ Common practive: Looking for a /logout/i String ■ The problem: Inefficient. Likely to cause false positives ■ There has to be a better way: ■ Introduction “Strategies”
  • 27. Strategy.Authentication Step 1: Identification 27 ■ Identifying a login form (3-way approach, input[type=password], geometry, [...])
  • 28. Strategy.Authentication Step 2: Error messages (Why a browser engines rocks) 28 ■ Verifying wrong credentials - Random strings - Failed login #BA.... -> #E4...
  • 29. Strategy.Authentication Step 3: Going in: .login(“..”, “..”) 29 ■ Verifying valid credentials - Behaviour should not be similiar to the behaviour of a invalid login
  • 30. Strategy.Authentication Step 4: Going out. .logout() 30 ■ Doing similiar work again for .logout() function seems obsolote ■ But it really isn’t. ■ It is the basis to a .is_still_loggedin() function ■ Which is really important to stay logged in during crawling ■ And if the scanner logged itself out, it can simply .login() again ■ That’s cool. :-)
  • 31. Exploitation and Privilege Escalation 31 ■ There is a whole universe besides injection vulnerabilities ■ Usually, scanners don’t detect them ■ But they should ■ And now they can: .login(“user1”, “...”); .logout(); .login(“user2”, “...”) ■ => Demo Time: Privilege Escalation, Multi-User Systems
  • 32. Geographically distributed scanning: Using the cloud 32 ■ When (injection) vulnerabilities are getting complicated: ■ Scenario 1: The backend of a website creates a log entry for every new IP address. It logs the USERAGENT. The log entries are kept in a SQL database. The function that creates the log entries, is vulnerable. The User-Agent is injectable. The problem is: ■ It only works once. As soon as the IP is in the database, the function won’t be executed anymore :-( ■ ==> SQLMap (and every other tool) will fail.
  • 33. Geographically distributed scanning: Using the cloud 33 ■ But they shouldn’t! ■ The limitation is totally detectable ■ And a new IP is just as far away as a single EC2 API call
  • 34. Geographically distributed scanning: Using the cloud 34 ■ Indeed! The cloud is a good thing for security :) ■ Demo Time: Introducing: sqlmap and w3af (on steroids)
  • 35. Combining “Strategies” and the distributed scanning 35 ■ Introducing next generation vulnerability scanning ■ Exploiting a really amazingly hard SQL Injection ■ Demo Time
  • 36. Further Research & Additional Ideas 36 ■ Country specific restrictions can be by-passed in a fully automatic manner ■ (Error) messages can be parsed and interpreted: Wolfram Alpha ■ Bloomfilters should be integrated ■ Other “Strategies” should be implemented (the limitations are gone)
  • 37. More Live Demos 37 ■ Demonstrating a logical layer beyond Authentication: .pay(“0000111122223333”, CVV=121, type=VISA) .search(“search query”) .sort(“DESC UNION SELECT [...]”) ■ Interpreting error messages ■ Pivoting on penetrated hosts - Spawning another scanner instance ■ And finally: Reporting!
  • 38. Thanks! 38