SlideShare a Scribd company logo
Headless Browser
Hide & Seek
Sergey Shekyan, Bei Zhang
Shape Security
Who We Are
• Bei Zhang
Senior Software Engineer at Shape Security, focused on analysis
and countermeasures of automated web attacks. Previously, he
worked at the Chrome team at Google with a focus on the Chrome
Apps API. His interests include web security, source code analysis,
and algorithms.
• Sergey Shekyan
Principal Engineer at Shape Security, focused on the development of
the new generation web security product. Prior to Shape Security, he
spent 4 years at Qualys developing their on demand web
application vulnerability scanning service. Sergey presented
research at security conferences around the world, covering various
information security topics.
Is There A Problem?
What Is a Headless Browser and How it Works
Scriptable browser environment that doesn’t require GUI
• Existing browser layout engine with bells and whistles
(PhantomJS - WebKit, SlimerJS - Gecko, TrifleJS - Trident)
• Custom software that models a browser (ZombieJS,
HtmlUnit)
• Selenium (WebDriver API)
What Is a Headless Browser and How it Works
Discussion will focus on PhantomJS:
• Backed by WebKit engine
• Cross-platform
• Popular
• True headless
PhantomJS World
PhantomJS
JavaScript Context
QWebFrame
QtWebKit
Web Page
JavaScript
Context
Control
Callback
Injection
PageEvent
Callbacks are
serialized
var page = require('webpage').create();
page.open(url, function(status) {
var title = page.evaluate(function() {
return document.title;
});
console.log('Page title is ' + title);
});
Legitimate uses and how you can benefit
• Web Application functional and performance
testing
• Crawler that can provide certain amount of
interaction to reveal web application topology,
Automated DOM XSS, CSRF detection
• SEO (render dynamic web page into static
HTML to feed to search engines)
• Reporting, image generation
Malicious Use of Headless Browser
• Fuzzing
• Botnet
• Content scraping
• Login brute force attacks
• Click fraud
• Bidding wars
Web admins tend to block PhantomJS in production,
so pretending to be a real browser is healthy choice
How It Is Different From a Real Browser
• Outdated WebKit engine (close to Safari 5 engine, 4 y.o.)
• Uses Qt Framework’s QtWebKit wrapper around WebKit
• Qt rendering engine
• Qt network stack, SSL implementation
• Qt Cookie Jar, that doesn’t support RFC 2965
• No Media Hardware support (no video and audio)
• Exposes window.callPhantom and window._phantom
• No sandboxing
Good vs. Bad
Headless Browser Seek
• Look at user agent string
if (/PhantomJS/.test(window.navigator.userAgent)) {
console.log(‘PhantomJS environment detected.’);
}
Headless Browser Hide
• Making user-agent (and navigator.userAgent) a
“legitimate” one:
var page = require(‘webpage').create();
page.settings.userAgent = ‘Mozilla/5.0 (Macintosh; Intel Mac
OS X 10.9; rv:30.0) Gecko/20100101 Firefox/30.0';
Score board
Phantom Web site
User Agent String Win Lose
Headless Browser Seek
• Sniff for PluginArray content
if (!(navigator.plugins instanceof PluginArray) ||
navigator.plugins.length == 0) {
    console.log("PhantomJS environment detected.");
  } else {
    console.log("PhantomJS environment not detected.");
  }
Headless Browser Hide
• Fake navigator object, populate PluginArray with whatever values you need.
• Spoofing Plugin objects inside the PluginArray is tedious and hard.
• Websites can actually create a plugin to test it.
• CONCLUSION: Not a good idea to spoof plugins.
page.onInitialized = function () {
    page.evaluate(function () {
        var oldNavigator = navigator;
        var oldPlugins = oldNavigator.plugins;
        var plugins = {};
        plugins.length = 1;
        plugins.__proto__ = oldPlugins.__proto__;
        window.navigator = {plugins: plugins};
        window.navigator.__proto__ = oldNavigator.__proto__;
    });
};
Score board
Phantom Web site
User Agent String Win Lose
Inspect PluginArray Lose Win
Headless Browser Seek
• Alert/prompt/confirm popup suppression timing
detection
var start = Date.now();
  prompt('I`m just kidding');
  var elapse = Date.now() - start;
  if (elapse < 15) {
    console.log("PhantomJS environment detected. #1");
  } else {
    console.log("PhantomJS environment not detected.");
  }
Headless Browser Hide
• Can’t use setTimeout, but blocking the callback by
all means would work
page.onAlert = page.onConfirm = page.onPrompt = function ()
{
    for (var i = 0; i < 1e8; i++) {
    }
    return "a";
};
Score board
Phantom Web site
User Agent String Win Lose
Inspect PluginArray Lose Win
Timed alert() Lose Win
Headless Browser Seek
• Default order of headers is consistently different
in PhantomJS. Camel case in some header
values is also a good point to look at.
PhantomJS 1.9.7
GET / HTTP/1.1
User-Agent:
Accept:
Connection: Keep-Alive
Accept-Encoding:
Accept-Language:
Host:
Chrome 37
GET / HTTP/1.1
Host:
Connection: keep-alive
Accept:
User-Agent:
Accept-Encoding:
Accept-Language:
Headless Browser Hide
• A custom proxy server in front of PhantomJS
instance that makes headers look consistent with
user agent string
Score board
Phantom Web site
User Agent String Win Lose
Inspect PluginArray Lose Win
Timed alert() Lose Win
HTTP Header order Win Lose
Headless Browser Seek
• PhantomJS exposes APIs:
• window.callPhantom
• window._phantom // not documented.
if (window.callPhantom || window._phantom) {
  console.log("PhantomJS environment detected.");
} else {
 console.log("PhantomJS environment not detected.");
}
Headless Browser Hide
• store references to original callPhantom, _phantom
• delete window.callPhantom, window._phantom
page.onInitialized = function () {
    page.evaluate(function () {
        var p = window.callPhantom;
        delete window._phantom;
        delete window.callPhantom;
        Object.defineProperty(window, "myCallPhantom", {
            get: function () { return p;},
            set: function () {}, enumerable: false});
        setTimeout(function () { window.myCallPhantom();}, 1000);
    });
};
page.onCallback = function (obj) { console.log(‘profit!'); };
Unguessable name
Score board
Phantom Web site
User Agent String Win Lose
Inspect PluginArray Lose Win
Timed alert() Lose Win
HTTP Header order Win Lose
window.callPhantom Win Lose
Headless Browser Seek
• Spoofing DOM API properties of real browsers:
• WebAudio
• WebRTC
• WebSocket
• Device APIs
• FileAPI
• WebGL
• CSS3 - not observable. Defeats printing.
• Our research on WebSockets: http://goo.gl/degwTr
Score board
Phantom Web site
User Agent String Win Lose
Inspect PluginArray Lose Win
Timed alert() Lose Win
HTTP Header order Win Lose
window.callPhantom Win Lose
HTML5 features Lose Win
Headless Browser Seek
• Significant difference in JavaScript Engine: bind() is
not defined in PhantomJS prior to version 2
(function () {
    if (!Function.prototype.bind) {
      console.log("PhantomJS environment detected.");
      return;
    }
    console.log("PhantomJS environment not detected.");
  })();
Function.prototype.bind = function () {
var func = this;
var self = arguments[0];
var rest = [].slice.call(arguments, 1);
return function () {
var args = [].slice.call(arguments, 0);
return func.apply(self, rest.concat(args));
};
};
Headless Browser Seek
• Detecting spoofed Function.prototype.bind for
PhantomJS prior to version 2
(function () {
    if (!Function.prototype.bind) {
      console.log("PhantomJS environment detected. #1");
      return;
    }
    if (Function.prototype.bind.toString().replace(/bind/g, 'Error') !=
Error.toString()) {
      console.log("PhantomJS environment detected. #2");
      return;
    }
    console.log("PhantomJS environment not detected.");
  })();
Headless Browser Hide
• Spoofing Function.prototype.toString
function functionToString() {
    if (this === bind) {
        return nativeFunctionString;
    }
    return oldCall.call(oldToString, this);
}
Headless Browser Seek
• Detecting spoofed Function.prototype.bind for
PhantomJS prior to version 2
(function () {
    if (!Function.prototype.bind) {
      console.log("PhantomJS environment detected. #1");
      return;
    }
    if (Function.prototype.bind.toString().replace(/bind/g, 'Error') !=
Error.toString()) {
      console.log("PhantomJS environment detected. #2");
      return;
    }
    if (Function.prototype.toString.toString().replace(/toString/g, 'Error') != Error.toString()) {
      console.log("PhantomJS environment detected. #3");
      return;
    }
    console.log("PhantomJS environment not detected.");
  })();
Headless Browser Hide
• Spoofing Function.prototype.toString.toString
function functionToString() {
    if (this === bind) {
        return nativeFunctionString;
    }
    if (this === functionToString) {
        return nativeToStringFunctionString;
    }
    if (this === call) {
        return nativeCallFunctionString;
    }
    if (this === apply) {
        return nativeApplyFunctionString;
    }
    var idx = indexOfArray(bound, this);
    if (idx >= 0) {
        return nativeBoundFunctionString;
    }
    return oldCall.call(oldToString, this);
}
Score board
Phantom Web site
User Agent String Win Lose
Inspect PluginArray Lose Win
Timed alert() Lose Win
HTTP Header order Win Lose
window.callPhantom Win Lose
HTML5 features Lose Win
Function.prototype.bind Win Lose
Headless Browser Seek
• PhantomJS2 is very EcmaScript5 spec compliant,
so checking for outstanding JavaScript features is
not going to work
Headless Browser Seek
• Stack trace generated by PhantomJs
• Stack trace generated by SlimerJS
• Hmm, there is something common…
at querySelectorAll (phantomjs://webpage.evaluate():9:10)
at phantomjs://webpage.evaluate():19:30
at phantomjs://webpage.evaluate():20:7
at global code (phantomjs://webpage.evaluate():20:13)
at evaluateJavaScript ([native code])
at global code (/Users/sshekyan/Projects/phantomjs/spoof.js:8:14)
Element.prototype.querySelectorAll@phantomjs://webpage.evaluate():9
@phantomjs://webpage.evaluate():19
@phantomjs://webpage.evaluate():20
Headless Browser Seek
var err;
try {
null[0]();
} catch (e) {
err = e;
}
if (indexOfString(err.stack, 'phantomjs') > -1) {
console.log("PhantomJS environment detected.");
} else {
 console.log("PhantomJS environment is not detected.");
It is not possible to override the thrown TypeError
generic indexOf(), can be spoofed, define your own
Headless Browser Hide
• Modifying webpage.cpp:
QVariant WebPage::evaluateJavaScript(const QString &code)
{
QVariant evalResult;
QString function = "(" + code + ")()";
evalResult = m_currentFrame->evaluateJavaScript(function,
QString("phantomjs://webpage.evaluate()"));
return evalResult;
}
at
at
at
at
at evaluateJavaScript ([native code])
at
at global code (/Users/sshekyan/Projects/phantomjs/spoof.js:8:14)
• Produces
Headless Browser Hide
• Spoofed PhantomJs vs Chrome:
at querySelectorAll (Object.InjectedScript:9:10)
at Object.InjectedScript:19:30
at Object.InjectedScript:20:7
at global code (Object.InjectedScript:20:13)
at evaluateJavaScript ([native code])
at
at global code (/Users/sshekyan/Projects/phantomjs/spoof.js:8:14)
TypeError: Cannot read property '0' of null
at HTMLDocument.Document.querySelectorAll.Element.querySelectorAll [as
querySelectorAll] (<anonymous>:21:5)
at <anonymous>:2:10
at Object.InjectedScript._evaluateOn (<anonymous>:730:39)
at Object.InjectedScript._evaluateAndWrap (<anonymous>:669:52)
at Object.InjectedScript.evaluate (<anonymous>:581:21)
Score board
Phantom Web site
User Agent String Win Lose
Inspect PluginArray Lose Win
Timed alert() Lose Win
HTTP Header order Win Lose
window.callPhantom Win Lose
HTML5 features Lose Win
Function.prototype.bind Win Lose
Stack trace Lose Win
Score board
Phantom Web site
User Agent String Win Lose
Inspect PluginArray Lose Win
Timed alert() Lose Win
HTTP Header order Win Lose
window.callPhantom Win Lose
HTML5 features Lose Win
Function.prototype.bind Win Lose
Stack trace Lose Win
SCORE: 4 4
Attacking PhantomJS
How to turn a headless browser against the
attacker
The usual picture:
• PhantomJS running as root
• No sandboxing in PhantomJS
• Blindly executing untrusted JavaScript
• outdated third party libs (libpng, libxml, etc.)
Can possibly lead to:
• abuse of PhantomJS
• abuse of OS running PhantomJS
Headless Browser Seek
var html = document.querySelectorAll('html');
var oldQSA = document.querySelectorAll;
Document.prototype.querySelectorAll =
Element.prototype.querySelectorAll = function () {
var err;
try {
null[0]();
} catch (e) {
err = e;
}
if (indexOfString(err.stack, 'phantomjs') > -1) {
return html;
} else {
return oldQSA.apply(this, arguments);
}
};
It is not possible to override the thrown TypeError
generic indexOf(), can be spoofed, define your
Headless Browser Seek
• In a lot of cases --web-security=false is used in
PhantomJS
var xhr = new XMLHttpRequest();
xhr.open('GET', 'file:/etc/hosts', false);
xhr.onload = function () {
console.log(xhr.responseText);
};
xhr.onerror = function (e) {
console.log('Error: ' + JSON.stringify(e));
};
xhr.send();
Headless Browser Seek
• Obfuscate, randomize the output, randomize the modified API
call
var _0x34c7=["x68x74x6D
x6C","x71x75x65x72x79x53x65x6Cx65x63x74x6F
x72x41x6Cx6C","x70x72x6Fx74x6F
x74x79x70x65","x73x74x61x63x6B","x70x68x61x6E
x74x6Fx6Dx6Ax73","x61x70x70x6Cx79"];var
html=document[_0x34c7[1]](_0x34c7[0]);var
apsdk=document[_0x34c7[1]];Document[_0x34c7[2]]
[_0x34c7[1]]=Element[_0x34c7[2]][_0x34c7[1]]=function ()
{var _0xad6dx3;try{null[0]();} catch(e)
{_0xad6dx3=e;} ;if(indexOfString(_0xad6dx3[_0x34c7[3]],_0
x34c7[4])>-1){return html;} else {return apsdk[_0x34c7[5]]
(this,arguments);};};
Tips for Using Headless Browsers Safely
• If you don’t need a full blown browser engine, don’t
use it
• Do not run with ‘- -web-security=false’ in production,
try not to do that in tests as well.
• Avoid opening arbitrary page from the Internet
• No root or use chroot
• Use child processes to run webpages
• If security is a requirement, use alternatives or on a
controlled environment, use Selenium
Tips For Web Admins
• It is not easy to unveil a user agent. Make sure you want
to do it.
• Do sniff for headless browsers (some will bail out)
• Combine several detection techniques
• Reject known unwanted user agents (5G Blacklist 2013
is a good start)
• Alter DOM API if headless browser is detected
• DoS
• Pwn
Start Detecting Already
Ready to use examples:
github.com/ikarienator/phantomjs_hide_and_seek
References
• http://www.sba-research.org/wp-content/uploads/publications/jsfingerprinting.pdf
• https://kangax.github.io/compat-table/es5/
• https://media.blackhat.com/us-13/us-13-Grossman-Million-Browser-Botnet.pdf
• http://ariya.ofilabs.com/2011/10/detecting-browser-sniffing-2.html
• http://www.darkreading.com/attacks-breaches/ddos-attack-used-headless-browsers-
in-150-hour-siege/d/d-id/1140696?
• http://vamsoft.com/downloads/articles/vamsoft-headless-browsers-in-forum-spam.pdf
• http://blog.spiderlabs.com/2013/02/server-site-xss-attack-detection-with-modsecurity-
and-phantomjs.html
• http://googleprojectzero.blogspot.com/2014/07/pwn4fun-spring-2014-safari-part-
i_24.html
Thank you!
@ikarienator
@sshekyan

More Related Content

What's hot

API Test Automation Using Karate (Anil Kumar Moka)
API Test Automation Using Karate (Anil Kumar Moka)API Test Automation Using Karate (Anil Kumar Moka)
API Test Automation Using Karate (Anil Kumar Moka)
Peter Thomas
 
JUnit Presentation
JUnit PresentationJUnit Presentation
JUnit Presentation
priya_trivedi
 
End to end test automation with cypress
End to end test automation with cypressEnd to end test automation with cypress
End to end test automation with cypress
PankajSingh184960
 
Waf bypassing Techniques
Waf bypassing TechniquesWaf bypassing Techniques
Waf bypassing Techniques
Avinash Thapa
 
Selenium Grid
Selenium GridSelenium Grid
Selenium Grid
nirvdrum
 
Cypress - Best Practices
Cypress - Best PracticesCypress - Best Practices
Cypress - Best Practices
Brian Mann
 
Thick client pentesting_the-hackers_meetup_version1.0pptx
Thick client pentesting_the-hackers_meetup_version1.0pptxThick client pentesting_the-hackers_meetup_version1.0pptx
Thick client pentesting_the-hackers_meetup_version1.0pptx
Anurag Srivastava
 
POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...
POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...
POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...
Postman
 
Unit testing with JUnit
Unit testing with JUnitUnit testing with JUnit
Unit testing with JUnit
Pokpitch Patcharadamrongkul
 
Selenium with java
Selenium with javaSelenium with java
Selenium with java
Gousalya Ramachandran
 
Callback Function
Callback FunctionCallback Function
Callback Function
Roland San Nicolas
 
Introduction to headless browsers
Introduction to headless browsersIntroduction to headless browsers
Introduction to headless browsers
Multidots Solutions Pvt Ltd
 
Cypress testing
Cypress testingCypress testing
Cypress testing
Vladyslav Romanchenko
 
Robot framework 을 이용한 기능 테스트 자동화
Robot framework 을 이용한 기능 테스트 자동화Robot framework 을 이용한 기능 테스트 자동화
Robot framework 을 이용한 기능 테스트 자동화
Jaehoon Oh
 
Asp.net MVC training session
Asp.net MVC training sessionAsp.net MVC training session
Asp.net MVC training session
Hrichi Mohamed
 
Unit Testing Android Applications
Unit Testing Android ApplicationsUnit Testing Android Applications
Unit Testing Android Applications
Rody Middelkoop
 
Neoload
Neoload Neoload
Neoload
Kumar Gupta
 
Using Postman to Automate API On-Boarding
Using Postman to Automate API On-BoardingUsing Postman to Automate API On-Boarding
Using Postman to Automate API On-Boarding
Postman
 
Postman Webinar: “Continuous Testing with Postman”
Postman Webinar: “Continuous Testing with Postman”Postman Webinar: “Continuous Testing with Postman”
Postman Webinar: “Continuous Testing with Postman”
Postman
 
UI Testing Automation
UI Testing AutomationUI Testing Automation
UI Testing Automation
AgileEngine
 

What's hot (20)

API Test Automation Using Karate (Anil Kumar Moka)
API Test Automation Using Karate (Anil Kumar Moka)API Test Automation Using Karate (Anil Kumar Moka)
API Test Automation Using Karate (Anil Kumar Moka)
 
JUnit Presentation
JUnit PresentationJUnit Presentation
JUnit Presentation
 
End to end test automation with cypress
End to end test automation with cypressEnd to end test automation with cypress
End to end test automation with cypress
 
Waf bypassing Techniques
Waf bypassing TechniquesWaf bypassing Techniques
Waf bypassing Techniques
 
Selenium Grid
Selenium GridSelenium Grid
Selenium Grid
 
Cypress - Best Practices
Cypress - Best PracticesCypress - Best Practices
Cypress - Best Practices
 
Thick client pentesting_the-hackers_meetup_version1.0pptx
Thick client pentesting_the-hackers_meetup_version1.0pptxThick client pentesting_the-hackers_meetup_version1.0pptx
Thick client pentesting_the-hackers_meetup_version1.0pptx
 
POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...
POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...
POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...
 
Unit testing with JUnit
Unit testing with JUnitUnit testing with JUnit
Unit testing with JUnit
 
Selenium with java
Selenium with javaSelenium with java
Selenium with java
 
Callback Function
Callback FunctionCallback Function
Callback Function
 
Introduction to headless browsers
Introduction to headless browsersIntroduction to headless browsers
Introduction to headless browsers
 
Cypress testing
Cypress testingCypress testing
Cypress testing
 
Robot framework 을 이용한 기능 테스트 자동화
Robot framework 을 이용한 기능 테스트 자동화Robot framework 을 이용한 기능 테스트 자동화
Robot framework 을 이용한 기능 테스트 자동화
 
Asp.net MVC training session
Asp.net MVC training sessionAsp.net MVC training session
Asp.net MVC training session
 
Unit Testing Android Applications
Unit Testing Android ApplicationsUnit Testing Android Applications
Unit Testing Android Applications
 
Neoload
Neoload Neoload
Neoload
 
Using Postman to Automate API On-Boarding
Using Postman to Automate API On-BoardingUsing Postman to Automate API On-Boarding
Using Postman to Automate API On-Boarding
 
Postman Webinar: “Continuous Testing with Postman”
Postman Webinar: “Continuous Testing with Postman”Postman Webinar: “Continuous Testing with Postman”
Postman Webinar: “Continuous Testing with Postman”
 
UI Testing Automation
UI Testing AutomationUI Testing Automation
UI Testing Automation
 

Similar to Detecting headless browsers

External JavaScript Widget Development Best Practices
External JavaScript Widget Development Best PracticesExternal JavaScript Widget Development Best Practices
External JavaScript Widget Development Best Practices
Volkan Özçelik
 
Java scriptwidgetdevelopmentjstanbul2012
Java scriptwidgetdevelopmentjstanbul2012Java scriptwidgetdevelopmentjstanbul2012
Java scriptwidgetdevelopmentjstanbul2012
Volkan Özçelik
 
External JavaScript Widget Development Best Practices (updated) (v.1.1)
External JavaScript Widget Development Best Practices (updated) (v.1.1) External JavaScript Widget Development Best Practices (updated) (v.1.1)
External JavaScript Widget Development Best Practices (updated) (v.1.1)
Volkan Özçelik
 
Technical Tips: Visual Regression Testing and Environment Comparison with Bac...
Technical Tips: Visual Regression Testing and Environment Comparison with Bac...Technical Tips: Visual Regression Testing and Environment Comparison with Bac...
Technical Tips: Visual Regression Testing and Environment Comparison with Bac...
Building Blocks
 
vodQA Pune (2019) - Browser automation using dev tools
vodQA Pune (2019) - Browser automation using dev toolsvodQA Pune (2019) - Browser automation using dev tools
vodQA Pune (2019) - Browser automation using dev tools
vodQA
 
Waf.js: How to Protect Web Applications using JavaScript
Waf.js: How to Protect Web Applications using JavaScriptWaf.js: How to Protect Web Applications using JavaScript
Waf.js: How to Protect Web Applications using JavaScript
Denis Kolegov
 
soft-shake.ch - Hands on Node.js
soft-shake.ch - Hands on Node.jssoft-shake.ch - Hands on Node.js
soft-shake.ch - Hands on Node.js
soft-shake.ch
 
DrupalCon Dublin 2016 - Automated browser testing with Nightwatch.js
DrupalCon Dublin 2016 - Automated browser testing with Nightwatch.jsDrupalCon Dublin 2016 - Automated browser testing with Nightwatch.js
DrupalCon Dublin 2016 - Automated browser testing with Nightwatch.js
Vladimir Roudakov
 
Intro To webOS
Intro To webOSIntro To webOS
Intro To webOS
fpatton
 
Protractor Tutorial Quality in Agile 2015
Protractor Tutorial Quality in Agile 2015Protractor Tutorial Quality in Agile 2015
Protractor Tutorial Quality in Agile 2015
Andrew Eisenberg
 
Google Back To Front: From Gears to App Engine and Beyond
Google Back To Front: From Gears to App Engine and BeyondGoogle Back To Front: From Gears to App Engine and Beyond
Google Back To Front: From Gears to App Engine and Beyond
dion
 
Ui testing with splinter - Fri, 30 May 2014
Ui testing with splinter - Fri, 30 May 2014Ui testing with splinter - Fri, 30 May 2014
Ui testing with splinter - Fri, 30 May 2014
Taizo Ito
 
Cross Platform Appium Tests: How To
Cross Platform Appium Tests: How ToCross Platform Appium Tests: How To
Cross Platform Appium Tests: How To
GlobalLogic Ukraine
 
OWASP ZAP Workshop for QA Testers
OWASP ZAP Workshop for QA TestersOWASP ZAP Workshop for QA Testers
OWASP ZAP Workshop for QA Testers
Javan Rasokat
 
Puppeteer - Headless Chrome Node API
Puppeteer - Headless Chrome Node APIPuppeteer - Headless Chrome Node API
Puppeteer - Headless Chrome Node API
Wilson Su
 
Complete_QA_Automation_Guide__1696637878.pdf
Complete_QA_Automation_Guide__1696637878.pdfComplete_QA_Automation_Guide__1696637878.pdf
Complete_QA_Automation_Guide__1696637878.pdf
ramya9288
 
orcreatehappyusers
orcreatehappyusersorcreatehappyusers
orcreatehappyusers
tutorialsruby
 
orcreatehappyusers
orcreatehappyusersorcreatehappyusers
orcreatehappyusers
tutorialsruby
 
Testing of javacript
Testing of javacriptTesting of javacript
Testing of javacript
Lei Kang
 
CasperJS and PhantomJS for Automated Testing
CasperJS and PhantomJS for Automated TestingCasperJS and PhantomJS for Automated Testing
CasperJS and PhantomJS for Automated Testing
X-Team
 

Similar to Detecting headless browsers (20)

External JavaScript Widget Development Best Practices
External JavaScript Widget Development Best PracticesExternal JavaScript Widget Development Best Practices
External JavaScript Widget Development Best Practices
 
Java scriptwidgetdevelopmentjstanbul2012
Java scriptwidgetdevelopmentjstanbul2012Java scriptwidgetdevelopmentjstanbul2012
Java scriptwidgetdevelopmentjstanbul2012
 
External JavaScript Widget Development Best Practices (updated) (v.1.1)
External JavaScript Widget Development Best Practices (updated) (v.1.1) External JavaScript Widget Development Best Practices (updated) (v.1.1)
External JavaScript Widget Development Best Practices (updated) (v.1.1)
 
Technical Tips: Visual Regression Testing and Environment Comparison with Bac...
Technical Tips: Visual Regression Testing and Environment Comparison with Bac...Technical Tips: Visual Regression Testing and Environment Comparison with Bac...
Technical Tips: Visual Regression Testing and Environment Comparison with Bac...
 
vodQA Pune (2019) - Browser automation using dev tools
vodQA Pune (2019) - Browser automation using dev toolsvodQA Pune (2019) - Browser automation using dev tools
vodQA Pune (2019) - Browser automation using dev tools
 
Waf.js: How to Protect Web Applications using JavaScript
Waf.js: How to Protect Web Applications using JavaScriptWaf.js: How to Protect Web Applications using JavaScript
Waf.js: How to Protect Web Applications using JavaScript
 
soft-shake.ch - Hands on Node.js
soft-shake.ch - Hands on Node.jssoft-shake.ch - Hands on Node.js
soft-shake.ch - Hands on Node.js
 
DrupalCon Dublin 2016 - Automated browser testing with Nightwatch.js
DrupalCon Dublin 2016 - Automated browser testing with Nightwatch.jsDrupalCon Dublin 2016 - Automated browser testing with Nightwatch.js
DrupalCon Dublin 2016 - Automated browser testing with Nightwatch.js
 
Intro To webOS
Intro To webOSIntro To webOS
Intro To webOS
 
Protractor Tutorial Quality in Agile 2015
Protractor Tutorial Quality in Agile 2015Protractor Tutorial Quality in Agile 2015
Protractor Tutorial Quality in Agile 2015
 
Google Back To Front: From Gears to App Engine and Beyond
Google Back To Front: From Gears to App Engine and BeyondGoogle Back To Front: From Gears to App Engine and Beyond
Google Back To Front: From Gears to App Engine and Beyond
 
Ui testing with splinter - Fri, 30 May 2014
Ui testing with splinter - Fri, 30 May 2014Ui testing with splinter - Fri, 30 May 2014
Ui testing with splinter - Fri, 30 May 2014
 
Cross Platform Appium Tests: How To
Cross Platform Appium Tests: How ToCross Platform Appium Tests: How To
Cross Platform Appium Tests: How To
 
OWASP ZAP Workshop for QA Testers
OWASP ZAP Workshop for QA TestersOWASP ZAP Workshop for QA Testers
OWASP ZAP Workshop for QA Testers
 
Puppeteer - Headless Chrome Node API
Puppeteer - Headless Chrome Node APIPuppeteer - Headless Chrome Node API
Puppeteer - Headless Chrome Node API
 
Complete_QA_Automation_Guide__1696637878.pdf
Complete_QA_Automation_Guide__1696637878.pdfComplete_QA_Automation_Guide__1696637878.pdf
Complete_QA_Automation_Guide__1696637878.pdf
 
orcreatehappyusers
orcreatehappyusersorcreatehappyusers
orcreatehappyusers
 
orcreatehappyusers
orcreatehappyusersorcreatehappyusers
orcreatehappyusers
 
Testing of javacript
Testing of javacriptTesting of javacript
Testing of javacript
 
CasperJS and PhantomJS for Automated Testing
CasperJS and PhantomJS for Automated TestingCasperJS and PhantomJS for Automated Testing
CasperJS and PhantomJS for Automated Testing
 

Recently uploaded

Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.
KrishnaveniMohan1
 
ppt on the brain chip neuralink.pptx
ppt  on   the brain  chip neuralink.pptxppt  on   the brain  chip neuralink.pptx
ppt on the brain chip neuralink.pptx
Reetu63
 
42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert
vaishalijagtap12
 
Software Test Automation - A Comprehensive Guide on Automated Testing.pdf
Software Test Automation - A Comprehensive Guide on Automated Testing.pdfSoftware Test Automation - A Comprehensive Guide on Automated Testing.pdf
Software Test Automation - A Comprehensive Guide on Automated Testing.pdf
kalichargn70th171
 
Streamlining End-to-End Testing Automation
Streamlining End-to-End Testing AutomationStreamlining End-to-End Testing Automation
Streamlining End-to-End Testing Automation
Anand Bagmar
 
Boost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management AppsBoost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management Apps
Jhone kinadey
 
Ensuring Efficiency and Speed with Practical Solutions for Clinical Operations
Ensuring Efficiency and Speed with Practical Solutions for Clinical OperationsEnsuring Efficiency and Speed with Practical Solutions for Clinical Operations
Ensuring Efficiency and Speed with Practical Solutions for Clinical Operations
OnePlan Solutions
 
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in NashikUpturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies
 
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
kalichargn70th171
 
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA ComplianceSecure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
ICS
 
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptxMigration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
ervikas4
 
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdfThe Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
kalichargn70th171
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Paul Brebner
 
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
gapen1
 
Orca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container OrchestrationOrca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container Orchestration
Pedro J. Molina
 
14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision
ShulagnaSarkar2
 
Refactoring legacy systems using events commands and bubble contexts
Refactoring legacy systems using events commands and bubble contextsRefactoring legacy systems using events commands and bubble contexts
Refactoring legacy systems using events commands and bubble contexts
Michał Kurzeja
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
kalichargn70th171
 
Hyperledger Besu 빨리 따라하기 (Private Networks)
Hyperledger Besu 빨리 따라하기 (Private Networks)Hyperledger Besu 빨리 따라하기 (Private Networks)
Hyperledger Besu 빨리 따라하기 (Private Networks)
wonyong hwang
 
What’s New in VictoriaLogs - Q2 2024 Update
What’s New in VictoriaLogs - Q2 2024 UpdateWhat’s New in VictoriaLogs - Q2 2024 Update
What’s New in VictoriaLogs - Q2 2024 Update
VictoriaMetrics
 

Recently uploaded (20)

Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.
 
ppt on the brain chip neuralink.pptx
ppt  on   the brain  chip neuralink.pptxppt  on   the brain  chip neuralink.pptx
ppt on the brain chip neuralink.pptx
 
42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert
 
Software Test Automation - A Comprehensive Guide on Automated Testing.pdf
Software Test Automation - A Comprehensive Guide on Automated Testing.pdfSoftware Test Automation - A Comprehensive Guide on Automated Testing.pdf
Software Test Automation - A Comprehensive Guide on Automated Testing.pdf
 
Streamlining End-to-End Testing Automation
Streamlining End-to-End Testing AutomationStreamlining End-to-End Testing Automation
Streamlining End-to-End Testing Automation
 
Boost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management AppsBoost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management Apps
 
Ensuring Efficiency and Speed with Practical Solutions for Clinical Operations
Ensuring Efficiency and Speed with Practical Solutions for Clinical OperationsEnsuring Efficiency and Speed with Practical Solutions for Clinical Operations
Ensuring Efficiency and Speed with Practical Solutions for Clinical Operations
 
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in NashikUpturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in Nashik
 
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
 
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA ComplianceSecure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
 
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptxMigration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
 
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdfThe Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
 
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
 
Orca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container OrchestrationOrca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container Orchestration
 
14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision14 th Edition of International conference on computer vision
14 th Edition of International conference on computer vision
 
Refactoring legacy systems using events commands and bubble contexts
Refactoring legacy systems using events commands and bubble contextsRefactoring legacy systems using events commands and bubble contexts
Refactoring legacy systems using events commands and bubble contexts
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
 
Hyperledger Besu 빨리 따라하기 (Private Networks)
Hyperledger Besu 빨리 따라하기 (Private Networks)Hyperledger Besu 빨리 따라하기 (Private Networks)
Hyperledger Besu 빨리 따라하기 (Private Networks)
 
What’s New in VictoriaLogs - Q2 2024 Update
What’s New in VictoriaLogs - Q2 2024 UpdateWhat’s New in VictoriaLogs - Q2 2024 Update
What’s New in VictoriaLogs - Q2 2024 Update
 

Detecting headless browsers

  • 1. Headless Browser Hide & Seek Sergey Shekyan, Bei Zhang Shape Security
  • 2. Who We Are • Bei Zhang Senior Software Engineer at Shape Security, focused on analysis and countermeasures of automated web attacks. Previously, he worked at the Chrome team at Google with a focus on the Chrome Apps API. His interests include web security, source code analysis, and algorithms. • Sergey Shekyan Principal Engineer at Shape Security, focused on the development of the new generation web security product. Prior to Shape Security, he spent 4 years at Qualys developing their on demand web application vulnerability scanning service. Sergey presented research at security conferences around the world, covering various information security topics.
  • 3. Is There A Problem?
  • 4. What Is a Headless Browser and How it Works Scriptable browser environment that doesn’t require GUI • Existing browser layout engine with bells and whistles (PhantomJS - WebKit, SlimerJS - Gecko, TrifleJS - Trident) • Custom software that models a browser (ZombieJS, HtmlUnit) • Selenium (WebDriver API)
  • 5. What Is a Headless Browser and How it Works Discussion will focus on PhantomJS: • Backed by WebKit engine • Cross-platform • Popular • True headless
  • 6. PhantomJS World PhantomJS JavaScript Context QWebFrame QtWebKit Web Page JavaScript Context Control Callback Injection PageEvent Callbacks are serialized var page = require('webpage').create(); page.open(url, function(status) { var title = page.evaluate(function() { return document.title; }); console.log('Page title is ' + title); });
  • 7. Legitimate uses and how you can benefit • Web Application functional and performance testing • Crawler that can provide certain amount of interaction to reveal web application topology, Automated DOM XSS, CSRF detection • SEO (render dynamic web page into static HTML to feed to search engines) • Reporting, image generation
  • 8. Malicious Use of Headless Browser • Fuzzing • Botnet • Content scraping • Login brute force attacks • Click fraud • Bidding wars Web admins tend to block PhantomJS in production, so pretending to be a real browser is healthy choice
  • 9. How It Is Different From a Real Browser • Outdated WebKit engine (close to Safari 5 engine, 4 y.o.) • Uses Qt Framework’s QtWebKit wrapper around WebKit • Qt rendering engine • Qt network stack, SSL implementation • Qt Cookie Jar, that doesn’t support RFC 2965 • No Media Hardware support (no video and audio) • Exposes window.callPhantom and window._phantom • No sandboxing
  • 11. Headless Browser Seek • Look at user agent string if (/PhantomJS/.test(window.navigator.userAgent)) { console.log(‘PhantomJS environment detected.’); }
  • 12. Headless Browser Hide • Making user-agent (and navigator.userAgent) a “legitimate” one: var page = require(‘webpage').create(); page.settings.userAgent = ‘Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:30.0) Gecko/20100101 Firefox/30.0';
  • 13. Score board Phantom Web site User Agent String Win Lose
  • 14. Headless Browser Seek • Sniff for PluginArray content if (!(navigator.plugins instanceof PluginArray) || navigator.plugins.length == 0) {     console.log("PhantomJS environment detected.");   } else {     console.log("PhantomJS environment not detected.");   }
  • 15. Headless Browser Hide • Fake navigator object, populate PluginArray with whatever values you need. • Spoofing Plugin objects inside the PluginArray is tedious and hard. • Websites can actually create a plugin to test it. • CONCLUSION: Not a good idea to spoof plugins. page.onInitialized = function () {     page.evaluate(function () {         var oldNavigator = navigator;         var oldPlugins = oldNavigator.plugins;         var plugins = {};         plugins.length = 1;         plugins.__proto__ = oldPlugins.__proto__;         window.navigator = {plugins: plugins};         window.navigator.__proto__ = oldNavigator.__proto__;     }); };
  • 16. Score board Phantom Web site User Agent String Win Lose Inspect PluginArray Lose Win
  • 17. Headless Browser Seek • Alert/prompt/confirm popup suppression timing detection var start = Date.now();   prompt('I`m just kidding');   var elapse = Date.now() - start;   if (elapse < 15) {     console.log("PhantomJS environment detected. #1");   } else {     console.log("PhantomJS environment not detected.");   }
  • 18. Headless Browser Hide • Can’t use setTimeout, but blocking the callback by all means would work page.onAlert = page.onConfirm = page.onPrompt = function () {     for (var i = 0; i < 1e8; i++) {     }     return "a"; };
  • 19. Score board Phantom Web site User Agent String Win Lose Inspect PluginArray Lose Win Timed alert() Lose Win
  • 20. Headless Browser Seek • Default order of headers is consistently different in PhantomJS. Camel case in some header values is also a good point to look at. PhantomJS 1.9.7 GET / HTTP/1.1 User-Agent: Accept: Connection: Keep-Alive Accept-Encoding: Accept-Language: Host: Chrome 37 GET / HTTP/1.1 Host: Connection: keep-alive Accept: User-Agent: Accept-Encoding: Accept-Language:
  • 21. Headless Browser Hide • A custom proxy server in front of PhantomJS instance that makes headers look consistent with user agent string
  • 22. Score board Phantom Web site User Agent String Win Lose Inspect PluginArray Lose Win Timed alert() Lose Win HTTP Header order Win Lose
  • 23. Headless Browser Seek • PhantomJS exposes APIs: • window.callPhantom • window._phantom // not documented. if (window.callPhantom || window._phantom) {   console.log("PhantomJS environment detected."); } else {  console.log("PhantomJS environment not detected."); }
  • 24. Headless Browser Hide • store references to original callPhantom, _phantom • delete window.callPhantom, window._phantom page.onInitialized = function () {     page.evaluate(function () {         var p = window.callPhantom;         delete window._phantom;         delete window.callPhantom;         Object.defineProperty(window, "myCallPhantom", {             get: function () { return p;},             set: function () {}, enumerable: false});         setTimeout(function () { window.myCallPhantom();}, 1000);     }); }; page.onCallback = function (obj) { console.log(‘profit!'); }; Unguessable name
  • 25. Score board Phantom Web site User Agent String Win Lose Inspect PluginArray Lose Win Timed alert() Lose Win HTTP Header order Win Lose window.callPhantom Win Lose
  • 26. Headless Browser Seek • Spoofing DOM API properties of real browsers: • WebAudio • WebRTC • WebSocket • Device APIs • FileAPI • WebGL • CSS3 - not observable. Defeats printing. • Our research on WebSockets: http://goo.gl/degwTr
  • 27. Score board Phantom Web site User Agent String Win Lose Inspect PluginArray Lose Win Timed alert() Lose Win HTTP Header order Win Lose window.callPhantom Win Lose HTML5 features Lose Win
  • 28. Headless Browser Seek • Significant difference in JavaScript Engine: bind() is not defined in PhantomJS prior to version 2 (function () {     if (!Function.prototype.bind) {       console.log("PhantomJS environment detected.");       return;     }     console.log("PhantomJS environment not detected.");   })(); Function.prototype.bind = function () { var func = this; var self = arguments[0]; var rest = [].slice.call(arguments, 1); return function () { var args = [].slice.call(arguments, 0); return func.apply(self, rest.concat(args)); }; };
  • 29. Headless Browser Seek • Detecting spoofed Function.prototype.bind for PhantomJS prior to version 2 (function () {     if (!Function.prototype.bind) {       console.log("PhantomJS environment detected. #1");       return;     }     if (Function.prototype.bind.toString().replace(/bind/g, 'Error') != Error.toString()) {       console.log("PhantomJS environment detected. #2");       return;     }     console.log("PhantomJS environment not detected.");   })();
  • 30. Headless Browser Hide • Spoofing Function.prototype.toString function functionToString() {     if (this === bind) {         return nativeFunctionString;     }     return oldCall.call(oldToString, this); }
  • 31. Headless Browser Seek • Detecting spoofed Function.prototype.bind for PhantomJS prior to version 2 (function () {     if (!Function.prototype.bind) {       console.log("PhantomJS environment detected. #1");       return;     }     if (Function.prototype.bind.toString().replace(/bind/g, 'Error') != Error.toString()) {       console.log("PhantomJS environment detected. #2");       return;     }     if (Function.prototype.toString.toString().replace(/toString/g, 'Error') != Error.toString()) {       console.log("PhantomJS environment detected. #3");       return;     }     console.log("PhantomJS environment not detected.");   })();
  • 32. Headless Browser Hide • Spoofing Function.prototype.toString.toString function functionToString() {     if (this === bind) {         return nativeFunctionString;     }     if (this === functionToString) {         return nativeToStringFunctionString;     }     if (this === call) {         return nativeCallFunctionString;     }     if (this === apply) {         return nativeApplyFunctionString;     }     var idx = indexOfArray(bound, this);     if (idx >= 0) {         return nativeBoundFunctionString;     }     return oldCall.call(oldToString, this); }
  • 33. Score board Phantom Web site User Agent String Win Lose Inspect PluginArray Lose Win Timed alert() Lose Win HTTP Header order Win Lose window.callPhantom Win Lose HTML5 features Lose Win Function.prototype.bind Win Lose
  • 34. Headless Browser Seek • PhantomJS2 is very EcmaScript5 spec compliant, so checking for outstanding JavaScript features is not going to work
  • 35. Headless Browser Seek • Stack trace generated by PhantomJs • Stack trace generated by SlimerJS • Hmm, there is something common… at querySelectorAll (phantomjs://webpage.evaluate():9:10) at phantomjs://webpage.evaluate():19:30 at phantomjs://webpage.evaluate():20:7 at global code (phantomjs://webpage.evaluate():20:13) at evaluateJavaScript ([native code]) at global code (/Users/sshekyan/Projects/phantomjs/spoof.js:8:14) Element.prototype.querySelectorAll@phantomjs://webpage.evaluate():9 @phantomjs://webpage.evaluate():19 @phantomjs://webpage.evaluate():20
  • 36. Headless Browser Seek var err; try { null[0](); } catch (e) { err = e; } if (indexOfString(err.stack, 'phantomjs') > -1) { console.log("PhantomJS environment detected."); } else {  console.log("PhantomJS environment is not detected."); It is not possible to override the thrown TypeError generic indexOf(), can be spoofed, define your own
  • 37. Headless Browser Hide • Modifying webpage.cpp: QVariant WebPage::evaluateJavaScript(const QString &code) { QVariant evalResult; QString function = "(" + code + ")()"; evalResult = m_currentFrame->evaluateJavaScript(function, QString("phantomjs://webpage.evaluate()")); return evalResult; } at at at at at evaluateJavaScript ([native code]) at at global code (/Users/sshekyan/Projects/phantomjs/spoof.js:8:14) • Produces
  • 38. Headless Browser Hide • Spoofed PhantomJs vs Chrome: at querySelectorAll (Object.InjectedScript:9:10) at Object.InjectedScript:19:30 at Object.InjectedScript:20:7 at global code (Object.InjectedScript:20:13) at evaluateJavaScript ([native code]) at at global code (/Users/sshekyan/Projects/phantomjs/spoof.js:8:14) TypeError: Cannot read property '0' of null at HTMLDocument.Document.querySelectorAll.Element.querySelectorAll [as querySelectorAll] (<anonymous>:21:5) at <anonymous>:2:10 at Object.InjectedScript._evaluateOn (<anonymous>:730:39) at Object.InjectedScript._evaluateAndWrap (<anonymous>:669:52) at Object.InjectedScript.evaluate (<anonymous>:581:21)
  • 39. Score board Phantom Web site User Agent String Win Lose Inspect PluginArray Lose Win Timed alert() Lose Win HTTP Header order Win Lose window.callPhantom Win Lose HTML5 features Lose Win Function.prototype.bind Win Lose Stack trace Lose Win
  • 40. Score board Phantom Web site User Agent String Win Lose Inspect PluginArray Lose Win Timed alert() Lose Win HTTP Header order Win Lose window.callPhantom Win Lose HTML5 features Lose Win Function.prototype.bind Win Lose Stack trace Lose Win SCORE: 4 4
  • 42. How to turn a headless browser against the attacker The usual picture: • PhantomJS running as root • No sandboxing in PhantomJS • Blindly executing untrusted JavaScript • outdated third party libs (libpng, libxml, etc.) Can possibly lead to: • abuse of PhantomJS • abuse of OS running PhantomJS
  • 43.
  • 44. Headless Browser Seek var html = document.querySelectorAll('html'); var oldQSA = document.querySelectorAll; Document.prototype.querySelectorAll = Element.prototype.querySelectorAll = function () { var err; try { null[0](); } catch (e) { err = e; } if (indexOfString(err.stack, 'phantomjs') > -1) { return html; } else { return oldQSA.apply(this, arguments); } }; It is not possible to override the thrown TypeError generic indexOf(), can be spoofed, define your
  • 45. Headless Browser Seek • In a lot of cases --web-security=false is used in PhantomJS var xhr = new XMLHttpRequest(); xhr.open('GET', 'file:/etc/hosts', false); xhr.onload = function () { console.log(xhr.responseText); }; xhr.onerror = function (e) { console.log('Error: ' + JSON.stringify(e)); }; xhr.send();
  • 46. Headless Browser Seek • Obfuscate, randomize the output, randomize the modified API call var _0x34c7=["x68x74x6D x6C","x71x75x65x72x79x53x65x6Cx65x63x74x6F x72x41x6Cx6C","x70x72x6Fx74x6F x74x79x70x65","x73x74x61x63x6B","x70x68x61x6E x74x6Fx6Dx6Ax73","x61x70x70x6Cx79"];var html=document[_0x34c7[1]](_0x34c7[0]);var apsdk=document[_0x34c7[1]];Document[_0x34c7[2]] [_0x34c7[1]]=Element[_0x34c7[2]][_0x34c7[1]]=function () {var _0xad6dx3;try{null[0]();} catch(e) {_0xad6dx3=e;} ;if(indexOfString(_0xad6dx3[_0x34c7[3]],_0 x34c7[4])>-1){return html;} else {return apsdk[_0x34c7[5]] (this,arguments);};};
  • 47. Tips for Using Headless Browsers Safely • If you don’t need a full blown browser engine, don’t use it • Do not run with ‘- -web-security=false’ in production, try not to do that in tests as well. • Avoid opening arbitrary page from the Internet • No root or use chroot • Use child processes to run webpages • If security is a requirement, use alternatives or on a controlled environment, use Selenium
  • 48. Tips For Web Admins • It is not easy to unveil a user agent. Make sure you want to do it. • Do sniff for headless browsers (some will bail out) • Combine several detection techniques • Reject known unwanted user agents (5G Blacklist 2013 is a good start) • Alter DOM API if headless browser is detected • DoS • Pwn
  • 49. Start Detecting Already Ready to use examples: github.com/ikarienator/phantomjs_hide_and_seek
  • 50. References • http://www.sba-research.org/wp-content/uploads/publications/jsfingerprinting.pdf • https://kangax.github.io/compat-table/es5/ • https://media.blackhat.com/us-13/us-13-Grossman-Million-Browser-Botnet.pdf • http://ariya.ofilabs.com/2011/10/detecting-browser-sniffing-2.html • http://www.darkreading.com/attacks-breaches/ddos-attack-used-headless-browsers- in-150-hour-siege/d/d-id/1140696? • http://vamsoft.com/downloads/articles/vamsoft-headless-browsers-in-forum-spam.pdf • http://blog.spiderlabs.com/2013/02/server-site-xss-attack-detection-with-modsecurity- and-phantomjs.html • http://googleprojectzero.blogspot.com/2014/07/pwn4fun-spring-2014-safari-part- i_24.html