vodQA Pune (2019) - Browser automation using dev tools

1. BROWSER AUTOMATION USING
DEVTOOLS
VODQA Pune 2019
By :
Shilpa G
QA at ThoughtWorks
LinkedIn - /gshilpa3
Github - /Shilpagopal

2. WHY
DEVTOOLS?
Limitation of existing tools to automate all
the browser supported features.
■ Handling HTTP basic authentications
■ Getting events from browser
■ Setting fake geo location
■ Updating User-agent
■ Interception web responses
■ Network Simulation
■ Screen casting chrome session
Chrome Devtools unlocks the possibility
to automate most of the browser
supported features.
2
“We can’t solve
our problems with
the same thinking
we used when we
created them” -
Albert Einstein

3. BROWSER INSTRUMENTATION WITH DEVTOOLS
3

4. Chrome DevTools Protocol
allows for tools to instrument,
inspect, debug and profile
chromium, chrome and any blink
based browsers. Chrome Devtools
uses this protocol to perform any
action on browser.
4

5. 5
Demo
Listening to the protocol

6. SNIFFING CHROME DEVTOOLS PROTOCOL
Devtools via Chrome Extension
Devtools as Protocol Client.
6

7. DEBUGGING REMOTE INSTANCE OF CHROME
7
$ chrome --remote-debugging-port=9222 https://twpune.github.io/vodqa/

8. 8
Demo
Attaching DevTools frontend to
the remotely running Chrome
instance for debugging

9. const WebSocket = require('ws');
var fs = require('fs');
ws = new
WebSocket("ws://localhost:9222/devtools/page/387D2CABAF785990F1694
0EEF89BF032");
ws.onopen = function () {
console.log('websocket is connected ...');
ws.send('{"id":1,"method":"Page.enable","params":{}}', (err) => {
if (err) {
console.log(err);
}else{
ws.send('{"id":2,"method":"Page.navigate","params":{"url":"https://www.googl
e.com/"}}', (err) => {
if (err) {
console.log(err);
}else{
ws.send('{"id":3,"method":"Page.captureScreenshot","params":{}}', (err) => {
if (err) {console.log(err);}
});
}
});
}
});
}
ws.onmessage = function (ev) {
response = JSON.parse(ev.data);
if(response.id == 3){const buffer = new Buffer(response.result.data,
'base64');
fs.writeFile("./screenshots/ws_screenshot.png", buffer, function(err) {
if(err) {return console.log(err);}
console.log("The ScreenShot is Captured");
ws.close();
});
}
}
const puppeteer = require('puppeteer');
puppeteer.launch().then(async browser => {
const page = await browser.newPage();
await page.goto('https://www.google.com/');
await
page.screenshot({path:'./screenshots/homePage.png'});
await browser.close();
});

10. PUPPETEER
Node library which provides a high-level API to control Chrome or Chromium
over the DevTools Protocol
1
0

11. PYRAMID OF PUPPETEER
1
1
Script

12. PUPPETEER
Installation:
■ Zero configuration
■ Bundles latest chromium
■ High level API’s for the common use cases
■ asyn/await latest JS features
Execution:
1
2
“A Puppeteer is a
person who
manipulates an
inanimate object
that might be
shaped like a
human, animal or
mythical creature
or another object
to create the
illusion that the
puppet is alive”
npm i puppeteer --save
node test_name.js

13. HOW TO LAUNCH CHROME IN HEADFUL MODE
1
3
const browser = await puppeteer.launch({
headless: false,
executablePath: '/usr/bin/chrome'
});

14. 1
4
Test optimization using
puppeteer

15. DON’T HAVE TO WAIT FOR THE ENTIRE PAGE TO LOAD
1
5
const url = 'http://automationpractice.com/'
await page.goto(url, {waitUntil:
'domcontentloaded'})
// Resolve When <div id='home_page'>
exists in DOM
await page.waitForSelector('#home_page');

16. ABORT UNNECESSARY REQUESTS
1
6
// 1. Intercept network requests.
await page.setRequestInterception(true);
page.on('request', interceptedRequest => {
// 2. Ignore requests for resources that don't produce DOM
const whitelist = ['document', 'script', 'xhr', 'fetch'];
if (!whitelist.includes(interceptedRequest.resourceType()))
interceptedRequest.abort();
else
// 3. Pass through all other requests.
interceptedRequest.continue();
});
await page.goto('http://automationpractice.com/');

17. 1
7
Advanced puppeteer
examples

18. HANDLING HTTP BASIC AUTHENTICATION
1
8
{
"id": 2,
"method": "Network.setExtraHTTPHeaders",
"params": {
"headers": {
"Authorization": "Basic
YWRtaW46YWRtaW4="
}
}
}
Websocket JSON request

19. NETWORK EMULATION
1
9
{
"id": 7,
"method": "Network.emulateNetworkConditions",
"params": {
"offline": true,
"latency": 300,
"downloadThroughput": 250,
"uploadThroughput": 750
}
}
Websocket JSON request

20. MOCK GEO LOCATION
2
0
{
"id": 3,
"method": "Page.setGeolocationOverride",
"params": {
"latitude": 27.1752868,
"longitude": 78.040009,
"accuracy": 100
}
}
Websocket JSON request

21. MAKE YOUR BROWSER TALK
2
1

22. OTHER EXAMPLES OF PUPPETEER
■ Scrap link from website
■ Device emulation - Keyboard API
■ Capture performance data
■ Capture timeline trace
■ Testing Offline
■ Create pdf of the website
■ A/B testing
2
2

23. Can my existing tests suit in
selenium take advantage of
chrome Devtools??
2
3

24. SELENIUM, REMOTE DEBUGGING THE BROWSER
2
4

25. MULTI-CLIENT REMOTE DEBUGGING SUPPORT
2
5

26. ENABLING CHROME DRIVER LOGS
2
6
System.setProperty(ChromeDriverService.CHROME_DRIVER_LOG_P
ROPERTY, System.getProperty("user.dir") +
"/target/chromedriver.log");
System.setProperty(ChromeDriverService.CHROME_DRIVER_EXE_P
ROPERTY, System.getProperty("user.dir") +
"/driver/chromedriver.exe");
ChromeDriverService service = new ChromeDriverService.Builder()
.usingAnyFreePort()
.withVerbose(true)
.build();
service.start();

27. SELENIUM
AND
DEVTOOLS
■ Enable chrome driver logs by setting system
property “ CHROME_DRIVER_LOG_PROPERTY”
■ Launch a chrome browser using
ChromeDriverService.
■ Parse the chromedriver logs to get the port on which
remote debugger is running.
■ Use the debugger port to make websocket socket
connections.
2
7
{ "description": "",
"devtoolsFrontendUrl":
"/devtools/inspector.html?ws=localhost:12326/devtools/page/4f500019-edde-4c1
e-a464-699638eb5fce",
"id": "4f500019-edde-4c1e-a464-699638eb5fce",
"title": "data:,",
"type": "page",
"url": "data:,",
"webSocketDebuggerUrl":
"ws://localhost:12326/devtools/page/4f500019-edde-4c1e-a464-699638eb5fce"}

28. REMOTE DEBUG
RemoteDebug
Is an initiative to bring a common
remote debugging protocol to
today’s browser. The vision is to
enable a new generation of
DevTools and editors that are able
to communicate independently of
their vendors.
- Kenneth Auchenberg
2
8

29. Conclusion
Chrome Debugging Protocol has unlocked
possibility to automate web application
using devtool.
Puppeteer made our life easier by providing
high level API to write highly reliable tests and
perform non functional testing like network
throttling, device emulation, performance tests
and etc.
2
9
“Conclusion is the
place you get tired
of thinking.”

30. REFERENCE
https://github.com/transitive-bullshit/awesome-puppeteer
https://github.com/dhamaniasad/HeadlessBrowsers
https://developers.google.com/web/updates/2017/04/headless-chro
me
https://developers.google.com/web/tools/puppeteer/
https://github.com/checkly/puppeteer-recorder
https://github.com/ShilpaGopal/puppeteer-examples-vodqa
https://github.com/sahajamit/chrome-devtools-webdriver-integratio
n
Try Puppeteer online : https://try-puppeteer.appspot.com/
Puppeteer documentation : https://pptr.dev/
3
0

31. THANK YOU
Shilpa G
shilpag.398ckm@gmail.com
LinkedIn - /gshilpa3
Github - /Shilpagopal