Automated testing of
Embedded Web Applications
Lauro Moura
lmoura@igalia.com
1 / 28
About me
At Igalia since 2020
WebKit team
QA, WebDriver
Other work done in the past:
EFL (JS and C# bindings)
Maemo (Python bindings, Nix WebKit port)
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
2 / 28
About Igalia
Specialized Open Source consultancy, founded in
2001
Fully remote, HQ in A Coruña (Spain). Flat
structure.
Top contributors to all the main Web Rendering
Engines
WebKit, Chromium, Gecko and Servo
Active contributor to other areas and OSS
projects
V8, SpiderMonkey, JSC, LLVM, Node.js, GStreamer,
Mesa, Linux Kernel...
Members of several working groups:
W3C, WHATWG, WPT, TC39, OpenJS, Test262,
Khronos... Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
3 / 28
Agenda
Embedded Web Apps
Some Challenges in Web Apps
Challenges of Embedded Web Apps
Browser automation overview
Browser automation tools
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
4 / 28
Web Applications
"Write once, run anywhere!"
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
5 / 28
Web Applications
Fast development cycles
Fast deployment
Large ecosystem
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
6 / 28
Web Apps in real life
"Write once, run _mostly_ anywhere!"
"Works better with $BROWSER"
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
7 / 28
Why does this happen?
Engine differences: Blink vs WebKit vs Gecko
Browser diferences: Chrome vs Brave vs Edge, Safari vs GNOME Web
Versioning: Canary, Stable, ESR
Browser engines != Desktop browsers != Mobile browsers
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
8 / 28
It's also about hardware
Desktop? Tablet? Phone?
4k? 1080p? Scaled or native resolution?
What kind of interaction? Mouse? Gestures?
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
9 / 28
Embedded Web Applications
Not just a generic site running on a generic mobile browser
Load specific resources...
(e.g. using a local server)
... from specific browsers
We can expand the browser with new APIs
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
10 / 28
Embedded Web
Applications in real life
But we might have different devices...
...with different hardware capabilities...
(e.g. different GPUs, screens)
Or different browser versions
Thankfully, still less variables
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
11 / 28
Other differences
Generic websites are usually sandboxed
Embedded web apps might need to interact with the system
Testing the web app in a live system is a must
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
12 / 28
But how to test a web app?
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
13 / 28
Automation != Testing
Automation is a tool
Crawling the web
Filing taxes
Running test steps
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
14 / 28
Originally, mimick user actions
Navigate to $URL
Find input for value "name"
Enter text "John Doe"
Find button with label "Confirm"
Click this button
Search for message "User created"
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
15 / 28
Brief history of browser automation
Early 2000's efforts
Selenium + WebDriver = Selenium WebDriver
W3C WebDriver standard in 2018
Supported by major browsers
WebDriver "Classic"
HTTP-based protocol (remote connections)
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
16 / 28
WebDriver Classic, under the hood
Commands
Responds
User Browser
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
17 / 28
Mimicking the user is not enough
Logic moving into the frontend
Asynchronous tasks
Polling is tricky
Developer tools/inspectors evolution
Enabling two-way communication with the browser
Automation tools based on these protocols
Browser-specific solutions?
in active development o/
W3C WebDriver BiDi
WebKit support incoming
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
18 / 28
WebDriver BiDi, under the hood
Commands
Events
User Browser
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
19 / 28
WebDriver Bidi
Based on WebSockets
Extends WebDriver Classic
Still perform user actions
Monitor the browser state without polling
Monitor messages
log
Observe changes
DOM
Intercept and mock requests
network
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
20 / 28
What about Browser
Automation Tools?
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
21 / 28
Selenium
One of the most well known tools around
Deeply influenced the WebDriver standard
JavaScript, Python, Java, Ruby, C#...
Selenium Grid
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
22 / 28
WebDriverIO
Also based on the W3C WebDriver
Written in JavaScript
Built-in support for Cucumber, Mocha and Jasmine test frameworks
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
23 / 28
Playwright
Puppeteer background
Developed by Microsoft
Autowaits feature
Depends on patched Firefox or WebKit, though...
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
24 / 28
Recap
Embedded Web Apps run in controlled scenarios
But testing how it integrates in the system is crucial
WebDriver BiDi features allows better testing of such apps
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
25 / 28
Questions?
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
26 / 28
Thank you!
Automated testing for Web-based embedded systems
Lauro Moura, 2023-04-18
27 / 28
28 / 28

Automated Testing for Web-based Systems on Embedded Devices

  • 1.
    Automated testing of EmbeddedWeb Applications Lauro Moura lmoura@igalia.com 1 / 28
  • 2.
    About me At Igaliasince 2020 WebKit team QA, WebDriver Other work done in the past: EFL (JS and C# bindings) Maemo (Python bindings, Nix WebKit port) Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 2 / 28
  • 3.
    About Igalia Specialized OpenSource consultancy, founded in 2001 Fully remote, HQ in A Coruña (Spain). Flat structure. Top contributors to all the main Web Rendering Engines WebKit, Chromium, Gecko and Servo Active contributor to other areas and OSS projects V8, SpiderMonkey, JSC, LLVM, Node.js, GStreamer, Mesa, Linux Kernel... Members of several working groups: W3C, WHATWG, WPT, TC39, OpenJS, Test262, Khronos... Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 3 / 28
  • 4.
    Agenda Embedded Web Apps SomeChallenges in Web Apps Challenges of Embedded Web Apps Browser automation overview Browser automation tools Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 4 / 28
  • 5.
    Web Applications "Write once,run anywhere!" Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 5 / 28
  • 6.
    Web Applications Fast developmentcycles Fast deployment Large ecosystem Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 6 / 28
  • 7.
    Web Apps inreal life "Write once, run _mostly_ anywhere!" "Works better with $BROWSER" Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 7 / 28
  • 8.
    Why does thishappen? Engine differences: Blink vs WebKit vs Gecko Browser diferences: Chrome vs Brave vs Edge, Safari vs GNOME Web Versioning: Canary, Stable, ESR Browser engines != Desktop browsers != Mobile browsers Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 8 / 28
  • 9.
    It's also abouthardware Desktop? Tablet? Phone? 4k? 1080p? Scaled or native resolution? What kind of interaction? Mouse? Gestures? Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 9 / 28
  • 10.
    Embedded Web Applications Notjust a generic site running on a generic mobile browser Load specific resources... (e.g. using a local server) ... from specific browsers We can expand the browser with new APIs Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 10 / 28
  • 11.
    Embedded Web Applications inreal life But we might have different devices... ...with different hardware capabilities... (e.g. different GPUs, screens) Or different browser versions Thankfully, still less variables Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 11 / 28
  • 12.
    Other differences Generic websitesare usually sandboxed Embedded web apps might need to interact with the system Testing the web app in a live system is a must Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 12 / 28
  • 13.
    But how totest a web app? Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 13 / 28
  • 14.
    Automation != Testing Automationis a tool Crawling the web Filing taxes Running test steps Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 14 / 28
  • 15.
    Originally, mimick useractions Navigate to $URL Find input for value "name" Enter text "John Doe" Find button with label "Confirm" Click this button Search for message "User created" Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 15 / 28
  • 16.
    Brief history ofbrowser automation Early 2000's efforts Selenium + WebDriver = Selenium WebDriver W3C WebDriver standard in 2018 Supported by major browsers WebDriver "Classic" HTTP-based protocol (remote connections) Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 16 / 28
  • 17.
    WebDriver Classic, underthe hood Commands Responds User Browser Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 17 / 28
  • 18.
    Mimicking the useris not enough Logic moving into the frontend Asynchronous tasks Polling is tricky Developer tools/inspectors evolution Enabling two-way communication with the browser Automation tools based on these protocols Browser-specific solutions? in active development o/ W3C WebDriver BiDi WebKit support incoming Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 18 / 28
  • 19.
    WebDriver BiDi, underthe hood Commands Events User Browser Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 19 / 28
  • 20.
    WebDriver Bidi Based onWebSockets Extends WebDriver Classic Still perform user actions Monitor the browser state without polling Monitor messages log Observe changes DOM Intercept and mock requests network Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 20 / 28
  • 21.
    What about Browser AutomationTools? Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 21 / 28
  • 22.
    Selenium One of themost well known tools around Deeply influenced the WebDriver standard JavaScript, Python, Java, Ruby, C#... Selenium Grid Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 22 / 28
  • 23.
    WebDriverIO Also based onthe W3C WebDriver Written in JavaScript Built-in support for Cucumber, Mocha and Jasmine test frameworks Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 23 / 28
  • 24.
    Playwright Puppeteer background Developed byMicrosoft Autowaits feature Depends on patched Firefox or WebKit, though... Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 24 / 28
  • 25.
    Recap Embedded Web Appsrun in controlled scenarios But testing how it integrates in the system is crucial WebDriver BiDi features allows better testing of such apps Automated testing for Web-based embedded systems Lauro Moura, 2023-04-18 25 / 28
  • 26.
    Questions? Automated testing forWeb-based embedded systems Lauro Moura, 2023-04-18 26 / 28
  • 27.
    Thank you! Automated testingfor Web-based embedded systems Lauro Moura, 2023-04-18 27 / 28
  • 28.