3. How we test Web Apps today
Technical level
Unit tests for separate components
E2E tests that involve backend
Organizational level
E2E Test are created and operated by
a QA team
4. What is the problem ?
Catches defects late
Reporting is hard
Local reproduction is hard
Retesting is slow
Keeping tests consistent is hard
Covering new features is hard
Covering fixes is unlikely
INEFFICIENT
5. How should web apps be tested ?
Move UI tests in the app
development cycle
Ask developers to write them
Execute frequently
Catches regressions very
early
Reporting is automatic
Reproduction is very easy
Tests are consistent by design
Very easy to cover new
features and fixes
6. What is preventing us from doing it ?
UI tests are written in a
different language than than
the web app
UI tests require external tools
and complicated local setup
UI tests are hard to debug
UI tests are unstable, frequent
false positives
7. How to fix it ?
Write tests in JavaScript
Make the UI tests run
completely in browser
Debug tests in the browser
dev tools
Make UI tests inherently
stable by automatic
synchronization with the UI
framework
8. Application Testing with OPA
Testing framework for UI5 based applications
UI5 based
application
Fully browser-based,
NO selenium
HTTP
Server
OPA script
Application testing
Developer centric
Mocked backend
MockServer
9. OPA – launching the test
Bootstrap UI5 and QUnit
Start the execution
Load the test site
Plain HTML
10. OPA – test suite
The test suite loads the
test cases
Global OPA configuration
11. OPA – test case
Loading Page Objects
We use BDD-line syntax
opaTest() is a wrapper
for Qunit.test()
Opa starts and stops the
the application
Test looks synchronous
Semantical actions and
assertions
12. OPA – page objects
Work with controls, not
the DOM
OPA helper for creating
Page Objects
waitFor() is the basic
building block
Report will contain nice
error messages
This is the stable ID from
the XML template
13. OPA – autoWaiter
Wait for all XHR requests to complete
Wait for all promises to be resolved
Wait for all timeouts to be completed
Application pollings are not a problem
Wait for UI5 rendering to be completed
Wait for UI5 NavContainer, OverflowToolbar, etc
specific behavior
15. Limitations
Same domain policy means that both test and
app should be deployed on the same origin, not
suitable for productive systems
In-browser operation mean that authentications
and browser redirects are not possible
OPA is a developer friendly but requires both
JavaScript and UI5 proficiency to use effectively
Web applications are browser-based javascript applications, frequently built with advanced frameworks like UI5, Angular, React.
Web applications bring many benefits to the overall user experience but also provide serious challenges, particularly for automated testing.
This reminder is important as most of the automated tools we use today are designed for automating the testing of web pages and not web applications.
And particularly, the tools are depending on the interactions being mapped to HTTP communication because that is how the web has initially designed.
But the web has changed and this mapping is no longer valid.
The de-factor standard tool for automated browser UI testing is Selenium. There are numerous selenium-based frameworks that strive to optimize some aspect of selenium. There are also commercial tools line QTP.
All those tools have a basic deficiency – they work on a low level and treat the application as a web page. And this is causing their major issues with reliability of the automated tests.
So it is time to evolve the testing tools and start testing the web apps and not the container web page.
Lets look how we test today
Unit tests
Rare
Qunit for UI5, Jasmin for Angular
E2E or integration testing
selenium, sikuli, commercial tools
QA team owns the tests
QA team operates the tests and reports issue
How to have a better quality – usuallty boils down to write more automated tests
Long Feedback Cycle
Process is hard - less likely to be followed under the pressure
Test quality deuterates over time and UI tests become a burden and not an asset.
Shorten the feedback cycle
use application tests as part of test-driven development - the test is a formal specification
write the test together with the code and commit them together
Those problems are so common that many teams consider automated UI test as impractical and avoid them.
So what is left is to live with manual UI regression testing.
Manual testing is NOT feasible for regression testing because it is slow and does not scale - could not be executed after each commit.
Remove the obstacles and provide value to developer
No single open-source UI testing framework so we at SAP UI5 team had to build it ourselves.
Testing framework for UI5 based applications – intended for UI testing of UI5 applications, no application tweaking is required
Fully browser-based, NO selenium – not based on Selenium, does not require any external deployments, only browser and http server to bootstrap the test and the app.
Application testing – Application testing in this context means UI testing of single web application with mocked backend. We provide support for ODATA backend mocking with integrated mocking support – MockServer
Developer centric – OPA is developer centric because uses inside knowledge of the application to speedup the test creation. Simple test execution and debugging.
Reliable by design – Reliability is a challenge, especially for UI tests. Common experience is that UI test are flaky and require special experts or whole teams to write and support them. With OPA, we are challenging the perception. Due to the integration with UI5 core, the OPA test is inherently reliable.
Opa test is just one html page – open it in the browser and execute it.
It will show the progress, you will see the test opening the app and clicking around, at the end you will see the report.
Same applies for automation – use any Qunit runner.
