Downloaded 14 times
![20260201 [FOSDEM] gomodjail - library sandboxing for Go modules.pdf](https://cdn.slidesharecdn.com/ss_thumbnails/20260201fosdemgomodjail-librarysandboxingforgomodules-260201225659-76609ec4-thumbnail.jpg?width=640&height=640&fit=bounds)
E2E testing with UIVeri5
- 1. Maxim Naidenov, TsvetelinaAleksandrova, SAP UI5 development June 28, 2019 E2E Testing with UIVeri5
- 2. 2 Testing Pyramid Manual Tests E2E Tests ApplicationTests OPA5 UIVeri5 Exploratory Testing Cost Speed
- 3. 3 UIVeri5 Introduction UIVeri5 isan E2E testing framework for UI5- based applications. Device Test UIVeri5 Execution environment Uses selenium to drive a real browser and interacts with your application as a real user would. Heavily inspired by Protractor and brings most of its benefits to UI5 applications. Unique benefits for UI5 applications
- 4. 4 But we haveOPA5 OPA5 is an in-browser testing framework. Both the test and the application reside in the same browser. OPA Tests UI5 Application MockServer npm serve No need for external tools or setup. Extremely fast Same origin for the test and the application. No authentication handling.
- 5. 5 But we haveSelenium Selenium is an established and widely used technology. Hard to write reliable tests for SPAs. Unreliable test is not an asset but a liability. DOM-level selectors but DOM is not UI5 API. Typically uses java and tests are hard to read for JS developers.
- 6. 6 Tests arewritten in JavaScript, same as the application. Tests use the the popular Jasmine syntax. Tests looks and feels synchronous. Unique integration with UI5. UIVeri5 Basics
- 7. 7 Advantages - Synchronization It is easy to create highly reliable tests because the test is automatically synchronized with the app. Synchronization logic from OPA5. The test is reliable and works as fast as the app can handle interactions.
- 8. 8 Test workson the same level of abstraction as the application. Test effectively uses the control API that is guaranteed to stay compatible. Building the locators is much simper and hence productive. Tool support for inspecting controls is available in UI5 Inspector. Advantages – Control Locators
- 9. 9 Supports mostcommon authentication scenarios with declarative configuration. Reliable, configurable authentication of all common flows. Extensible for advanced E2E use- cases. Advantages - Authentication
- 10.
- 11. 11 Support forPage Objects similar to OPA. UIVeri5 Summary HTML report with screenshots of every step. Automatic chromedriver download. UIVeri5 is open-source, fork it and extend for your specific needs.
- 12.
Editor's Notes
- #2 Hi everyone. My name is Maxim Naidenov and here with me is Tsvetelina Aleksandrova. We are part of UI5 development and we are developing testing tools and technologies for UI5. Today we want to introduce our new testing tool – uiveri5. The name stans for UI5 application verification but also nicely spells as console command so we stick with it.
- #3 Lets first see how a new testing tool fits into the overall picture. Here we have the agile testing pyramid, adapted for a typical UI5 application. On the lowest level we usually have unit tests. The typical ui5 app doesn’t have so much generic logic that could be tested with a classical unit tests. But for a UI app, the units are the individual screens or views. So on this level we would like to test individual screens of the application in mocked environment so we could explore maximum usecases of interaction with this screen. For this level we have the OPA5 framework. Most likely you have heard it or even used it as it is available for the last few years. Having most of testing on this level have a lot of benefits but in it is usually no enough to guarantee the overall product quality. On the middle level we have integration tests that exercise the UI, together with a backend. That’s why they are frequently called E2E as you test the whole system. This level is typically dominated by Selenium and for years there was no real alternative. But now we introduce UIVeri5 as a tool better suited for automating ui5 applications. On the top level we usually have manual tests. In theory we would strive to have minimum manual tests as they are executed by humans and so have all the typical drawbacks of manual labor – slow, expensive, can’t easily scale. But in practice we very frequently see projects that depend primary on manual testing and sometimes this is caused by technical limitations of the underlaying layer and particularly the reliability of the automated tests. Of course, it should be noted that manual tests also have the major benefit of manual labor and this is the creativity and that’s why we frequently refer to them as exploratory testing. So maybe it is not a good idea to eliminate the exploratory testing completely but we definitely must have the majority of testing done on the underlying levels.
- #4 So what is UIVeri5. This is an e2e testing framework for UI5 based applications. Using selenkum, we are able to automate a real browser and interact with the application as a real user would do. This includes mobile browsers. UIVeri5 is heavily inspired by Protractor. You may heard it, Protractor is an e2e testing tool for angular applications and it introduced some excellent ideas for creating good E2E tests. We adopted most of those ideas and added some more to provide unique benefits for UI5 applications.
- #5 But before looking in those unique benefits, lets discuss a question that maybe you already have in your mind. If we have OPA and we could use it to create reliable UI tests in javascripts, why do we need another framework ?. OPA is an in-browser testing framework, in nutshell you have the test encapsulated in an html file, OPA uses Qunit as a test runner. The OPA test opens the application in an iframe and simulates user interactions. This is great as it has some uniques benefits. First is that you don’t need any external tools or setup. You use the same local server that serves the application, you load the test html and it runs. And it runs extremely fast as you don’t have anything else in between the test and the app. But the in-browser execution also poses two major limitations that prevents OPA usage for E2E scenarios. First, both the application and the tests should share a common origin e.g., the test should reside on the same server. This convenient for development but frequently is not acceptable with productive applications. Second, again because OPA test is exеcuted in the same browser as the application, the test can’t handle authentications. And in practice, even inside the corporate network, most test systems are protected with some authentications.
- #6 You might also wander why do we need one more Selenium framework. Selenium is an established and widely used technology. Till recently we also suggested the use of Selenium for E2E tests in UI5 Documentation. But selenium has some drawbacks and the major one is that it is hard to write reliable test for single-page applications. It is certainly possible to write reliable UI tests but this requires significant efforts and expertise by the test developer. And it is very easy to create a test that works most of the time but fails randomly, for example when the system is loaded. The second major problem with selenium for UI5 apps are the selectors. Selenium uses DOM-level selectors like id,css,xpath and this is fine for manually created web pages. But with UI5, you have a control tree that is rendered to DOM. So the DOM is a result of the rendring and not the source of truth. Using dom-level selectors is generally hard and it is very easy to create a fragile test that breaks with minor changes in the app or in UI5. And one common problem is that selenium is typically considered as java technology and most selenium tests are actually written in java. And even more, as both java and selenium are established technologies, it is typical to overuse design patterns and this makes the tests hardly readable for Javascript developers. And the effect is that typically it is a separate team that writes and supports the UI tests, disconnected by the application development. And this common setup makes it impossible to include the e2e test in an agile development methodologies like test driven development.
- #7 So lets now look at UIVeri5 and see why it is a better choice for E2E tests for UI5 application. The first advantage of UIVeri5 is that tests are written in JavaScript. And it is not particularly the language but the fact that if the tests are in the same language, it is much simpler for the application developer to create and maintain them. It is much easier to keep the tests consistent with the code and this greatly benefits the quality of the application. With good tests that developers understand it is much easier to establish modern development practices like Test Driven Development or Continuous Delivery. The second advantage of UIVeri5 is that it uses the jasmine syntax. Jasmine is a simple, modern syntaxis with only 3 keywords. Next, tests with UIVeri5 looks synchronous. I use the word “look” because javascript is inherently asynchronous and in nodejs communication with the browser is asynchronous. But this async behavior is hidden behind an execution flow abstraction that executes them sequentially. This is a very similar to the way jasmine or qunit async tests work but in UIVewri5 we have a second level so the browser operations are also synchronized. The effect is that it is very convenient to comprehend the test scenario. And last, as uiveri5 is specially intended for UI5 applications, we have some unique integrations between the test framework and the UI5 runtime that give some major advantages. So lets have a closed look at those integrations.
- #8 The most important advantage of UIVeri5 is that it is easy to create highly reliable tests. And the reason is that the test framework synchronizes the test execution with ui5 rendering. In practice this means that you compose the tests of only action and assertion statements but the statements are executed only when the UI5 rendering is complete and the UI is stale. Imagine a typical scenario of a button click that starts a backend call that returns and triggers a data binding to update the UI. The test can reliably assert the state of the UI only when all those steps are complete. But there are many factors like the network speed, the backend load, the browser rendering speed that would influence the duration of this operation. The typical e2e test waits for a fixed amount of time but this is prone to reliability issues. A better approach is to wait for some specific element to appear. But then you need explicit wait statements and synchronization code could easily become overwhelming part of the test. And even then, there are many cases in which, due to the dynamic UI generation it is hard to compose a reliable wait condition. So we implemented an automatic synchronization. Whenever the test tries to assert a value, we explicitly ask UI5 runtime if the UI is stable and wait till all rendering is complete Detecting the rendering completion is a non-trival task we already developed reliable algorithms for this in the context of OPA5 testing framework. It actually works by tracking all the ways to defer a job in browser. We keep track of XHR calls, timeouts and native promises and once all they are complete, we could say rendering is complete and UI is stale. There are further aspects to make this basic idea work in practice – we have special handling for UIs doing polling of backend and for animations being part of a navigation sequence. This automatic synchronization merges nicely in the execution flow of the browser interactions. The net effect is that while writing the test, you don’t need to think of synchronization, you just describe the scenario in sequential and synchronous way and the test just works reliably. The test will work as fast as the backend could respond and the browser is able to render the UI changes. This makes the test simple to write and simple to maintain and most importantly, it is reliable by design.
- #9 The second major advantage of UIVeri5 are the control locators. Control locators allow selecting elements on their control properties and not with their DOM attributes. And because UI5 applications are ultimately composed of controls, this has the benefit that the test is much nearer to the application so it is simpler to create and maintain it. You can use the view names and stable ID given by the developer in the template. Or you can use unique property or relation of the control to locate it reliably. We also have advanced locators like locate a control with specific label or with specific binding path. Another advantage of control locators is that using them, you comply with the UI5 compatibility promise that covers control API but now generated DOM or CSS. This makes it possible to create stable tests that don’t need adaptations when you upgrade UI5 minor versions. Building control locators is much more productive when you have access to the application sources but even without using the sources, it is easy to derive them with the help of UI5 Inspector. If you remember, this is our chrome extension that could visualize the the tree of controls and inspect their properties.
- #10 The third major advantage is the handling of authentication. By definition, E2E tests are supposed to test an application working against a real backend. In nearly all such scenarios, the applications are configured to require authentication and respectively the test should authenticate before proceeding with the scenario. With todays cloud-based landscapes it is common to use advanced authentication flows like OAuth2 or delegation flows. Driving the browser to complete such flow is a tedious and error prone task and adapting to different setups of same application on different landscapes is even more tedious. To simplify such scenarios, UIVeri5 supports a declarative authentication that handles the most common authentication flows for Fiori-based applications reliably. You only need to declare the authentication type and provide the credentials and the framework with execute the authentication before starting your test. You can also customize the authentication configuration to accommodate custom login form or to define your own that extends the predefined ones. We also support advanced authentication scenarios like programmatic login, browser redirects and page refresh that allow an E2E test that covers extensive scenario that spans several applications.
- #11 And to see uiveri5 in action, Tsvetelina will give a short demo how easy is to create an e2e test.
- #12 Now you had a glimpse how simple is to create a reliable E2E test for UI5 application. But there are a lot more features in uiveri5. We have support for Page Objects, similar to OPA. PO are a great way to structure your real-life tests and further builds on the idea to make the test simple to comprehend and maintain. We have nice html report with screenshots of every step of the test. It is valuable tool to make a fast analisys what is the rootcause of the failure. We have automatic chromedriver download. Typically the webdrivers should be downloaded and kept uptodate manually on every machine where e2e tests would be developed or executed. But this is tedious, especially with browsers that autoupdate frequently and webdrivers compatible with single version of the browser like chrome and chromedriver. So we have a mechanism to automatically download the correct version of chromedriver or the latest version of the geckodriver for firefox. And finally, UIVeri5 is open-source so you can fork and extend it for your specific needs. Please consider contributing them if you feel they could be of interest to the community.
- #13 Thank you for your attention. I hope you will enjoy UIVeri5 as much as we enjoy working on it. We will be glad to hear your opinion or ideas about UI5 Testing tools, you can meet us at the experts corner.