Pre-Con Ed: Who Moved My UI? API Testing in a Headless World.
•
0 likes•545 views
Pre-Con Ed: Who Moved My UI? API Testing in a Headless World.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (11)
Similar to Pre-Con Ed: Who Moved My UI? API Testing in a Headless World.
Similar to Pre-Con Ed: Who Moved My UI? API Testing in a Headless World. (20)
More from CA Technologies
More from CA Technologies (20)
Recently uploaded
Recently uploaded (20)
Pre-Con Ed: Who Moved My UI? API Testing in a Headless World.
- 1. World® ’16 Pre-Con: Who Moved My UI? API Testing in a Headless World Ulrich Vogt – Sr Principal Eng. Services Architect - CA Technologies DO5X08E DEVOPS Heather Peyton - Principal Marketing Manager - CA Technologies
- 2. 2 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD For Informational Purposes Only Terms of this Presentation © 2016 CA. All rights reserved. All trademarks referenced herein belong to their respective companies. The presentation provided at CA World 2016 is intended for information purposes only and does not form any type of warranty. Some of the specific slides with customer references relate to customer's specific use and experience of CA products and solutions so actual results may vary. Certain information in this presentation may outline CA’s general product direction. This presentation shall not serve to (i) affect the rights and/or obligations of CA or its licensees under any existing or future license agreement or services agreement relating to any CA software product; or (ii) amend any product documentation or specifications for any CA software product. This presentation is based on current information and resource allocations as of November 1, 2016, and is subject to change or withdrawal by CA at any time without notice. The development, release and timing of any features or functionality described in this presentation remain at CA’s sole discretion. Notwithstanding anything in this presentation to the contrary, upon the general availability of any future CA product release referenced in this presentation, CA may make such release available to new licensees in the form of a regularly scheduled major product release. Such release may be made available to licensees of the product who are active subscribers to CA maintenance and support, on a when and if- available basis. The information in this presentation is not deemed to be incorporated into any contract.
- 7. 7 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD API Testing – Value-Driven? APIs now serve as the primary interface for application logic1 If an API breaks, it can break a whole end to end value chain, not just an app! 1 - https://en.wikipedia.org/wiki/API_testing
- 8. 8 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Continuous Testing How do You Get There? Testing Technology Adoption and Maturity Optimized Organization for DevOps *Application release automation view AGILE MATURITY Silos, manual handovers, waterfall, one release/year, monolithic apps, long term project/resource planning, error prone release processes DevOps, end-to-end delivery, incremental agile/sprint release cadence, multi- component micro apps, automation enabled, the right people with the right skills. Where are you? UI Test Automation Modeling & Test Generation Automation API Test Automation & Service Virtualization Continuous Testing
- 9. 9 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Service Virtualization Application Test DevTest Portal DevTest Workstation Application Test DevTest Portal DevTest Workstation + Additional Run Time and AppTest Licenses Sold Service Virtualization Application Test DevTest Portal DevTest Workstation CAI – Auto Generate Virtual Services CAI – Find APIs, Generate API Tests CAI Application Test DevTest Portal DevTest Workstation CA Application Test + CAI – Find APIs, Generate API Tests Service Virtualization CA Service Virtualization Power User + Additional Run Time and AppTest Licenses Sold DevTest Portal DevTest Workstation App Test + CAI – Find APIs, Generate API Tests CA SV Power User + CAI + CAI – Auto Generate Virtual Services CA Application Test CA Service Virtualization Power User DevTest 9.5 DevTest 10.0
- 12. 12 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Changes to Web Sites – Static to Dynamic § Early 2000’s HTTP post to HTML response – <Form Action=mypage.jsp> – POST or GET name=value pairs – Simple formatting of HTML response § Mid 2005’s AJAX based web pages – SOAP or XML data requested – There is now a “application with bugs” in the browser parsing the data and presenting data § Enter 2010’s HTML 5 applications – RESTful request for data – Response XML or JSON – Browser code is still a “application with bugs” Static Dynamic
- 13. 13 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Why Create Tests that Follow the User Journey? § User Interface (UI) testing follows the movement of a user through the application. § The test data for a UI test starts with input and then “builds” with more user data or system returned data. § Data presented on each screen adding new data creates the journey
- 14. 14 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD What is the Generated Data and How it is Handled? § Input data on first page becomes a dataset § Create properties bases on “value” for sharing § Data returned to the page via API is potential data into next step Show me cars Show me mileage Acura Buick … VIN Nbr Acura Get a car loan Terms VIN Nbr Acura
- 16. 16 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Key Considerations When Looking at the Layer § What business rules and validation is at the Mobile layer? § Are the API/Service end points changing data? § ESB all the data on the wire? § Backend systems – have they changed? ESB
- 17. 17 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Considerations for API and Service Delivery Layers § Where’s the UI? – SOAP XML, REST JSON – HTTP, TP & Message Queue’s § Testing orchestration of multiple service calls to accomplish a requirement § API calls are less brittle than UI testing § Nonfunctional load testing to understand composite response times ESB
- 18. 18 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD How do I Find the API’s and the Ordering for Generation? § Use the CA Application Test (CAI) Chrome plugin to see API’s – ‘Developer tool’ in Chrome – Plugin lives in browser and see’s requests and response data – View before encryption and signatures – Works with any backend server, not only java based Acura Buick /get/cars TL RTX /get/cars/model/Acura /get/cars/Vin/ABC123DEF Miles Not salvage Color +Install Plugin CA Continuous Application Insight Server Generate CA Application Tests Transactions
- 19. 19 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD How do I Find the API’s and the Ordering for Generation? § Start Document Transactions – Limit scope of transactions by IP address – Associate transactions with start and stop time – Works for any client applications – Requires java agent in backend 10:25 am 192.168.168.44 “Acura” “Buick” GET /get/cars “TL” “RTX” GET /get/cars/model/Acura GET /get/cars/Vin/ABC123DEF “Miles” “Not salvage” “Color” 10:25 am 192.168.168.44 J2EE DevTest Agent
- 22. 22 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Extending and Embedding Test Cases Automatically § Need a standard prefix to your test for authentication? § Want to validate the database after executing a test step that updates the database? § Want to run a test with additional data?
- 28. 28 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Recommended Sessions SESSION # TITLE DATE/TIME DO5X09E So you implemented a service buss? How ya' gonna test that monster? 11/15/2016 at 10:45 am Lagoon K DO5X10E Sharing the response time? No thank you. I’ll have another piece of pie! 11/15/2016 at 1:00 pm Lagoon K DO5T12T Integrated Testing from A to Z (SV/Test) 11/16/2016 at 11:30 am DevOps Theater
- 29. 29 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Must See Demos Modernize App Delivery Integrated CD Theater 5 - DOV501P Deliver Test Data Faster Test Data Manager Theater 5 - DOV511P Deliver Better Apps Service Virtualization Theater 5 - DOV507P Orchestrate Your Release Release Automation Theater 5 - DOV513P
- 31. 31 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Baseline Test Creation Using CA Continuous Application Insight – On Youtube § Environment Introduction: Baseline Test Case Automation with CA CAI 8 - Part 1 - https://www.youtube.com/watch?v=juH9yVJ9fVY § Filtering Transactions and Creation Options: Baseline Test Case Automation with CA CAI 8 - Part 2 – https://www.youtube.com/watch?v=i6_bLJeGmKQ § Create a Baseline: Baseline Test Case Automation with CA CAI 8 - Part 3 - https://www.youtube.com/watch?v=84vcPiMvfPY § Review a Baseline: Baseline Test Case Automation with CA CAI 8 - Part 4 - https://www.youtube.com/watch?v=BAetSXMfpxA § Advanced Baseline Review: Baseline Test Case Automation with CA CAI 8 - Part 5 - https://www.youtube.com/watch?v=vycy3hzLX5E § Execute Expanded Baselines: Baseline Test Case Automation with CA CAI 8 - Part 6 - https://www.youtube.com/watch?v=2UOCh-yjAc0 § Execute Data-Driven Baselines: Baseline Test Case Automation with CA CAI 8 - Part 7 - https://www.youtube.com/watch?v=jyJD4mUDafg
- 32. 32 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Thank you. Stay connected at communities.ca.com
- 33. @CAWORLD #CAWORLD © 2016 CA. All RIGHTS RESERVED.33 @CAWORLD #CAWORLD DevOps – Continuous Delivery For more information on DevOps – Continuous Delivery, please visit: http://cainc.to/PiTFpu
Editor's Notes
- As you build out your presentation, here are a few dos and don’ts.
- Regression tests are used to verify that the system under test behaves as expected, assuring that changes at the backend do not have a negative impact to the frontend. Usually, a regression test proves things that you already know are working. Even nowadays many customers and QA teams do regression tests manually. Some create the tests scripts manually and run them in some automated fashion. Others maintain their test cases in a spreadsheet an execute them manually by clicking through the UIs. Both require engineers with in-depth knowledge of the complex application. These are all time consuming tasks and if you want to execute a couple of hundred test cases each time a piece of code has changed on a two-weeks release cycle a lot of time is wasted on those manual tasks just to verify that it is working as expected. In this session we want to demonstrate how simple and easy it is with DevTest to create test cases for regression tests automatically saving an enormous amount of time to create or execute those test cases manually. At first we will look into what needs to be tested. Then we will identify the internal and external APIs that can be tested. After introducing the two basic approaches in DevTest to create test cases we will demonstrate how to do this.
- Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate. The key word is “automated”. Automation is the key. Every organization is in a different place when it comes to automation. Automation requires new skill sets and can also create new problems. When you shift skill sets from knowing a process to a tool new checks/reviews need to be put in place to ensure the “tool” is working correctly. Orgs early in the process with large manual testing teams usually focus on IT Test Automation. Even the best automation frameworks tend to bring you back to manual test creation, either in the form of script creation or keyword selection. The time spent converting test cases to automated tests can often then outweigh the time saved executing them and maintenance can create a further bottleneck. With optimized test cases and the ability to auto execute more tests in shorter periods the problem then become availability – how then are you hitting all the systems you need to run you tests.. are they available? Do they behave like the real systems? Can you vary/control run conditions? . This is where Service Virtualization comes into play. When the system or requirements change, brittle automated tests must be updated, or otherwise you risk automated test failures and wasteful over-testing. This time spent identifying the impact of a change on tests and updating them can have a huge impact on speed and quality. Service Virtualization creates life like copies of APIs, services and systems that can be used for testing and development without the need to access production systems, purchase hardware/software for test environments or pay 3rp party access services. Without continuous access to the systems and services you need to test its impossible to continuously test.
- In its early days, web sites and pages were rather static, meant to read information, jump into detailed explanation back and forth. Today, web applications are dynamic in the sense that you can interact with the web pages, they are more complex, they keep track what you have read and done. Collected data are used to create user profiles, create recommendations and maintain shopping carts.
- How do we test these complex applications? Starting from an initial web page the web application collects more and more data during a user’s journey through web site. This information is used to provide more complex services: on a web shop you start looking for type of shoes, narrowing down on size and color, selecting a pair or two to your shopping cart and move on to the purchase and shipping process. How do we test complex applications, where data returned from from one action on a web page is used in a subsequent step? Data is built up along the way by watching the user journey. Becomes the input for the next page. Data needs to be correlated to run with the correct API and back-end supporting systems.
- Creating a test case for this kind of application, we have to be aware of where the data come from, how are they generated, what is happening between two steps, and how are they carried over from one step to the other. Automating a test case for automatic regression tests means that we need to be able to account for where we make the changes, when we have to carry over data from one step to another, and how we are going to use them. The input on first web page becomes the starting dataset Data on return can become the input for next step Data can be shared between steps
- In a complex web application multiple components interact with each other utilizing their APIs. A common architecture has a UI layer implemented as apps in mobile devices or as rich browser clients. They usually talk to a second tier web services using SOAP or REST APIs.
- The second tier web services implement the business logic and validation and often access backend systems by an Enterprise Service bus to retrieve data records. ESB testing that the messages are being sent to the right place. Example – Ring the doorbell does the garage door open..
- Regression tests apply to the second tier level. The client applications at the UI level invoke APIs of the Services on the second level, and the testing the orchestration of API calls of the Service Level. Testing the APIs instead of the UI bears two advantages: API tests are less brittle than UI tests, and API tests can easily be used for functional tests and load performance tests. Using our SV tools we are able to record the user’s journey through an application by several means. We can hook between client and server application and record the traffic exchanged between those. We can also deploy our DevTest agent into the Java application environments and capture calls to the various Java APIs. Testing the orchestration of multiple services at this level. Testing at the API layer is less brittle than testing at the API. You can vary easily do non-functional and load testing to measure response time.
- When we hook our tool between the client and the server application we record the user’s journey through the application, and it is easily identified by the order of the API calls. You can download the plugin using the sandbox in DevTest portal. It
- If using the DevTest agent, the DevTest agent resides on the server side. Sitting there it may see API calls from many different users. So we have to scope down the traffic seen by the agent in order to identify a user’s journey through an application. The tool filters the traffic for the IP address of the requests and associates recorded transactions with start and end time to be able to determine the order of the transactions.
- This demo shows usage of the CAI Chrome plugin. Following the user‘s journey through the application the Chrome plugin records transactions initiated from client to the service. Once completed, the user can select the transactions to create a functional test by putting the selected transaction on a shelf. From there functions are available to create the functional test and – optionally- to generate a report about the shelved transactions.
- Similarly, a functional test following the user‘s journey through an application can be recorded by the feature of Deocument Transactions. Different to the Chrome plugin this approach requires the deployment of an agent to the application. The agent captures web traffic similarly, but in addition to the Chrome plugin, it can also capture internal APIs. These internal data grant an insight into the strcuture of the application and the user gains important insights into the appliaction‘s architecture.
- With recordings from Document Transactions the user has even more options to create tests automatically. If you need to log in into your application before running a test, you can create a test template that does the authentication and add the actual test to the template after the authentication steps. If your test modifies a database by adding, updating or deleting, you can add test steps automatically to verify if those changes were actually applied to the database.
- Auto generate test cases at full velocity Discover APIs automatically Auto generate baseline and directly invoke & verify ESB responses Invoke and verify across all layers with a single test Stabilize environment with service virtualization Multi consumer virtual service generation Subset/mask and/or create/reserve test data