Download to read offline
More Related Content
![[WSO2 API Day Chicago 2019] Sustainable Competitive Advantage](https://cdn.slidesharecdn.com/ss_thumbnails/wso2apidaychicago2019-sustainablecompetitiveadvantage-190626104303-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Applying AI to API Testing across the lifecycle - Integration Tuesdays -Final copy (1) (1).PPTX
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
![Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/codedagentsdeck-251215155422-5497c599-thumbnail.jpg?width=640&height=640&fit=bounds)
Applying AI to API Testing across the lifecycle - Integration Tuesdays -Final copy (1) (1).PPTX
- 1. Applying AI/Automation toAPI Testing across the lifecycle — Swetha Sridharan Product Manager, IBM API Connect Krithika Prakash STSM, IBM API Connect and Gateways
- 2. IBM API Connectpowers digital applications by unlocking business data and assets as APIs API Management Socialize Empower application developers to explore and consume your APIs using branded self- service portals Secure Easily apply built-in and extensible policies to secure, control and mediate the delivery of APIs protecting data and business assets Create & Test Automatically create APIs to expose data, microservices, enterprise applications, and SaaS services using open standards Manage Rapidly organize, publish and analyze any API through the full lifecycle from design to retire
- 3.
- 4. 4 To deliver APIs fasterwith high quality Mock API Simulate API responses quickly to speed up the creation Test API Perform unit, functional, integration tests continuously Create API Design, develop and debug APIs with flexible tooling
- 5. Development Test Pre-ProdProduction Testing across Lifecycle Iterative testing Testing guaranteed scenarios as part of CI/CD pipeline Testing to identify gaps
- 6. Development Test Pre-ProdProduction Testing across Lifecycle Iterative testing Testing guaranteed scenarios as part of CI/CD pipeline Testing to identify gaps Auto Test Assist Automated test generation with (add on) Smart Generation AI Insights
- 7. 7 Auto Test Assist •Improves developer productivity by automatically generate, schedule and test APIs during its creation without write test cases • Uses “Biased Random” request generation, strongly biased towards correct requests unlike plain random requests • Send 1000s of requests to auto-test APIs during its creation with no test data needed • Validate and test APIs with just the OpenAPI document (API endpoint) • Comprehensive reports on coverage and easy drill down of results Development
- 8. • Open API documentis used to generate tests • A well-designed API spec is required to get started Example Development
- 9. 9 Automated test generationwith add on Smart Generation • Automated API Test generation from Open API definition or data on wire • Intelligence to generate assertions for headers and payload for validating API behavior • Feature rich visual test editor • Seamlessly integrate with DevOps pipeline through admin REST API • Insights into test runs and API quality Test/Pre-Prod
- 10. 10 Telemetry Production workload Test workload Telemetry Coverage Data Production API TestAPI API Test Generation AI Challenges (simplified): • Understanding relationships (e.g. Create orderline requires an order) • Condensing production scenarios into ‘gaps’ (clustering) • Generating valid synthetic test workload (prod data is sensitive) New Tests - Test 1 - Test 2 - Test … AI Insights Driven test generation • Identify gaps in test case coverage by finding unique patterns that are not already covered in existing API test cases • Save developer time by automatically generating comprehensive test cases for discovered scenarios • Recommend new tests comparing production trace data Production
- 11. Test without writing testcases during development using Auto Test Assist Automatically generate test cases for long term testing needs of API lifecycle Integrate tests into the DevOps process Use production data to continuously learn and improve coverage Applying testing across lifecycle Auto Test Assist Smart generation of test cases AI Insights Development Test Pre-Prod Production
- 12. API Management withchoice and flexibility IBM API Connect SaaS Consume as a service with multiple options for isolation and availability IBM API Connect Software Deploy on-prem or any cloud, bring your own OpenShift, other K8s platforms or VMware IBM Cloud Pak for Integration Consume API Management integration capabilities in a unified integration platform, on OpenShift anywhere
- 13. IBM Cloud Pak forIntegration IBM API Connect Software IBM API Connect SaaS on AWS (multi-tenant on AWS) IBM API Connect Reserved Instance (single tenant on IBM Cloud) Auto Test Assist Smart Generation Automated Test Generation(non-AI) AI Insights What’s in where?
- 14. Resources on theWeb API Connect Sales Kit: ibm.biz/apicsk→ White Papers: Can you trust your APIs?→ Random test generation→ External: IBM API Connect product page → Knowledge Center → Blog - API test creation using Smart generation→ Blog - Auto Test Assist→ Blog - Automated test generation→ Additional learning: – Free Trial on API Connect SaaS – Email us at: askapi@us.ibm.com People Resources Tom Campbell WW Technical Sales Leader campbeth@us.ibm.com Sales Mike McGee mmcgee@us.ibm.com Steve Cawn scawn@us.ibm.com Product Management Michael Thompson mcthomps@us.ibm.com Swetha Sridharan swetha.sridharan@ibm.com Dan Temkin dtemkin@us.ibm.com Parisa Lotfi parisal@ibm.com Sonia Mehta sonia.mehta@ibm.com David Pooley Ecosystem: Integration Sales dpooley@ibm.com Shruthi Savla Services venkatas@us.ibm.com Cathy Daly Technology Sales Enablement cdaly@us.ibm.com Sangeeth Varadaraj Pricing sangeeth.varadaraj@ibm.com
- 15.
- 16.
Editor's Notes
- #4 Now, before we talk about all fun stuff, I want to sent the context as to why this topic is important and relevant. With enterprises adopting digital transformation at a pace that we have never seen before, enterprises are creating business applications to stay ahead of competition. These applications are nothing microservices defined through set of APIS. They are creating business value and new models by adding microservices/APIs from third party vendors making it even more important that they are reliable and trustworthy.
- #5 A development flow starts with mocking to simulate API responses to speed up the creation, followed by design and development of APIs. Now comes the least favorite task for any developer – it’s testing the code/API. Having written the code, a developer knows what works and what doesn’t. So, a developer is mildly motivated to test the business use-cases and different scenarios APIs would interact.
- #8 With Auto test assist, you can test APIs without having to write test cases. It will analyze your API, then automatically generate random requests using biased random model and reports about any conformance errors and server-side errors that it observes. This greatly improves the developer productivity as it does not require any test cases and increases the accuracy by testing for corner cases automatically. It starts by creating auto test profile that spells out how the Auto Test Assist should run, what areas to test, endpoints to test etc. It then analyzes the API document to derive the dependencies between the resources before starting to generate the test requests. The results are displayed in the reports tab and can be drilled down further as needed. This is particularly useful if the API is being developed as testing to identity errors happens without having to write test cases. This can also be used for doing regression testing.
- #9 All you need is an API specification that is defined according to the best practices with clearly marked parameters and well thought operatio s and resources with dependency maintained. Here you can see that the how well a customer detail resource has been definied in the spec that’s been used by auto test assist to generate the calls while testing
- #10 With automated test generation, you can automatically generate test cases with a single click using the API contract or with data on wire. It generates test assertions based on the semantics of the API to make sure the API behaves the way it should. It comes with a visual editor making it easy not just for the developers but also your QA team to quickly modify test cases. The biggest advantage is these test cases can be fired from your CI/CD pipeline. Every time a developer checks in code, you can automatically run the test cases against the API. The results are displayed in the reports for easy troubleshooting. We have made this This comes with an add on Smart Generation. It does the exact same thing as the automated test generation but the difference being this is smart enough to create assertions for multi-step API calls including deletion to ensure data is kept clean. This uses a technology Open API Analyser that extracts the resources and their dependencies from the Open API specification. Then, uses natural language processing to determine if two operations are dependent on each other to create a sequence of test assertions where output from one operation is provided as input to another. This eliminates the need to manually sequence API calls to already generated test cases.
- #11 The last feature we have here is the AI Insights driven test generation to identify gaps in the test coverage and recommend new tests. As APIs are being consumed in the real world, we discover some API interactions (with various backend systems) that are untested. In the recent times, we have seen how Retail has responded to the changing user needs. An API call that makes an order might result in a delivery microservice based on the user type. A premium user gets an expedited shipping while a standard user gets a 5 – 7 business day shipping. In addition to that, a user can choose in-store pickup irrespective of these choices. API calls we observe from production workload are a good starting point to identify these unique patterns that are not covered in the existing test cases and recommends new tests with least manual intervention.
- #12 Now, aligning these features back to the lifecycle, AutoTest Assist is the starting point to identify errors early in the development phase without having to write any testcases. As APIs get promoted to different environments, auto test generation can be used to test for guaranteed, repeatable scenario and as part of your CI/CD pipeline. As things mature and APIs move to production, AI Insights can be used to detect gaps in the test coverage by comparing the production trace patterns with the existing test cases.
- #15 This chart provides a starter list of web sites and contacts. Build your network by adding geography contacts and other websites of interest.