Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Applitools
•
4 likes•1,851 views
The document discusses AI tools for software testing such as ChatGPT, Github Copilot, and Applitools Visual AI. It provides an overview of each tool and how they can help with testing tasks like test automation, debugging, and handling dynamic content. The document also covers potential challenges with AI like data privacy issues and tools having superficial knowledge. It emphasizes that AI should be used as an assistance to humans rather than replacing them and that finding the right balance and application of tools is important.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Applitools
Similar to Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Applitools (20)
More from Applitools
More from Applitools (20)
Recently uploaded
Recently uploaded (20)
Unlocking the Power of ChatGPT and AI in Testing - NextSteps, presented by Applitools
- 2. ABOUT ME Software Quality Evangelist BagmarAnand linkedin.com/in/anandbagmar/
- 3. AGENDA • Recap • What’s changed? • Test Automation & its relationship with Flaky Tests • AI to the rescue – ChatGPT, Github CoPilot, Self-healing Execution Cloud • Pitfalls of AI • Summary & Next Steps • Q&A
- 4. ChatGPT in Testing Example: Testing an eCommerce application – Amazon (USA) http://applitools.info/8eh
- 5. Testing Approach • Getting started • Framework • Design patterns • Code optimization • Reporting • Metrics • Test scenarios • Test cases • Test data • Risk • Coverage • Metrics Strategy Test Design Automation Test execution
- 6. Testing - Automation • Getting started • The 1st Test • Design patterns • Reporting • Metrics • Test scenarios • Test cases • Test data • Risk • Coverage • Metrics Strategy Test Design Automation Test execution
- 7. Automation – Strategy Next question: What is a good way to automate the above test cases?
- 8. Automation – The 1st Test Next question: Implement the 1st test case using Selenium Java with Junit
- 9. Automation – With WebDriverManager Next question: Use WebDriverManager in the above example
- 10. Automation – Design Patterns Next question: Implement the above test using Page- Object-Model design pattern
- 11. Testing – Test Execution • Getting started • The 1st Test • Design patterns • Reporting • Metrics • Test scenarios • Test cases • Test data • Risk • Coverage • Metrics Strategy Test Design Automation Test execution
- 12. Test Execution – Reports Next question: How to generate reports and metrics for automated tests? This is not what I was looking for!
- 16. Writing code Question asked: Implement code for count-and-say https://leetcode.com/problems/count-and-say/
- 17. Writing code Question asked: Implement code for count-and-say https://leetcode.com/problems/count-and-say/
- 18. Writing code Follow-up asked: Give above solution in Java This actually works J
- 20. Refactor code Question asked: Given a link to code in Github, refactor it I have not verified the validity of this refactoring
- 21. Refactor code Question asked: Given a snippet of code, refactor it Large output pauses abruptly ”continue” proceeds
- 22. Complex code Question asked: Given a complex code, asked ChatGPT to refactor the code, and then reduce the complexity
- 23. Complex code Question asked: Given a complex code, asked ChatGPT to refactor the code, and then reduce the complexity
- 25. Debugging Question asked: Given a buggy code, fix it
- 26. Debugging Question asked: Given a buggy code, fix it Not only fixed the code, ChatGPT also explained the error
- 27. Applitools Visual AI Applitools Ultrafast Cloud applitools.info/b8h
- 28. Visual AI simplifies functional testing
- 29. Visual AI simplifies functional testing
- 30. Visual AI handles dynamic content
- 31. Visual AI handles dynamic content
- 32. Visual AI accelerates cross browser/device testing Capture the entire UI with a single snapshot DOM & CSS sent to Applitools Ultrafast Test Cloud Applitools Eyes Visual AI analysis Ultrafast Grid renders DOM snapshots in parallel Login to Eyes to view test results . Firewall 18.2x faster than traditional cloud execution grids. Applitools Ultrafast Test Cloud
- 33. Take-aways from the earlier webinar: • ChatGPT and many other tools in the AI-space are solving very interesting challenges • Understand how to use them in your context • Ask the right question to get a good answer! • Tools have limitations – choose them wisely • Human mind & experience are “still” the essential ingredients in SDLC
- 34. What has changed in these few weeks?
- 35. ChatGPT on Laptop, Phone & Raspberry Pi https://arstechnica.com/information-technology/2023/03/you-can-now-run-a-gpt-3-level-ai-model-on-your-laptop-phone-and-raspberry-pi/
- 37. Google’s Bard
- 38. Google’s Bard
- 39. Google’s Bard
- 40. Google’s Bard
- 41. Google’s Bard
- 48. Impact of GPT-4
- 49. Impact of GPT-4 • New & powerful ways of interaction
- 50. Impact of GPT-4 • New & powerful ways of interaction Question asked: https://leetcode.com/problems/count-and-say/
- 51. Impact of GPT-4 • New & powerful ways of interaction Question asked: https://leetcode.com/problems/count-and-say/
- 52. Impact of GPT-4 • New & powerful ways of interaction Question asked: https://leetcode.com/problems/count-and-say/
- 53. Impact of GPT-4 • New & powerful ways of interaction Question asked: Implement in Java https://leetcode.com/problems/count-and-say/
- 54. Impact of GPT-4 • New & powerful ways of interaction Question asked: Implement in Java https://leetcode.com/problems/count-and-say/
- 55. Impact of GPT-4 • New & powerful ways of interaction
- 56. Impact of GPT-4 • New & powerful ways of interaction
- 57. Impact of GPT-4 • New & powerful ways of interaction
- 58. Impact of GPT-4 • New & powerful ways of interaction
- 59. Impact of GPT-4 • New & powerful ways of interaction
- 60. Impact of GPT-4 • New & powerful ways of interaction
- 61. Impact of GPT-4 • Plugin ecosystem
- 62. Impact of GPT-4 • New products leveraging the power of ChatGPT-4
- 64. More errors noticed in ChatGPT
- 65. Quality of responses seems inconsistent
- 66. Quality of responses seems inconsistent
- 67. Quality of responses seems inconsistent
- 68. Quality of responses seems inconsistent
- 69. Quality of responses seems inconsistent
- 70. Github Copilot & Github Copilot X https://github.com/features/preview/copilot-x
- 74. AutoGPT • AutoGPT is a remarkable AI technology that utilizes GPT-4 and GPT-3.5 through API • It can create full-fledged projects by iterating on its own prompts and building upon them in each iteration. • It can read and write files, browse the web, review the results of its prompts, and combine them with the prompt history. Example: • It can be given an AI name such as RecipeBuilder and 5 goals that it has to meet. Once the goals are set, AutoGPT can start working on the project until completion. • AutoGPT is prone to fall into loops and make pointless requests when given complicated tasks. • For simple jobs, the outcomes are amazing. • AutoGPT uses credits from your OpenAI account, and the free version includes $18. • AutoGPT asks for permission after every prompt, enabling you to test it extensively before it costs you a dollar. https://openaimaster.com/how-to-use-autogpt/
- 75. Why am I talking about AutoGPT? https://github.com/Torantulino/Auto-GPT/
- 80. @BagmarAnand Challenges in Automation • Slow and flaky test execution • Sub-optimal, in-efficient automation • Incorrect, non-contextual test data
- 83. @BagmarAnand How to identify flaky tests?
- 84. @BagmarAnand ReportPortal.io – AI based reporting
- 85. @BagmarAnand Flakiness due to locator changes
- 87. The Applitools Execution Cloud The World’s First Self-Healing Test Infrastructure 88
- 88. What is the Execution Cloud? 1. Run functional or visual tests against browsers or devices in the cloud. 2. Self-heal any flakiness caused from problems in navigation during the test. 89 Applitools Execution Cloud is the world’s first intelligent testing infrastructure, enabling self-healing of open-source tests. It enables teams to:
- 89. Benefits of the Execution Cloud EASIER SET UP Launching and running your tests completely in the cloud is hassle free and doesn’t require any local testing set- up or infrastructure. 90 LESS FLAKINESS Execution Cloud self-heals tests that fail from incorrect locators used during navigation, meaning less of your tests will flake out. FASTER EXECUTION All tests execute in the cloud on our lightning fast infrastructure, so your pipelines will finish faster
- 92. Execution Cloud Will Self-Heal Tests Even If: ● Element properties change (e.g., ID, class, tag name, custom, etc) ● Text changes (clickable text, input value, label, placeholder) ● DOM position changes (hierarchy, position in list) ● Size and location changes 93
- 94. @BagmarAnand • Create a new test • Auto-generate code Github Copilot
- 95. @BagmarAnand • WebInspector for locators • PageObjects • https://youtu.be/z7Yjl2Agrwg https://www.jetbrains.com/aqua/ JetBrains Aqua
- 98. @BagmarAnand • JetBrains Aqua • Github Copilot • ChatGPT Test Data
- 100. @BagmarAnand Potential pitfalls of AI • Thinking AI will solve all problems! https://twitter.com/ProtonPrivacy/status/1619007351750627342
- 101. @BagmarAnand Potential pitfalls of AI • Thinking AI will solve all problems! • Results in superficial knowledge https://twitter.com/ProtonPrivacy/status/1619007351750627342
- 102. @BagmarAnand Potential pitfalls of AI • Thinking AI will solve all problems! • Results in superficial knowledge • AI will take away a lot of jobs! https://twitter.com/ProtonPrivacy/status/1619007351750627342
- 104. @BagmarAnand Potential pitfalls of AI • Thinking AI will solve all problems! • Results in superficial knowledge • AI will take away a lot of jobs! • Data Privacy https://twitter.com/ProtonPrivacy/status/1619007351750627342
- 105. @BagmarAnand Potential pitfalls of AI • Data Privacy https://arstechnica.com/information-technology/2023/02/ai-powered-bing-chat-spills-its-secrets-via-prompt-injection-attack/
- 106. @BagmarAnand Potential pitfalls of AI • Data Privacy https://www.msn.com/en-us/news/technology/whoops-samsung-workers-accidentally-leaked-trade-secrets-via-chatgpt/ar-AA19ygnA
- 107. @BagmarAnand Potential pitfalls of AI • Data Privacy https://twitter.com/ProtonPrivacy/status/1619007351750627342
- 108. @BagmarAnand Potential pitfalls of AI • Data Privacy
- 109. Summary • AI in Automation is getting very “interesting” • Step up, or you will get left behind! • Manually repeated work SHOULD definitely be avoided • Automation Cocktail is the way forward! • Your taste, your recipe • Build Vs Buy Vs Reuse à Focus on ROI, Value à Choose wisely • Be Agile à Evolve, Iterate, Communicate, Collaborate
- 110. Some tips for you Use this opportunity to: • Upskill yourself in new tools and technologies • Evolve your thought process • Be creative and use new information to come with unique solutions • Be innovative and learn how to use tools and technologies for different purposes
- 111. My approach How can I complete my 40 hours of work in 35 hours?
- 112. Thank You!
- 114. • https://openai.com/blog/chatgpt/ • https://applitools.com/blog/ai-the-magical-helping-hand-in-testing/ • https://applitools.com • https://github.com/features/copilot • https://githubnext.com/ • https://en.wikipedia.org/wiki/OpenAI Resources