Myth vs Reality: Understanding AI/ML for QA Automation - w/ Jonathan Lipps
•
1 like•1,398 views
** Full webinar recording -- https://youtu.be/ihpAsmRtGuM ** Artificial Intelligence and Machine Learning (AI/ML) have seen application in a variety of fields, including the automation of QA tasks. But what are they exactly? What distinguishes different instances and applications of AI, for example? What are the horizons of these technologies in the field of QA? The promise of AI/ML must be understood correctly to be harnessed appropriately. As with any buzzword, many technologies and products are offered under the guise of AI/ML without satisfying the definition. The industry is reforming itself around the promise that AI/ML holds often without a clear understanding of the technical limitations that give the promise its boundaries. In this webinar, test automation guru Jonathan Lipps gives a detailed overview of the concepts that underpin AI/ML, and discuss their ramifications for the work of QA automation. In addition to a discussion of AI/ML in general, Jonathan looks at examples from the QA industry. These examples will help give attendees the basic understanding required to cut through the marketing language. so we can clearly evaluate AI/ML solutions, and calibrate expectations about the benefit of AI/ML in QA, both as it stands today and in the future.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Myth vs Reality: Understanding AI/ML for QA Automation - w/ Jonathan Lipps
Similar to Myth vs Reality: Understanding AI/ML for QA Automation - w/ Jonathan Lipps (20)
More from Applitools
More from Applitools (20)
Recently uploaded
Recently uploaded (20)
Myth vs Reality: Understanding AI/ML for QA Automation - w/ Jonathan Lipps
- 1. Myth vs Reality: Understanding AI/ML for QA Automation Jonathan Lipps • Founding Principal • Cloud Grey @AppiumDevs • @cloudgrey_io • @jlipps • appiumpro.com Applitools Webinar · The Internet January 31, 2020
- 2. Founding Principal Architect, Maintainer Jonathan Lipps • Founding Principal • Cloud Grey @AppiumDevs • @cloudgrey_io • @jlipps • appiumpro.com
- 3. Intro
- 5. @jlipps · cloudgrey.io AI == BS? 😮
- 6. @jlipps · cloudgrey.io “The core feature of a B.S.-industrial complex is that every member of the ecosystem knows about the charade, but is incentivized to keep shoveling.” source:https://hackerfall.com/story/ai-bs-industrial-complex-and-its-discontents
- 8. @jlipps · cloudgrey.io How is AI any different from any other software technology?
- 9. What are AI & ML?
- 10. @jlipps · cloudgrey.io AI: anything a computer does that seems smart (not very helpful)
- 11. @jlipps · cloudgrey.io ML: “field of study that gives computers the ability to learn without being explicitly programmed.” - Arthur Samuel (ok that’s a bit better)
- 12. @jlipps · cloudgrey.io Category of ML Main Idea Supervised Learning Learn a function based on tagged inputs Unsupervised Learning Learn classifications and patterns in untagged data Reinforcement Learning Learn by trial-and-error in a scenario that generates reward feedback Deep Learning A specific take on the use of neural networks
- 13. @jlipps · cloudgrey.io Category of ML Use Cases Supervised Learning Classify a new instance of data based on a trained model. Predict a numeric quantity from new data. Unsupervised Learning Find patterns in a dataset that are difficult for human researchers to spot. Reinforcement Learning Develop human-like reasoning in a well-defined task environment. Deep Learning Attack problems with very complex input data.
- 14. @jlipps · cloudgrey.io Category of ML Examples Supervised Learning Given information about a flower’s petal length, shape, and other details, predict the species of flower. Unsupervised Learning Given a huge library of popular music, find natural groupings of songs and see if they correspond to human understandings of genre. Reinforcement Learning Teach a bot to compete against humans in a video game. Deep Learning Given an image of an animal, classify it according to the animal’s species.
- 15. @jlipps · cloudgrey.io Example ML Algorithms / Approaches Main Idea Linear Regression Used in supervised learning to learn a function which can be applied to new inputs to get a scalar output value. k-Means Clustering Used in unsupervised learning to partition an n-dimensional space based on natural groupings of data. Neural Networks Used in a variety of applications. Simulates the operations of neurons to learn the weight of different values in an input vector. Generative Adversarial Networks Pit two neural networks against each other in a kind of ‘imitation game’ in order to produce fake data that passes for real data.
- 16. @jlipps · cloudgrey.io source: https://medium.com/machine-learning-for-humans/supervised-learning-740383a2feab
- 17. @jlipps · cloudgrey.io source: https://medium.com/machine-learning-for-humans/unsupervised-learning-f45587588294
- 18. @jlipps · cloudgrey.io source: https://www.youtube.com/watch?v=3lp9eN5JE2A&t=1631s
- 19. @jlipps · cloudgrey.io source: https://towardsdatascience.com/understanding-generative-adversarial-networks-gans-cd6e4651a29
- 20. AI/ML in QA
- 29. @jlipps · cloudgrey.io Categories of “AI” solutions in QA Main Idea AI in marketing only Intelligently designed software that doesn’t use machine learning models. AI/ML in a supporting role ML models are used to support features, not as a replacement for test authoring. AI/ML as the primary driver of automation Tests are written and bugs found by autonomous bots acting on pre- or post-trained ML models.
- 30. @jlipps · cloudgrey.io Categories of “AI” solutions in QA Example AI in marketing only Scrape production user activity logs to generate test cases. Capture multiple selectors for elements to increase test robustness. AI/ML in a supporting role Image recognition models to detect visual differences. Video quality models give feedback on user-perceived quality. AI/ML as the primary driver of automation You hand off the app to the AI with no additional metadata and it sends you back bug reports.
- 31. Conclusion
- 32. @jlipps · cloudgrey.io AI == BS* (with a few exceptions)
- 33. @jlipps · cloudgrey.io Do you need “AI” in your testing? Why?
- 34. @jlipps · cloudgrey.io Evaluate technologies based on their actual ROI, not how well they claim the hype of the zeitgeist.
- 35. @jlipps · cloudgrey.io A handy question to probe a product with: “What corpus did you use to train your ML model?”
- 36. @jlipps · cloudgrey.io Prediction: most actual ROI will be from AI/ ML in supporting roles, for a while yet.
- 37. Thank You! Don’t forget to sign up for Your free weekly Appium newsletter appiumpro.com Jonathan Lipps • Founding Principal • Cloud Grey @AppiumDevs • @cloudgrey_io • @jlipps • appiumpro.com