Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Modern Functional Test Automation Through Visual AI - webinar w/ Raja Rao

4,627 views

Published on

** Full webinar recording here: https://youtu.be/EaISHnCjNGY **

"I am confident that once you give this approach a try, you will rethink your entire current code-based approach" -- Raja Rao, Head of Test Automation University

In this webinar, you'll see the modern way or the intelligent way of doing web and mobile testing. Specifically, functional, end-to-end UI testing.

The analogy is a gasoline car versus an electric car: both are cars, both need tires, seats, breaks, etc... but the core engine that moves the car is different -- which makes a huge difference.

The main idea here is that, once the functionality in an app happens (for example: logging into an app), you simply take a screenshot of the resulting page or resulting state of the app, and take screenshots every time you run the test and compare them with the original screenshot using Visual AI (instead of pixel-by-pixel comparison, or DOM-diffing). If there is a difference, then the AI will highlight only meaningful differences and ignores differences that we humans ignore.

You'll see that by using this approach where you delegate a lot of work the Visual AI, you'll see exponential benefits, such as up to 5X increase in the number of bugs found, up to 10X less code and so on.

In this webinar, Raja Rao compares several typical functional testing use cases to show how it actually works.

Talking points:
* What is modern functional testing
* What is "Visual AI" -- and why you need it
* Deeply analyze legacy code based functional test and compare it with the modern approach (number of lines, locators, labels, etc...)
* Compare legacy versus modern code by going over some use cases and approaches, such as Data-driven testing, Sorting an HTML table, Testing a dynamic bar chart, Testing iFrames, Testing dynamic pages, etc…

Published in: Software
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1url.pw/zTkvR ◀ ◀ ◀ ◀
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Modern Functional Test Automation Through Visual AI - webinar w/ Raja Rao

  1. 1. Modern Functional Testing Through Visual AI Raja Rao DV Sr. Director and Head of Growth, Applitools Head of TestAutomationU @rajaraodv
  2. 2. Login page to test
  3. 3. Functional Testing - Legacy Line of Navigation code 1 Line of Button click code 1 Lines of Validation code 16 Total LOC 18 Locators like Ids, XPaths etc that can change 14 Hard-coded texts that are used to compare labels 7 Total things that can change and break! 21 Lines of code Validation code that can break 30-45 minutes to create
  4. 4. State 1 (e.g. Login page) State 2 (e.g. Login page With error) Action (Click on Login button) Take a screenshot here (After the functionality) State Diagram
  5. 5. Line of Navigation code 1 Line of Button click code 1 Lines of Validation code 3 (vs 16) Total LOC 5 (vs 18) Locators like Ids, XPaths etc that can change 1 (vs 14) Hard-coded texts that are used to compare labels 0 (vs 7) Total things that can change 1 (vs 21) Lines of code Validation code that can break 2 minutes to create (vs 30 minutes!) Functional Testing - Visual AI
  6. 6. Login page (After with new version) Real bugs 1 Visual bugs [misalignments, cutoffs, etc] 1 Differences due to enhancements 2 New feature 1 Total differences 5 Real bug Placeholder text has changed Placeholder text has changed New feature Visual bug
  7. 7. Test Result: Traditional Functional Testing Real bug Placeholder text has changed Placeholder text has changed New feature Visual bug The real bug has escaped because you didn’t add code for logo! -1 Visual bug escaped because you were only testing for error’s existence (and not position) -1 Both enhancements were caught 2 Misses the New feature -1 1 False positive! Because of “our terms and conditions” link, the XPath of Facebook is broken. The test reports an additional failure! -1 Total -2 Note: 1 point for catching bugs/diffs -1 point for missing bugs or false-positives
  8. 8. Test Result: Visual AI based Functional testing Summary: Accurately catches all the differences. Very easy to analyze for yourself without having to re-run any tests since it literally highlights the diffs! The real bug is caught 1 The Visual bug is caught 1 Both enhancements were caught 2 New feature caught 1 Total false positives! 0 Total 6
  9. 9. Test Result: Visual AI based Functional testing Summary: Accurately catches all the differences. Very easy to analyze for yourself without having to re-run any tests since it literally highlights the diffs! The real bug is caught 1 The Visual bug is caught 1 Both enhancements were caught 2 New feature caught 1 Total false positives! 0 Total 6
  10. 10. Bugs caught: Legacy vs Visual AI Count Difference type Legacy Visual AI 1 Real bugs -1 1 1 Visual bugs -1 1 2 Differences due to enhancements to placeholder text 2 2 1 New feature (Terms and conditions) -1 1 1 NO FALSE POSITIVES -1 0 5 Total diffs caught* -2 +6 [8X better] *This is just an example, but the reality is not from this
  11. 11. Test Maintenance Cost: Legacy vs Visual AI Count Difference type Maintenance Action taken Legacy (time in minutes) Visual AI 1 Real bugs Do manual test to find these bugs, write new tests, ensure they catch these bugs 15 mins 0 mins 1 Visual bugs Need to add CSS related tests 15 mins 0 mins 2 Differences due to enhancements to placeholder text Update tests 10 mins 2 seconds (Simply click the Thumbs up button to create a new “Baseline”) 1 New feature (Terms and conditions) Add new test 5 mins 2 seconds (Simply click the Thumbs up button to create a new “Baseline”) FALSE POSITIVES Manually rerun tests, add breakpoints, figure out why they are broken, figure out new XPaths, ensure new XPaths work, submit new tests 15 mins 0 Total maintenance cost 60 mins! 4 seconds
  12. 12. Functional Testing cost (1 browser & 1 page) Cost type Legacy (time in minutes) Visual AI Time to create tests 30 minutes 5 minutes [6X faster] Time to maintain tests 60 minutes 4 seconds [1000X faster] Total time taken 90 minutes 5 minutes [18X faster] Test development and maintenance cost Cost type Dom based Visual AI Success rate -2 bugs caught +6 [8X more] #Bugs found Cost type Dom based (Number of locators) Visual AI Dependency on Locators and Labels 20 (locators and labels) 1 [20X more stable] Test Stability cost Note: These number will vary dramatically for different use cases
  13. 13. So what’s the build and maintenance cost for this?!!
  14. 14. Functional Testing (DOM based) Functional testing with Visual AI Things that are not Visual and are better suited to test as part of API testing and other backend testing. Functional testing using Visual AI is a superset of legacy functional testing Virtually all functional testing can be done by taking screenshots
  15. 15. Description Legacy Automation testing Modern Automation testing Remarks Browser Automation (Button clicks, data entry) Selenium, Cypress, TestCafe, Puppeteer etc Selenium, Cypress, TestCafe, Puppeteer etc No difference Test Frameworks JUnit, TestNG,etc JUnit, TestNG,etc No difference Assertion libraries Chai, Assertions libs that comes with JUnit, TestNG and other libraries Applitools Visual AI ● 10X faster to develop and maintain functional tests ● 5X more bugs caught Test running Infrastructure SauceLabs, BrowserStack, localhost or custom cloud SauceLabs, BrowserStack, localhost or custom cloud No difference (but if you use Visual Grid, you need to run just one variation because Visual Grid takes care of rendering tests in different browsers and viewports) Browser rendering infrastructure [Optional] N/A Visual Grid ● 30X faster cross-browser testing ● 50% less cost Ecosystem support Jira, Jenkins, Github Jira, Jenkins, Github No difference Education and community Paid and/or non-existent Test Automation University ● Largest free training website for QA professionals ● 2nd largest Slack community to help answer questions Tech stack comparison
  16. 16. Test creation phase: 1. Fewer tests to create 2. Quicker to create tests as you don't need to search for locators and labels etc. Test maintenance phase: 1. Fewer tests to maintain because: 1. Fewer tests to begin with 2. Fewer tests break because of less dependency on locators 2. Easy to update existing tests if the UI changes because you can just update the baseline image instead of changing code and rewriting the tests 1. This is because you are simply taking screenshots and not depending on code to validate Bug reporting phase: 1. Improved test coverage finds 6X more bugs 2. Reliable results. With virtually no false-positives means no need to double-check the test results 3. Easy to verify results. Because you take screenshots, it's very easy to verify test results and make changes 4. Direct integration with Jira and Github means easy to file bugs with Screenshot attached. Debugging phase (Developers): 1. If you are using RCA, it's easy for developers to debug the issue Benefits
  17. 17. Thank You Raja Rao DV Director of Growth, Applitools Head of TestAutomationU @rajaraodv
  18. 18. Questions Raja Rao DV Director of Growth, Applitools Head of TestAutomationU @rajaraodv

×