Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

QA Fest 2017. Jeremias Rößler. Applying AI to testing

236 views

Published on

If the future of testing is AI, I will show you how.
Various automation tools are available, but due to high creation and maintenance efforts for tests, GUI-Testing is still mainly a manual task. Meanwhile overall testing effort has risen to make up 30% of a typical software budget. Is crowd-testing the answer? What if we could have automated test cases be created automatically?
Monkey Testing is not a new idea. But combined with a manually trainable AI and an innovative new testing approach (dubbed ""difference testing""), we can now not only have the monkey search for technical bugs (i.e. crashes) but generate functional test cases that are optimized towards several goals and are even better than manually created ones.
Visit the future of testing and see how AI can help us create better software!

Published in: Education
  • Be the first to comment

QA Fest 2017. Jeremias Rößler. Applying AI to testing

  1. 1. @roesslerj 1 Applying AI to Testing Dr. Jeremias Rößler Kyiv 2017
  2. 2. @roesslerj 2
  3. 3. @roesslerj 3
  4. 4. @roesslerj 4
  5. 5. @roesslerj 5
  6. 6. @roesslerj 6 Oracle Problem
  7. 7. @roesslerj 7 How do you bring AIinto testing?
  8. 8. @roesslerj 8 AI in Testing? – so far I do not see anyone doing it in a useful way. Antony Edwards, CTO of TestPlant “
  9. 9. @roesslerj “ 9 AI in Testautomation is a hype. Wolfgang Platz, Founder & CPO of Tricentis
  10. 10. @roesslerj 10 I think there is a world market for maybe five computers.“ Thomas J. Watson CEO of IBM
  11. 11. @roesslerj 11 Goal of Testing?
  12. 12. @roesslerj 12 When is a bug a bug? When it’s not a feature!
  13. 13. @roesslerj 13 1.if (name[0] == '.') continue; Is it a bug? . .. text.c other.files
  14. 14. @roesslerj 14 Is it a bug?
  15. 15. @roesslerj 15 Is it a bug?
  16. 16. @roesslerj 16 Without specification, there are no bugs — only surprises. Brian Kernighan “
  17. 17. @roesslerj 17 Why do we test? After implementation? 1 + 1 = 3 2 + 2 = 4
  18. 18. @roesslerj 18
  19. 19. @roesslerj 20
  20. 20. @roesslerj 21 Why do we test? After implementation? 1 + 1 = 3 2 + 2 = 4 After change? 1 + 1 = 3 1 + 1 = 5
  21. 21. @roesslerj regression-test ≠ test 22
  22. 22. @roesslerj regression-test = version control 23
  23. 23. @roesslerj 24
  24. 24. @roesslerj 25
  25. 25. @roesslerj 26
  26. 26. @roesslerj 27
  27. 27. @roesslerj 28
  28. 28. @roesslerj 29
  29. 29. @roesslerj 30
  30. 30. @roesslerj 31
  31. 31. @roesslerj 32
  32. 32. @roesslerj 33
  33. 33. @roesslerj 34
  34. 34. @roesslerj 35
  35. 35. @roesslerj 36
  36. 36. @roesslerj 37 How do you bring AIinto testing?
  37. 37. @roesslerj 38 Infinite-Monkey-Theorem: A monkey hitting keys at random on a typewriter for a long enough time will type the complete works of William Shakespeare.
  38. 38. @roesslerj 39 We replace the typewriter with a computer…
  39. 39. @roesslerj Infinite Monkey 40 public static void main(String... args) throws Exception { Robot robot = new Robot(); while (true) { robot.mouseMove(random.nextInt(maxX), random.nextInt(maxY)); robot.mousePress(InputEvent.BUTTON1_DOWN_MASK); robot.mouseRelease(InputEvent.BUTTON1_DOWN_MASK); robot.delay(200); for (char inputChar : randomString().toCharArray()) { robot.keyPress((int) inputChar); robot.keyRelease((int) inputChar); robot.delay(10); } robot.keyPress(KeyEvent.VK_ENTER); robot.keyRelease(KeyEvent.VK_ENTER); } } 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16.
  40. 40. @roesslerj 41 DEMO Infinite Monkey
  41. 41. @roesslerj 43
  42. 42. @roesslerj 44
  43. 43. @roesslerj Smart Monkey 45 public static void main(String... args) throws Exception { WebDriver driver = new FirefoxDriver(); driver.get("http://www.retest.de"); while (true) { List<WebElement> links = driver.findElements(By.tagName("a")); links.get(random.nextInt(links.size())).click(); Thread.sleep(500); List<WebElement> fields = driver.findElements(By.xpath("//input[@type='text']")); WebElement field = fields.get(random.nextInt(fields.size())); field.sendKeys(randomString()); Thread.sleep(500); } } 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14.
  44. 44. @roesslerj 46 Smart Monkey DEMO
  45. 45. @roesslerj Smart Monkey 48
  46. 46. @roesslerj 49 - gremlins.js - NetFlix Chaos Monkey - UI/Application Exerciser Monkey Open Source Products
  47. 47. @roesslerj 50
  48. 48. @roesslerj 51
  49. 49. @roesslerj 52 Afraid of AI?
  50. 50. @roesslerj 53 How many blondes does it need to screw in a light bulb? http://www.smbc-comics.com/comic/humor None, because automation has eliminated the need for humans.
  51. 51. @roesslerj “ 54 AI is perfectly suited to replace testing. Jason Arbon, Founder & CPO of AppDiff
  52. 52. @roesslerj 55 Not so fast…
  53. 53. @roesslerj 56
  54. 54. @roesslerj 57 Be afraid if: - test automation engineer - manual automate
  55. 55. @roesslerj 58
  56. 56. @roesslerj 59 Intelligenter Affe Summary Monkey-Testing: cheaper faster better multiplies manual tests Difference Testing little maintenance no wasteful testing yet more complete
  57. 57. @roesslerj 60 November 2017
  58. 58. @roesslerj 61 @Test public void test_login() throws Exception { final WebDriver driver = new FirefoxDriver(); driver.get( "http://www.retest.de" ); recheck.check( driver ); driver.findElement( By.id( "username" ) ).sendKeys( username ); driver.findElement( By.id( "pwd" ) ).sendKeys( password ); driver.findElement( By.className( "button" ) ).submit(); recheck.check( driver ); }
  59. 59. @roesslerj 62 PDF Excel XML HTML Proprietary
  60. 60. @roesslerj 63 http://retest.rocks @ReTest_en
  61. 61. @roesslerj Questions? https://www.facebook.com/retest.de http://retest.rocksroessler@retest.de The largest software testing conference in Ukraine

×