Jason Arbon, Jonathan Lipps and Chris Navrides discuss the benefits of adding AI to Appium and introduce an open source classifier plugin for Appium at the Selenium Conference in Chicago (Oct 2018).

1. AI For Element Selection
Jason Arbon
jason@test.ai
/in/jasonarbon
@jarbon

6. If Appium
had a brain...

11. driver.find_element_by_xpath('//*[@id="main
bar"]/div[1]/div/div[1]/div[3]/img')
http://www.automationtestinghub.com/appium-inspector/
Look Familiar?

13. driver.elementByCustom(‘ai:cart’)
New AI-Powered Way

14. Speed?

17. Training Data + Models
http://www.automationtestinghub.com/appium-inspector/
Open Source

18. driver.elementByCustom(‘<X>:<semantic_label>’)
Build your Own Classifiers and Plug them in!
Extensible

19. Add Your Custom App Images to Training Data
Customizable

20. https://github.com/appium/appium-base-driver/pull/268

21. https://github.com/testdotai/appium-classifier-plugin

24. Under the hood

25. What is TensorFlow?

26. LOTS of data

27. Pre-trained model

28. MOBILENET

29. Training

37. // Hello World
// Untested on Windows, Linux
// Perf limited by JS Tensorflow
// Statistical, not guaranteed!
// Hard to debug
// Have Fun!
Caveats

38. AITesting.org

40. jason@test.ai
/in/jasonarbon
@jarbon
Let’s Connect!