Download as PDF, PPTX
Don't fear our new robot overlords – A new way to test on mobile
- 1. Don’t fear ournew Robot Overlords! A new way to test on mobile devices ! ! Philip Brechler • europython 2014
- 2.
- 3. Agenda • Problems whentesting on mobile devices
- 4. Agenda • Problems whentesting on mobile devices • Idea
- 5. Agenda • Problems whentesting on mobile devices • Idea • Concept
- 6. Agenda • Problems whentesting on mobile devices • Idea • Concept • Demo
- 7. Agenda • Problems whentesting on mobile devices • Idea • Concept • Demo • Conclusion and Learnings
- 8. Problems in general CC-BY-SA:Javier Enjuto
- 9. Problems in general •You need to check how the app looks CC-BY-SA: Javier Enjuto
- 10. Problems in general •You need to check how the app looks • You can check coordinates, fonts etc. CC-BY-SA: Javier Enjuto
- 11. Problems in general •You need to check how the app looks • You can check coordinates, fonts etc. • In the end you never know if it looks right CC-BY-SA: Javier Enjuto
- 12. Problems in general •You need to check how the app looks • You can check coordinates, fonts etc. • In the end you never know if it looks right • The human need to double check everything CC-BY-SA: Javier Enjuto
- 13. Problems with simulators CC-BY:Steve Jurvetson
- 14. Problems with simulators •To much computation power CC-BY: Steve Jurvetson
- 15. Problems with simulators •To much computation power • No memory issues CC-BY: Steve Jurvetson
- 16. Problems with simulators •To much computation power • No memory issues • No touch interface, mouse pointers are to exact CC-BY: Steve Jurvetson
- 17. Problems with simulators •To much computation power • No memory issues • No touch interface, mouse pointers are to exact • Different browser engines CC-BY: Steve Jurvetson
- 18. Problems with simulators •To much computation power • No memory issues • No touch interface, mouse pointers are to exact • Different browser engines • Some APIs not available CC-BY: Steve Jurvetson
- 19. Problems with emulators CC-BY-SA:Marshall Astor
- 20. Problems with emulators •Incredibly slow CC-BY-SA: Marshall Astor
- 21. Problems with emulators •Incredibly slow • Hard to interact with CC-BY-SA: Marshall Astor
- 22. Problems with emulators •Incredibly slow • Hard to interact with • Android: Every manufacturer makes their own interface and browser CC-BY-SA: Marshall Astor
- 23. Problems with emulators •Incredibly slow • Hard to interact with • Android: Every manufacturer makes their own interface and browser • Android: No Google Play Services CC-BY-SA: Marshall Astor
- 24. Best solution: Testing onreal devices with real inputs and a checking eye
- 25.
- 26.
- 27. Idea • Saw atalk about Sikuli for automated testing of websites
- 28. Idea • Saw atalk about Sikuli for automated testing of websites • Idea: Sikuli for mobile device testing but with a CI server
- 30.
- 31. Sikuli • Sikuli isa tool for automating computer tasks with screenshots
- 32. Sikuli • Sikuli isa tool for automating computer tasks with screenshots • A very python like script language
- 33. Sikuli • Sikuli isa tool for automating computer tasks with screenshots • A very python like script language • Invented at MIT, now developed at Sikuli Labs (University of Colorado)
- 34. Sikuli • Sikuli isa tool for automating computer tasks with screenshots • A very python like script language • Invented at MIT, now developed at Sikuli Labs (University of Colorado) • Can be used for CI testing with Robot Framework or similar
- 35. Idea • Saw atalk about Sikuli for automated testing of websites • Idea: Sikuli for mobile device testing but with a CI server
- 36. Idea • Saw atalk about Sikuli for automated testing of websites • Idea: Sikuli for mobile device testing but with a CI server • In short: Make a screenshot, use CV for detecting icon, tap icon
- 37. Idea • Saw atalk about Sikuli for automated testing of websites • Idea: Sikuli for mobile device testing but with a CI server • In short: Make a screenshot, use CV for detecting icon, tap icon • With a robot
- 38.
- 39. Making a screenshot •Android: Use adb command and some bash magic
- 40. Making a screenshot •Android: Use adb command and some bash magic • iOS: Making screenshots of devices is possible using Xcode but not with the terminal
- 41. Making a screenshot •Android: Use adb command and some bash magic • iOS: Making screenshots of devices is possible using Xcode but not with the terminal • But there is an open source project that reverse engineered some of iTunes’s and Xcode’s magic called libimobiledevice
- 42.
- 43. Detecting an icon •Use OpenCV to do a template recognition (finding an image in an image)
- 44. Detecting an icon •Use OpenCV to do a template recognition (finding an image in an image) • OpenCV is written in C/C++ but has great python wrappers
- 45. Detecting an icon •Use OpenCV to do a template recognition (finding an image in an image) • OpenCV is written in C/C++ but has great python wrappers • Quite simple API for such a complex task
- 46. Detecting an icon •Use OpenCV to do a template recognition (finding an image in an image) • OpenCV is written in C/C++ but has great python wrappers • Quite simple API for such a complex task • Get the coordinate in the screenshot of your searched icon
- 47.
- 48.
- 49. The robot • Tapsterbot •Open Source delta robot
- 50. The robot • Tapsterbot •Open Source delta robot • Made with Arduino, 3 Servos and 3D printed parts
- 51. The robot • Tapsterbot •Open Source delta robot • Made with Arduino, 3 Servos and 3D printed parts • Costs: ~250€ (With printing)
- 52. The robot • Tapsterbot •Open Source delta robot • Made with Arduino, 3 Servos and 3D printed parts • Costs: ~250€ (With printing) • Driver written in node.js
- 53. Talking to arobot
- 54. Talking to arobot • The robot is connected via WebSocket
- 55. Talking to arobot • The robot is connected via WebSocket • The robot is driven by a node.js „app“
- 56. Talking to arobot • The robot is connected via WebSocket • The robot is driven by a node.js „app“ • Theoretically the robot could be exchanged quite easily
- 57. Talking to arobot • The robot is connected via WebSocket • The robot is driven by a node.js „app“ • Theoretically the robot could be exchanged quite easily • More robots?
- 58. Tightening it alltogether
- 59. Tightening it alltogether • Project Goldeneye
- 60. Tightening it alltogether • Project Goldeneye • To test an app or a mobile website just write a standard python unittest test
- 61. Tightening it alltogether • Project Goldeneye • To test an app or a mobile website just write a standard python unittest test • All the heavy lifting is abstracted away from you
- 62. Tightening it alltogether • Project Goldeneye • To test an app or a mobile website just write a standard python unittest test • All the heavy lifting is abstracted away from you • Just tell the test to tap and to check
- 63. Tightening it alltogether • Project Goldeneye • To test an app or a mobile website just write a standard python unittest test • All the heavy lifting is abstracted away from you • Just tell the test to tap and to check • Writing such tests is quite easy, even your QA can do this
- 64.
- 65.
- 66. Conclusion • Goldeneye couldbe useful for testing mobile apps and especially websites in a CI system
- 67. Conclusion • Goldeneye couldbe useful for testing mobile apps and especially websites in a CI system • Testing of existing projects, especially large web projects could be easier
- 68. Conclusion • Goldeneye couldbe useful for testing mobile apps and especially websites in a CI system • Testing of existing projects, especially large web projects could be easier • Robots never get tired
- 69. Conclusion • Goldeneye couldbe useful for testing mobile apps and especially websites in a CI system • Testing of existing projects, especially large web projects could be easier • Robots never get tired • The robot’s hardware is fine, the software isn’t
- 70.
- 71. Learnings • A functionin a function in a function isn’t normal, but in node.js it is
- 72. Learnings • A functionin a function in a function isn’t normal, but in node.js it is • Never trust a github project with one maintainer
- 73. Learnings • A functionin a function in a function isn’t normal, but in node.js it is • Never trust a github project with one maintainer • OpenCV is fun and not that hard to learn
- 74.