Automated visual testing for android and ios

1. Enrich Your Mobile Automation
With
Visual Assertions
http://clipartzebraz.com/cliparts/eye-clipart/cliparti1_eye-clipart_10.jpg
Rajdeep Varma
Sr. Consultant, ThoughtWorks

2. About Myself
• Rajdeep Varma
• 6 years in Quality Engineering. I work at
ThoughtWorks
• Active open source contributor
• Passionate about test automation and agile practices
• Tested a number of web apps and recently started
testing mobile apps

3. I will talk about
• The Problem: Limitation of functional testing tools
and Need of Automated visual regression.
• Our Approach: “Nakal”
• Some Challenges and solutions
• Future Plans

4. WHY ??

5. version-3.1 version-3.2

6. Difference

7. Common
Codebase
App 1 App 2

8. The Problem
• UI changes done for App-1, frequently
break UI of App-2
• There is no automated regression to
catch this!

9. Tech Radar
• CSS Critic
• dpxdt
• Huxley
• PhantomCSS
• Wraith
NOT FOR
MOBILE

10. Options
• Manual UI regression against past build:
Error prone.
It SUCKS!
• Proprietary cloud based tools
• Create a tool that meets our need
http://images.clipartpanda.com/choice-clipart-fork-choice-hi.png

11. My Need
• No new framework. Use existing tests.
• Easy to extend any test for visual validations
• Easy to integrate in our CI
• On-premise and easy to debug failures locally

12. My tool-set

13. “nakal”

14. Demo

15. How it works?
• ADB for capturing screen on android.
• AppleScript for capturing iOS Simulator
screenshots
• ImageMagick for comparison

16. Installation and Configuration
•Install ImageMagick
•Include in Gemfile
gem ‘nakal'
•In env.rb or any equivalent file:
require “nakal/cucumber" OR require "nakal" ; include Nakal::DSL
Nakal.platform = :androidOR Nakal.platform = :ios

17. To add visual assertion
nakal_execute(“home_screen")
•For the very first time, to create baseline, run your
test with environment variable
NAKAL_MODE=build
•Now onwards, run your test with environment
variable NAKAL_MODE=compare

18. Some Improvements
• Implicit wait till current screen matches baseline
• Mask a dynamic region
• Crop Notification and scroll area
• Embedded screenshots in cucumber reports
• Driver independent. Can work with appium or calabash etc
• Easily integrate in CI by setting NAKAL_MODE environment
variable
• Use ImageMagick fuzz for better comparison

20. Nakal Configuration
default_device:
top: 70
right: 0
left: 0
bottom: 0
login_screen: {region_foo: [390, 112,512,156]}

21. Best Practices
• Use mocking to make your components reliable
• Use directories to organize your baseline. e.g:
nakal_execute(“login_flow/home_screen”)
• Don’t use if UI is under very active development
• Use only when there is genuine need

22. Coming Soon..
• Better masking
• Java version

23. https://github.com/rajdeepv/nakal
rubygems.org/gems/nakal
nakal

24. References:
https://www.thoughtworks.com/radar
https://rubygems.org/gems/rmagick/versions/2.15.4
http://rmagick.github.io/
https://developer.android.com/tools/help/shell.html
https://github.com/Huddle/PhantomCSS

25. ?
Rajdeep Varma
Sr. Consultant, ThoughtWorks
Email: mail.rajvarma@gmail.com