Skip to main content
BUILDING YOUR OWN NATIVE APP
VISUAL TESTING MODULE WITH APPIUM
Wim Selles, 

Sr. Solutions Architect @Sauce Labs
IT’S NOT ABOUT ME
! :@wswebcreation
" : https://github.com/wswebcreation
IDEA
IDEA
REQUIREMENTS
• Native App support
• Cross platform
• Integrate WebdriverIO
• Paid tools === trial first
• Support blockouts
• Support element screenshots/bounding box
FOR A VISUAL REGRESSION / TESTING SOLUTION
BLOCKOUTS
Status-/navigation-/

home indicator bar
WHY …?
Animations Dynamic text
RESEARCH
VISUAL REGRESSION / TESTING FOR NATIVE APPS
iOSSnapshotTestCase
Nakal
sightr.io
screenshot-tests-for-android
RESULT
VISUAL REGRESSION / TESTING FOR NATIVE APPS
Requirements
screenshot-tests-

for-android
iOSSnapshot

TestCase
Nakal sightr.io VisWiz Applitools
Native apps support Yes Yes Yes Yes Yes Yes
WebdriverIO support

- JS

- No extra env

dependencies
No No No No Yes
Not for 

native apps
Trial? Open source Open source
Open
source
Beta Yes Yes
Image diff Yes, basic Yes, basic Yes Yes Yes, basic Yes, extensive
Blockouts No No Yes and no No no Yes
Element screenshots No No No No No No
Bounding box No No No No No Yes
RESULT
VISUAL REGRESSION / TESTING FOR NATIVE APPS
Requirements
screenshot-tests-

for-android
iOSSnapshot

TestCase
Nakal sightr.io VisWiz Applitools
Native apps support Yes Yes Yes Yes Yes Yes
WebdriverIO support

- JS

- No extra env

dependencies
No No No No Yes
Not for 

native apps
Trial? Open source Open source
Open
source
Beta Yes Yes
Image diff Yes, basic Yes, basic Yes Yes Yes, basic Yes, extensive
Blockouts No No Yes and no No no Yes
Element screenshots No No No No No No
Bounding box No No No No No Yes
RESULT
VISUAL REGRESSION / TESTING FOR NATIVE APPS
Requirements
screenshot-tests-

for-android
iOSSnapshot

TestCase
Nakal sightr.io VisWiz Applitools
Native apps support Yes Yes Yes Yes Yes Yes
WebdriverIO support

- JS

- No extra env

dependencies
No No No No Yes
Not for 

native apps
Trial? Open source Open source
Open
source
Beta Yes Yes
Image diff Yes, basic Yes, basic Yes Yes Yes, basic Yes, extensive
Blockouts No No Yes and no No no Yes
Element screenshots No No No No No No
Bounding box No No No No No Yes
RESULT
VISUAL REGRESSION / TESTING FOR NATIVE APPS
Requirements
screenshot-tests-

for-android
iOSSnapshot

TestCase
Nakal sightr.io VisWiz Applitools
Native apps support Yes Yes Yes Yes Yes Yes
WebdriverIO support

- JS

- No extra env

dependencies
No No No No Yes
Not for 

native apps
Trial? Open source Open source
Open
source
Beta Yes Yes
Image diff Yes, basic Yes, basic Yes Yes Yes, basic Yes, extensive
Blockouts No No Yes and no No no Yes
Element screenshots No No No No No No
Bounding box No No No No No Yes
RESULT
VISUAL REGRESSION / TESTING FOR NATIVE APPS
Requirements
screenshot-tests-

for-android
iOSSnapshot

TestCase
Nakal sightr.io VisWiz Applitools
Native apps support Yes Yes Yes Yes Yes Yes
WebdriverIO support

- JS

- No extra env

dependencies
No No No No Yes
Not for 

native apps
Trial? Open source Open source
Open
source
Beta Yes Yes
Image diff Yes, basic Yes, basic Yes Yes Yes, basic Yes, extensive
Blockouts No No Yes and no No no Yes
Element screenshots No No No No No No
Bounding box No No No No No Yes
?
CONCLUSION
WHAT TO DO?
LOGIC
WHO SHOULD DO WHAT?
Requirements Module Comparison engine
Actively maintained - Yes
Environment dependencies No No
Blockout(s) Yes, determine and provide Yes
Element screenshot Yes -
Create baseline Yes -
Screenshots per device Yes -
Support compare options

(for example antialiasing)
Yes Yes
LOGIC
WHO SHOULD DO WHAT?
Requirements Module Comparison engine
Actively maintained - Yes
Environment dependencies No No
Blockout(s) Yes, determine and provide Yes
Element screenshot Yes -
Create baseline Yes -
Screenshots per device Yes -
Support compare options

(for example antialiasing)
Yes Yes
JAVASCRIPT PIXEL-BY-PIXEL LIBRARIES
js-imagediff
image-diff
Rembrandt.JS
LooksSamepixelmatch
node-opencv
img-diff-js
Pixel-Diff
Resemble.JS
JAVASCRIPT PIXEL-BY-PIXEL LIBRARIES
js-imagediff
image-diff
Rembrandt.JS
LooksSamepixelmatch
node-opencv
img-diff-js
Pixel-Diff
Resemble.JS
JAVASCRIPT PIXEL-BY-PIXEL LIBRARIES
js-imagediff
image-diff
Rembrandt.JS
LooksSamepixelmatch
node-opencv
img-diff-js
Pixel-Diff
Resemble.JS
JAVASCRIPT PIXEL-BY-PIXEL LIBRARIES
js-imagediff
image-diff
Rembrandt.JS
LooksSamepixelmatch
node-opencv
img-diff-js
Pixel-Diff
Resemble.JS
JAVASCRIPT PIXEL-BY-PIXEL LIBRARIES
js-imagediff
image-diff
Rembrandt.JS
LooksSamepixelmatch
node-opencv
img-diff-js
Pixel-Diff
Resemble.JS
CONCLUSION
CONCLUSION
• saveScreen
WHAT SHOULD IT DO?
• saveScreen
• compareScreen
WHAT SHOULD IT DO?
• saveScreen
• compareScreen
• saveElement
WHAT SHOULD IT DO?
• saveScreen
• compareScreen
• saveElement
• compareElement
WHAT SHOULD IT DO?
SAVESCREEN
WHAT SHOULD IT DO?
COMPARESCREEN
WHERE COULD IT GO WRONG, WHERE COULD IT CAUSE FLAKINESS?
Navigation bar Home bar indicatorStatus bar
STATUS BAR
• Android and iOS differ in height
• iOS OS versions / devices differ
• Android OS versions / devices differ
• Cross platform / device solution
DEMO
NAVIGATION BAR
• Android OS versions / devices differ
• Not each device has a tool/navigation bar
• Cross device solution
FOR ANDROID
HOME BAR INDICATOR
● Currently 2 sizes
● The behaviour can be adjusted
IPHONE X SERIES
SAVEELEMENT
WHAT SHOULD IT DO?
SAVEELEMENT
WHAT SHOULD IT DO?
SAVEELEMENT
WHAT SHOULD IT DO?
SAVEELEMENT
EXTRA POWER
COMPAREELEMENT
● Comparing (options) the 

same as for compareScreen
● Element block-outs 

more complex
WHAT SHOULD IT DO?
USE CASE
React Native
0.54
React Native
0.57
USE CASE
React Native
0.54
React Native
0.57
USE CASE
DISCLAIMER
Cons
● Basic comparison
● Impact on project size
● Limited blockout options
● No dashboard
Pro’s
● Open Source
● The (module) logic is easy
● Easy cross platform/device blockout
logic
driver.compareScreen(
'compareScreen-elementBlockOuts', {
elementBlockOuts: [
// block out element 1
{ element: LoginScreen.loginButton },
// block out element 2 with margin
{ element: LoginScreen.email, margin: 50},
],
}
);
WANT TO USE IT?
https://github.com/wswebcreation/wdio-native-app-compare
WANT TO USE IT?
THANK YOU.