The global shift to mobile applications as the primary means of accessing the internet is undeniable and has caused enterprises to quickly develop mobile channels for their new and existing business applications. With the many other challenges this trend has presented, it was time for enterprises to lasso the wild west of mobile application development and find a way to increase the velocity of their mobile app releases to keep pace with these ever changing mobile market dynamics all without compromising quality. And, with every passing iOS upgrade and Android feature, comes greater end-user expectations around mobile apps’ performance and usability. As a result a new industry standard was born – Continuous Quality – where mobile app development times are reduced from once or twice a year to once or twice an hour to address the real-time nature of monitoring and maintaining an app’s quality performance. As Developers and Dev/Test teams evaluate the wide range of test automation frameworks and tools available, they will realize there is no white horse around to save their damsel in distress. So, it’s important that they look for an environment that has the following capabilities to achieve a Continuous Quality process: • Real end user environment • Enterprise security • Open and Integrated platform – so they can easily leverage existing tools and frameworks • Global and collaborative lab • An Hybrid Cloud deployment model (Private cloud, Public, Local – USB) • Elastic for capacity on demand Key Takeaways: • How to include testing in every sprint and bake Performance and network conditions into their test code • How to efficiently automate their tests across platforms in a cross-platform test code • How to move beyond automating test code into automating the entire CI process • How to leverage post production monitoring of your mobile apps for continuous quality

1. Continuous Quality As a Key for A 5 Star
Mobile Apps Delivery
@esconfs
#esconfs
Eran Kinsbruner
Perfecto Mobile, Israel

2. About Me
Eran Kinsbruner is a director & mobile technical evangelist at PerfectoMobile,
one of the leading mobile cloud and automation companies. Formerly CTO for
mobile testing and Texas Instruments project manager at Matrix, Eran has
been in testing since 1999 with experience that includes managing teams at
Qulicke & Soffa, Sun Microsystems, General Electric, and NeuStar. The co-
inventor of a test exclusion automated mechanism for mobile J2ME testing at
Sun Microsystems, Eran has experience in the mobile testing world. You can
find Eran on Facebook, Twitter @ek121268, LinkedIn, and his professional
mobile testing blog at ek121268.wordpress.com.
Contact Me: Erank@perfectomobile.com

3. Agenda
3
Mobile
market
START
Mobile
Testing
Coverage
Mobile Projects
Challenges
Mobile Test
automation
CI & Best
Practices

4. Easily Applied for Browsers …
Mobile Create Challenges …
UI
Integration
Unit Testing

5. UI
Integration
Unit Testing
Performance Security
Mobile Requires Shifting the Plan …
Target
Devices
Target
Env.OS
OS versions
Form factors
CPU & Memory
Manufacturers
Wearable
Networks
Orientation
Events
Device events
Locations
Notifications

6. Common Mobile Project Challenges/Pains
6

7. 7
• Pain:
• Market pressure compressing delivery schedules
• Growing App complexity driving test case growth
• Rapidly evolving test matrix
• Goal:
• Deliver feedback to developers faster
• Complete test coverage within each sprint cycle
• Strategy
• Increase automation
• Adopt continuous integration
The Challenges – Velocity vs. Quality
And
Or

8. What does it take to cover the REAL
end user experience?
v
Device & Platform
Model
OS Version
Screen Size
Form Factors v
Environment
Location
Network
v
Conditions
Signal Strength
Load
Others

9. Q1 2015 – EU5 Index
50% Preview
Supports:
• Sufficient Device Mix
• Legacy, Reference, New
• Allows sufficient OS coverage
• Support mix of Screen Sizes
• Support mix of hardware’s

10. Application name position
Labels
Menu panel
Button images
Text colors
and images
Cross Platform Differences

11. Functional Automation – Mobile Objects
100% accurate native
object rec.
Much faster than OCR
Support all languages
Does not often detect
GUI defects
Imperative for end
user’s experience
Imperative for GUI
glitches
Implement the
WYSIWYG
Slower then object
level analysis
Dependent on 3rd party
Does not support all
languages
Object level analysis
Visual analysis (OCR)
May require
Instrumentation

12. Mobile Test Automation That Works
• Develop for testability
• Object Recognition
– Mix ‘n match methods
• Image recognition
• DOM
• Object id
• XPath
• CSS
• Robust
– Error handling
– Conditional flow
– Reusable functions
– Data driven
Reliable
Repeatable Maintainable
Scalable

13. CI Blockers: Inefficient & Brittle AutomationWhat
• Automated testing takes
too much time to develop
• Constant maintenance
Why
• Advance coding skills
• No cross platform tests
• Lack coding guidelines
How
• Build stable framework
• Use predefined code
functions and existing
product
• Separate test code from
execution mgt

14. CI Blockers: Poor Execution Optimization
What
• Automated testing run
slow
• Too many tests, not
enough time
• Not stable on mobile
devices Why
• Lack of execution
management
• Single threaded
automation
• Device errors
How
• Parallel execution
• Tune the code
• Apply automation
best practices
• Manage tests like code

15. Error handling Remove CI/Automation Blockers
System level control
Application control
Device control

17. Mobile Continuous Integration Recommended Work Flow
Dev Team
Source Control
Code Commit
CI Server
ReportingUnit Tests
Launch Perfecto Job
(Release Package)
Support today:
PM Native &
Selenium
Automated App Deployment
On Real Devices
Functional Testing
Integration Testing
Performance Testing
Continuous Feedback
Test Planning & Test Execution
Bug Reporting

18. Manage the Time Budget …
Build
Frequency
Test
Duration
Test Scope
Device
Coverage
Unit System Smoke Regression Performance
Hourly 20-40 Min ±4
3-4 times a
day
30-60 Min ±6
Nightly 2-7 Hours ±10
Weekend 10-48 Hours ±20

19. Velocity! Quality!
Strategies for
achieving 5 stars

20. 1. Assure device coverage

21. Real
Devices
Optimal
mix to
cover
market
Max
Elasticity
My app analytics Mobile Market
My Space
App Control
System Control
Device control
Network Control
Burst up/down
Always available
1. Assure device/OS coverage

22. April OCTOBERJUNE JULY AUGUST SEPTEMBER
Sony Xperia
Z4 Tablet
New iPhones & iPads
Samsung Note 5
Motorola Moto X (New)
iOS 9 Beta
Windows 10
for Phones
DEVICES
PLATFORM
Mobile Market Calendar 2015
Android M ?
MAY
Apple Watch Release
Samsung Galaxy S6
Samsung Galaxy S6 Edge
LG G4
Nokia Lumia 640 XL
ZTE Blade S6 Plus
Huawei P8
Lenovo A6000 Plus
New trending devices,
First Samsung’s to run
Android L OTS
You can already start Beta
testing in June!
Seek for Emulators of
iPhone 6S and more close to
the release date!
Seek for early Beta through
Android Studio
New nexus devices
Perfecto Mobile’s Q2
Index Release!!

23. 2. Built an adaptive test plan
 Update your test plan
according to customer
feedback
 Automate to run regularly

24. Mobile Ready
Automation
Fast Impact
utilizing existing
knowledge
Test on real
devices
Sustainable –
doesn’t break
CI Ready
Device
Agnostic, any
app type
Unattended
3. Mobile-Ready
Automation is Key
Become a Jedi
Master you will

25. Does your Lab come with
“Environment Controls”?
Are you testing “in the
Wild”?
4. Test in Real-World Conditions

26. Build Your Mobile App Test Plan Properly
Functional
• Validation of Functionality
• Smoke / Regressions Testing
• Offline access testing
• Negative Testing
Non Functional
• Network Strength / Outage /
Recovery
• Different NW Types
• Peripheral Testing
• Services
Interoperability (IOP)
• Voice / SMS interrupts
• Notifications
• Battery /Cable Removal
Memory Leak
• Memory Usage
• Memory Leaks
• Garbage Collection
Installation Testing
• New App Install
• Uninstall and Reinstall
• Upgrade testing
Language Testing
• Validation for Locales (French &
English)
• Images and Text
• Currencies, time zones etc.
• Context
Performance Testing
• CPU Usage testing
• Network Usage
• Page Render time or activity
Render time
Security Testing
• OWASP Vulnerabilities
• Sensitive data (App/Device)
• Dynamic Testing
• Static Code Analysis
• Data Encryption
Usability Testing
• User Experience
• Competitive Analysis
• Expert Review

27. Eran Kinsbruner
Dir, Technical Evangelist
Perfecto Mobile
Erank@perfectomobile.com

28. Get 15% off using Discount
Code:
MWS15
(until June 30th)