SlideShare a Scribd company logo
!
!
W4#
Session!
6/24/2015! !
10:15!AM!
!
!
!
!
“Testing#Mobile#App#Performance”##
!
Presented#by:#
#
Brad#Stoner#
Neotys#
#
#
#
#
#
Brought#to#you#by:#
#
#
#
#
#
#
340!Corporate!Way,!Suite!300,!Orange!Park,!FL!32073!
888D268D8770!E!904D278D0524!E!sqeinfo@sqe.com!E!www.sqe.com!
!
!
!
!
!
!!!!! !
!!!!
Brad Stoner is a senior performance engineer at Neotys. Previously, he managed the load
and performance team at H&R Block where he spent seven years leading his five-person team
in the pursuit of improved application performance and quality. Brad and his team managed
the performance testing process for more than fifty projects annually. He founded Sandbreak
Digital Solutions, a consulting company specializing in web application performance testing,
web page optimization, front end optimization, capacity testing, infrastructure validation, and
cloud testing. In his twelve years of IT experience, Brad has held roles from systems
engineering to operations management.
Brad Stoner
Neotys
www.neotys.com© 2015 Neotys. All Rights Reserved.
Ensuring the Performance of Mobile Apps
On Every Device and Network…
StarCANADA 2015 Brad Stoner, Neotys USA
www.neotys.com© 2015 Neotys. All Rights Reserved.
BIOGRAPHY
NEOTYS USA
Senior Performance Engineer
H&R Block
IT Manager – Load and Performance Team
Sandbreak Digital Solutions LLC
Former Founder/Principal Consultant
Webinars on neotys.com
www.neotys.com© 2015 Neotys. All Rights Reserved.
OBJECTIVES
Review of the Problem
Why is performance testing necessary?
How does it apply mobile?
Types of Mobile Testing
How they relate to performance
Mobile Testing Challenges
Overview of Performance Testing
Mobile specifics
Suggested Approaches
www.neotys.com© 2015 Neotys. All Rights Reserved.
PERFORMANCE MATTERS! From a Nice-to-Have to a MUST-Have
1 Second Could Cost
Amazon $1.6 Billion In Sales
US Online Businesses =
$1.7 trillion in 2013
100K$ TV ADS WEBSITE CRASHES!
The Obamacare Website
Failed For The Same Reason
The Soviet Union Did
Failures
Obamacare Apps
Bank of America
Facebook IPO
Poor Performance =
Lost Revenues
Meanwhile…
Entertainment, video &
mobile usage are exploding!
Website = Brand
Secure Marketing Investments
www.neotys.com© 2015 Neotys. All Rights Reserved.
MARKET DRIVERS: BUSINESS
Web and Mobile Applications are Business Critical
Organizations compete via the internet
Software and Brand are synonymous
Technology usage growing
Failure and poor performance are catastrophic and costly
Time To Market pressure increasing
End user expectations are rising
Investments in Software Quality are Up
Load testing is a must-have
Time pressure on dev & tests teams
18%
23%
28%
CY 2012 CY 2013 CY 2014
% IT BUDGET FOR APP QUALITY
Source: World Quality Report 2013-2014
0%
40%
20%
www.neotys.com© 2015 Neotys. All Rights Reserved.
MOBILE IS GROWING!
Unprecedented User
Expectations
Exploding Expansion
of Devices
• 1.2 billion new smartphones
over the next 5 years
• Smartphones over
40% of all handset
shipments
• 85% of users
expect mobile app
to load at least as
quickly as desktop
sites
• 25% of users abandoned a
mobile app after 3 second
delay
www.neotys.com© 2015 Neotys. All Rights Reserved.
PERFORMANCE PERSPECTIVE
How can we tell where the problem is?
Combinations of functional and performance testing but…
“End User Experience”
• Time between a client initiating a transaction and the
time the result is rendered back in the device
• Stand-alone application - EUE is determined completely
on the device
• “Other” application: - EUE can be measured on device
but is sum of:
- Round trip time from device to server and back
- Device-side rendering
Does it matter where the problem is?
(device, server, network, etc.)
• To the business…maybe not.
• To you…maybe (probably) yes!!
www.neotys.com© 2015 Neotys. All Rights Reserved.
CHALLENGES/CONSIDERATIONS – DEVICE TYPE
DEVICE CHALLENGES
Operating Systems
App Technologies
Screen Sizes
iOS Android Windows
Design of what content is
above/below the fold
When that content
gets delivered
Different content for
different devices
Web SocketHTML 5 Streaming
Video/Audio
Google SPDY
www.neotys.com© 2015 Neotys. All Rights Reserved.
MOBILE APPLICATION TYPES
WEB
Browser-Based
NATIVE
Client Downloaded From
App Store or Google Play
• Usually connected
to the internet
- If not, “Standalone”
• Generally faster (client-side)
than browser app
• Often a different application
site than if accessed via
desktop browser
HYBRID
Combo of Web
and Native
• Usually
wrappers for
web clients
www.neotys.com© 2015 Neotys. All Rights Reserved.
TYPES OF MOBILE TESTING
PLATFORM TESTING
LOAD TESTING
FUNCTIONAL
TESTING
FEATURES
DEVICE
PERFORMANCE
• Does the app do what
it’s expected to do?
How does the app work on
this device?
• Supported?
• CPU, Battery consumption
How fast is it displayed?
• Speed of use
• Can 1000 users use this app?
• What is the user experience
under load?
• What about 500 users on 3G
and 500 on 4G LTE?
www.neotys.com© 2015 Neotys. All Rights Reserved.
TESTING STANDALONE APPLICATIONS
Functional / Platform Testing Will Suffice
• All logic on the device itself
• No back-end server
Metrics to Correlate
• Custom Timers
• Device “Vitals” (CPU, Battery, Throughput)
Automated or Manual?
• Hard to do it manually for all different configurations!!
• How to get Vitals manually
Solution Requirements
• Consider an automation tool / vendor that allows:
- ON DEVICE testing
- Multi-device testing
- Vitals
www.neotys.com© 2015 Neotys. All Rights Reserved.
TESTING NATIVE, WEB, OR HYBRID APPLICATIONS
Remember: EUE = device-side rendering + server-side performance
How do we test each?
DEVICE-SIDE
Performance impacted by concurrent user load
Similar to browser-based apps but with nuances
SERVER-SIDE
Essentially a stand-alone application
Measurements made similar to prior slide
However performance impacted by server
Have To Separate Device-Side From Server-Side
How do we correlate the two?
www.neotys.com© 2015 Neotys. All Rights Reserved.
LOAD AND PERFORMANCE TESTING – OVERVIEW
What?
• The process of putting demand on a system or
device and measuring its response
• Non-functional
• Bridges, roller coasters, elevators…
• Software applications: Generating network traffic
• (Impossible to do manually!!)
Why?
• To avoid:
o Loss of revenue
o Customer “migration”
o Decreased productivity
• To understand an application’s
o Scalability
o Stability
www.neotys.com© 2015 Neotys. All Rights Reserved.
LOAD / PERFORMANCE TESTING SOLUTIONS
1
2
3
4
HOW THEY WORK
Record Usage Scenarios
Measure Response Times
Generate the Network Load
Analyze Results
- Network traffic between client and server
- “Models” of real-world activity
- Round-trip times from client to server and back
www.neotys.com© 2015 Neotys. All Rights Reserved.
LOAD / PERFORMANCE TESTING SOLUTIONS
Cloud Load Generators
Controller
On-Premise Load Generators
Firewall
Application
Server
Database
Server
LOAD TESTING SOLUTION APPLICATION INFRASTRUCTURE
LOAD
www.neotys.com© 2015 Neotys. All Rights Reserved.
LOAD / PERFORMANCE TESTING SOLUTIONS
Cloud Load Generators
Controller
On-Premise Load Generators
Firewall
Application
Server
Database
Server
LOAD TESTING SOLUTION APPLICATION INFRASTRUCTURE
LOAD
Monitoring
Engine
Monitoring
www.neotys.com© 2015 Neotys. All Rights Reserved.
DEVICE CHALLENGES FOR PERFORMANCE TESTING
Recording the Application
• Type of app
- Native apps
- Web apps through mobile browsers
- Hybrid apps
• Technologies used in the app
- HTML 5
- Web Socket
- Streaming Video/audio
- Google SPDY
Replay scenarios for specific devices and browsers
• Different content is delivered depending on the specific device or browser
• Different browsers and devices have different numbers of parallel connections that
affect performance
www.neotys.com© 2015 Neotys. All Rights Reserved.
CHALLENGES/CONSIDERATIONS – NETWORK CONDITIONS
Bandwidth
• How are users with different download and
upload capacities experiencing performance on
the app?
Latency
• How are network delays affecting the
performance of the app?
Packet Loss
• How are users experiencing the app when
switching cell towers, traveling, losing signal, etc.
SHOULD WE EMULATE THESE CONDITIONS?
YES!!!
www.neotys.com© 2015 Neotys. All Rights Reserved.
LATENCY: THE REAL BOTTLENECK
Bandwidth Isn’t Everything
• Benefits of increasing bandwidth over HTTP
caps out around 5 Mbps
• Increasing bandwidth from 5 to 10 Mbps
decreases page load time by only about 5%
Latency is Worse for Mobile
• In 2012, the average round trip time to
Google was ~100ms worldwide and ~50-
60ms within the US
• On mobile, RTT is around 100-1000ms
• If a device has been idle and requires the
radio to turn on first, add 1000-2000ms to
that time
Benefit of Adding an Extra Mbps
Bandwidth
PercentImprovement
20
0
10
30
40
www.neotys.com© 2015 Neotys. All Rights Reserved.
PACKET LOSS INFLUENCE
Packet Loss
Consequences
Packet retransmission
End user experience degraded
because packet loss increases jitter
Errors could occur if the packet loss
is in pure data transmission
Packet Loss
Inevitable
Mobility of the user
Quickly switching
to the next cell tower
Temporary lose the signal due to
congestion of the current cell
www.neotys.com© 2015 Neotys. All Rights Reserved.
CHALLENGES/CONSIDERATIONS - HIGH, DISTRIBUTED LOAD
High Number of Users
• Mobile applications usually have large user bases
• High user growth creates need for highly scalable load generation infrastructure
Geographical Distributed Load
• Users are accessing apps from remote
locations
• Distance and location greatly affect
performance
Mix of Web & Mobile Users
• Many mobile apps have a “partnered” web app with a
shared backend
• High load on the web app can affect performance on
mobile
TIP: USE CLOUD-BASED LOAD GENERATION
www.neotys.com© 2015 Neotys. All Rights Reserved.
MOBILE PERFORMANCE TESTING – SOLUTION REQUIREMENTS
Record from Any Device
• From real devices and emulators
• Native or browser-based applications
Simulate Any Device or Browser
• Ensures the right content is delivered for
that device or browser
• Simulates parallel connections for each
specific device and browser
Emulate Realistic Network Conditions
• Bandwidth
• Latency
• Packet Loss
Generate Load from the Cloud
• Geographic realism
• Test full delivery chain
Load
Command
3GH+ 4G WiFi
APPMonitor
Network
Emulation
& Device
Simulation
Cloud Load Generators On-premise Load Generators
Database ServerApp Server Web Server
www.neotys.com© 2015 Neotys. All Rights Reserved.
WHAT HAPPENED TO EUE?
The Problem:
• Performance solutions don’t provide on-device measures
• Functional/platform solutions don’t separate round-trip or server-side measures
HOLY GRAIL
SOLUTION
COMBINE THE TWO!
(Manual stop watch approach not feasible)
Automated
Functional/Platform
Automated
Performance
On-device measures
Device Vitals
“Round-Trip” measures
INTEGRATED FOR RESULTS CORRELATION
www.neotys.com© 2015 Neotys. All Rights Reserved.
LOAD + FUNCTIONAL TESTING
• Automatically trigger
functional test with the
start of a load test
• See real device rendering
times and vitals while the
backend of the
application is under load
• Results correlated for
“end-to-end”
Load
3GH+
4G WiFi
APPMonitor
Cloud LGsOn-premise LGs
Database ServerApp Server Web Server
Controller
Real
Devices &
Networks
Functional
Simulated
Devices
w/
Network
Emulation
Real Device Metrics
4G
3GH+
Real Device Metrics Under Load
www.neotys.com© 2015 Neotys. All Rights Reserved.
COMPLETE MOBILE EUE
Real Device Users
Virtual Users Load
Real Device
Testing Tool
Load
Testing Tool Application
Server
Load End User
Experience
Round-Trip
Time
00:00:00 00:00:30 00:01:00 00:01:30
www.neotys.com© 2015 Neotys. All Rights Reserved.
SUMMARY
Mobile devices are everywhere. Application Performance is critical.
Performance usually means “End User Experience”
Test with a combo of
AUTOMATED & INTEGRATED
solutions
MEASURE & CORRELATE
www.neotys.com© 2015 Neotys. All Rights Reserved.
QUESTIONS?
Feel free to contact me:
Email: brad.stoner@neotys.com

More Related Content

What's hot

Mobile Application testing
Mobile Application testingMobile Application testing
Mobile Application testing
Mukta Gupta
 
SauceCon 2017: Making Your Mobile App Automatable
SauceCon 2017: Making Your Mobile App AutomatableSauceCon 2017: Making Your Mobile App Automatable
SauceCon 2017: Making Your Mobile App Automatable
Sauce Labs
 
Testing Techniques for Mobile Applications
Testing Techniques for Mobile ApplicationsTesting Techniques for Mobile Applications
Testing Techniques for Mobile Applications
IndicThreads
 
Ensure Your Mobile Channels Generate Revenue on Holiday Peak Days
Ensure Your Mobile Channels Generate Revenue on Holiday Peak DaysEnsure Your Mobile Channels Generate Revenue on Holiday Peak Days
Ensure Your Mobile Channels Generate Revenue on Holiday Peak Days
Bitbar
 
Mobile Apps Delivery - Evolution and Tools
Mobile Apps Delivery - Evolution and ToolsMobile Apps Delivery - Evolution and Tools
Mobile Apps Delivery - Evolution and Tools
Asaf Saar
 
How to Find Vulnerabilities and Bugs in Mobile Applications
How to Find Vulnerabilities and Bugs in Mobile ApplicationsHow to Find Vulnerabilities and Bugs in Mobile Applications
How to Find Vulnerabilities and Bugs in Mobile Applications
Josiah Renaudin
 
Mobile Testing Challenges and Solutions XBOSoft Webinar
Mobile Testing Challenges and Solutions XBOSoft WebinarMobile Testing Challenges and Solutions XBOSoft Webinar
Mobile Testing Challenges and Solutions XBOSoft Webinar
XBOSoft
 
Consigue una App 5 estrellas
Consigue una App 5 estrellasConsigue una App 5 estrellas
Consigue una App 5 estrellas
Globe Testing
 
SauceCon 2017: test.allTheThings(): Digital Edition
SauceCon 2017: test.allTheThings(): Digital EditionSauceCon 2017: test.allTheThings(): Digital Edition
SauceCon 2017: test.allTheThings(): Digital Edition
Sauce Labs
 
10 things you didnt know about appium + whats new in appium 1.5
10 things you didnt know about appium + whats new in appium 1.510 things you didnt know about appium + whats new in appium 1.5
10 things you didnt know about appium + whats new in appium 1.5
Sauce Labs
 
Starting Your DevOps Journey – Practical Tips for Ops
Starting Your DevOps Journey – Practical Tips for OpsStarting Your DevOps Journey – Practical Tips for Ops
Starting Your DevOps Journey – Practical Tips for Ops
Dynatrace
 
An Introduction to Appium Desktop
An Introduction to Appium DesktopAn Introduction to Appium Desktop
An Introduction to Appium Desktop
Sauce Labs
 
Sauce Labs+Applitools - Automated Visual Testing in the Cloud
Sauce Labs+Applitools - Automated Visual Testing in the CloudSauce Labs+Applitools - Automated Visual Testing in the Cloud
Sauce Labs+Applitools - Automated Visual Testing in the Cloud
Sauce Labs
 
Webinar learn how to test any mobile app style from within eclipse using real...
Webinar learn how to test any mobile app style from within eclipse using real...Webinar learn how to test any mobile app style from within eclipse using real...
Webinar learn how to test any mobile app style from within eclipse using real...Perfecto Mobile
 
Ciprian balea automated performance-testing
Ciprian balea   automated performance-testingCiprian balea   automated performance-testing
Ciprian balea automated performance-testingRomania Testing
 
Selenium Camp 2016
Selenium Camp 2016Selenium Camp 2016
Selenium Camp 2016
Dan Cuellar
 
Mobile test automation perfecto star east
Mobile test automation perfecto star eastMobile test automation perfecto star east
Mobile test automation perfecto star east
Perfecto Mobile
 
How Digital Changed the Game... and how to cross platform test for it
How Digital Changed the Game... and how to cross platform test for itHow Digital Changed the Game... and how to cross platform test for it
How Digital Changed the Game... and how to cross platform test for it
Lizzy Guido (she/her)
 
SauceCon 2017: Building a Continuous Delivery Pipeline with Testing in Mind
SauceCon 2017: Building a Continuous Delivery Pipeline with Testing in MindSauceCon 2017: Building a Continuous Delivery Pipeline with Testing in Mind
SauceCon 2017: Building a Continuous Delivery Pipeline with Testing in Mind
Sauce Labs
 
Launch High Performing Mobile Apps with Appurify
Launch High Performing Mobile Apps with AppurifyLaunch High Performing Mobile Apps with Appurify
Launch High Performing Mobile Apps with Appurify
Manish Lachwani
 

What's hot (20)

Mobile Application testing
Mobile Application testingMobile Application testing
Mobile Application testing
 
SauceCon 2017: Making Your Mobile App Automatable
SauceCon 2017: Making Your Mobile App AutomatableSauceCon 2017: Making Your Mobile App Automatable
SauceCon 2017: Making Your Mobile App Automatable
 
Testing Techniques for Mobile Applications
Testing Techniques for Mobile ApplicationsTesting Techniques for Mobile Applications
Testing Techniques for Mobile Applications
 
Ensure Your Mobile Channels Generate Revenue on Holiday Peak Days
Ensure Your Mobile Channels Generate Revenue on Holiday Peak DaysEnsure Your Mobile Channels Generate Revenue on Holiday Peak Days
Ensure Your Mobile Channels Generate Revenue on Holiday Peak Days
 
Mobile Apps Delivery - Evolution and Tools
Mobile Apps Delivery - Evolution and ToolsMobile Apps Delivery - Evolution and Tools
Mobile Apps Delivery - Evolution and Tools
 
How to Find Vulnerabilities and Bugs in Mobile Applications
How to Find Vulnerabilities and Bugs in Mobile ApplicationsHow to Find Vulnerabilities and Bugs in Mobile Applications
How to Find Vulnerabilities and Bugs in Mobile Applications
 
Mobile Testing Challenges and Solutions XBOSoft Webinar
Mobile Testing Challenges and Solutions XBOSoft WebinarMobile Testing Challenges and Solutions XBOSoft Webinar
Mobile Testing Challenges and Solutions XBOSoft Webinar
 
Consigue una App 5 estrellas
Consigue una App 5 estrellasConsigue una App 5 estrellas
Consigue una App 5 estrellas
 
SauceCon 2017: test.allTheThings(): Digital Edition
SauceCon 2017: test.allTheThings(): Digital EditionSauceCon 2017: test.allTheThings(): Digital Edition
SauceCon 2017: test.allTheThings(): Digital Edition
 
10 things you didnt know about appium + whats new in appium 1.5
10 things you didnt know about appium + whats new in appium 1.510 things you didnt know about appium + whats new in appium 1.5
10 things you didnt know about appium + whats new in appium 1.5
 
Starting Your DevOps Journey – Practical Tips for Ops
Starting Your DevOps Journey – Practical Tips for OpsStarting Your DevOps Journey – Practical Tips for Ops
Starting Your DevOps Journey – Practical Tips for Ops
 
An Introduction to Appium Desktop
An Introduction to Appium DesktopAn Introduction to Appium Desktop
An Introduction to Appium Desktop
 
Sauce Labs+Applitools - Automated Visual Testing in the Cloud
Sauce Labs+Applitools - Automated Visual Testing in the CloudSauce Labs+Applitools - Automated Visual Testing in the Cloud
Sauce Labs+Applitools - Automated Visual Testing in the Cloud
 
Webinar learn how to test any mobile app style from within eclipse using real...
Webinar learn how to test any mobile app style from within eclipse using real...Webinar learn how to test any mobile app style from within eclipse using real...
Webinar learn how to test any mobile app style from within eclipse using real...
 
Ciprian balea automated performance-testing
Ciprian balea   automated performance-testingCiprian balea   automated performance-testing
Ciprian balea automated performance-testing
 
Selenium Camp 2016
Selenium Camp 2016Selenium Camp 2016
Selenium Camp 2016
 
Mobile test automation perfecto star east
Mobile test automation perfecto star eastMobile test automation perfecto star east
Mobile test automation perfecto star east
 
How Digital Changed the Game... and how to cross platform test for it
How Digital Changed the Game... and how to cross platform test for itHow Digital Changed the Game... and how to cross platform test for it
How Digital Changed the Game... and how to cross platform test for it
 
SauceCon 2017: Building a Continuous Delivery Pipeline with Testing in Mind
SauceCon 2017: Building a Continuous Delivery Pipeline with Testing in MindSauceCon 2017: Building a Continuous Delivery Pipeline with Testing in Mind
SauceCon 2017: Building a Continuous Delivery Pipeline with Testing in Mind
 
Launch High Performing Mobile Apps with Appurify
Launch High Performing Mobile Apps with AppurifyLaunch High Performing Mobile Apps with Appurify
Launch High Performing Mobile Apps with Appurify
 

Similar to Testing Mobile App Performance

Innovate 2013 session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3Innovate 2013   session 1243 mobile testing.v3
Innovate 2013 session 1243 mobile testing.v3
Leigh Williamson
 
Optimizing Global Application Delivery Webinar
Optimizing Global Application Delivery WebinarOptimizing Global Application Delivery Webinar
Optimizing Global Application Delivery Webinar
Ginny Au
 
Webinar effective mobile performance testing using real devices
Webinar effective mobile performance testing using real devicesWebinar effective mobile performance testing using real devices
Webinar effective mobile performance testing using real devicesPerfecto Mobile
 
Automating things you didn't know you could automate
Automating things you didn't know you could automateAutomating things you didn't know you could automate
Automating things you didn't know you could automate
Lizzy Guido (she/her)
 
Software QS Tag White Paper
Software QS Tag White PaperSoftware QS Tag White Paper
Software QS Tag White Paper
Devendra Singh
 
[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...
[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...
[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...
NITHIN S.S
 
Agile Network India | New to Automation? Learn what, how & when | TRIPTI MITTAL
Agile Network India | New to Automation? Learn what, how & when | TRIPTI MITTALAgile Network India | New to Automation? Learn what, how & when | TRIPTI MITTAL
Agile Network India | New to Automation? Learn what, how & when | TRIPTI MITTAL
AgileNetwork
 
The Art of Mobile Performance (Mobiconf 2015)
The Art of Mobile Performance (Mobiconf 2015)The Art of Mobile Performance (Mobiconf 2015)
The Art of Mobile Performance (Mobiconf 2015)
David Low
 
Mobile Performance Testing Approaches and Challenges
Mobile Performance Testing Approaches and ChallengesMobile Performance Testing Approaches and Challenges
Mobile Performance Testing Approaches and Challenges
Nous Infosystems
 
IBM Digital Experience 2015 - APPLICATION MODERNIZATION IN THE DIGITAL EXPERI...
IBM Digital Experience 2015 - APPLICATION MODERNIZATION IN THE DIGITAL EXPERI...IBM Digital Experience 2015 - APPLICATION MODERNIZATION IN THE DIGITAL EXPERI...
IBM Digital Experience 2015 - APPLICATION MODERNIZATION IN THE DIGITAL EXPERI...
John Head
 
Modern Load Testing: Move Your Load Testing from the Past to the Present
Modern Load Testing: Move Your Load Testing from the Past to the PresentModern Load Testing: Move Your Load Testing from the Past to the Present
Modern Load Testing: Move Your Load Testing from the Past to the Present
SOASTA
 
Modern Load Testing: Move Your Load Testing from the Past to the Present
Modern Load Testing: Move Your Load Testing from the Past to the PresentModern Load Testing: Move Your Load Testing from the Past to the Present
Modern Load Testing: Move Your Load Testing from the Past to the Present
SOASTA
 
The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...
The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...
The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...
Bitbar
 
Mobile Automation: Lessons From The Trenches
Mobile Automation: Lessons From The TrenchesMobile Automation: Lessons From The Trenches
Mobile Automation: Lessons From The Trenches
Ashok Karania
 
How to stop fingerpointing when your application is down
How to stop fingerpointing when your application is downHow to stop fingerpointing when your application is down
How to stop fingerpointing when your application is downCompuware ASEAN
 
Experitest & Wipro Co-Webinar
Experitest & Wipro Co-Webinar Experitest & Wipro Co-Webinar
Experitest & Wipro Co-Webinar
Experitest
 
Think tank event mobile app testing v1.3
Think tank event   mobile app testing v1.3Think tank event   mobile app testing v1.3
Think tank event mobile app testing v1.3
Samer Desouky
 
Bringing Your Web Apps to IBM Digital Experience
Bringing Your Web Apps to IBM Digital ExperienceBringing Your Web Apps to IBM Digital Experience
Bringing Your Web Apps to IBM Digital Experience
John Head
 
Maximize the Benefits from Your Test Automation Investment
Maximize the Benefits from Your Test Automation InvestmentMaximize the Benefits from Your Test Automation Investment
Maximize the Benefits from Your Test Automation Investment
Bitbar
 
Using Automation to Meet Demands for Performance and Quality
Using Automation to Meet Demands for Performance and QualityUsing Automation to Meet Demands for Performance and Quality
Using Automation to Meet Demands for Performance and Quality
Neotys
 

Similar to Testing Mobile App Performance (20)

Innovate 2013 session 1243 mobile testing.v3
Innovate 2013   session 1243 mobile testing.v3Innovate 2013   session 1243 mobile testing.v3
Innovate 2013 session 1243 mobile testing.v3
 
Optimizing Global Application Delivery Webinar
Optimizing Global Application Delivery WebinarOptimizing Global Application Delivery Webinar
Optimizing Global Application Delivery Webinar
 
Webinar effective mobile performance testing using real devices
Webinar effective mobile performance testing using real devicesWebinar effective mobile performance testing using real devices
Webinar effective mobile performance testing using real devices
 
Automating things you didn't know you could automate
Automating things you didn't know you could automateAutomating things you didn't know you could automate
Automating things you didn't know you could automate
 
Software QS Tag White Paper
Software QS Tag White PaperSoftware QS Tag White Paper
Software QS Tag White Paper
 
[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...
[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...
[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...
 
Agile Network India | New to Automation? Learn what, how & when | TRIPTI MITTAL
Agile Network India | New to Automation? Learn what, how & when | TRIPTI MITTALAgile Network India | New to Automation? Learn what, how & when | TRIPTI MITTAL
Agile Network India | New to Automation? Learn what, how & when | TRIPTI MITTAL
 
The Art of Mobile Performance (Mobiconf 2015)
The Art of Mobile Performance (Mobiconf 2015)The Art of Mobile Performance (Mobiconf 2015)
The Art of Mobile Performance (Mobiconf 2015)
 
Mobile Performance Testing Approaches and Challenges
Mobile Performance Testing Approaches and ChallengesMobile Performance Testing Approaches and Challenges
Mobile Performance Testing Approaches and Challenges
 
IBM Digital Experience 2015 - APPLICATION MODERNIZATION IN THE DIGITAL EXPERI...
IBM Digital Experience 2015 - APPLICATION MODERNIZATION IN THE DIGITAL EXPERI...IBM Digital Experience 2015 - APPLICATION MODERNIZATION IN THE DIGITAL EXPERI...
IBM Digital Experience 2015 - APPLICATION MODERNIZATION IN THE DIGITAL EXPERI...
 
Modern Load Testing: Move Your Load Testing from the Past to the Present
Modern Load Testing: Move Your Load Testing from the Past to the PresentModern Load Testing: Move Your Load Testing from the Past to the Present
Modern Load Testing: Move Your Load Testing from the Past to the Present
 
Modern Load Testing: Move Your Load Testing from the Past to the Present
Modern Load Testing: Move Your Load Testing from the Past to the PresentModern Load Testing: Move Your Load Testing from the Past to the Present
Modern Load Testing: Move Your Load Testing from the Past to the Present
 
The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...
The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...
The Best of Both Worlds - Combining Performance and Functional Mobile App Tes...
 
Mobile Automation: Lessons From The Trenches
Mobile Automation: Lessons From The TrenchesMobile Automation: Lessons From The Trenches
Mobile Automation: Lessons From The Trenches
 
How to stop fingerpointing when your application is down
How to stop fingerpointing when your application is downHow to stop fingerpointing when your application is down
How to stop fingerpointing when your application is down
 
Experitest & Wipro Co-Webinar
Experitest & Wipro Co-Webinar Experitest & Wipro Co-Webinar
Experitest & Wipro Co-Webinar
 
Think tank event mobile app testing v1.3
Think tank event   mobile app testing v1.3Think tank event   mobile app testing v1.3
Think tank event mobile app testing v1.3
 
Bringing Your Web Apps to IBM Digital Experience
Bringing Your Web Apps to IBM Digital ExperienceBringing Your Web Apps to IBM Digital Experience
Bringing Your Web Apps to IBM Digital Experience
 
Maximize the Benefits from Your Test Automation Investment
Maximize the Benefits from Your Test Automation InvestmentMaximize the Benefits from Your Test Automation Investment
Maximize the Benefits from Your Test Automation Investment
 
Using Automation to Meet Demands for Performance and Quality
Using Automation to Meet Demands for Performance and QualityUsing Automation to Meet Demands for Performance and Quality
Using Automation to Meet Demands for Performance and Quality
 

More from TechWell

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and Recovering
TechWell
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization
TechWell
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build Architecture
TechWell
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good Start
TechWell
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test Strategy
TechWell
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for Success
TechWell
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlow
TechWell
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your Sanity
TechWell
 
Ma 15
Ma 15Ma 15
Ma 15
TechWell
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps Strategy
TechWell
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOps
TechWell
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—Leadership
TechWell
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile Teams
TechWell
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile Game
TechWell
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
TechWell
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps Implementation
TechWell
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery Process
TechWell
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to Automate
TechWell
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for Success
TechWell
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile Transformation
TechWell
 

More from TechWell (20)

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and Recovering
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build Architecture
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good Start
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test Strategy
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for Success
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlow
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your Sanity
 
Ma 15
Ma 15Ma 15
Ma 15
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps Strategy
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOps
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—Leadership
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile Teams
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile Game
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps Implementation
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery Process
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to Automate
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for Success
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile Transformation
 

Recently uploaded

top nidhi software solution freedownload
top nidhi software solution freedownloadtop nidhi software solution freedownload
top nidhi software solution freedownload
vrstrong314
 
Into the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdfInto the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdf
Ortus Solutions, Corp
 
Quarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden ExtensionsQuarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden Extensions
Max Andersen
 
Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024
Globus
 
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Shahin Sheidaei
 
GlobusWorld 2024 Opening Keynote session
GlobusWorld 2024 Opening Keynote sessionGlobusWorld 2024 Opening Keynote session
GlobusWorld 2024 Opening Keynote session
Globus
 
Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604
Fermin Galan
 
Large Language Models and the End of Programming
Large Language Models and the End of ProgrammingLarge Language Models and the End of Programming
Large Language Models and the End of Programming
Matt Welsh
 
How Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptxHow Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptx
wottaspaceseo
 
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
Juraj Vysvader
 
A Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of PassageA Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of Passage
Philip Schwarz
 
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Globus
 
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdfDominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
AMB-Review
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Globus
 
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfEnhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
Jay Das
 
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteAI Pilot Review: The World’s First Virtual Assistant Marketing Suite
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
Google
 
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
Tier1 app
 
Accelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with PlatformlessAccelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with Platformless
WSO2
 
Graphic Design Crash Course for beginners
Graphic Design Crash Course for beginnersGraphic Design Crash Course for beginners
Graphic Design Crash Course for beginners
e20449
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
Globus
 

Recently uploaded (20)

top nidhi software solution freedownload
top nidhi software solution freedownloadtop nidhi software solution freedownload
top nidhi software solution freedownload
 
Into the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdfInto the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdf
 
Quarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden ExtensionsQuarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden Extensions
 
Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024
 
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
 
GlobusWorld 2024 Opening Keynote session
GlobusWorld 2024 Opening Keynote sessionGlobusWorld 2024 Opening Keynote session
GlobusWorld 2024 Opening Keynote session
 
Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604Orion Context Broker introduction 20240604
Orion Context Broker introduction 20240604
 
Large Language Models and the End of Programming
Large Language Models and the End of ProgrammingLarge Language Models and the End of Programming
Large Language Models and the End of Programming
 
How Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptxHow Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptx
 
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
 
A Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of PassageA Sighting of filterA in Typelevel Rite of Passage
A Sighting of filterA in Typelevel Rite of Passage
 
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
 
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdfDominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
 
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfEnhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdf
 
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteAI Pilot Review: The World’s First Virtual Assistant Marketing Suite
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
 
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
 
Accelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with PlatformlessAccelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with Platformless
 
Graphic Design Crash Course for beginners
Graphic Design Crash Course for beginnersGraphic Design Crash Course for beginners
Graphic Design Crash Course for beginners
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
 

Testing Mobile App Performance

  • 2. ! ! !!!!! ! !!!! Brad Stoner is a senior performance engineer at Neotys. Previously, he managed the load and performance team at H&R Block where he spent seven years leading his five-person team in the pursuit of improved application performance and quality. Brad and his team managed the performance testing process for more than fifty projects annually. He founded Sandbreak Digital Solutions, a consulting company specializing in web application performance testing, web page optimization, front end optimization, capacity testing, infrastructure validation, and cloud testing. In his twelve years of IT experience, Brad has held roles from systems engineering to operations management. Brad Stoner Neotys
  • 3. www.neotys.com© 2015 Neotys. All Rights Reserved. Ensuring the Performance of Mobile Apps On Every Device and Network… StarCANADA 2015 Brad Stoner, Neotys USA
  • 4. www.neotys.com© 2015 Neotys. All Rights Reserved. BIOGRAPHY NEOTYS USA Senior Performance Engineer H&R Block IT Manager – Load and Performance Team Sandbreak Digital Solutions LLC Former Founder/Principal Consultant Webinars on neotys.com
  • 5. www.neotys.com© 2015 Neotys. All Rights Reserved. OBJECTIVES Review of the Problem Why is performance testing necessary? How does it apply mobile? Types of Mobile Testing How they relate to performance Mobile Testing Challenges Overview of Performance Testing Mobile specifics Suggested Approaches
  • 6. www.neotys.com© 2015 Neotys. All Rights Reserved. PERFORMANCE MATTERS! From a Nice-to-Have to a MUST-Have 1 Second Could Cost Amazon $1.6 Billion In Sales US Online Businesses = $1.7 trillion in 2013 100K$ TV ADS WEBSITE CRASHES! The Obamacare Website Failed For The Same Reason The Soviet Union Did Failures Obamacare Apps Bank of America Facebook IPO Poor Performance = Lost Revenues Meanwhile… Entertainment, video & mobile usage are exploding! Website = Brand Secure Marketing Investments
  • 7. www.neotys.com© 2015 Neotys. All Rights Reserved. MARKET DRIVERS: BUSINESS Web and Mobile Applications are Business Critical Organizations compete via the internet Software and Brand are synonymous Technology usage growing Failure and poor performance are catastrophic and costly Time To Market pressure increasing End user expectations are rising Investments in Software Quality are Up Load testing is a must-have Time pressure on dev & tests teams 18% 23% 28% CY 2012 CY 2013 CY 2014 % IT BUDGET FOR APP QUALITY Source: World Quality Report 2013-2014 0% 40% 20%
  • 8. www.neotys.com© 2015 Neotys. All Rights Reserved. MOBILE IS GROWING! Unprecedented User Expectations Exploding Expansion of Devices • 1.2 billion new smartphones over the next 5 years • Smartphones over 40% of all handset shipments • 85% of users expect mobile app to load at least as quickly as desktop sites • 25% of users abandoned a mobile app after 3 second delay
  • 9. www.neotys.com© 2015 Neotys. All Rights Reserved. PERFORMANCE PERSPECTIVE How can we tell where the problem is? Combinations of functional and performance testing but… “End User Experience” • Time between a client initiating a transaction and the time the result is rendered back in the device • Stand-alone application - EUE is determined completely on the device • “Other” application: - EUE can be measured on device but is sum of: - Round trip time from device to server and back - Device-side rendering Does it matter where the problem is? (device, server, network, etc.) • To the business…maybe not. • To you…maybe (probably) yes!!
  • 10. www.neotys.com© 2015 Neotys. All Rights Reserved. CHALLENGES/CONSIDERATIONS – DEVICE TYPE DEVICE CHALLENGES Operating Systems App Technologies Screen Sizes iOS Android Windows Design of what content is above/below the fold When that content gets delivered Different content for different devices Web SocketHTML 5 Streaming Video/Audio Google SPDY
  • 11. www.neotys.com© 2015 Neotys. All Rights Reserved. MOBILE APPLICATION TYPES WEB Browser-Based NATIVE Client Downloaded From App Store or Google Play • Usually connected to the internet - If not, “Standalone” • Generally faster (client-side) than browser app • Often a different application site than if accessed via desktop browser HYBRID Combo of Web and Native • Usually wrappers for web clients
  • 12. www.neotys.com© 2015 Neotys. All Rights Reserved. TYPES OF MOBILE TESTING PLATFORM TESTING LOAD TESTING FUNCTIONAL TESTING FEATURES DEVICE PERFORMANCE • Does the app do what it’s expected to do? How does the app work on this device? • Supported? • CPU, Battery consumption How fast is it displayed? • Speed of use • Can 1000 users use this app? • What is the user experience under load? • What about 500 users on 3G and 500 on 4G LTE?
  • 13. www.neotys.com© 2015 Neotys. All Rights Reserved. TESTING STANDALONE APPLICATIONS Functional / Platform Testing Will Suffice • All logic on the device itself • No back-end server Metrics to Correlate • Custom Timers • Device “Vitals” (CPU, Battery, Throughput) Automated or Manual? • Hard to do it manually for all different configurations!! • How to get Vitals manually Solution Requirements • Consider an automation tool / vendor that allows: - ON DEVICE testing - Multi-device testing - Vitals
  • 14. www.neotys.com© 2015 Neotys. All Rights Reserved. TESTING NATIVE, WEB, OR HYBRID APPLICATIONS Remember: EUE = device-side rendering + server-side performance How do we test each? DEVICE-SIDE Performance impacted by concurrent user load Similar to browser-based apps but with nuances SERVER-SIDE Essentially a stand-alone application Measurements made similar to prior slide However performance impacted by server Have To Separate Device-Side From Server-Side How do we correlate the two?
  • 15. www.neotys.com© 2015 Neotys. All Rights Reserved. LOAD AND PERFORMANCE TESTING – OVERVIEW What? • The process of putting demand on a system or device and measuring its response • Non-functional • Bridges, roller coasters, elevators… • Software applications: Generating network traffic • (Impossible to do manually!!) Why? • To avoid: o Loss of revenue o Customer “migration” o Decreased productivity • To understand an application’s o Scalability o Stability
  • 16. www.neotys.com© 2015 Neotys. All Rights Reserved. LOAD / PERFORMANCE TESTING SOLUTIONS 1 2 3 4 HOW THEY WORK Record Usage Scenarios Measure Response Times Generate the Network Load Analyze Results - Network traffic between client and server - “Models” of real-world activity - Round-trip times from client to server and back
  • 17. www.neotys.com© 2015 Neotys. All Rights Reserved. LOAD / PERFORMANCE TESTING SOLUTIONS Cloud Load Generators Controller On-Premise Load Generators Firewall Application Server Database Server LOAD TESTING SOLUTION APPLICATION INFRASTRUCTURE LOAD
  • 18. www.neotys.com© 2015 Neotys. All Rights Reserved. LOAD / PERFORMANCE TESTING SOLUTIONS Cloud Load Generators Controller On-Premise Load Generators Firewall Application Server Database Server LOAD TESTING SOLUTION APPLICATION INFRASTRUCTURE LOAD Monitoring Engine Monitoring
  • 19. www.neotys.com© 2015 Neotys. All Rights Reserved. DEVICE CHALLENGES FOR PERFORMANCE TESTING Recording the Application • Type of app - Native apps - Web apps through mobile browsers - Hybrid apps • Technologies used in the app - HTML 5 - Web Socket - Streaming Video/audio - Google SPDY Replay scenarios for specific devices and browsers • Different content is delivered depending on the specific device or browser • Different browsers and devices have different numbers of parallel connections that affect performance
  • 20. www.neotys.com© 2015 Neotys. All Rights Reserved. CHALLENGES/CONSIDERATIONS – NETWORK CONDITIONS Bandwidth • How are users with different download and upload capacities experiencing performance on the app? Latency • How are network delays affecting the performance of the app? Packet Loss • How are users experiencing the app when switching cell towers, traveling, losing signal, etc. SHOULD WE EMULATE THESE CONDITIONS? YES!!!
  • 21. www.neotys.com© 2015 Neotys. All Rights Reserved. LATENCY: THE REAL BOTTLENECK Bandwidth Isn’t Everything • Benefits of increasing bandwidth over HTTP caps out around 5 Mbps • Increasing bandwidth from 5 to 10 Mbps decreases page load time by only about 5% Latency is Worse for Mobile • In 2012, the average round trip time to Google was ~100ms worldwide and ~50- 60ms within the US • On mobile, RTT is around 100-1000ms • If a device has been idle and requires the radio to turn on first, add 1000-2000ms to that time Benefit of Adding an Extra Mbps Bandwidth PercentImprovement 20 0 10 30 40
  • 22. www.neotys.com© 2015 Neotys. All Rights Reserved. PACKET LOSS INFLUENCE Packet Loss Consequences Packet retransmission End user experience degraded because packet loss increases jitter Errors could occur if the packet loss is in pure data transmission Packet Loss Inevitable Mobility of the user Quickly switching to the next cell tower Temporary lose the signal due to congestion of the current cell
  • 23. www.neotys.com© 2015 Neotys. All Rights Reserved. CHALLENGES/CONSIDERATIONS - HIGH, DISTRIBUTED LOAD High Number of Users • Mobile applications usually have large user bases • High user growth creates need for highly scalable load generation infrastructure Geographical Distributed Load • Users are accessing apps from remote locations • Distance and location greatly affect performance Mix of Web & Mobile Users • Many mobile apps have a “partnered” web app with a shared backend • High load on the web app can affect performance on mobile TIP: USE CLOUD-BASED LOAD GENERATION
  • 24. www.neotys.com© 2015 Neotys. All Rights Reserved. MOBILE PERFORMANCE TESTING – SOLUTION REQUIREMENTS Record from Any Device • From real devices and emulators • Native or browser-based applications Simulate Any Device or Browser • Ensures the right content is delivered for that device or browser • Simulates parallel connections for each specific device and browser Emulate Realistic Network Conditions • Bandwidth • Latency • Packet Loss Generate Load from the Cloud • Geographic realism • Test full delivery chain Load Command 3GH+ 4G WiFi APPMonitor Network Emulation & Device Simulation Cloud Load Generators On-premise Load Generators Database ServerApp Server Web Server
  • 25. www.neotys.com© 2015 Neotys. All Rights Reserved. WHAT HAPPENED TO EUE? The Problem: • Performance solutions don’t provide on-device measures • Functional/platform solutions don’t separate round-trip or server-side measures HOLY GRAIL SOLUTION COMBINE THE TWO! (Manual stop watch approach not feasible) Automated Functional/Platform Automated Performance On-device measures Device Vitals “Round-Trip” measures INTEGRATED FOR RESULTS CORRELATION
  • 26. www.neotys.com© 2015 Neotys. All Rights Reserved. LOAD + FUNCTIONAL TESTING • Automatically trigger functional test with the start of a load test • See real device rendering times and vitals while the backend of the application is under load • Results correlated for “end-to-end” Load 3GH+ 4G WiFi APPMonitor Cloud LGsOn-premise LGs Database ServerApp Server Web Server Controller Real Devices & Networks Functional Simulated Devices w/ Network Emulation Real Device Metrics 4G 3GH+ Real Device Metrics Under Load
  • 27. www.neotys.com© 2015 Neotys. All Rights Reserved. COMPLETE MOBILE EUE Real Device Users Virtual Users Load Real Device Testing Tool Load Testing Tool Application Server Load End User Experience Round-Trip Time 00:00:00 00:00:30 00:01:00 00:01:30
  • 28. www.neotys.com© 2015 Neotys. All Rights Reserved. SUMMARY Mobile devices are everywhere. Application Performance is critical. Performance usually means “End User Experience” Test with a combo of AUTOMATED & INTEGRATED solutions MEASURE & CORRELATE
  • 29. www.neotys.com© 2015 Neotys. All Rights Reserved. QUESTIONS? Feel free to contact me: Email: brad.stoner@neotys.com