Is your mobile website ready for load? Can your mobile app's back-end scale to service peak demand? Mobile web and app usage is growing at a breakneck pace! Between 2014 and 2015, traffic from mobile devices increased from 40% to 75% of total retail traffic, an increase of 35%. In 2015, mobile commerce accounted for $115 billion worldwide. At the end of this year, that number is projected to reach $142 billion. Join us at this webinar to learn: • The different types of mobile traffic generated by browsing and apps • How to load test your mobile web site for mobile browsers • Why mobile apps need load testing, too, and how to create tests Whether your peak time is Black Friday and Cyber Monday, monthly filing rush, quarterly reporting, annual open enrollment, tax filing, or any other peak, this webinar will give you the steps to build load tests so your mobile site and apps will be ready for mobile peak.

1. ©2016 SOASTA, All rights reserved.
Tom Chavez @tomchavez
Mike Ostenberg @mikeostenberg
How to Load Test for your Mobile Peak

2. ©2016 SOASTA, All rights reserved.
• How important is Mobile?
• Mobile Web and Apps
• The need for load testing
• Measure everything
• What to test?
• Test early and often – shift
left and right
• How to test?

3. Importance
of Mobile

4. ©2016 SOASTA, All rights reserved. 4
Mobile is growing!
• In 2015, mobile commerce
accounted for $115B worldwide
• In 2016, mobile commerce
is project to reach $142B

5. ©2016 SOASTA, All rights reserved. 5
How is your site now?
• mCommerce is global
o US is not the benchmark
• 39% of mobile users
are unhappy with their online experiences
o Top reasons: slow pages and site freezes/crashes
• 57% have experienced problems when using mobile
• 3 out of 4 consider mobile slower than desktop
• 46% will not return to a site that performs poorly
• 27% of smartphone users exceed data cap every month

6. ©2016 SOASTA, All rights reserved. 6
When is your Peak Season?
• Peak is year-round
o Super Bowl Sunday – 110 million viewers
o Bi-weekly or monthly payroll processing
o Valentine’s Day, President’s Day, Mother’s Day, Father’s Day
o Quarterly processing and filing
o U.S. Tax filing
o Spring, Summer, Fall & Winter season sales
o July 4th, Christmas in July
o Back to School
o Olympics
o Halloween  We are here
o Election Day
o Singles Day – world’s biggest shopping day, $9B in 12 hours
o Black Friday, Cyber Monday and Holidays
Back to top to repeat again next year!

7. Mobile Web
and Mobile Apps

8. ©2016 SOASTA, All rights reserved. 8
Mobile Web
Responsive web site WebSite
• Desktop Users
• Mobile Users
• Same or similar content
o Adapts to different
screen sizes
Mobile optimized
• “m-dot” site for mobile
• Custom built for mobile devices
All have html, images, JavaScript, css, plus third-party content

9. ©2016 SOASTA, All rights reserved. 9
Native App
• Android – Java
• iOS – Objective C/Swift
• Mobile app framework
o Same source code
Hybrid App
• Mix of native code + html
All have backend calls for login/authentication, account
settings, database calls, transactions, plus third-party SDK
calls
Mobile Apps
WebSite/Mobile
Backend

10. ©2016 SOASTA, All rights reserved. 10
The Need for Load Testing
• Everyone runs load testing on their website, right?
o Load and Performance testing is often not done or too late
o Black Friday, Cyber Monday, Holiday failures
• What’s the need for mobile web load testing?
o Mobile is growing faster than desktop
o Responsive sites and mobile sites deliver different content to
mobile devices over mobile network
• What about mobile app load testing?
o Mobile app back ends serve different content to mobile apps
o Remember Pokemon Go? It wasn’t the app that failed.

11. Measure Everything

12. ©2016 SOASTA, All rights reserved. 12
Measure Real and Virtual Users
• Real Users (RUM) provide deep data about your visitors
o No other way to know everything about your users on your site
and in your app
SOASTA mPulse collects and analyzes RUM data from your web
and mobile app users
• Virtual Users tell you how your site or app will do at scale
o You can’t get a million friends to visit your site or app at the same
time!
SOASTA CloudTest lets you easily build and execute load tests at
scales from hundreds to millions of virtual users

13. ©2016 SOASTA, All rights reserved. 13
Other Things to Measure with RUM
• Session paths
o How are your users going through your site
• Mobile App Performance
o How are your mobile apps performing?
o What are users doing in your mobile apps?
• Revenue
o How much money are you making from each user and where?

14. ©2016 SOASTA, All rights reserved. 14
Average Performance

15. ©2016 SOASTA, All rights reserved. 15
Performance by Device

16. ©2016 SOASTA, All rights reserved. 16
Creating a Test Plan – Peak Load
What was the busiest Day, Hour, Minute, Second?

17. ©2016 SOASTA, All rights reserved. 17
Creating a Web Test Plan – Page Flows
What were most popular page flows with think times? Which APIs?

18. ©2016 SOASTA, All rights reserved. 18
Network Emulation
• Testing to match realistic situations
o Users are not all from one location on a fast network
o Packets get delayed, lost, re-ordered
• Apps perform differently as network conditions worsen
o Reduced throughput
o Timeouts occur, connections are lost
o Transactions take longer, longer sessions
Adding real network modeling to load and performance testing

19. Test Early
and Often

20. ©2016 SOASTA, All rights reserved. 20
Don’t Wait Til the Last Minute to Test!

21. ©2016 SOASTA, All rights reserved. 21
Test Left and Test Right, even into Production
• Performance Testing in development
o Set performance SLAs for new code and test in development,
same as unit and functional tests which code must pass
• Performance Testing in staging
o Higher loads in near-production systems
• Performance Testing in Production
o Full scale load tests to expected load and beyond
o Find weak and stress points and prioritize fixes into project
Performance Testing is Continuous

22. ©2016 SOASTA, All rights reserved. 22
Test in Production
Some issues will only be found at scale load on production systems
Don’t forget your third party providers: CDNs, external scripts

23. ©2016 SOASTA, All rights reserved. 23
Measure Mobile App Performance Under Load
All mobile apps perform well when there is no load

24. ©2016 SOASTA, All rights reserved. 24
Automate Your Testing
Use Continuous Integration tools to automate testing

25. Architecture
and Demo

26. ©2016 SOASTA, All rights reserved. 26
Now That the Test is Over
• Analyze results for future areas to address
o Weak or stress areas that aren’t likely this time but …
• Identify third parties that were affected by load
o Coordinate with suppliers to address issues
o Align schedules or at least be aware of their changes
Even if the test passed, you’re not done

27. ©2016 SOASTA, All rights reserved. 27
Issues Will Happen
• “I am 100% confident that everything we tested will work
just fine.” – Nordstrom Performance Leader
o When loads are different than what you modeled, may have
problems
o E.g. At Nordstrom, during a recent anniversary sale, mobile
sign-ins were 9x greater than anticipated
o Used RUM to get to the root cause of the issue
Realize that Issues Will Happen Where You Can’t Predict Them

28. Q & A
Resources
Next Steps

29. ©2016 SOASTA, All rights reserved. 29
CloudTest – Load Testing for mobile web, apps
• Load testing from development to production
o Continuous testing at any scale, on-prem or in the cloud
• Real-time analytics with customizable dashboards
o Safely test in production with full visibility
• Run tests live or automatically
o Full hands-on control or hands-off automation
• Full end-to-end view
o Front-end performance to back-end systems
Fast, scalable, affordable load testing for performance optimization

30. ©2016 SOASTA, All rights reserved. 30
SOASTA Digital Performance Management Platform

31. ©2016 SOASTA, All rights reserved. 31
Questions, Resources & Next Steps
• Holiday Readiness
o soasta.com/holiday-readiness
• The Performance Beacon
o soasta.com/blog
o soasta.com/blog/2015-popular-posts-web-performance
• Webinars
o soasta.com/webinars
o Load Testing for Peak; RUM-Based Testing
• CloudTest on-Demand
o soasta.com/cloudtest: SOASTA will build & run tests for you
Resources

32. ©2016 SOASTA, All rights reserved. 32
“The leader in performance analytics.”
www.soasta.com
SOASTA