Webinar: Performance Testing for Mobile Applications by RTTS

3,251 views

Published on

Mobile applications are being developed and deployed rapidly, causing a new set of challenges for QA teams in every industry.
Most teams tasked with testing a mobile application are trying to overcome their company's cost constraints by putting a quick process in place to validate the end user's experience.
In this webinar, we will discuss a surefire approach for ensuring mobile application performance without increasing costs.

What You Will Learn:
+ The current state of the mobile testing space
+ The caveats of applying traditional performance testing methods to mobile application development projects
+ How to build a performance test plan tailored for a mobile application
+ How to implement an solution with the testing tools you may already have

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,251
On SlideShare
0
From Embeds
0
Number of Embeds
1,974
Actions
Shares
0
Downloads
22
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Webinar: Performance Testing for Mobile Applications by RTTS

  1. 1. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Performance Testing for Mobile Applications Moderator Laura Poggi Marketing Manager Matthew Adcock Division Manager, Performance & Scalability Presenter
  2. 2. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 The Software Quality Experts 1996 Founded Locations New York | Philadelphia | Atlanta | Phoenix 400 + Projects
  3. 3. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 What we offer Extensive knowledge of the leading test tools Unmatched experts for both strategy and implementation Innovative services for all project sizes and lengths 1 2 3
  4. 4. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Objectives • Review the current mobile load testing ecosystem & challenges. • Provide paths for interim load testing solutions for mobile applications. • Limit additional software costs related to mobile load testing requirements. • Enable collaboration amongst peers. "Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for a lifetime."
  5. 5. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Load Testing in Our Context HTTP HTTP Resource Monitoring Test Agent Virtual Users Test Coordinator Test Agent Test Results Virtual Users Web Application
  6. 6. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 What’s Happening? • Mobile usage is taking off. Consumers have high expectations on usability and performance. • Consumers are using tablets and smart phones; i.e. there are many platforms to support. • The technology is evolving rapidly; i.e. HTML5 vs. Native Apps.
  7. 7. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 There are Billions of Installs
  8. 8. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 What’s the Future of Mobile? The future is complex: • HTML5 • Single codebase running across many platforms using a web browser. • Inherent maintenance benefits & upgrade pathways. • Possibly shorter time to market. • Native Apps • Multiple codebases each targeting a specific platform. • Monetary incentives for software developers. • Increased brand management. • Sophisticated user interface.
  9. 9. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Testing is Always Catching Up
  10. 10. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Mobile Load Testing Future? Business Challenges + Technical Challenges Bumpy Ride • Mobile load testing is nascent; i.e. we’re in an early phase. • There’s a myriad of choices and/or vendors cropping up weekly. • Technology is changing rapidly.
  11. 11. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Business Challenges • Load test tool licensing constraints and/or budget limitations. • Load test tool procurement lengthy and/or cumbersome. • Accelerating time-to-market; i.e. “hurry-up and test”. • A plethora of target devices & operating systems to support.
  12. 12. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Technical Challenges • Mobile architecture not supported out the box. • Limited features of load testing tool; i.e. not keeping pace. • Creating scripts may be unorthodox and/or unclear. • Emulating large numbers of concurrent users may be required. • Incorporating network conditions becomes important.
  13. 13. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Some Existing Tools Commercial w/ Traditional Install Hosted/SAAS Open Source
  14. 14. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Short-Term Solutions • Scrutinize the mobile architecture; know the pain points and challenges. • Be creative and use what you have to limit costs related to tool transition, vendor selection, and/or constraints within your organization. • Leverage open source technology. • Take advantage of free utilities and web-based services.
  15. 15. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Our Focus Primary Focus • Creating automated test scripts for mobile applications. Honorable Mentions • Emulating network conditions of mobile devices. • Scaling up load generation to support increasing consumption of on-demand services.
  16. 16. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Test Automation Challenges • Many load test tools run on desktops NOT mobile devices. • Mobile network traffic occurs on a Wi-Fi or Cellular network. Desktop Web Application LAN LAN
  17. 17. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Test Automation Solutions 1. Spoof mobile device from a desktop web browser. 2. Proxy recording using actual mobile device. 3. Packet captures (i.e. PCAP) using actual mobile device. Script Capture
  18. 18. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Spoofing Mobile Devices Use Case: Applicable if the target application runs within a mobile web browser. This can’t be used for native apps. Strategy • Most web browsers can be modified to appear to be running on a mobile device. • With the web browser configured to imitate a mobile device, script recording can occur from the desktop where the load testing tool is installed.
  19. 19. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Spoofing Relies on Metadata • HTTP clients must use standardized metadata in the header of a HTTP request to indicate the client’s OS, browser type and version, along with extensions. • The “User-Agent” field is the key to this solution. • Many web applications use this metadata to redirect customers to mobile-specific URLs and/or mobile-specific content. Desktop Web Application HTML, CSS, JPEG JSON, XML
  20. 20. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 User-Agent Metadata GET http://www.gasbuddy.com/ HTTP/1.1 Host: www.gasbuddy.com User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:18.0) Gecko/20100101 Firefox/18.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Connection: keep-alive Mozilla/[version] ([system and browser information of client]) [browser platform] ([browser platform details]) [extensions] Example: The User-Agent is composed of the following tokens: Desktop
  21. 21. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 User-Agent for Windows XP GET http://www.gasbuddy.com/ HTTP/1.1 Host: www.gasbuddy.com User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:18.0) Gecko/20100101 Firefox/18.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Connection: keep-alive Desktop
  22. 22. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 User-Agent Glossary
  23. 23. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 User-Agent for Google Nexus GET http://gasbuddy.com/ HTTP/1.1 Host: gasbuddy.com Connection: keep-alive User-Agent: Mozilla/5.0 (Linux; Android 4.2.2; Nexus 7 Build/JDQ39) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.166 Safari/535.19 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Encoding: gzip,deflate,sdch Accept-Language: en-US,en;q=0.8 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
  24. 24. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 User-Agent Makes a Difference Desktop
  25. 25. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Modifying the Web Browser Capabilities differ by web browser and version: • Firefox has configuration web page. • Earlier versions of Internet Explorer required Add-ons; latter versions have built-in features using its Developer Tools menu item. • Google Chrome has Developer Tools; earlier version required command-line. • Safari as a Develop menu bar that can toggle the User-Agent. Detailed Instructions: http://www.howtogeek.com/113439/how-to-change-your-browsers-user-agent-without-installing-any- extensions/
  26. 26. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Make Firefox an iPad
  27. 27. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Firefox Meets iPad • Customer goes to the GasBuddy home page at http://www.gasbuddy.com/. • The GasBuddy application detects that the browser session emanates from a mobile device and redirects the web browser to the mobile version of the web site. • The mobile functionality is now accessible within a desktop browser, such as location aware browsing. Begin test script capture from the load testing tool…
  28. 28. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 User-Agent Spoofing Caveat Some desktop web browsers may not behave the same as a mobile web browser… • Application may require device-specific information, such as GPS. • UI may want to use location aware browsing There are other approaches if this is the case…
  29. 29. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Proxy Recording Use Case: Applicable for browser-based applications on mobile devices and native apps. The mobile device and load testing tool desktop must use Wi-Fi. Strategy • “Point” the client’s user interface to an intermediate proxy running on your desktop. • Some load testing tools can capture HTTP traffic directly by its recording mechanism. • Ancillary utilities can capture the HTTP traffic, if the load testing tool is not capable to so.
  30. 30. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Proxy Recording Steps Desktop Web Application LAN 1. Set the Wi-Fi settings on the mobile device to use a HTTP Proxy. 2. Launch the load testing tool recording mechanism or launch the web proxy utility on your desktop. 3. Navigate through the mobile application; either via a web browser or native app. 4. After recording work with the automated test script or transform the captured conversation into the format for the load testing tool.
  31. 31. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Enabling a Proxy for a Device • Proxy settings are found within the Wi-Fi options of the SSID. • Access to the settings is dependent upon the device. • Enter the Wi-Fi IP address and port of the web proxy or test tool.
  32. 32. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 JMeter Proxy Recording • JMeter uses a web proxy for script capture. • The proxy runs on the desktop on user-defined port number. • The mobile device needs to “point” to this desktop’s IP address and port number to be captured by JMeter.
  33. 33. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 JMeter Proxy Recording Desktop Web Application
  34. 34. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Example: JMeter Test Plan
  35. 35. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Fiddler Proxy Recording What happens if the load testing tools don’t support proxy recording? • Use an ancillary tool (that’s free) to capture the HTTP conversation. • Manually transform the HTTP conversation into an automated test script. • Programmatically convert the HTTP conversation into an automated test script. http://www.fiddler2.com/fiddler2/
  36. 36. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Fiddler Setup
  37. 37. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Fiddler Proxy Recording Desktop Web Application
  38. 38. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Fiddler Session • Captures all HTTP requests. • The raw details of each HTTP request are stored.
  39. 39. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Fiddler Session to Script • Manually edit load test tool script with Fiddler Session data. Or • Use Fiddler Plug-ins to automate the transformation process. Support for Visual Studio Web Tests already exists. Custom formats can be created using any .NET language.
  40. 40. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Packet Capture (PCAP) Refers to the act of capturing network traffic, as well as a well-defined structure for saving and manipulating the data. • Libraries exist for both UNIX-like systems (libcap) and Windows systems (WinPcap); i.e. network capture is not tied to your Windows desktop and can even be performed on the mobile device. • Network captures can be enabled via tools, such as tcpdump, WireShark, and Microsoft Network Monitor (all are free!). • There is an API built in C. There are other programming languages that have a wrapper API built; i.e. network captures can be programmatically manipulated for our scripting purposes.
  41. 41. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Packet Capture (PCAP) Use Case: Applicable for browser-based applications on mobile devices and native apps. You only need access to the PCAP traces; i.e. the mobile device and your desktop do not need to be coupled. Strategies • Setup a Wi-Fi hotspot on a desktop computer with a packet capture mechanism. Point the mobile device to the hotspot. You can also use the Wi- Fi proxy strategy that was discussed previously. • Enable PCAP capture on mobile device. This will require the mobile device to be rooted. • Plug packet capture device into network switch that will eventually “see” the mobile traffic.
  42. 42. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 PCAP Using Wireshark Desktop Web Application
  43. 43. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Using Wireshark • Ability to open existing captures, filter conversations, and view packet details. • Ability to launch new packet captures.
  44. 44. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Starting a Wireshark Capture • You need to pick the network interface to capture packets off of. • We need to make sure that the wireless network interface is used, if we are using the Wi-Fi hotspot or web proxy setup. • Start and Stop the capture when appropriate.
  45. 45. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 A Wireshark Capture
  46. 46. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Filtering a Wireshark Capture
  47. 47. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 What’s in Wireshark Capture?
  48. 48. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 There is PCAP in the “Cloud”
  49. 49. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 What Can We Do? • Wirehark provides useful views into the HTTP packets. • This information can be manually transformed into an automated test script using any load testing tool. • The transformation can be automated, if you are comfortable with programming.
  50. 50. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 PCAP Programming PCAP traces can be programmatically parsed. We can use this functionality to automate the test script transformation process. • Pcap.NET: http://pcapdotnet.codeplex.com/ • jNetPcap: http://jnetpcap.com/ • dpkt: http://code.google.com/p/dpkt/ • node-pcap-parser: https://github.com/nearinfinity/node-pcap- parser/blob/master/test/pcap-parser-test.js
  51. 51. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 PCAP Programming Examples http://jnetpcap.com/tutorial/usage http://jon.oberheide.org/blog/2008/10/15/dpkt-tutorial-2-parsing-a-pcap-file/
  52. 52. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Test Automation Re-Cap 1. Spoof mobile device from a desktop web browser. Relies on modifying your browser’s User-Agent metadata. 2. Proxy recording using actual mobile device. Record directly into test tool or use web proxy debugger. 3. Packet captures (i.e. PCAP) using actual mobile device. Use packet capture to get HTTP contents and/or program a transformation solution. Script Capture
  53. 53. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Emulating Network Conditions • Networks can influence the end- user experience, as well as the scalability of the mobile application. • Typical testing lab environments do not introduce network variability into the performance equation.
  54. 54. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Emulating Network Conditions Characteristics to consider: • Latency • Bandwidth • Packet Loss • Jitter Challenges: • Not a lot of “accessible” options; i.e. open source and/or turnkey. • Mostly commercial vendors.
  55. 55. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Emulating Network Conditions Network Emulator for Windows Toolkit (NEWT) Commercial Solutions Open Source Solutions
  56. 56. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Scaling Up Load Generation Typical Requirements: • Large numbers of users requires hardware to emulate. • Hardware may only be needed on-demand. • Requirement to geographically distribute virtual users. Solutions: • Use cloud services to build out infrastructure. • Leverage self-service solutions w/ pre-configured infrastructure. • Nothing is “free” here.
  57. 57. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Scaling Up Load Generation
  58. 58. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Moving Forward • We’re accessible for further discussions. • Potential for future webinars to continue our discussion. • Foster collaboration. We’ve created a LinkedIn User Group in order to expand upon this discussion.
  59. 59. © 2013 Real-Time Technology Solutions, Inc. 360 Lexington Ave. FL 9, New York, NY 10017 www.rttsweb.com | (212) 240-9050 Thank you! Questions? Linkedin Group Performance Testing - Mobile Applications Matthew Adcock madcock@rttsweb.com Website www.rttsweb.com Call us (646) 329-9309

×