Downloaded 49 times
Uploaded byStephen Thair
Measuring mobile performance (@LDNWebPerf Version)
The document discusses the nuances and challenges of measuring mobile web performance, emphasizing that mobile conditions differ significantly from desktop settings. It highlights various methods for assessing performance, including the use of proxies, browser plug-ins, and custom tools, while noting the limitations of these approaches on different mobile platforms. Ultimately, it underscores the importance of understanding location, device, and network factors when evaluating mobile performance.
More Related Content
![[TTT Meetup] Enhance mobile app testing with performance-centric strategies (...](https://cdn.slidesharecdn.com/ss_thumbnails/tttenhancemobileapptestingwithperformance-centricstrategies1-220425045726-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Measuring mobile performance (@LDNWebPerf Version)
Measuring mobile performance (@LDNWebPerf Version)
- 1. Stephen Thair, SeritiConsulting, @TheOpsMgrLondon Web Performance Meetup Group, @LDNWebPerfMeasuring mobile performance14/09/2011© Seriti Consulting1
- 2. Before we start…3Thingsto keep in mind14/09/2011© Seriti Consulting2
- 3.
- 4. A Java appcalling a java app calling a C++ app using JNI to call a java app….14/09/2011© Seriti Consulting4http://assets.en.oreilly.com/1/event/60/Analyzing%20the%20Performance%20of%20Mobile%20Web_%20Challenges%20and%20Techniques%20Presentation.pdf
- 5.
- 6. WiFivs HSDPA Performance14/09/2011© Seriti Consulting6
- 7.
- 8. Different Phones =Different Performance14/09/2011© Seriti Consulting8http://www.anandtech.com/show/4144/lg-optimus-2x-nvidia-tegra-2-review-the-first-dual-core-smartphone/8
- 9. Different operating systems14/09/2011©Seriti Consulting9
- 10. Different Browsers…14/09/2011© SeritiConsulting10http://www.webdevelopersnotes.com/articles/mobile-web-browsers-list.php
- 11. Different network types…14/09/2011©Seriti Consulting11http://developer.android.com/reference/android/telephony/TelephonyManager.html
- 12. Mobile Performance islocation-dependent14/09/2011© Seriti Consulting12http://mytests.3pmobile.com/context/?WyIyUFVDZHlWV2ErZHFISmxCSWFXRENnPT0iLCJEYnFYVVp3SEN4MD0iXQ%3D%3D
- 13. Page Load times(along the thames)14/09/2011© Seriti Consulting13Off the chart (2m36s)
- 14. HSDPA… on a #!%&$@*ing train14/09/2011© Seriti Consulting14Max RTT = 20266mshttp://blog.davidsingleton.org/mobiletcp
- 15. How important isperformance on mobile, anyway?14/09/2011© Seriti Consulting15
- 16. You only have1.5 seconds on mobile!!!14/09/2011© Seriti Consulting16
- 17. How many waysare there to measure web performance?And how useful are they on Mobile?14/09/2011© Seriti Consulting17
- 18.
- 19. 6Browser plug-ins14/09/2011© SeritiConsulting19As used in HTTPWatch, DynatraceAjax Edition etcBut only some mobile browsers support plug-ins… Android & iPhone = No, WinMo & Blackberry = Yes.
- 20. 5Proxy Timing14/09/2011© SeritiConsulting20E.g. Fiddler Proxy (http://www.fiddler2.com/Fiddler2/version.asp)There is no proxy settings in Android unless you root your phone (http://android-proxy.blogspot.com/)Only useful for WiFi…Beware the “observer effect”
- 21. 4Web Server Module14/09/2011©Seriti Consulting21As used by many APM solutions e.g. Dynatrace, New RelicInstall mod_ or ISAPI filter Measure the RTT between web server and DB/Application tierGreat for measuring your back-end performanceNot much good for front end…
- 22. 3JavaScript Timing14/09/2011© SeritiConsulting22As used in Boomerang, Episodes, GA, WebTuna etcOK for web pages, better with new WebTiming and NavigationTiming APIs not that mobile browsers support them…Relies on JavaScript and Cookies… which is problematic…
- 23. 2 Network levelTiming14/09/2011© Seriti Consulting23As used by PCAPPerf and by Atomic Labs Pion, Tealeaf, CoradiantPCAPPERF limited to over WiFi (otherwise you can’t capture the packets)*Pion, Tealeaf etc require network tap…
- 24. 1 Custom browser14/09/2011©Seriti Consulting24As used by 3PMobile.com (on the phone) and synthetic agents (by active monitoring providers e.g. Site Confidence)At present this is the only way to get real OTA performance data.
- 25. 14/09/2011© Seriti Consulting25Sowhat to use?
- 26. 14/09/2011© Seriti Consulting26Dependson your needs
- 27. 14/09/2011© Seriti Consulting27Developmentvs“Real World”
- 28. 14/09/2011© Seriti Consulting28Networkperspective vsBrowser Perspective?
- 29. PCAPPerf Testing 14/09/2011© SeritiConsulting29WiFi Connection to PCPC acting as Wifi hotspot and capturing packets with WiresharkUpload pcap file to pcapperf.appspot.comAndView the waterfall Uploadhttp://code.google.com/p/pcaphar/wiki/CaptureMobileTraffics
- 30. Mobitest – blaze.io/mobile 14/09/2011©Seriti Consulting30“PCAPPerfas SaaS” + some on-device event capture + screenshotsWiFi onlyTethered machines in Ottawa, DC, Amsterdam
- 31. Mobitest results screen14/09/2011©Seriti Consulting31Download timePage SizeWaterfallScreenshotLink to HAR
- 32. Mobitest / PCAPPerfwaterfall14/09/2011© Seriti Consulting32
- 33. 3PMobile.com “Browser2”14/09/2011© SeritiConsulting33Android Native Browser code instrumented for performance measurementTest Anywhere you have signal…Full Mobile context – device & location & network. WiFi & 3G.Browser-level perspectiveScreenshotsAutomated TestingHTML, CSS, Script source code
- 34. 3PMobile.com “Browser2” Waterfall14/09/2011©Seriti Consulting34Waterfall of www.ft.com (redirected to http://m.ft.com)
- 35.
- 36. Device Context14/09/2011© SeritiConsulting36
- 37.
- 38.
- 39. Download the HTML,Script, CSS source14/09/2011© Seriti Consulting39
- 40. Sign up forfree Beta @ www.3pmobile.com14/09/2011© Seriti Consulting40
- 41. 14/09/2011© Seriti Consulting41Sowhat else is there?
- 42. TCP Dump onRooted phone14/09/2011© Seriti Consulting42http://www.vbsteven.be/blog/android-debugging-inspectin-network-traffic-with-tcpdump/
- 43. JavaScript Bookmarklets14/09/2011© SeritiConsulting43Steve Souder’s Mobile Performance Bookmarlethttp://stevesouders.com/mobileperf/mobileperfbkm.phpNone of these measure performance per se, but they can give you more information about the page (but beware of a strong “observer effect”)
- 44. Keynote MITE (Emulation)14/09/2011©Seriti Consulting44The free Keynote MITE emulatorWebkit-based emulation based on device profiles
- 45. Debugging tools14/09/2011© SeritiConsulting45Most SDK’s include debugging tools and profilers that can help you understand what’s going onBut you are tethered to your PC (normally)
- 46. Watchmouse“IAM” library14/09/2011© SeritiConsulting46In-App Monitoring (IAM) is a set of libraries to help instrument your web apps. Currently iPhone only. Replace your app WebView calls with IAMWebViewIt will gather the performance data and beacon it out for youhttps://bitbucket.org/watchmouse/iam/
- 47. BBC / EpitroSignal Strength Survey14/09/2011© Seriti Consulting47http://www.bbc.co.uk/news/technology-14644507
- 48. 14/09/2011© Seriti Consulting48Andthat’s all I’ve got…
- 49. 14/09/2011© Seriti Consulting49Questions?StephenThair, Seriti Consulting, @TheOpsMgrStephen.thair@seriticonsulting.comwww.seriticonsulting.com/blogLondon Web Performance Meetup Group, @LDNWebPerfMonthly Meetups in Central London (near Bank)http://www.meetup.com/London-Web-Performance-Group/
Editor's Notes
- #4 Forget what you thought you knew…
- #5 Here is one reason why a mobile browser isn’t the same as a desktop browser….(on Android) the “web browser” is a Java App calling a C/C++ app which calls another C app…
- #6 Different bandwidthDifferent latencyDifferent network layer entirely (Backhaul probably isn’t even IP-based…)
- #9 And if you throw in CyanogenMod you can get even better performance…
- #11 And if you throw in CyanogenMod you can get even better performance…
- #12 What type of network is it connected to?
- #13 I was on the Thames Clipper on my way back from the O2 after a concert. So I ran some automated testing and this graphs my results… Average32.859Median12.071Std Deviation45.691Min9.275Max156.520
- #14 Test MarkerTime (Sec)Signal Strength (ASU)19.2753212.681339.68274156.52018512.071969.28513752.86814810.1849923.1658Average32.859Median12.071Std Deviation45.691
- #15 “Snakes… on a motherf**king plane!” Network latency varies enormously…. What can we see here?The connection is relatively good at sending and receiving data. The green crosses represent packets that were actually lost which only happened for 75 of 1000.There are some crazy-high round trip times. The minimum round trip time was 107ms (which would put my home cable connection to shame) and even the median is pretty awesome at 239ms but the maximum was a whopping 20226 ms - that's more than 20 seconds!
- #19 1.JavaScript timing e.g. WebTuna, Boomerang, Episodes2.Browser plug-in e.g. HTTPwatch3.Custom browser variant e.g. A Webkit-build (often used as a monitoring agent)4.Proxy timing e.g. Fiddler proxy5.Web Server Module e.g. APM solutions like AppDynamics, Dynatraceetc6.Network-level e.g. Atomic Labs Pion, Tealeaf, Coradiant
- #20 2.Browser plug-in e.g. HTTPwatchBut only some mobile browsers have a plug-in architecture… and exactly what timing info that would provide… I have no idea…
- #21 4.Proxy timing e.g. Fiddler proxy or Charles ProxyObserver Affect #1 = http://www.fiddler2.com/fiddler/perf/PerfMeasure.aspObserver Affect #2= http://insidehttp.blogspot.com/2005/06/using-fiddler-for-performance.html
- #22 5.Web Server Module e.g. APM solutions like AppDynamics, Dynatraceetc
- #23 1.JavaScript timing e.g. WebTuna, Boomerang, EpisodesYou can also use YUI Profiler I believe http://developer.yahoo.com/yui/profilerI have seen JavaScript warnings appear due to “excessive time in the onBeforeUnload event” – so keep in mind that JavaScript is a lot slower in mobile devices and hence the observer effect is larger (and the timing more accurate).
- #24 Network tap based solutions are great for measuring your own site, not so great for measuring 3rd party sites.PCAPPerf is limited to WiFi… which isn’t anything like the real “mobile” experience. The mobile network stack is NOT the same as the desktop stack, and the client side delay is larger also. *In theory you can use tcpdump on a rooted phone. More on that later…
- #25 The mobile network stack is NOT the same as the desktop stack, and the client side delay is larger also.
- #28 Is a tethered, rooted phone over wifi adequate to your needs?
- #29 Is knowing what’s happening “on the wire” enough, or do you want to understand what’s happening at the browser level?And the “mobile context” …So lets look at a few in detail…
- #30 Flexible solution (in theory works with every WiFi enabled phone)But WiFi isn’t mobile…And it’s a faff…
- #31 Essentially the PCAPPerf approach as SaaSBut it does support iPhone and Android.
- #34 Currently Android only but PoC code for Blackberry and iPhone exists. iPhone client in 90 to 120 days, all being well.
- #43 Rooted and Tethered, again…
- #45 Even a PC-based version of Webkit isn’t the same as webkit running on a mobile device. But useful none the less. Lots of other features and unsurprisingly there is a (expensive) Pro version “The MITE Pro annual user license is $5,000 and includes 100 hours of free remote testing. To learn more about single- and multiple-license purchases, please contact us today.”
- #47 The mobile network stack is NOT the same as the desktop stack, and the client side delay is larger also.