• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Measuring mobile performance (@LDNWebPerf Version)
 

Measuring mobile performance (@LDNWebPerf Version)

on

  • 3,784 views

A presentation to the London Web Performance User Group covering the different ways of measuring Mobile web performance and some of the strength & weaknesses of each, depending on your needs.

A presentation to the London Web Performance User Group covering the different ways of measuring Mobile web performance and some of the strength & weaknesses of each, depending on your needs.

Statistics

Views

Total Views
3,784
Views on SlideShare
2,731
Embed Views
1,053

Actions

Likes
3
Downloads
41
Comments
0

7 Embeds 1,053

http://www.seriticonsulting.com 1046
https://mail.google.com 2
http://twitter.com 1
http://us-w1.rockmelt.com 1
http://a0.twimg.com 1
http://paper.li 1
http://seriti.squarespace.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Forget what you thought you knew…
  • Here is one reason why a mobile browser isn’t the same as a desktop browser….(on Android) the “web browser” is a Java App calling a C/C++ app which calls another C app…
  • Different bandwidthDifferent latencyDifferent network layer entirely (Backhaul probably isn’t even IP-based…)
  • And if you throw in CyanogenMod you can get even better performance…
  • And if you throw in CyanogenMod you can get even better performance…
  • What type of network is it connected to?
  • I was on the Thames Clipper on my way back from the O2 after a concert. So I ran some automated testing and this graphs my results… Average32.859Median12.071Std Deviation45.691Min9.275Max156.520
  • Test MarkerTime (Sec)Signal Strength (ASU)19.2753212.681339.68274156.52018512.071969.28513752.86814810.1849923.1658Average32.859Median12.071Std Deviation45.691
  • “Snakes… on a motherf**king plane!” Network latency varies enormously…. What can we see here?The connection is relatively good at sending and receiving data. The green crosses represent packets that were actually lost which only happened for 75 of 1000.There are some crazy-high round trip times. The minimum round trip time was 107ms (which would put my home cable connection to shame) and even the median is pretty awesome at 239ms but the maximum was a whopping 20226 ms - that's more than 20 seconds!
  • 1.JavaScript timing e.g. WebTuna, Boomerang, Episodes2.Browser plug-in e.g. HTTPwatch3.Custom browser variant e.g. A Webkit-build (often used as a monitoring agent)4.Proxy timing e.g. Fiddler proxy5.Web Server Module e.g. APM solutions like AppDynamics, Dynatraceetc6.Network-level e.g. Atomic Labs Pion, Tealeaf, Coradiant
  • 2.Browser plug-in e.g. HTTPwatchBut only some mobile browsers have a plug-in architecture… and exactly what timing info that would provide… I have no idea…
  • 4.Proxy timing e.g. Fiddler proxy or Charles ProxyObserver Affect #1 = http://www.fiddler2.com/fiddler/perf/PerfMeasure.aspObserver Affect #2= http://insidehttp.blogspot.com/2005/06/using-fiddler-for-performance.html
  • 5.Web Server Module e.g. APM solutions like AppDynamics, Dynatraceetc
  • 1.JavaScript timing e.g. WebTuna, Boomerang, EpisodesYou can also use YUI Profiler I believe http://developer.yahoo.com/yui/profilerI have seen JavaScript warnings appear due to “excessive time in the onBeforeUnload event” – so keep in mind that JavaScript is a lot slower in mobile devices and hence the observer effect is larger (and the timing more accurate).
  • Network tap based solutions are great for measuring your own site, not so great for measuring 3rd party sites.PCAPPerf is limited to WiFi… which isn’t anything like the real “mobile” experience. The mobile network stack is NOT the same as the desktop stack, and the client side delay is larger also. *In theory you can use tcpdump on a rooted phone. More on that later…
  • The mobile network stack is NOT the same as the desktop stack, and the client side delay is larger also.
  • Is a tethered, rooted phone over wifi adequate to your needs?
  • Is knowing what’s happening “on the wire” enough, or do you want to understand what’s happening at the browser level?And the “mobile context” …So lets look at a few in detail…
  • Flexible solution (in theory works with every WiFi enabled phone)But WiFi isn’t mobile…And it’s a faff…
  • Essentially the PCAPPerf approach as SaaSBut it does support iPhone and Android.
  • Currently Android only but PoC code for Blackberry and iPhone exists. iPhone client in 90 to 120 days, all being well.
  • Rooted and Tethered, again…
  • Even a PC-based version of Webkit isn’t the same as webkit running on a mobile device. But useful none the less. Lots of other features and unsurprisingly there is a (expensive) Pro version “The MITE Pro annual user license is $5,000 and includes 100 hours of free remote testing. To learn more about single- and multiple-license purchases, please contact us today.”
  • The mobile network stack is NOT the same as the desktop stack, and the client side delay is larger also.

Measuring mobile performance (@LDNWebPerf Version) Measuring mobile performance (@LDNWebPerf Version) Presentation Transcript

  • Stephen Thair, Seriti Consulting, @TheOpsMgr
    London Web Performance Meetup Group, @LDNWebPerf
    Measuring mobile performance
    14/09/2011
    © Seriti Consulting
    1
  • Before we start…
    3
    Things to keep in mind
    14/09/2011
    © Seriti Consulting
    2
  • Desktop

    Mobile
    14/09/2011
    © Seriti Consulting
    3
  • A Java app calling a java app calling a C++ app using JNI to call a java app….
    14/09/2011
    © Seriti Consulting
    4
    http://assets.en.oreilly.com/1/event/60/Analyzing%20the%20Performance%20of%20Mobile%20Web_%20Challenges%20and%20Techniques%20Presentation.pdf
  • Wifi

    Mobile
    14/09/2011
    © Seriti Consulting
    5
  • WiFivs HSDPA Performance
    14/09/2011
    © Seriti Consulting
    6
  • Mobile

    Mobile
    14/09/2011
    © Seriti Consulting
    7
  • Different Phones = Different Performance
    14/09/2011
    © Seriti Consulting
    8
    http://www.anandtech.com/show/4144/lg-optimus-2x-nvidia-tegra-2-review-the-first-dual-core-smartphone/8
  • Different operating systems
    14/09/2011
    © Seriti Consulting
    9
  • Different Browsers…
    14/09/2011
    © Seriti Consulting
    10
    http://www.webdevelopersnotes.com/articles/mobile-web-browsers-list.php
  • Different network types…
    14/09/2011
    © Seriti Consulting
    11
    http://developer.android.com/reference/android/telephony/TelephonyManager.html
  • Mobile Performance is location-dependent
    14/09/2011
    © Seriti Consulting
    12
    http://mytests.3pmobile.com/context/?WyIyUFVDZHlWV2ErZHFISmxCSWFXRENnPT0iLCJEYnFYVVp3SEN4MD0iXQ%3D%3D
  • Page Load times (along the thames)
    14/09/2011
    © Seriti Consulting
    13
    Off the chart (2m36s)
  • HSDPA… on a #!%&$@*ing train
    14/09/2011
    © Seriti Consulting
    14
    Max RTT = 20266ms
    http://blog.davidsingleton.org/mobiletcp
  • How important is performance on mobile, anyway?
    14/09/2011
    © Seriti Consulting
    15
  • You only have 1.5 seconds on mobile!!!
    14/09/2011
    © Seriti Consulting
    16
  • How many ways are there to measure web performance?
    And how useful are they on Mobile?
    14/09/2011
    © Seriti Consulting
    17
  • 6
    14/09/2011
    © Seriti Consulting
    18
  • 6Browser plug-ins
    14/09/2011
    © Seriti Consulting
    19
    As used in HTTPWatch, DynatraceAjax Edition etc
    But only some mobile browsers support plug-ins… Android & iPhone = No, WinMo & Blackberry = Yes.
  • 5Proxy Timing
    14/09/2011
    © Seriti Consulting
    20
    E.g. Fiddler Proxy (http://www.fiddler2.com/Fiddler2/version.asp)
    There is no proxy settings in Android unless you root your phone (http://android-proxy.blogspot.com/)
    Only useful for WiFi…
    Beware the “observer effect”
  • 4Web Server Module
    14/09/2011
    © Seriti Consulting
    21
    As used by many APM solutions e.g. Dynatrace, New Relic
    Install mod_ or ISAPI filter
    Measure the RTT between web server and DB/Application tier
    Great for measuring your back-end performance
    Not much good for front end…
  • 3JavaScript Timing
    14/09/2011
    © Seriti Consulting
    22
    As used in Boomerang, Episodes, GA, WebTuna etc
    OK for web pages, better with new WebTiming and NavigationTiming APIs not that mobile browsers support them…
    Relies on JavaScript and Cookies… which is problematic…
  • 2 Network level Timing
    14/09/2011
    © Seriti Consulting
    23
    As used by PCAPPerf and by Atomic Labs Pion, Tealeaf, Coradiant
    PCAPPERF limited to over WiFi (otherwise you can’t capture the packets)*
    Pion, Tealeaf etc require network tap…
  • 1 Custom browser
    14/09/2011
    © Seriti Consulting
    24
    As used by 3PMobile.com (on the phone) and synthetic agents (by active monitoring providers e.g. Site Confidence)
    At present this is the only way to get real OTA performance data.
  • 14/09/2011
    © Seriti Consulting
    25
    So what to use?
  • 14/09/2011
    © Seriti Consulting
    26
    Depends on your needs
  • 14/09/2011
    © Seriti Consulting
    27
    Development
    vs
    “Real World”
  • 14/09/2011
    © Seriti Consulting
    28
    Network perspective vsBrowser Perspective?
  • PCAPPerf Testing
    14/09/2011
    © Seriti Consulting
    29
    WiFi Connection to PC
    PC acting as Wifi hotspot and capturing packets with Wireshark
    Upload pcap file to pcapperf.appspot.com
    And
    View the waterfall
    Upload
    http://code.google.com/p/pcaphar/wiki/CaptureMobileTraffics
  • Mobitest – blaze.io/mobile
    14/09/2011
    © Seriti Consulting
    30
    “PCAPPerfas SaaS” + some on-device event capture + screenshots
    WiFi only
    Tethered machines in Ottawa, DC, Amsterdam
  • Mobitest results screen
    14/09/2011
    © Seriti Consulting
    31
    Download time
    Page Size
    Waterfall
    Screenshot
    Link to HAR
  • Mobitest / PCAPPerf waterfall
    14/09/2011
    © Seriti Consulting
    32
  • 3PMobile.com “Browser2”
    14/09/2011
    © Seriti Consulting
    33
    Android Native Browser code instrumented for performance measurement
    Test Anywhere you have signal…
    Full Mobile context – device & location & network. WiFi & 3G.
    Browser-level perspective
    Screenshots
    Automated Testing
    HTML, CSS, Script source code
  • 3PMobile.com “Browser2” Waterfall
    14/09/2011
    © Seriti Consulting
    34
    Waterfall of www.ft.com (redirected to http://m.ft.com)
  • Screenshots
    14/09/2011
    © Seriti Consulting
    35
  • Device Context
    14/09/2011
    © Seriti Consulting
    36
  • 14/09/2011
    © Seriti Consulting
    37
    Location Context
  • 14/09/2011
    © Seriti Consulting
    38
    Network Context
  • Download the HTML, Script, CSS source
    14/09/2011
    © Seriti Consulting
    39
  • Sign up for free Beta @ www.3pmobile.com
    14/09/2011
    © Seriti Consulting
    40
  • 14/09/2011
    © Seriti Consulting
    41
    So what else is there?
  • TCP Dump on Rooted phone
    14/09/2011
    © Seriti Consulting
    42
    http://www.vbsteven.be/blog/android-debugging-inspectin-network-traffic-with-tcpdump/
  • JavaScript Bookmarklets
    14/09/2011
    © Seriti Consulting
    43
    Steve Souder’s Mobile Performance Bookmarlet
    http://stevesouders.com/mobileperf/mobileperfbkm.php
    None of these measure performance per se, but they can give you more information about the page (but beware of a strong “observer effect”)
  • Keynote MITE (Emulation)
    14/09/2011
    © Seriti Consulting
    44
    The free Keynote MITE emulator
    Webkit-based emulation based on device profiles
  • Debugging tools
    14/09/2011
    © Seriti Consulting
    45
    Most SDK’s include debugging tools and profilers that can help you understand what’s going on
    But you are tethered to your PC (normally)
  • Watchmouse“IAM” library
    14/09/2011
    © Seriti Consulting
    46
    In-App Monitoring (IAM) is a set of libraries to help instrument your web apps. Currently iPhone only.
    Replace your app WebView calls with IAMWebView
    It will gather the performance data and beacon it out for you
    https://bitbucket.org/watchmouse/iam/
  • BBC / Epitro Signal Strength Survey
    14/09/2011
    © Seriti Consulting
    47
    http://www.bbc.co.uk/news/technology-14644507
  • 14/09/2011
    © Seriti Consulting
    48
    And that’s all I’ve got…
  • 14/09/2011
    © Seriti Consulting
    49
    Questions?
    Stephen Thair, Seriti Consulting, @TheOpsMgr
    Stephen.thair@seriticonsulting.com
    www.seriticonsulting.com/blog
    London Web Performance Meetup Group, @LDNWebPerf
    Monthly Meetups in Central London (near Bank)
    http://www.meetup.com/London-Web-Performance-Group/