Presentation from 17/3/2011 at the NY Web Performance Chapter about the iPhone/Android Comparison Study by Blaze.io (http://www.blaze.io), presented by Guy Podjarny
2. Agenda Why Mobile Web Performance Matters Challenges in Mobile WPO Intro to Mobitest State of Mobile Web Performance Study Anecdotes & Insights Coming Soon Summary Q&A 2
3. Why Mobile Web Perf Matters Because Web Performance matters Many studies show rev/perf correlation Because Mobile users are impatient 58% of users expect desktop speed on mobile Tablets take away the “Mobile Site” solution Because Mobile browsing is growing quickly Mobile Browsing will pass Desktop by 2014 31% of US Mobile Users have a Smartphone
6. Challenges in Mobile WPO Slower Clients Less Powerful Devices, Slower Networks Fragmentation OS Fragmentation Browser Fragmentation Device Fragmentation Form-Factor Fragmentation Network Fragmentation Non-Smartphones (not in this presentation) Lack of Tools Emphasizes challenges above
7. Current Mobile Performance Tools Blaze Mobitest Details to follow Pcapperf (pcapperf.appspot.com) Upload captured mobile network dump Analysis Bookmarklets Mobile Perf, DOM Monster, etc. New: YSlowBookmarklet (Alpha) JDrop Collect on Device, Analyze Online 7
9. Mobitest - Overview Free service for measuring mobile page load Measures on real devices, using custom apps Available devices include: iPhone 4.3 iPhone 4.2 Android 2.3 (Nexus S) Android 2.2 (Galaxy S) Currently only measures over WiFi 3G works, but not yet included in service 9
10. How does it work? Mobile WebPageTest Agents Custom apps, created for each platform Poll WPT to receive jobs Load pages while monitoring activities Package and upload results Supports multiple runs, first/repeat view & video Private Modified WPT Instance Supports different results format 10
11. Getting the Data Mobile Devices are locked down Getting the info out is hard Each OS required different approaches And presented different limitations iPhone: Embedded Browser Callbacks Android: Browser Callbacks + On-device sniffer + pcap2har Blackberry (Alpha): Networking API 11
13. State of Mobile Web Perf Study Goal: Statistically Strong Performance Bakeoff What’s unique about it? Scale: Measured 45,000 Page Loads Automation: More accurate than manual timing Real Devices: Not measuring on simulators Questions Answered: Who’s Faster: Android or iPhone? What’s the impact of the latest versions? How does WiFi compare to 3G? 13
14. Methdology Measuring Software: Mobitest Test Sites: Fortune 1,000 Devices: iPhone 4, Nexus S, Galaxy S Network: Mostly WiFi, one 3G test Process: Measure sites on a device 3 times, save median Repeat 3 times on different days Use median results of medians 14
15. iPhone vs. Android - Average Android 52% Faster On Mobile Sites, Android only 3% Faster Effectively tied On Desktop Sites, Android is 59% Faster Will manifest even more in Tablets 15
16. iPhone vs. Android – Races Android Faster on 84% of Sites iPhone won 16% of the races On Mobile Sites, Android won 71% of the time Average gap decreased to 1.1 seconds 16
17. Android Version Comparison Android 2.3 is 10% Faster Than 2.2 2.144 Seconds vs. 2.370 Android 2.2 was still faster on 42% of sites Average gap merely 65 milliseconds JavaScript in 2.3 is 37% Faster Based on SunSpider JavaScript Benchmark Conclusions: JavaScript Speed != Browsing Speed We Optimize What We Measure 17
18. iPhone Version Comparison iPhone 4.3 Not Faster than 4.2 4.3 is 2% Slower: 3.253 vs. 3.182 4.3 was faster on 51% of Sites Average gap is an even zero JavaScript Engine is 2.5x Faster! And yet, no browsing speed improvement Theory: Nitro not always applied Doesn’t manifest on full-screen websites, home page web apps, embedded browsers 18
19. WiFi vs. 3G Compared iPhone 4.2 on WiFi & 3G Results: WiFi 13% Faster (3.6 vs. 3.2) WiFi faster on 82% of Sites Tests done under great 3G conditions: Bell’s HSPA Network, Night time, good reception Measured Mbps: 6 Download, 1.6 Upload 100 mid-day loads increase gap to 60% 19
20. Mobile vs. Regular Site Definition of a Mobile Site (in this study): iPhone requires 30+ less resources than IE8 Result: Mobile Sites Loaded 39% Faster 66% Faster on iPhone, 8% Faster on Android Simple Sites 2xFaster Than Complex Simple = 30- Requests 20
25. WSJ - Theories Android downloads resources faster Seen on other sites, more efficient network? Speedtest.net shows the same up/down speeds Android reorders image requests? Consistently seen in the WSJ waterfall Has some constraints – domains do repeat later Complex sites make room for optimizations 25
28. Velocity Conf - # Connections iPhone 4.3 More connections 6 per host, 35 max More than 4.2 Android 4.2 Less connections 4 per host, 4 max 28
29. Velocity Conf – JS/CSS In Parallel iPhone CSS & JS block each other Android CSS & JS don’t appear to block 29
30. Velocity Conf – Conclusions More connections doesn’t mean faster Apparently even more on mobile devices Doesn’t match with current desktop wisdom Smart parallel actions matter CSS & JS often mix together Blocking resources are a major slowdown Browser speed has many moving parts… 30