The Mobile Difference – In NumbersGuy Podjarny@guypodhttp://www.guypo.com/          Akamai Confidential
Mobile Isn’t Desktop•  It’s different in many ways…•  This presentation focuses on sizing the gap •  How big is the gap be...
Faster ForwardTM   ©2012 Akamai
Mobile Computing Power               !=Desktop/Laptop Computing Power                                 Akamai Confidential
Apple Laptop vs. Mobile, CPU/RAM                       Faster ForwardTM   ©2012 Akamai
Fast Mobile Browsing is all the rage                          Faster ForwardTM   ©2012 Akamai
Better CPU matters in Benchmarks                                                         Quad Core, ~1.5 Ghz              ...
Perf of iOS5 across different hardware                               SunspiderJSBenchmark,iOS5Devices                     ...
Test Time!Faster ForwardTM        ©2012 Akamai
Test Time!•  Question:   Does mobile computing power impact page load time?•  Tested Sites: Alexa US Top 100•  Test Tool: ...
Results: Hardware Load Time Impact                                    Sunspider*&*Load*Time*Per*Device*   4000"   3500"   ...
CPU Difference - Conclusions•  Mobile CPU is still underpowered compared to desktop•  It indeed impacts web performance, e...
Cellular Network         !=Broadband Network                    Akamai Confidential
Claim: Cellular Networks are SSSLLLOOOOWWWW    Low  Bandwidth High Latency High Packet    Loss   Volatility               ...
Bandwidthhttp://www.flickr.com/photos/pasukaru76/4228438307/                                         Faster Forward       ...
Cellular Network Speed – WikipediaProtocol           Max Downlink                Max UplinkGPRS               56-115 KbpsC...
You only get those speeds it if you’re this guy…http://www.flickr.com/photos/23081877@N02/2798378080/                     ...
PCWorld/Novarum      •  3G          •  Download: 0.59 – 3.84 Mbps          •  Upload: 0.56 – 1.44 Mbps      •  4G         ...
Mobile Network Speed, Akamai SOTI, Q4 2011http://www.akamai.com/stateoftheinternet/                               Faster F...
Average Broadband Speedshttp://www.netindex.com/download/http://www.akamai.com/stateoftheinternet/   Faster ForwardTM   ©2...
Cellular Network Speed – Conclusions•  Theoretical limits are NOT the reality•  Cellular bandwidth definitely less than br...
Latency   Faster ForwardTM   ©2012 Akamai
Broadband Latency (FCC)        “Fiber-to-the-home services provided 17 ms round-trip        latency on average, while cabl...
Ofcom study, UK Mobile Networks, May ‘11                            “Latency… was an average of 117 milliseconds. Nearly t...
Cisco Cloud Index, 2010-2015“Global average fixed latency is 125 and average mobilelatency is 290… Global mobile consumer ...
Yep, Mobile Latency is different                     Latency(by(Connec,on(Type(•  Chart showing latency on Mobile vs Broad...
Cellular Network Latency - Conclusions•  Not a pretty sight…•  MUCH higher than broadband latency•  Measured averages arou...
Test Time!Faster ForwardTM        ©2012 Akamai
Test Time!•  Question:   How much impact does the difference in Bandwidth & Latency   have on web performance?•  Tested Si...
Download BW Impact on Load Time                     Poor Cell Conn                         DSL                            ...
Upload BW Impact on Load Time                     Poor Cell Conn            Avg Cell Conn                                 ...
Latency Impact on Load Time                                                  Poor Cell Conn                               ...
Bandwidth/Latency Difference - Conclusions•  Upload difference doesn’t matter much•  Download difference DOES matter•  Lat...
Packet LossFaster ForwardTM          ©2012 Akamai
Packet Loss – Broadband vs Mobile            Average Packet Loss   Mobile                     0.6%Broadband             0....
Packet Loss, Akamai/MIT Research  •  Tested 149 Million connections in 51 countries     •  Compared PL for mobile networks...
Packet Loss - Conclusions•  Packet Loss is significantly higher in Mobile, But… •  It’s small (even in Mobile) •  It usual...
Apple Network Link Conditioner                                             780/330 Kbps                                   ...
Network Link Conditioner in iOS 6? (via @jedisct1)                         Faster ForwardTM            ©2012 Akamai
VolatilityFaster ForwardTM          ©2012 Akamai
Different protocols lead to different speedshttp://www.ericsson.com/res/docs/2012/traffic_and_market_report_june_2012.pdf ...
Signal Strength affects download speedhttp://stakeholders.ofcom.org.uk/binaries/research/telecoms-research/bbspeeds2010/Mo...
Different Carriers have different average speedsSource (PCWorld): http://bit.ly/JX7MQw   Source: Ofcom                    ...
Volatility - Conclusions•  Stop Guessing!  •  You can’t predict your user’s speeds, today or in the future•  Little value ...
Cellular Networks - Summary•  Cellular Networks are kinda crappy •  Can’t live with ‘em, can’t live without ‘em…•  You can...
Mobile Browser       !=Desktop Browser                  Akamai Confidential
Actually, they’re not that different…•  Mobile & Desktop browsers are actually similar•  Many share the same underlying en...
HTTP PipeliningFaster ForwardTM   ©2012 Akamai
Mobile Software: Pipelining     •  HTTP Pipelining is around since HTTP 1.1      •  Send multiple requests on connection b...
Network Capture•  Samsung Galaxy S  •  Max Conn: 12  •  Conn Per Host: 12  •  Max Piped Reqs: 6  •  Max “In-Flight” Reqs: ...
Test Time!Faster ForwardTM        ©2012 Akamai
Test Time!•  Question:   Does HTTP Pipelining help make websites faster?•  Tested Sites: Alexa US Top 500•  Test Tool: Web...
Test Results – Pipelining has little effect…                                      Pipelining"Off"                          ...
Why doesn’t pipelining have an effect?•  Are you sure pipelining was used? •  Yes… confirmed via tcpdump and various other...
Summary          Akamai Confidential
Summary•  Those were A LOT of numbers…•  Mobile is truly different, and it matters for web performance •  Who saw that com...
Thank You!                    Questions?  The Mobile Difference – In NumbersGuy Podjarny@guypodhttp://www.guypo.com/      ...
Upcoming SlideShare
Loading in...5
×

Quantifying The Mobile Difference

12,457

Published on

We all know Mobile is different, but by how much?

This presentation attempts to quantify the difference between mobile and non-mobile, focusing on CPU, network and browser differences.

Published in: Technology, News & Politics
0 Comments
16 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
12,457
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
178
Comments
0
Likes
16
Embeds 0
No embeds

No notes for slide

Quantifying The Mobile Difference

  1. 1. The Mobile Difference – In NumbersGuy Podjarny@guypodhttp://www.guypo.com/ Akamai Confidential
  2. 2. Mobile Isn’t Desktop•  It’s different in many ways…•  This presentation focuses on sizing the gap •  How big is the gap between Mobile & Non-Mobile? •  How much does it impact performance?•  We can’t cover everything, so we’ll focus on three topics: •  Mobile CPU != Desktop/Laptop CPU •  Cellular Networks != Broadband Networks •  Mobile Browsers != Desktop Browsers Faster ForwardTM ©2012 Akamai
  3. 3. Faster ForwardTM ©2012 Akamai
  4. 4. Mobile Computing Power !=Desktop/Laptop Computing Power Akamai Confidential
  5. 5. Apple Laptop vs. Mobile, CPU/RAM Faster ForwardTM ©2012 Akamai
  6. 6. Fast Mobile Browsing is all the rage Faster ForwardTM ©2012 Akamai
  7. 7. Better CPU matters in Benchmarks Quad Core, ~1.5 Ghz Dual Core, ~1 Ghzhttp://www.anandtech.com/show/5810/samsung-galaxy-s-iii-performance-preview Faster Forward TM ©2012 Akamai
  8. 8. Perf of iOS5 across different hardware SunspiderJSBenchmark,iOS5Devices 10x 24yroldMBP,2x2.7Ghz 230 Better iPhone4S,2x800Mhz 2250 37% Better iPhone4,800Mhz 3545 25% Better iPhone3GS,600Mhz 4737 0" 500" 1000" 1500" 2000" 2500" 3000" 3500" 4000" 4500" 5000" Faster ForwardTM ©2012 Akamai
  9. 9. Test Time!Faster ForwardTM ©2012 Akamai
  10. 10. Test Time!•  Question: Does mobile computing power impact page load time?•  Tested Sites: Alexa US Top 100•  Test Tool: Mobitest•  Details: •  Test iPhone 4, iPhone 4S and iOS Simulator on 2-yr old MacBook Pro •  Note: iOS Simulator uses the actual hardware •  Tested overnight •  High speed Cable connection, WiFi Faster ForwardTM ©2012 Akamai
  11. 11. Results: Hardware Load Time Impact Sunspider*&*Load*Time*Per*Device* 4000" 3500" 3000" 2500" Sunspider" 2000" Load"Time" 1500" 1000" 500" 0" iPhone"4,"800"Mhz" iPhone"4S,"2x800"Mhz" 24yr"old"MBP,"2x2.7"Ghz" iPhone 4 iPhone 4S Measure vs. vs. iPhone 4S iOS Simulator Sunspider 37% 90% Load Time 15% 48% Faster ForwardTM ©2012 Akamai
  12. 12. CPU Difference - Conclusions•  Mobile CPU is still underpowered compared to desktop•  It indeed impacts web performance, even for regular sites•  Impact on load time is lighter than on benchmarks •  Roughly 25-50% of impact on benchmarks•  Computing power is more than just CPU •  At least GPU and RAM also have an impact •  Stick around for Ariya’s Presentation!•  Recommendation: Test on old devices too Faster ForwardTM ©2012 Akamai
  13. 13. Cellular Network !=Broadband Network Akamai Confidential
  14. 14. Claim: Cellular Networks are SSSLLLOOOOWWWW Low Bandwidth High Latency High Packet Loss Volatility Faster ForwardTM ©2012 Akamai
  15. 15. Bandwidthhttp://www.flickr.com/photos/pasukaru76/4228438307/ Faster Forward TM ©2012 Akamai
  16. 16. Cellular Network Speed – WikipediaProtocol Max Downlink Max UplinkGPRS 56-115 KbpsCDMA200 (1X) 153.6 KbpsEdge 236.8 Kbps3G Bar 2 Mbps stationary client 384 Kbps moving clientEV-DO 2.5-4.9 Mbps 0.15-1.8 MbpsEDGE Evolution 1.6 Mbps 0.5 MbpsHSDPA 2 Mbps 200 KbpsHSPA+ 14.4 Mbps4G Bar 1 Gbps stationary client 100 Mbps moving clientHSPA+ Release 11 21-672 Mbps 5.8-168 MbpsLTE 100-300 Mbps 50-75 MbpsWiMax 37-365 Mbps 17-376 Mbps Faster ForwardTM ©2012 Akamai
  17. 17. You only get those speeds it if you’re this guy…http://www.flickr.com/photos/23081877@N02/2798378080/ Faster Forward TM ©2012 Akamai
  18. 18. PCWorld/Novarum •  3G •  Download: 0.59 – 3.84 Mbps •  Upload: 0.56 – 1.44 Mbps •  4G •  Download: 2.81 – 9.12 Mbps •  Upload: 0.97 – 5.86 Mbps •  Date: Apr, 2012Source (PCWorld): http://bit.ly/JX7MQw Faster ForwardTM ©2012 Akamai
  19. 19. Mobile Network Speed, Akamai SOTI, Q4 2011http://www.akamai.com/stateoftheinternet/ Faster ForwardTM ©2012 Akamai
  20. 20. Average Broadband Speedshttp://www.netindex.com/download/http://www.akamai.com/stateoftheinternet/ Faster ForwardTM ©2012 Akamai
  21. 21. Cellular Network Speed – Conclusions•  Theoretical limits are NOT the reality•  Cellular bandwidth definitely less than broadband•  There is no single number for cellular network speed•  But we need SOME numbers for testing…•  So here’s a SWAG at numbers to use for testing •  Merging many data points, and rounding numbers for convenience Test Case Download (Kbps) Upload (Kbps) Poor Cell Network 400 250 Average Cell Network 1200 600 Faster ForwardTM ©2012 Akamai
  22. 22. Latency Faster ForwardTM ©2012 Akamai
  23. 23. Broadband Latency (FCC) “Fiber-to-the-home services provided 17 ms round-trip latency on average, while cable-based services averaged 28 ms, and DSL-based services averaged 44 ms.” Connection RTT Type Fiber 17ms Cable 28ms DSL 44ms Faster ForwardTMhttp://www.fcc.gov/measuring-broadband-america ©2012 Akamai
  24. 24. Ofcom study, UK Mobile Networks, May ‘11 “Latency… was an average of 117 milliseconds. Nearly two Static Tests: thirds of tests took more than 100 milliseconds” User Panel: “Average latency was 192 milliseconds and around a third of tests delivered latency of higher than 200 milliseconds”http://stakeholders.ofcom.org.uk/binaries/research/telecoms-research/bbspeeds2010/Mobile_BB_performance.pdf Faster ForwardTM ©2012 Akamai
  25. 25. Cisco Cloud Index, 2010-2015“Global average fixed latency is 125 and average mobilelatency is 290… Global mobile consumer average latency is307.3 ms” Faster ForwardTM ©2012 Akamai
  26. 26. Yep, Mobile Latency is different Latency(by(Connec,on(Type(•  Chart showing latency on Mobile vs Broadband, Ofcom 350" numbers 300" 250" 200" 150" 100" 50" 0" Fiber" Cable" DSL" Mobile" Mobile" Mobile" Mobile" (Ofcom"Tests)" (Ofcom" (Cisco)" Consumer" Panel)" (Cisco)" Faster ForwardTM ©2012 Akamai
  27. 27. Cellular Network Latency - Conclusions•  Not a pretty sight…•  MUCH higher than broadband latency•  Measured averages around 100-300 ms•  Updated table for suggested testing numbers: Test Case Download Upload Latency (Kbps) (Kbps) (ms) Poor Cell Network 500 250 300 Average Cell Network 1200 600 200 Faster ForwardTM ©2012 Akamai
  28. 28. Test Time!Faster ForwardTM ©2012 Akamai
  29. 29. Test Time!•  Question: How much impact does the difference in Bandwidth & Latency have on web performance?•  Tested Sites: Alexa US Top 100•  Test Tool: WebPageTest•  Details: •  Increase Latency, Download & Upload Bandwidth separately •  Set no limits on other two variables •  Run each test 3 times, use median result Faster ForwardTM ©2012 Akamai
  30. 30. Download BW Impact on Load Time Poor Cell Conn DSL Cable FIBER Avg Cell Conn Faster ForwardTM ©2012 Akamai
  31. 31. Upload BW Impact on Load Time Poor Cell Conn Avg Cell Conn DSL Cable FIBER Faster ForwardTM ©2012 Akamai
  32. 32. Latency Impact on Load Time Poor Cell Conn Avg Cell Conn DSL Cable FIBER Faster ForwardTM ©2012 Akamai
  33. 33. Bandwidth/Latency Difference - Conclusions•  Upload difference doesn’t matter much•  Download difference DOES matter•  Latency difference matters the most•  Tip #1: Test using Cell Network speeds •  Know what you’re getting•  Tip #2: Reduce number of requests •  Especially blocking requests, e.g scripts, CSS•  Tip #3: Reduce number of bytes Faster ForwardTM ©2012 Akamai
  34. 34. Packet LossFaster ForwardTM ©2012 Akamai
  35. 35. Packet Loss – Broadband vs Mobile Average Packet Loss Mobile 0.6%Broadband 0.017% Faster ForwardTM ©2012 Akamai
  36. 36. Packet Loss, Akamai/MIT Research •  Tested 149 Million connections in 51 countries •  Compared PL for mobile networks from non-mobile networks •  Most connections have no packet loss (Median PL is 0%). •  75th percentile usually zero too •  Average connection packet loss: 2.5% •  Fixed network average is 2.2% •  Implies connections are usually fine, but sometimes terrible •  The average PL for the non-zero half is 5% •  Suggests “small” connections have higher Packet Loss •  HTTP Connections are often smallhttp://pam2012.ftw.at/papers/PAM2012paper6.pdf Faster ForwardTM ©2012 Akamai
  37. 37. Packet Loss - Conclusions•  Packet Loss is significantly higher in Mobile, But… •  It’s small (even in Mobile) •  It usually doesn’t happen (even in Mobile)•  Smaller connections lose more packets•  Recommendations: •  Don’t worry about Packet Loss too much… •  Include some Packet Loss in your tests •  Ideal: Lots of caching, good user-feedback on error and no SPOFs SWAG Test Case Download Upload Latency Packet (Kbps) (Kbps) (ms) Loss Poor Cell Network 500 250 300 1% Average Cell Network 1200 600 200 0% Faster ForwardTM ©2012 Akamai
  38. 38. Apple Network Link Conditioner 780/330 Kbps 200ms RTT850/420 Kbps 190ms RTT 780/330 Kbps 200ms RTT 1% Packet Loss Faster ForwardTM ©2012 Akamai
  39. 39. Network Link Conditioner in iOS 6? (via @jedisct1) Faster ForwardTM ©2012 Akamai
  40. 40. VolatilityFaster ForwardTM ©2012 Akamai
  41. 41. Different protocols lead to different speedshttp://www.ericsson.com/res/docs/2012/traffic_and_market_report_june_2012.pdf Faster Forward TM ©2012 Akamai
  42. 42. Signal Strength affects download speedhttp://stakeholders.ofcom.org.uk/binaries/research/telecoms-research/bbspeeds2010/Mobile_BB_performance.pdf Faster ForwardTM ©2012 Akamai
  43. 43. Different Carriers have different average speedsSource (PCWorld): http://bit.ly/JX7MQw Source: Ofcom http://stakeholders.ofcom.org.uk/binaries/research/telecoms-research/bbspeeds2010/Mobile_BB_performance.pdf Faster ForwardTM ©2012 Akamai
  44. 44. Volatility - Conclusions•  Stop Guessing! •  You can’t predict your user’s speeds, today or in the future•  Little value in doing something “only on cellular”•  Strategy #1: Aim at the lowest common denominator •  Design for a usable experience in a slow network •  Enhance the page when possible•  Strategy #2: Adapt to Network Conditions •  E.g. reduce image quality when conditions are bad •  Not easy to do, but doable •  Check your options with your Server/Host/CDN Faster ForwardTM ©2012 Akamai
  45. 45. Cellular Networks - Summary•  Cellular Networks are kinda crappy •  Can’t live with ‘em, can’t live without ‘em…•  You can assume latency will be high •  And download won’t be too awesome•  Don’t worry about Packet Loss & Upload rate •  You have bigger things to worry about•  Make Peace with it – you can’t predict cellular network speed •  Best bet is to aim low, and adapt to network conditions SWAG Test Case Download Upload Latency Packet (Kbps) (Kbps) (ms) Loss Poor Cell Network 500 250 300 1% Average Cell Network 1200 600 200 0% Faster ForwardTM ©2012 Akamai
  46. 46. Mobile Browser !=Desktop Browser Akamai Confidential
  47. 47. Actually, they’re not that different…•  Mobile & Desktop browsers are actually similar•  Many share the same underlying engines (e.g. WebKit)•  Some share the browser itself (e.g. Safari, Chrome, Firefox)•  How are they different, then? •  HTTP Pipelining •  Connection Models •  Cache Sizes •  Fragmentation Faster ForwardTM ©2012 Akamai
  48. 48. HTTP PipeliningFaster ForwardTM ©2012 Akamai
  49. 49. Mobile Software: Pipelining •  HTTP Pipelining is around since HTTP 1.1 •  Send multiple requests on connection before receiving response •  Most useful in high latency environment •  Big in Mobile •  All big mobile browsers • iOS 5+, Android, Opera •  Hardly used on Desktop • Opera Only (Default) • Chrome, Firefox (Opt-in)49 Faster ForwardTM ©2012 Akamai
  50. 50. Network Capture•  Samsung Galaxy S •  Max Conn: 12 •  Conn Per Host: 12 •  Max Piped Reqs: 6 •  Max “In-Flight” Reqs: 72 (!!!)•  Full Details: http://www.guypo.com/technical/http-pipelining- request-distribution-algorithms/50 Faster ForwardTM ©2012 Akamai
  51. 51. Test Time!Faster ForwardTM ©2012 Akamai
  52. 52. Test Time!•  Question: Does HTTP Pipelining help make websites faster?•  Tested Sites: Alexa US Top 500•  Test Tool: WebPageTest•  Details: •  Use Firefox agent, which supports pipelining •  Turn pipelining on and off •  Set bandwidth to DSL, Cable & FIOS •  Increase Latency to see if impact changes Faster ForwardTM ©2012 Akamai
  53. 53. Test Results – Pipelining has little effect… Pipelining"Off" Pipelining"Off"Load%Time,%DSL%Bandwidth% Pipelining"On" Load%Time,%Cable%Bandwidth% Pipelining"On"14,000"12,000" 10,000"10,000" 8,000" 8,000" 6,000" 6,000" 4,000" 4,000" 2,000" 2,000" 0" 0" 50" 100" 200" 300" 50" 100" Pipelining"Off" Load%Time,%FIOS%Bandwidth% Pipelining"On" 12,000" 10,000" 8,000" 6,000" 4,000" 2,000" 0" 50" 100" 200" 300" Faster ForwardTM ©2012 Akamai
  54. 54. Why doesn’t pipelining have an effect?•  Are you sure pipelining was used? •  Yes… confirmed via tcpdump and various other metrics•  I don’t know for sure, but I can guess… •  18 domains referenced on a page (average) •  9 domains used only for one request (average) •  3-4 domains used only for two requests (average) •  Browsers prefer new connection to piping a request •  Other bottlenecks…•  Saw similar results for SPDY in recent test •  Details: http://bit.ly/MptaBV•  Conclusion: Not much you can/should do… Faster ForwardTM ©2012 Akamai
  55. 55. Summary Akamai Confidential
  56. 56. Summary•  Those were A LOT of numbers…•  Mobile is truly different, and it matters for web performance •  Who saw that coming?!•  Main differences for Web Performance: •  CPU delta, but its effect is not linear •  Network delta in Latency & Download BW, not upload/Packet-Loss•  Main takeaways for web developers: •  Test performance on old devices too •  Test with at least average and poor cell network performance •  Provide a decent experience in the worst of conditions•  Remember - we only looked at a handful of aspects… Faster ForwardTM ©2012 Akamai
  57. 57. Thank You! Questions? The Mobile Difference – In NumbersGuy Podjarny@guypodhttp://www.guypo.com/ Akamai Confidential
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×