Building a Performance Labfor Mobile Apps in a DayAshray MathurVenkatesh KatariSusie Xia
Salesforce Mobile AppsMobile CRM(Hybrid)Chatter(iOS & Android)Touch (HTML5 & Hybrid)
Mobile Performance ChallengesiOS AndroidWar StoriesMobile Web
Mobile Performance Challenges
Mobile Performance - The Challengeshttp://www.html5rocks.com/en/tutorials/internals/howbrowserswork/
Mobile Performance - The Challenges
Mobile Performance - The Challenges
Mobile Performance - The Challenges
Build your own mobile performance lab !
Setting Baselines and Targets
Mobile Usability Engineering 101Delay User Reaction< 1s Feels Instant1 - 3s App is working3s+ Mental context switch10s+ Il...
Mobile Web Apps
Key Performance MetricsPage Load TimesServer / Client MetricsNetwork MetricsNav Timing Metrics
App	  Instrumenta-on	  Mobile	  Infrastructure	  Mobile	  Automa-on	  3 Essential Steps for Building a Performance Lab
•  WebDriver is the standard for browser automationo  Extensively used for functional testingo  Libraries in almost every ...
App Instrumentation : boomerang.js•  Steps:	  •  Git:	  	  •  h9ps://github.com/lognormal/boomerang/	  •  Include:	  •  In...
App Instrumentation : boomerang.js•  Page	  Load	  Times	  •  Bandwidth	  and	  Latency	  •  Perceived	  Performance	  •  ...
Mobile Infrastructure
•  Webdriver for interactive scenarioso  Multi page flows•  No Webdriver within instrumented intervalso  Click -> Wait -> ...
Continuous Integration
Simulator Based CI
Code Checkin Build/deploy WarmupLaunchSimulator andrun testsSimulator Based CI
iOS
AutomationUI Automation FrameworkFrank Testing FrameworkiOS-Driver (selenium based!!)
LoggingTime spent in server + networkTotal time spent on client SideTime to first actionable screen
Caching metrics: Time spent instoring/retrieving data, size ofcache, cache eviction counter.
Data processing metrics: Time spentin serializing/de-serializing JSONresponse, massaging data etc.
Time spent in Controller/datalayer, massaging the data etc
My precious….
Stress /longevity testing-Apple script to generate memory warnings <git repo>-UI Automation script to continuously go over...
Instruments –iOS ProfilingMemory Leaks !CPU monitorProfiling
Android
Build an Android lab that can collect real deviceperformance metrics in a fully automatic way!
With Selendroid + iWebDriver/ChromeDriverWebNative
Heap Usage on Android•  Dalvik runtime is garbage collected•  Heap size limit is increasing on deviceso  Device specifico ...
Never tell a woman that she will get alarger wardrobe…
GC Log Message is NOT enough!
Heap Dumps•  Get a heap dump:o  Dump hprof File in DDMSo  android.os.Debug.dumpHprofData()•  Convert hprof file to standar...
War Stories
PerceivedPerformance
Do Things Early or Late!•  Early:o  Important elemento  Viewpoint contento  Start actions early•  Late:o  Async cleanup
Preloading LoginFool your browser for faster speed.
Fool your browser for faster speed•  Request Javascript as images while user typing password!•  Observation:o  Typing pass...
Mobile Lab EvolutionWhat the heckis mobileperformance?Can I try withstop clock?Haha, I havecustominstrumentation.I am buil...
Takeaways•  Mobile Performance is hard, but exciting•  Decide what and how to measure your apps•  Consider the app stabili...
Future Series•  Building a Performance Lab for Mobile Apps in a Dayo  Tutorial Session at Velocity New York, 3:30pm Oct 14...
Booth #215
Velocity   building a performance lab for mobile apps in a day - final
Velocity   building a performance lab for mobile apps in a day - final
Velocity   building a performance lab for mobile apps in a day - final
Velocity   building a performance lab for mobile apps in a day - final
Upcoming SlideShare
Loading in...5
×

Velocity building a performance lab for mobile apps in a day - final

1,060
-1

Published on

velocity mobile performance lab

Published in: Technology, Design
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,060
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
44
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Velocity building a performance lab for mobile apps in a day - final

  1. 1. Building a Performance Labfor Mobile Apps in a DayAshray MathurVenkatesh KatariSusie Xia
  2. 2. Salesforce Mobile AppsMobile CRM(Hybrid)Chatter(iOS & Android)Touch (HTML5 & Hybrid)
  3. 3. Mobile Performance ChallengesiOS AndroidWar StoriesMobile Web
  4. 4. Mobile Performance Challenges
  5. 5. Mobile Performance - The Challengeshttp://www.html5rocks.com/en/tutorials/internals/howbrowserswork/
  6. 6. Mobile Performance - The Challenges
  7. 7. Mobile Performance - The Challenges
  8. 8. Mobile Performance - The Challenges
  9. 9. Build your own mobile performance lab !
  10. 10. Setting Baselines and Targets
  11. 11. Mobile Usability Engineering 101Delay User Reaction< 1s Feels Instant1 - 3s App is working3s+ Mental context switch10s+ Ill come back later ...Breaking the 1s mobile barrier : http://bit.ly/mobile-barrier
  12. 12. Mobile Web Apps
  13. 13. Key Performance MetricsPage Load TimesServer / Client MetricsNetwork MetricsNav Timing Metrics
  14. 14. App  Instrumenta-on  Mobile  Infrastructure  Mobile  Automa-on  3 Essential Steps for Building a Performance Lab
  15. 15. •  WebDriver is the standard for browser automationo  Extensively used for functional testingo  Libraries in almost every languageo  W3C working draft•  Plain functional test, nothing specialo  Device Drivero  iOS – iWebDriver, iOS Drivero  Android - AndroidDriver, ChromeDriverMobile Web Automation•  Dont reinvent the wheel !
  16. 16. App Instrumentation : boomerang.js•  Steps:  •  Git:    •  h9ps://github.com/lognormal/boomerang/  •  Include:  •  Ini-alize:  BOOMR.init({      beacon_url:  "h9p://beacons.yoursite.com/path/to/beacon.gif",      site_domain:  "yoursite.com”  });  <script  src="/javascript/boomerang.js"  type="text/javascript"></script>  Documentation - http://lognormal.github.io/boomerang/doc/
  17. 17. App Instrumentation : boomerang.js•  Page  Load  Times  •  Bandwidth  and  Latency  •  Perceived  Performance  •  Page  Components  Load  Time    •  Custom  Measurements  BOOMR.plugins.RT.startTimer(“t_-mer”);    //some  foo  BOOMR.plugins.RT.endTimer(“t_-mer”);    BOOMR.plugins.RT.startTimer("t_head");!...!BOOMR.plugins.RT.endTimer(“t_head”).startTimer(“t_body”);!!!  BOOMR.page_ready();!  
  18. 18. Mobile Infrastructure
  19. 19. •  Webdriver for interactive scenarioso  Multi page flows•  No Webdriver within instrumented intervalso  Click -> Wait -> Measure•  Real device performance metricso  Beacon to server / loggingo  Webdriver to collect json•  Aggregate over multiple iterations !•  Simulate Networko  Bandwidth/Latency/Packet Loss•  Visualize Data•  Test -> Measure -> Analyze -> Fix -> RepeatTesting Methodology
  20. 20. Continuous Integration
  21. 21. Simulator Based CI
  22. 22. Code Checkin Build/deploy WarmupLaunchSimulator andrun testsSimulator Based CI
  23. 23. iOS
  24. 24. AutomationUI Automation FrameworkFrank Testing FrameworkiOS-Driver (selenium based!!)
  25. 25. LoggingTime spent in server + networkTotal time spent on client SideTime to first actionable screen
  26. 26. Caching metrics: Time spent instoring/retrieving data, size ofcache, cache eviction counter.
  27. 27. Data processing metrics: Time spentin serializing/de-serializing JSONresponse, massaging data etc.
  28. 28. Time spent in Controller/datalayer, massaging the data etc
  29. 29. My precious….
  30. 30. Stress /longevity testing-Apple script to generate memory warnings <git repo>-UI Automation script to continuously go over all the mostused scenarios-Monkey script <git repo>Stress/longevity testingSimulator longevity tests withsynthetic memory warnings usingapple scriptMonkey Script! It goes berserk onyour UI!Repeated user scenarios test onphysical device
  31. 31. Instruments –iOS ProfilingMemory Leaks !CPU monitorProfiling
  32. 32. Android
  33. 33. Build an Android lab that can collect real deviceperformance metrics in a fully automatic way!
  34. 34. With Selendroid + iWebDriver/ChromeDriverWebNative
  35. 35. Heap Usage on Android•  Dalvik runtime is garbage collected•  Heap size limit is increasing on deviceso  Device specifico  16MB, 48MB, 64MB...o  android:largeHeap="true"•  We are good, right?
  36. 36. Never tell a woman that she will get alarger wardrobe…
  37. 37. GC Log Message is NOT enough!
  38. 38. Heap Dumps•  Get a heap dump:o  Dump hprof File in DDMSo  android.os.Debug.dumpHprofData()•  Convert hprof file to standard formato  hprof-conv android.hprof standard.hprof•  Analyze hprof fileo  Eclipse Memory Analyzer (MAT)
  39. 39. War Stories
  40. 40. PerceivedPerformance
  41. 41. Do Things Early or Late!•  Early:o  Important elemento  Viewpoint contento  Start actions early•  Late:o  Async cleanup
  42. 42. Preloading LoginFool your browser for faster speed.
  43. 43. Fool your browser for faster speed•  Request Javascript as images while user typing password!•  Observation:o  Typing password on mobile devices takes time!•  Fact in Browser:o  Images, CSS are downloaded in parallelo  Javascript is downloaded single-threaded
  44. 44. Mobile Lab EvolutionWhat the heckis mobileperformance?Can I try withstop clock?Haha, I havecustominstrumentation.I am buildingan automatedmobile lab.Automatedeverything, offto get a BEER !
  45. 45. Takeaways•  Mobile Performance is hard, but exciting•  Decide what and how to measure your apps•  Consider the app stability•  Set up automation, get rid of stopclock
  46. 46. Future Series•  Building a Performance Lab for Mobile Apps in a Dayo  Tutorial Session at Velocity New York, 3:30pm Oct 14o  http://velocityconf.com/velocityny2013/public/schedule/detail/30135•  Dont Let Me Wait: Client-side Optimizationo  Workshop Session at Grace Hopper Conference, 5:15pm Oct 3o  http://gracehopper.org/2013/schedule-at-a-glance/10-3/
  47. 47. Booth #215
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×