Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Velocity building a performance lab for mobile apps in a day - final


Published on

velocity mobile performance lab

Published in: Technology, Design
  • Be the first to comment

Velocity building a performance lab for mobile apps in a day - final

  1. 1. Building a Performance Labfor Mobile Apps in a DayAshray MathurVenkatesh KatariSusie Xia
  2. 2. Salesforce Mobile AppsMobile CRM(Hybrid)Chatter(iOS & Android)Touch (HTML5 & Hybrid)
  3. 3. Mobile Performance ChallengesiOS AndroidWar StoriesMobile Web
  4. 4. Mobile Performance Challenges
  5. 5. Mobile Performance - The Challenges
  6. 6. Mobile Performance - The Challenges
  7. 7. Mobile Performance - The Challenges
  8. 8. Mobile Performance - The Challenges
  9. 9. Build your own mobile performance lab !
  10. 10. Setting Baselines and Targets
  11. 11. Mobile Usability Engineering 101Delay User Reaction< 1s Feels Instant1 - 3s App is working3s+ Mental context switch10s+ Ill come back later ...Breaking the 1s mobile barrier :
  12. 12. Mobile Web Apps
  13. 13. Key Performance MetricsPage Load TimesServer / Client MetricsNetwork MetricsNav Timing Metrics
  14. 14. App  Instrumenta-on  Mobile  Infrastructure  Mobile  Automa-on  3 Essential Steps for Building a Performance Lab
  15. 15. •  WebDriver is the standard for browser automationo  Extensively used for functional testingo  Libraries in almost every languageo  W3C working draft•  Plain functional test, nothing specialo  Device Drivero  iOS – iWebDriver, iOS Drivero  Android - AndroidDriver, ChromeDriverMobile Web Automation•  Dont reinvent the wheel !
  16. 16. App Instrumentation : boomerang.js•  Steps:  •  Git:    •  h9ps://  •  Include:  •  Ini-alize:  BOOMR.init({      beacon_url:  "h9p://",      site_domain:  "”  });  <script  src="/javascript/boomerang.js"  type="text/javascript"></script>  Documentation -
  17. 17. App Instrumentation : boomerang.js•  Page  Load  Times  •  Bandwidth  and  Latency  •  Perceived  Performance  •  Page  Components  Load  Time    •  Custom  Measurements  BOOMR.plugins.RT.startTimer(“t_-mer”);    //some  foo  BOOMR.plugins.RT.endTimer(“t_-mer”);    BOOMR.plugins.RT.startTimer("t_head");!...!BOOMR.plugins.RT.endTimer(“t_head”).startTimer(“t_body”);!!!  BOOMR.page_ready();!  
  18. 18. Mobile Infrastructure
  19. 19. •  Webdriver for interactive scenarioso  Multi page flows•  No Webdriver within instrumented intervalso  Click -> Wait -> Measure•  Real device performance metricso  Beacon to server / loggingo  Webdriver to collect json•  Aggregate over multiple iterations !•  Simulate Networko  Bandwidth/Latency/Packet Loss•  Visualize Data•  Test -> Measure -> Analyze -> Fix -> RepeatTesting Methodology
  20. 20. Continuous Integration
  21. 21. Simulator Based CI
  22. 22. Code Checkin Build/deploy WarmupLaunchSimulator andrun testsSimulator Based CI
  23. 23. iOS
  24. 24. AutomationUI Automation FrameworkFrank Testing FrameworkiOS-Driver (selenium based!!)
  25. 25. LoggingTime spent in server + networkTotal time spent on client SideTime to first actionable screen
  26. 26. Caching metrics: Time spent instoring/retrieving data, size ofcache, cache eviction counter.
  27. 27. Data processing metrics: Time spentin serializing/de-serializing JSONresponse, massaging data etc.
  28. 28. Time spent in Controller/datalayer, massaging the data etc
  29. 29. My precious….
  30. 30. Stress /longevity testing-Apple script to generate memory warnings <git repo>-UI Automation script to continuously go over all the mostused scenarios-Monkey script <git repo>Stress/longevity testingSimulator longevity tests withsynthetic memory warnings usingapple scriptMonkey Script! It goes berserk onyour UI!Repeated user scenarios test onphysical device
  31. 31. Instruments –iOS ProfilingMemory Leaks !CPU monitorProfiling
  32. 32. Android
  33. 33. Build an Android lab that can collect real deviceperformance metrics in a fully automatic way!
  34. 34. With Selendroid + iWebDriver/ChromeDriverWebNative
  35. 35. Heap Usage on Android•  Dalvik runtime is garbage collected•  Heap size limit is increasing on deviceso  Device specifico  16MB, 48MB, 64MB...o  android:largeHeap="true"•  We are good, right?
  36. 36. Never tell a woman that she will get alarger wardrobe…
  37. 37. GC Log Message is NOT enough!
  38. 38. Heap Dumps•  Get a heap dump:o  Dump hprof File in DDMSo  android.os.Debug.dumpHprofData()•  Convert hprof file to standard formato  hprof-conv android.hprof standard.hprof•  Analyze hprof fileo  Eclipse Memory Analyzer (MAT)
  39. 39. War Stories
  40. 40. PerceivedPerformance
  41. 41. Do Things Early or Late!•  Early:o  Important elemento  Viewpoint contento  Start actions early•  Late:o  Async cleanup
  42. 42. Preloading LoginFool your browser for faster speed.
  43. 43. Fool your browser for faster speed•  Request Javascript as images while user typing password!•  Observation:o  Typing password on mobile devices takes time!•  Fact in Browser:o  Images, CSS are downloaded in parallelo  Javascript is downloaded single-threaded
  44. 44. Mobile Lab EvolutionWhat the heckis mobileperformance?Can I try withstop clock?Haha, I havecustominstrumentation.I am buildingan automatedmobile lab.Automatedeverything, offto get a BEER !
  45. 45. Takeaways•  Mobile Performance is hard, but exciting•  Decide what and how to measure your apps•  Consider the app stability•  Set up automation, get rid of stopclock
  46. 46. Future Series•  Building a Performance Lab for Mobile Apps in a Dayo  Tutorial Session at Velocity New York, 3:30pm Oct 14o•  Dont Let Me Wait: Client-side Optimizationo  Workshop Session at Grace Hopper Conference, 5:15pm Oct 3o
  47. 47. Booth #215