Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Managing application performance by Kwame Thomison

92 views

Published on

Managing application performance is a huge challenge for most engineering organizations. The most difficult questions we have to answer often only indirectly related to shortening the critical path. How do you align stakeholders with competing priorities and agendas? How do you drum up interest, persuade your best engineers to focus on performance, and keep the team motivated? How should the the team be positioned within the larger organization? Where do you even start? You can't find good answers to any of those questions with a simple web search.

In this talk, Kwame Thomison shares tips for creating a perf team with a clear, compelling mission, a few frameworks he’s devised for thinking about web performance, and some of the lessons he’s learned working on perf teams at companies like Meebo, Facebook, and Asana.

Published in: Technology
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Managing application performance by Kwame Thomison

  1. 1. Cheat Codes for Managing Application Performance Kwame Thomison Magnetic Inc.
  2. 2. About Me Meebo: Web based instant messaging Facebook: Social network Asana: Collaboration software
  3. 3. What makes managing app performance so challenging?
  4. 4. What makes managing app performance so challenging? It’s not most engineers’ first choice No true north star for the team Requires your very best engineers Unique skillset Perf work isn’t prioritized early enough
  5. 5. Risks of failing to address those challenges Business metrics suffer Burnout Wasted time Endless cycle of improvement and regression
  6. 6. Agenda Telemetry and Tooling Building Alignment North Star
  7. 7. Telemetry and Tooling Metrics Reasoning About Data Building the Right Tools
  8. 8. Your architecture
  9. 9. What your API expert sees
  10. 10. What your product expert sees
  11. 11. What your product expert sees
  12. 12. Your architecture
  13. 13. Telemetry and Tooling: Metrics
  14. 14. Choosing the right metrics Approximate User Value Use Percentiles Correlative Metrics
  15. 15. Two Popular Perf Metrics Time to Interact (TTI) End to End (E2E)
  16. 16. Choosing the right metrics Approximate User Value Use Percentiles Correlative Metrics
  17. 17. Source: https://www.dynatrace.com/news/blog/why-averages-suck-and-percentiles-are-great/
  18. 18. Source: https://www.dynatrace.com/news/blog/why-averages-suck-and-percentiles-are-great/
  19. 19. Source: https://www.dynatrace.com/news/blog/why-averages-suck-and-percentiles-are-great/
  20. 20. Choosing the right metrics Approximate User Value Use Percentiles Correlative Metrics
  21. 21. Telemetry and Tooling: Reasoning About Data
  22. 22. A quick refresher on graphs
  23. 23. Time Series
  24. 24. Histogram
  25. 25. Cumulative Distribution Function
  26. 26. Time Series: Slow and Steady Degradation • Gradual increase over day/ weeks • No corresponding increase in system load • No discernible pattern when slicing by demographic • Slow rollout of a new service or feature
  27. 27. Time Series: The “blip” • Regression that looks like noise • w/w shows a periodic regression • No correlation with app usage • Cold/warm start mechanic
  28. 28. Histogram: Bimodal distribution • Histogram reveals multiple modes •No pattern when slicing by demographic •Indicates a relatively expensive code path being executed part of the time
  29. 29. CDF: Extreme outliers • The CDF levels off sharply at p99 • Increases even more at p99.9 • What’s broken? • Big fish or celebrities
  30. 30. Telemetry and Tooling: Building the Right Tools
  31. 31. Building the Right Tools: Tool Type Static Analyzers Profilers Predictive Modelers
  32. 32. Building the Right Tools: Develop Workflow Example • Dev in sandbox • Run test suite • Push to staging • Push to prod
  33. 33. Building the Right Tools: Render Pipeline Example • DB Query • Server Processing • Server Response • Client CPU
  34. 34. 3 Tool Types x 4 Steps in Dev Workflow x 4 Steps in Rendering Pipeline x = up to 48 tools So what do you build first?
  35. 35. Prioritizing Tools • Sort by time on the critical path for rendering pipeline • Sort by stages of dev workflow • Sort by tool type • Throw away moonshots
  36. 36. Building Alignment A Common Language Demonstrating Impact Performance Culture
  37. 37. Building Alignment: A Common Language • Perf experts speak a different language • Others often uses neologisms • Loss of trust over time
  38. 38. Building Alignment: A Common Language What do you call this? • JS file • JS bundle • Static asset
  39. 39. Building Alignment: A Common Language Primary concerns for perf • Why has perf regressed? • What impact does a given change have? • How is perf work prioritized?
  40. 40. How do you get people speaking the same language?
  41. 41. Define a perf grammar!
  42. 42. Example {magnitude} {sequence} {percentile} {stage of work}
  43. 43. -100ms p75 critical path client CPU
  44. 44. Building Alignment: Demonstrating Your Impact • Display measured TTI on Every Page Load • Share Experiments From Other Companies • AB Test Your Optimizations • Conduct Your Own Study
  45. 45. Conducting Your Own Study: Bucketing Users • Bucket sessions into perf classes (e.g. < 2s, > 2s) • Examine business metrics for each perf class • Infer causal relationship between perf and those business metrics • You may be misled into inferring a causal relationship where one doesn’t exist
  46. 46. Conducting Your Own Study: Artificial Perf Manipulation • Introduce an artificial perf regression/improvement • Measure differences between control and experiment groups • Infer causal relationship between perf and business metrics • You’re hurting the UX intentionally • You might infer a linear relationship where one doesn’t exist
  47. 47. Conducting Your Own Study: A Longterm Holdout Group • For a portion of your users, don’t ship any optimizations • Review business metrics after several weeks/months • Infer causal relationship between perf and business metrics • Potentially very complex to implement
  48. 48. Building Alignment: Cultivating a Performance Culture
  49. 49. What does a healthy culture of managing app performance look like?
  50. 50. People have a shared definition for and approach to achieving excellence
  51. 51. Cultivating a Performance Culture • Clear ownership • Prioritization • Tooling and Documentation • Levers
  52. 52. You’re out to build allies!
  53. 53. Unclear Priorities or Lack of Education or Lack of Tooling = Adversaries
  54. 54. Shared Priorities and Performance Education and Data, Tools, and APIs = Allies
  55. 55. The North Star
  56. 56. North Star • Architecture Strategy • Team Mission
  57. 57. North Star: Architecture Strategy • What’s your perf cost per incremental feature? • Use that lens to guide technology decisions • Give engineers control over rendering pipeline • Pay or Parallelize
  58. 58. North Star: Architecture Strategy • What’s your perf cost per incremental feature? • Use that lens to guide technology decisions • Give engineers control over rendering pipeline • Pay or Parallelize
  59. 59. North Star: Team Mission • Empower other engineers • Work on the hardest problems • Minimize perf cost per incremental feature • Focus on the big picture
  60. 60. Summing it all up • Tips for Telemetry and Tooling • Aligning the Company • Defining a North Star
  61. 61. Let’s make the entire web faster!
  62. 62. Questions? kwame@magnetic-inc.com

×