Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
@atlassian #atlascamp
Show Me the Numbers
Automated Browser Performance Testing

Chris Mountford
Senior Software Developer, JIRA
Atlassian
@chro...
Chris Mountford

2005
3.4.1
185
16660
3
Chris Mountford

4
Chris Mountford

4
Chris Mountford
Inline Edit

4
Chris Mountford
Inline Edit

4
Chris Mountford
Inline Edit

Search

4
Chris Mountford
Inline Edit

Search

4
Chris Mountford
Inline Edit

Search
Detail View
4
Chris Mountford
Inline Edit

Search
Detail View
4
Browser Performance
Testing
Welcome!
Browser Performance Testing

“of specific interactions for a single user
Measuring the perceived performance

”
6
Browser Performance Testing

“of specific interactions for a single user
Measuring the perceived performance
perceived

”
7
Outline
• Intro

• Optimisation Techniques

• JavaScript Renaissance
Engineering

• Perception

• Performance Factors

• G...
Outline
• Intro

• Optimisation Techniques

• JavaScript Renaissance
Engineering

• Perception

• Performance Factors

• G...
JavaScript Timeline
A history
lesson
1995

Classical JavaScript

10
Browser Wars

10
Browser Wars

Dark Ages

10
Dark Ages

Firebug

10
V8

Firebug

10
V8

Node.js

Firebug

10
Node.js

Hipsters

10
11
12
13
“Why do we split everything
into so many files?”

13
“Why do we split everything
into so many files?”

“Performance.”
13
14
“Why do we merge all the files
into large batches?”

14
“Why do we merge all the files
into large batches?”

“Performance.”
14
15
“Why...”

15
“Why...”

“Performance!”
15
“Why...”

“Performance!”
15
“

Show me the numbers!

”

– Not Jacky Chan

Hey it’s
Jacky

16
“ If all we have are opinions, let’s go
with mine.
”
If we have data, let’s look at data.

Netscape
CEO

– Jim Barksdale

...
Factors That Influence
Browser Performance
sad panda
Mobile Web
• Everything is the same but worse
• network / radio power mode steps, sleep, etc.
• cpu / battery impact
• mem...
Network
Time
20
Parse
Time
21
JavaScript Execution
• JS Engines
• native bind(), map() etc

• DOM Manipulation
• events

• JQuery
• Sizzle selectors
• m...
Blocking

23
Blocking
<script src=”wtf.js”>
23
Top Tools
Chrome DevTools
Firebug Lite
dynaTrace

24
Reflow
26
inspired by @stoyanstefanov

27
Will It Reflow?
inspired by @stoyanstefanov

27
Will It Reflow?
inspired by @stoyanstefanov

27
Yes!

Will It Reflow?
inspired by @stoyanstefanov

27
But wait, there’s more!
• CSS Transforms
• Hardware acceleration?

• Data weight
• Concurrent request count
• Inline Base6...
But wait, there’s more!
• CSS Transforms
• Hardware acceleration?

• Data weight
• Concurrent request count
• Inline Base6...
Designing the JIRA
Browser Performance
Test Suite
Sweet!
JIRA Browser Perf Tests

30
JIRA Browser Perf Tests

Page
Objects
30
JIRA Browser Perf Tests

Page
Objects

Soke
Framework
30
JIRA Browser Perf Tests

Page
Objects

Soke
Framework

Bamboo
30
JIRA Browser Perf Tests

Page
Objects

Soke
Framework

Plugin

Bamboo
30
JIRA Browser Perf Tests

Page
Objects

Soke
Framework

Plugin

DB

Bamboo
30
JIRA Browser Perf Tests

Hacking
Page
Objects

Soke
Framework

Plugin

DB

Bamboo
30
JIRA Browser Perf Tests
Koto
Charts
Hacking
Page
Objects

Soke
Framework

Plugin

DB

Bamboo
30
JIRA Browser Perf Tests
Koto
Charts

Psycho
Events
Hacking

Page
Objects

Soke
Framework

Plugin

DB

Bamboo
30
JIRA Browser Perf Tests
Koto
Charts

Psycho
Events

OnDemand
Analytics

Hacking
Page
Objects

Soke
Framework

Plugin

DB

...
JIRA Browser Perf Tests
Baselines
Koto
Charts

Psycho
Events

OnDemand
Analytics

Hacking
Page
Objects

Soke
Framework

Pl...
JIRA Browser Perf Tests
Baselines
Koto
Charts

Vigilance

Psycho
Events

OnDemand
Analytics

Hacking
Page
Objects

Soke
Fr...
Soke Interactions
• Define Timed Sections
• Number of Repetitions
• Personas

31
Soke Interactions
• Define Timed Sections
• Number of Repetitions
• Personas

Big ups to
Atlassian
PerfEng
Team

31
Soke Interactions in Scala

32
33
33
34
Optimisation Techniques
• Batching
• Atlassian Plugins, No Worries
• Predictive Fetching
• Guided by Analytics
• Caching
•...
“

I don’t know... it just feels slow.

”

– a JIRA Product Manager

Show me the
numbers!

36
Perception is Reality
No, it only
seems that way
38
Psycho Latency
• Ready for Action
• When does the User believe the app is waiting for her?

• Manual “psycho” event placem...
Gotchas
I’m gonna
click( )
Selenium / Webdriver

42
Selenium / Webdriver
• Chromedriver implemented by Chromium team

42
Selenium / Webdriver
• Chromedriver implemented by Chromium team
• mouse click on geometric centre of target

42
Selenium / Webdriver
• Chromedriver implemented by Chromium team
• mouse click on geometric centre of target
• W3C Webdriv...
Selenium / Webdriver
• Chromedriver implemented by Chromium team
• mouse click on geometric centre of target
• W3C Webdriv...
Selenium / Webdriver
• Chromedriver implemented by Chromium team
• mouse click on geometric centre of target
• W3C Webdriv...
Selenium / Webdriver

43
Selenium / Webdriver
• Windows and IE process control

43
Selenium / Webdriver
• Windows and IE process control
• Browser auto-upgrades

43
Selenium / Webdriver
• Windows and IE process control
• Browser auto-upgrades
• Changing what you measure

43
Selenium / Webdriver
• Windows and IE process control
• Browser auto-upgrades
• Changing what you measure
• Prefetch cache...
Take Aways
• Lots of factors
• Measure perceived reality
• Be holistic, quantitative, beware micro-optimisation trap
• Mon...
Thank you!
For resources:
#atlascamp
@chromosundrift
Upcoming SlideShare
Loading in …5
×

AtlasCamp 2013: Show Me Number! Automated Browser Performance Testing

904 views

Published on

Published in: Technology, Design
  • Be the first to comment

  • Be the first to like this

AtlasCamp 2013: Show Me Number! Automated Browser Performance Testing

  1. 1. @atlassian #atlascamp
  2. 2. Show Me the Numbers Automated Browser Performance Testing Chris Mountford Senior Software Developer, JIRA Atlassian @chromosundrift 2
  3. 3. Chris Mountford 2005 3.4.1 185 16660 3
  4. 4. Chris Mountford 4
  5. 5. Chris Mountford 4
  6. 6. Chris Mountford Inline Edit 4
  7. 7. Chris Mountford Inline Edit 4
  8. 8. Chris Mountford Inline Edit Search 4
  9. 9. Chris Mountford Inline Edit Search 4
  10. 10. Chris Mountford Inline Edit Search Detail View 4
  11. 11. Chris Mountford Inline Edit Search Detail View 4
  12. 12. Browser Performance Testing Welcome!
  13. 13. Browser Performance Testing “of specific interactions for a single user Measuring the perceived performance ” 6
  14. 14. Browser Performance Testing “of specific interactions for a single user Measuring the perceived performance perceived ” 7
  15. 15. Outline • Intro • Optimisation Techniques • JavaScript Renaissance Engineering • Perception • Performance Factors • Gotchas • Take Aways • JIRA Perf Tests 8
  16. 16. Outline • Intro • Optimisation Techniques • JavaScript Renaissance Engineering • Perception • Performance Factors • Gotchas • Take Aways • JIRA Perf Tests 8
  17. 17. JavaScript Timeline A history lesson
  18. 18. 1995 Classical JavaScript 10
  19. 19. Browser Wars 10
  20. 20. Browser Wars Dark Ages 10
  21. 21. Dark Ages Firebug 10
  22. 22. V8 Firebug 10
  23. 23. V8 Node.js Firebug 10
  24. 24. Node.js Hipsters 10
  25. 25. 11
  26. 26. 12
  27. 27. 13
  28. 28. “Why do we split everything into so many files?” 13
  29. 29. “Why do we split everything into so many files?” “Performance.” 13
  30. 30. 14
  31. 31. “Why do we merge all the files into large batches?” 14
  32. 32. “Why do we merge all the files into large batches?” “Performance.” 14
  33. 33. 15
  34. 34. “Why...” 15
  35. 35. “Why...” “Performance!” 15
  36. 36. “Why...” “Performance!” 15
  37. 37. “ Show me the numbers! ” – Not Jacky Chan Hey it’s Jacky 16
  38. 38. “ If all we have are opinions, let’s go with mine. ” If we have data, let’s look at data. Netscape CEO – Jim Barksdale 17
  39. 39. Factors That Influence Browser Performance sad panda
  40. 40. Mobile Web • Everything is the same but worse • network / radio power mode steps, sleep, etc. • cpu / battery impact • memory • standards compliance • tooling 19
  41. 41. Network Time 20
  42. 42. Parse Time 21
  43. 43. JavaScript Execution • JS Engines • native bind(), map() etc • DOM Manipulation • events • JQuery • Sizzle selectors • micro-optimisation ☠ • Memory leaks 22
  44. 44. Blocking 23
  45. 45. Blocking <script src=”wtf.js”> 23
  46. 46. Top Tools Chrome DevTools Firebug Lite dynaTrace 24
  47. 47. Reflow
  48. 48. 26
  49. 49. inspired by @stoyanstefanov 27
  50. 50. Will It Reflow? inspired by @stoyanstefanov 27
  51. 51. Will It Reflow? inspired by @stoyanstefanov 27
  52. 52. Yes! Will It Reflow? inspired by @stoyanstefanov 27
  53. 53. But wait, there’s more! • CSS Transforms • Hardware acceleration? • Data weight • Concurrent request count • Inline Base64 Encoded resources 28
  54. 54. But wait, there’s more! • CSS Transforms • Hardware acceleration? • Data weight • Concurrent request count • Inline Base64 Encoded resources 28
  55. 55. Designing the JIRA Browser Performance Test Suite Sweet!
  56. 56. JIRA Browser Perf Tests 30
  57. 57. JIRA Browser Perf Tests Page Objects 30
  58. 58. JIRA Browser Perf Tests Page Objects Soke Framework 30
  59. 59. JIRA Browser Perf Tests Page Objects Soke Framework Bamboo 30
  60. 60. JIRA Browser Perf Tests Page Objects Soke Framework Plugin Bamboo 30
  61. 61. JIRA Browser Perf Tests Page Objects Soke Framework Plugin DB Bamboo 30
  62. 62. JIRA Browser Perf Tests Hacking Page Objects Soke Framework Plugin DB Bamboo 30
  63. 63. JIRA Browser Perf Tests Koto Charts Hacking Page Objects Soke Framework Plugin DB Bamboo 30
  64. 64. JIRA Browser Perf Tests Koto Charts Psycho Events Hacking Page Objects Soke Framework Plugin DB Bamboo 30
  65. 65. JIRA Browser Perf Tests Koto Charts Psycho Events OnDemand Analytics Hacking Page Objects Soke Framework Plugin DB Bamboo 30
  66. 66. JIRA Browser Perf Tests Baselines Koto Charts Psycho Events OnDemand Analytics Hacking Page Objects Soke Framework Plugin DB Bamboo 30
  67. 67. JIRA Browser Perf Tests Baselines Koto Charts Vigilance Psycho Events OnDemand Analytics Hacking Page Objects Soke Framework Plugin DB Bamboo 30
  68. 68. Soke Interactions • Define Timed Sections • Number of Repetitions • Personas 31
  69. 69. Soke Interactions • Define Timed Sections • Number of Repetitions • Personas Big ups to Atlassian PerfEng Team 31
  70. 70. Soke Interactions in Scala 32
  71. 71. 33
  72. 72. 33
  73. 73. 34
  74. 74. Optimisation Techniques • Batching • Atlassian Plugins, No Worries • Predictive Fetching • Guided by Analytics • Caching • Watch for memory leaks 35
  75. 75. “ I don’t know... it just feels slow. ” – a JIRA Product Manager Show me the numbers! 36
  76. 76. Perception is Reality No, it only seems that way
  77. 77. 38
  78. 78. Psycho Latency • Ready for Action • When does the User believe the app is waiting for her? • Manual “psycho” event placement • and maintenance! • Yardstick tuning against a real system • Optimisation Example: Execution Reordering 39
  79. 79. Gotchas I’m gonna
  80. 80. click( )
  81. 81. Selenium / Webdriver 42
  82. 82. Selenium / Webdriver • Chromedriver implemented by Chromium team 42
  83. 83. Selenium / Webdriver • Chromedriver implemented by Chromium team • mouse click on geometric centre of target 42
  84. 84. Selenium / Webdriver • Chromedriver implemented by Chromium team • mouse click on geometric centre of target • W3C Webdriver wire protocol draft 42
  85. 85. Selenium / Webdriver • Chromedriver implemented by Chromium team • mouse click on geometric centre of target • W3C Webdriver wire protocol draft • OK Browsers, follow the standard! 42
  86. 86. Selenium / Webdriver • Chromedriver implemented by Chromium team • mouse click on geometric centre of target • W3C Webdriver wire protocol draft • OK Browsers, follow the standard! 42
  87. 87. Selenium / Webdriver 43
  88. 88. Selenium / Webdriver • Windows and IE process control 43
  89. 89. Selenium / Webdriver • Windows and IE process control • Browser auto-upgrades 43
  90. 90. Selenium / Webdriver • Windows and IE process control • Browser auto-upgrades • Changing what you measure 43
  91. 91. Selenium / Webdriver • Windows and IE process control • Browser auto-upgrades • Changing what you measure • Prefetch cache benefit in JIRA 6.0 43
  92. 92. Take Aways • Lots of factors • Measure perceived reality • Be holistic, quantitative, beware micro-optimisation trap • Monitor and Maintain your performance suite • Use Chrome DevTools etc. 44
  93. 93. Thank you!
  94. 94. For resources: #atlascamp @chromosundrift

×