SlideShare a Scribd company logo
1 of 24
Download to read offline
U S I N G
H E AT M A P S
T O D E F I N E
P E R F O R M A N C E
M E T R I C S
S E R G E Y C H E R N Y S H E V
J U N E 2 7 , 2 0 1 6 @ N E W Y O R K
W E B P E R F O R M A N C E M E E T U P
W H Y D O W E M E A S U R E S P E E D ?
• Monitor for degradations (Ops)
• Analyze code for perf issues (Devs)
• Verify improvements (Devs)
• Prioritize improvements (Business)
• Budget for WPO initiatives (Business)
S TAT S
• Google: +500ms => -25% searches (2006)
• Amazon: +100ms => -1% revenue (2008)
• Netflix: +GZip => -43% Traffic cost (2008)
• Shopzilla: -5s => +12% Conversion rate (2009)
• Google: +400ms => -0.21% searches after experiment! (2009)
• Mozilla: -2.2s => +15.4% Downloads (2010)
• Edmunds: -77% load time => +20% page views (2011)
• DoubleClick: -1 redirect => +12% CTR (2011)
• Etsy: +160Kb => +12% bounce rate (2014)
• Trainline: -0.3s => +$11.5M / year revenue (2016)
WPOStats.com
R E A L U S E R M E A S U R E M E N T
• Real users (a lot of them)
• A lot of data (need to store it)
• All noise you can get, requires filtering
• Metrics are distributions
• Can correlate to business KPIs
S Y N T H E T I C T E S T I N G &
A N A LY S I S
• From particular location
• Tester controls instrumentation
• One metric value
• Data can have lots of details for analysis
T O D AY ' S M E T R I C S
• DNS, SSL/TLS, TTFB
• Page Load / Document Complete / Fully Loaded
• First Paint
• AFT
• SpeedIndex
T I M E T O F I R S T PA I N T
• Shows when completely useless part is over
TTFP
3.5s
A F T ( A B O V E T H E F O L D T I M E )
• When everything is finally visible
AFT
15.3s
S P E E D I N D E X
• Unifies rendering progress in one number
Speed Index
8618
U S E R T I M I N G A P I
• Records custom JS timings on the page
• Measured by both RUM and Synthetic tools
• Can be hard to match with user's reality
• Requires JavaScript instrumentation
M E A S U R E U S E R E X P E R I E N C E
• Great experience for users
• "Fast" is just a component
• Correlate what you measure
to business KPIs
• Do not measure what's easy,
measure what matters
N O T T E C H N O L O G Y
H E AT
M A P S
D I S C O V E R A N D
D E F I N E
W H AT M AT T E R S
U S I N G
U S E R A C T I O N H E AT M A P S
• Eye tracking
• Click tracking
• Hard to capture
• Shows current state
• Can be automated
P R O D U C T F E AT U R E H E AT M A P S
• Hard to automate
• Each industry is different
• Usually multiple templates
that power many pages
• Business team already
knows the answers
I N T E R A C T I O N O N T H E W E B
1. Acknowledge action
2. Verify destination
3. Provide primary content
4. Allow interaction
5. Show secondary content
6. Invisible tasks
D E S I G N C O M P O N E N T S
• Core Branding &
Main Navigation
• Primary content
H E AT M A P C U T- O U T S
Core Branding & Main Navigation Primary content
C O R E B R A N D I N G H E AT M A P
C O R E B R A N D I N G H E AT M A P
Core Branding & Main Navigation
8s
8s
3.5s
15.3s
P R I M A RY C O N T E N T H E AT M A P
11s
P R I M A RY C O N T E N T H E AT M A P
11s
Primary Content
3.5s
15.3s
E X T E N D E D M E T R I C S
• Time To First Paint (TTFP) - 3.5s
• Core Branding & Main Navigation - 8s
• Primary content - 11s
• ...
• Above The Fold (AFT) - 15.3s
S E L E C T O R - B A S E D I M P L E M E N TAT I O N
• Uses CSS selectors to define active zones
• Executes "custom metric" function in WebPageTest to
get cutout coordinates
• Downloads test results and video frames using WPT API
• Manipulates video frames and feeds into
visualmetrics.py and uses 99% threshold to grab times
• Kudos to Patrick Meenan for awesome tools
T H A N K Y O U !
sergey.chernyshev@gmail.com
@SergeyChe

More Related Content

More from SergeyChernyshev

Fastest request is never made
Fastest request is never madeFastest request is never made
Fastest request is never madeSergeyChernyshev
 
Extending your applications to the edge with CDNs
Extending your applications to the edge with CDNsExtending your applications to the edge with CDNs
Extending your applications to the edge with CDNsSergeyChernyshev
 
What we can learn from CDNs about Web Development, Deployment, and Performance
What we can learn from CDNs about Web Development, Deployment, and PerformanceWhat we can learn from CDNs about Web Development, Deployment, and Performance
What we can learn from CDNs about Web Development, Deployment, and PerformanceSergeyChernyshev
 
Scalability vs. Performance
Scalability vs. PerformanceScalability vs. Performance
Scalability vs. PerformanceSergeyChernyshev
 
Web performance: beyond load testing
Web performance: beyond load testingWeb performance: beyond load testing
Web performance: beyond load testingSergeyChernyshev
 
Introduction to web performance @ IEEE
Introduction to web performance @ IEEEIntroduction to web performance @ IEEE
Introduction to web performance @ IEEESergeyChernyshev
 
Introduction to Web Performance
Introduction to Web PerformanceIntroduction to Web Performance
Introduction to Web PerformanceSergeyChernyshev
 
Performance anti patterns in ajax applications
Performance anti patterns in ajax applicationsPerformance anti patterns in ajax applications
Performance anti patterns in ajax applicationsSergeyChernyshev
 
Keeping track of your performance using Show Slow
Keeping track of your performance using Show SlowKeeping track of your performance using Show Slow
Keeping track of your performance using Show SlowSergeyChernyshev
 
Keeping track of your performance using show slow
Keeping track of your performance using show slowKeeping track of your performance using show slow
Keeping track of your performance using show slowSergeyChernyshev
 
Building your own CDN using Amazon EC2
Building your own CDN using Amazon EC2Building your own CDN using Amazon EC2
Building your own CDN using Amazon EC2SergeyChernyshev
 
Php Code Profiling Using X Debug
Php Code Profiling Using X DebugPhp Code Profiling Using X Debug
Php Code Profiling Using X DebugSergeyChernyshev
 
Open Source Hardware: A Start...
Open Source Hardware: A Start...Open Source Hardware: A Start...
Open Source Hardware: A Start...SergeyChernyshev
 

More from SergeyChernyshev (20)

Fastest request is never made
Fastest request is never madeFastest request is never made
Fastest request is never made
 
Extending your applications to the edge with CDNs
Extending your applications to the edge with CDNsExtending your applications to the edge with CDNs
Extending your applications to the edge with CDNs
 
Speed is feature #1
Speed is feature #1Speed is feature #1
Speed is feature #1
 
Tools of the trade 2014
Tools of the trade 2014Tools of the trade 2014
Tools of the trade 2014
 
What we can learn from CDNs about Web Development, Deployment, and Performance
What we can learn from CDNs about Web Development, Deployment, and PerformanceWhat we can learn from CDNs about Web Development, Deployment, and Performance
What we can learn from CDNs about Web Development, Deployment, and Performance
 
Scalability vs. Performance
Scalability vs. PerformanceScalability vs. Performance
Scalability vs. Performance
 
Web performance: beyond load testing
Web performance: beyond load testingWeb performance: beyond load testing
Web performance: beyond load testing
 
Introduction to web performance @ IEEE
Introduction to web performance @ IEEEIntroduction to web performance @ IEEE
Introduction to web performance @ IEEE
 
Introduction to Web Performance
Introduction to Web PerformanceIntroduction to Web Performance
Introduction to Web Performance
 
Performance anti patterns in ajax applications
Performance anti patterns in ajax applicationsPerformance anti patterns in ajax applications
Performance anti patterns in ajax applications
 
Taming 3rd party content
Taming 3rd party contentTaming 3rd party content
Taming 3rd party content
 
Velocity 2010 review
Velocity 2010 reviewVelocity 2010 review
Velocity 2010 review
 
Keeping track of your performance using Show Slow
Keeping track of your performance using Show SlowKeeping track of your performance using Show Slow
Keeping track of your performance using Show Slow
 
Keeping track of your performance using show slow
Keeping track of your performance using show slowKeeping track of your performance using show slow
Keeping track of your performance using show slow
 
In Search of Speed
In Search of SpeedIn Search of Speed
In Search of Speed
 
Building your own CDN using Amazon EC2
Building your own CDN using Amazon EC2Building your own CDN using Amazon EC2
Building your own CDN using Amazon EC2
 
Php Code Profiling Using X Debug
Php Code Profiling Using X DebugPhp Code Profiling Using X Debug
Php Code Profiling Using X Debug
 
Why RDFa?
Why RDFa?Why RDFa?
Why RDFa?
 
Introduction to RDFa
Introduction to RDFaIntroduction to RDFa
Introduction to RDFa
 
Open Source Hardware: A Start...
Open Source Hardware: A Start...Open Source Hardware: A Start...
Open Source Hardware: A Start...
 

Recently uploaded

ETHICAL HACKING dddddddddddddddfnandni.pptx
ETHICAL HACKING dddddddddddddddfnandni.pptxETHICAL HACKING dddddddddddddddfnandni.pptx
ETHICAL HACKING dddddddddddddddfnandni.pptxNIMMANAGANTI RAMAKRISHNA
 
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书zdzoqco
 
Unidad 4 – Redes de ordenadores (en inglés).pptx
Unidad 4 – Redes de ordenadores (en inglés).pptxUnidad 4 – Redes de ordenadores (en inglés).pptx
Unidad 4 – Redes de ordenadores (en inglés).pptxmibuzondetrabajo
 
IP addressing and IPv6, presented by Paul Wilson at IETF 119
IP addressing and IPv6, presented by Paul Wilson at IETF 119IP addressing and IPv6, presented by Paul Wilson at IETF 119
IP addressing and IPv6, presented by Paul Wilson at IETF 119APNIC
 
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书rnrncn29
 
『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书
『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书
『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书rnrncn29
 
TRENDS Enabling and inhibiting dimensions.pptx
TRENDS Enabling and inhibiting dimensions.pptxTRENDS Enabling and inhibiting dimensions.pptx
TRENDS Enabling and inhibiting dimensions.pptxAndrieCagasanAkio
 
Film cover research (1).pptxsdasdasdasdasdasa
Film cover research (1).pptxsdasdasdasdasdasaFilm cover research (1).pptxsdasdasdasdasdasa
Film cover research (1).pptxsdasdasdasdasdasa494f574xmv
 
Top 10 Interactive Website Design Trends in 2024.pptx
Top 10 Interactive Website Design Trends in 2024.pptxTop 10 Interactive Website Design Trends in 2024.pptx
Top 10 Interactive Website Design Trends in 2024.pptxDyna Gilbert
 
Company Snapshot Theme for Business by Slidesgo.pptx
Company Snapshot Theme for Business by Slidesgo.pptxCompany Snapshot Theme for Business by Slidesgo.pptx
Company Snapshot Theme for Business by Slidesgo.pptxMario
 
SCM Symposium PPT Format Customer loyalty is predi
SCM Symposium PPT Format Customer loyalty is prediSCM Symposium PPT Format Customer loyalty is predi
SCM Symposium PPT Format Customer loyalty is predieusebiomeyer
 

Recently uploaded (11)

ETHICAL HACKING dddddddddddddddfnandni.pptx
ETHICAL HACKING dddddddddddddddfnandni.pptxETHICAL HACKING dddddddddddddddfnandni.pptx
ETHICAL HACKING dddddddddddddddfnandni.pptx
 
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书
办理多伦多大学毕业证成绩单|购买加拿大UTSG文凭证书
 
Unidad 4 – Redes de ordenadores (en inglés).pptx
Unidad 4 – Redes de ordenadores (en inglés).pptxUnidad 4 – Redes de ordenadores (en inglés).pptx
Unidad 4 – Redes de ordenadores (en inglés).pptx
 
IP addressing and IPv6, presented by Paul Wilson at IETF 119
IP addressing and IPv6, presented by Paul Wilson at IETF 119IP addressing and IPv6, presented by Paul Wilson at IETF 119
IP addressing and IPv6, presented by Paul Wilson at IETF 119
 
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书
『澳洲文凭』买拉筹伯大学毕业证书成绩单办理澳洲LTU文凭学位证书
 
『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书
『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书
『澳洲文凭』买詹姆士库克大学毕业证书成绩单办理澳洲JCU文凭学位证书
 
TRENDS Enabling and inhibiting dimensions.pptx
TRENDS Enabling and inhibiting dimensions.pptxTRENDS Enabling and inhibiting dimensions.pptx
TRENDS Enabling and inhibiting dimensions.pptx
 
Film cover research (1).pptxsdasdasdasdasdasa
Film cover research (1).pptxsdasdasdasdasdasaFilm cover research (1).pptxsdasdasdasdasdasa
Film cover research (1).pptxsdasdasdasdasdasa
 
Top 10 Interactive Website Design Trends in 2024.pptx
Top 10 Interactive Website Design Trends in 2024.pptxTop 10 Interactive Website Design Trends in 2024.pptx
Top 10 Interactive Website Design Trends in 2024.pptx
 
Company Snapshot Theme for Business by Slidesgo.pptx
Company Snapshot Theme for Business by Slidesgo.pptxCompany Snapshot Theme for Business by Slidesgo.pptx
Company Snapshot Theme for Business by Slidesgo.pptx
 
SCM Symposium PPT Format Customer loyalty is predi
SCM Symposium PPT Format Customer loyalty is prediSCM Symposium PPT Format Customer loyalty is predi
SCM Symposium PPT Format Customer loyalty is predi
 

Using Heat Maps to improve Web Performance Metrics @ NY Web Performance Meetup

  • 1. U S I N G H E AT M A P S T O D E F I N E P E R F O R M A N C E M E T R I C S S E R G E Y C H E R N Y S H E V J U N E 2 7 , 2 0 1 6 @ N E W Y O R K W E B P E R F O R M A N C E M E E T U P
  • 2. W H Y D O W E M E A S U R E S P E E D ? • Monitor for degradations (Ops) • Analyze code for perf issues (Devs) • Verify improvements (Devs) • Prioritize improvements (Business) • Budget for WPO initiatives (Business)
  • 3. S TAT S • Google: +500ms => -25% searches (2006) • Amazon: +100ms => -1% revenue (2008) • Netflix: +GZip => -43% Traffic cost (2008) • Shopzilla: -5s => +12% Conversion rate (2009) • Google: +400ms => -0.21% searches after experiment! (2009) • Mozilla: -2.2s => +15.4% Downloads (2010) • Edmunds: -77% load time => +20% page views (2011) • DoubleClick: -1 redirect => +12% CTR (2011) • Etsy: +160Kb => +12% bounce rate (2014) • Trainline: -0.3s => +$11.5M / year revenue (2016) WPOStats.com
  • 4. R E A L U S E R M E A S U R E M E N T • Real users (a lot of them) • A lot of data (need to store it) • All noise you can get, requires filtering • Metrics are distributions • Can correlate to business KPIs
  • 5. S Y N T H E T I C T E S T I N G & A N A LY S I S • From particular location • Tester controls instrumentation • One metric value • Data can have lots of details for analysis
  • 6. T O D AY ' S M E T R I C S • DNS, SSL/TLS, TTFB • Page Load / Document Complete / Fully Loaded • First Paint • AFT • SpeedIndex
  • 7. T I M E T O F I R S T PA I N T • Shows when completely useless part is over TTFP 3.5s
  • 8. A F T ( A B O V E T H E F O L D T I M E ) • When everything is finally visible AFT 15.3s
  • 9. S P E E D I N D E X • Unifies rendering progress in one number Speed Index 8618
  • 10. U S E R T I M I N G A P I • Records custom JS timings on the page • Measured by both RUM and Synthetic tools • Can be hard to match with user's reality • Requires JavaScript instrumentation
  • 11. M E A S U R E U S E R E X P E R I E N C E • Great experience for users • "Fast" is just a component • Correlate what you measure to business KPIs • Do not measure what's easy, measure what matters N O T T E C H N O L O G Y
  • 12. H E AT M A P S D I S C O V E R A N D D E F I N E W H AT M AT T E R S U S I N G
  • 13. U S E R A C T I O N H E AT M A P S • Eye tracking • Click tracking • Hard to capture • Shows current state • Can be automated
  • 14. P R O D U C T F E AT U R E H E AT M A P S • Hard to automate • Each industry is different • Usually multiple templates that power many pages • Business team already knows the answers
  • 15. I N T E R A C T I O N O N T H E W E B 1. Acknowledge action 2. Verify destination 3. Provide primary content 4. Allow interaction 5. Show secondary content 6. Invisible tasks
  • 16. D E S I G N C O M P O N E N T S • Core Branding & Main Navigation • Primary content
  • 17. H E AT M A P C U T- O U T S Core Branding & Main Navigation Primary content
  • 18. C O R E B R A N D I N G H E AT M A P
  • 19. C O R E B R A N D I N G H E AT M A P Core Branding & Main Navigation 8s 8s 3.5s 15.3s
  • 20. P R I M A RY C O N T E N T H E AT M A P
  • 21. 11s P R I M A RY C O N T E N T H E AT M A P 11s Primary Content 3.5s 15.3s
  • 22. E X T E N D E D M E T R I C S • Time To First Paint (TTFP) - 3.5s • Core Branding & Main Navigation - 8s • Primary content - 11s • ... • Above The Fold (AFT) - 15.3s
  • 23. S E L E C T O R - B A S E D I M P L E M E N TAT I O N • Uses CSS selectors to define active zones • Executes "custom metric" function in WebPageTest to get cutout coordinates • Downloads test results and video frames using WPT API • Manipulates video frames and feeds into visualmetrics.py and uses 99% threshold to grab times • Kudos to Patrick Meenan for awesome tools
  • 24. T H A N K Y O U ! sergey.chernyshev@gmail.com @SergeyChe