useful tools to automate to use web performance metric

1. 1
Automate web performance metrics
how to measure cdn
performance?
Marc Cortinas – Production Service - Webops - August 2014

2. 2
Agenda
Why?
Increase web preformance = Sales
Motivation PS-9063
1. How to measure web performance metrics?
2. How to automated web testing?
3. Combine: automate + web performance
4. What’s NodeJS? PhantomJS? QtWebKit?
5. Phantomas – Features
6. Phantomas – Metric Choice
7. Arquitecture
8. Grafana Dashboard
9. Which CDN are we goind to measure?

3. Tools used to measure web performance metrics
MAIN OBJECTIVE:
Enumerate different tools to calculate web performance in our application
Tools:
1. Pluguins/Browser features:  Firebug, HTTP Tamper Data, Chrome
Developer Tools, Firefox Developer Tools, etc… 
2. applications: Fiddler, Har 101(maybe gained by
phantomjs)+Tools(yslow),phantomas-grunt
3. $$ External Services $$ : Iplabel, Monitis, Pingdom, Apica, AlertSite
4. Others solutions:
3
1. Nagios + check_http
2. Time + Curl/wget
3. Monit + http check
4. Munin + http_loadtime

4. 4
Tools used to automate web testing
MAIN OBJECTIVE:
Enumerate different tools to automate web testing
Tools:
1. Selenium Grid + Jenkins
2. Others +++ Help me!!! 

5. Combine: automate + web performance
MAIN OBJECTIVE:
Automate some web performance metric solution
Candidates: (all based on NodeJS)
1. PhantomJS + Yslow : HAR + YSLOW
 Yslow: less features than phantomas
5
2. Grunt-phantomas :
 Build-GUI from phantomas is so limited
 Data stored in JSON file
3. Our solution: Grafana + Phantomas
1. Phantomas to collect data in CSV files
2. phan2graph.py Python Script to send to graphite
3. One sensor in each AWS EC2 region (blog account)
 Micro instances is not allowed on VPC (ireland)
4. Sync whisper files(+backup) VS multiples datasources in
Grafana(+realtime)

6. What’s NodeJS? PhantomJS? QtWebKit?
MAIN OBJECTIVE:
Describe basic concepts to understand phantomas:
1. NodeJS: is a platform built on Chrome's JavaScript runtime for easily building
6
fast, scalable network applications
2. PhantomJS: PhantomJS is a headless WebKit scriptable with a JavaScript
API. It has fast and native support for various web standards: DOM handling,
CSS selector, JSON, Canvas, and SVG. Based on QtWebKit
3. QtWebKit: web content engine based on the Open Source WebKit project,
the same engine used by browsers such as Google Chrome and Safari from
Apple.

7. 7
Phantomas - Features
MAIN OBJECTIVE:
List of main useful features
Features:
1.  Command line tool 
2. Internal solution, I prefer that external solution
3. Full-load page,
4. 3rd parties calls can be discarted (explicit list of domain allowed)
5. Multiple runs to improve median datas
6. Differents output data formats: JSON, CSV, ElasticSearch,
7. Metrics can be sent to statds, elasticsearch (graphite too now! )
8. Timeout parameter
9. 2 web engine avaible: qtwebkit (default) or gecko (experimental)
10. User-agent switcher
11. HAR file export
12. Screenshot to SVG

8. 8
Phantomas – Metrics Choice
MAIN OBJECTIVE:
List of metrics: https://github.com/macbre/phantomas
Metrics selected to choice CDN:
1. Median Latency: median value of time to the first byte for all responses
2. Full Load Page Time: time it took to receive the last byte of the last HTTP
response
3. Number of assets not gziped: static assets that were not gzipped
4. Fastest Response: the time to the last byte of the fastest response
Metrics not analized yet but maybe is interesting:
timeToFirstCss: time it took to receive the last byte of the first CSS
timeToFirstJs: time it took to receive the last byte of the first JS
timeToFirstImage: time it took to receive the last byte of the first image
cachingNotSpecified: responses with no caching header sent (either Cache-Control or Expires)
cachingTooShort: responses with too short (less than a week) caching time
cachingDisabled: responses with caching disabled (max-age=0)
oldCachingHeaders: responses with old, HTTP 1.0 caching headers (Expires and Pragma)

9. 9
Architecture
CSV
phant2graph.py
Grafana
phant
AWS
Cogent
Colt
other..
Akamai
Fastly
webop
Graphite Graphite
rsync
whisper file whisper file
Code Name Graphite URL
ap-southeast-1 Singapore http://phant-singapore.services.odigeo.com/dashboard/ - Singapore
ap-southeast-2 Sydney http://phant-sydney.services.odigeo.com/dashboard/ - Sydney
eu-west-1 Ireland http://phant-ireland.services.odigeo.com/dashboard/ - Ireland
sa-east-1 Sao Paulo http://phant-saopaulo.services.odigeo.com/dashboard/ - Saopaulo
us-east-1 Northern Virginia http://phant-virginia.services.odigeo.com/dashboard/ - Virginia
us-west-1 Northern California http://phant-california.services.odigeo.com/dashboard/ - California

10. 10
Grafana Dashboard
• Web performance by CDN
• Web performance by Region

11. Which CDN are we goind to measure?
1. All content: the main domain browseable throught CDN
1. AWS Cloudfront
2. Fastly
3. Limelight
4. Akamai
5. EdgeCast
6. Own
7. …
8. Multi-CDN (JS detect the region and select CDN)
2. Only static: exclusive domain for static content
11
1. AWS Cloudfront
2. Fastly

12. 12
Thanks... Questions?