Your SlideShare is downloading. ×
0
Use 10,000 Browsers                      - or -  How to Crowdsource, Archive and Share User   Agent Data in the Open with ...
about:meLindsey SimonGoogle Front-End Developerelsigh@gmail.com / @elsighmon  ● AdWords  ● App Engine  ● i18n  ● Closure l...
Your Development Test Environment
Your Mobile Test Environment
http://en.wikipedia.org/wiki/Usage_share_of_web_browsers
understandingcapabilities and performance
Multi-Prong Attack Strategy
BrowserscopeOpen-source, community-driven project for profiling browsers.Explicit Goals: ● foster innovation by tracking f...
crowdsourcing the results
crowdsourcing● no dedicated test resources
crowdsourcing● no dedicated test resources● project runs in perpetuity
crowdsourcing● no dedicated test resources● project runs in perpetuity● real world test conditions
crowdsourcing● no dedicated test resources● project runs in perpetuity● real world test conditions● aggregating results re...
crowdsourcing● no dedicated test resources● project runs in perpetuity● real world test conditions● aggregating results re...
key characteristics ● more matrix than benchmark
key characteristics ● more matrix than benchmark ● reproducible tests and results (no video or control setup)
key characteristics ● more matrix than benchmark ● reproducible tests and results (no video or control setup) ● gather & d...
anatomy of a browserscope test1. create a web page that populates a JS object.2. login and "Add a test" on browserscope.3....
DEMO!http://goo.gl/LGRqA
Hosted Tests
hosted test categoriesNetwork Performance ● Steve Souders original UA Profiler tests ● connections, caching, parallel down...
hosted test categoriesSecurity ● Collin Jackson(CMU) & Adam Barth(Berkeley) & team ● JSON, x-origin, XSS, sandbox, etc..JS...
Network Results Timelinebrowserscope.org/timeline?category=summary
Acid3 Results Timelinebrowserscope.org/timeline?category=acid3
Summary Results Timelinebrowserscope.org/timeline?category=summary
User Tests
Modernizr 2.0.4 Results Timeline
HTML5 Ready - current desktopmodernizr2 results - browserscope.org/evolution?v=top-d
HTML5 Ready - edge desktopmodernizr2 results - browserscope.org/evolution?v=top-d-e
HTML5 Ready - mobilemodernizr2 results - browserscope.org/evolution?v=top-m
under the hood● App Engine● runtime median engine driven by task queues● memcache, lots of it● closure-library/compiler● g...
data points ● 2M tests run ● 100M discrete data points ● 1308 browsers (600K discrete user agent strings) ● 54K User tests...
roadmap ● code linting based on DOM property support ● mobile / OS distinction ● more data vizlinks:browserscope.orgcode.g...
Thank You Free   Software!
Browserscope oscon 2011
Upcoming SlideShare
Loading in...5
×

Browserscope oscon 2011

1,685

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,685
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
14
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Browserscope oscon 2011"

  1. 1. Use 10,000 Browsers - or - How to Crowdsource, Archive and Share User Agent Data in the Open with BrowserscopeOSCON 2011
  2. 2. about:meLindsey SimonGoogle Front-End Developerelsigh@gmail.com / @elsighmon ● AdWords ● App Engine ● i18n ● Closure library ● Search ● Google Translate ● Google+
  3. 3. Your Development Test Environment
  4. 4. Your Mobile Test Environment
  5. 5. http://en.wikipedia.org/wiki/Usage_share_of_web_browsers
  6. 6. understandingcapabilities and performance
  7. 7. Multi-Prong Attack Strategy
  8. 8. BrowserscopeOpen-source, community-driven project for profiling browsers.Explicit Goals: ● foster innovation by tracking functionality ● push browser innovation, uncover regressions ● historical resource for web developersBorn September 13th, 2009
  9. 9. crowdsourcing the results
  10. 10. crowdsourcing● no dedicated test resources
  11. 11. crowdsourcing● no dedicated test resources● project runs in perpetuity
  12. 12. crowdsourcing● no dedicated test resources● project runs in perpetuity● real world test conditions
  13. 13. crowdsourcing● no dedicated test resources● project runs in perpetuity● real world test conditions● aggregating results reduces bias
  14. 14. crowdsourcing● no dedicated test resources● project runs in perpetuity● real world test conditions● aggregating results reduces bias● new browsers show up immediately
  15. 15. key characteristics ● more matrix than benchmark
  16. 16. key characteristics ● more matrix than benchmark ● reproducible tests and results (no video or control setup)
  17. 17. key characteristics ● more matrix than benchmark ● reproducible tests and results (no video or control setup) ● gather & display historical data, trends
  18. 18. anatomy of a browserscope test1. create a web page that populates a JS object.2. login and "Add a test" on browserscope.3. add browserscope JS snippet(s) to your test page.4. get lots of users to hit your page
  19. 19. DEMO!http://goo.gl/LGRqA
  20. 20. Hosted Tests
  21. 21. hosted test categoriesNetwork Performance ● Steve Souders original UA Profiler tests ● connections, caching, parallel downloads, etc...Rich Text Edit Mode ● Annie Sullivan, Roland Steiner, Ehsan Akhgari ● i.e. cut and paste in browsers, w/ bold, italic, formattingAcid3 ● Ian Hicksons tests, added by Jacob Moon ● javascript, css, renderingSelectors API Test Suite ● John Resigs tests, added by Lindsey Simon ● querySelectorAll DOM retrieval syntax
  22. 22. hosted test categoriesSecurity ● Collin Jackson(CMU) & Adam Barth(Berkeley) & team ● JSON, x-origin, XSS, sandbox, etc..JSKB ● Mike Samuel (Google Caja) ● JavaScript compliance -> compilation ● similar to Joe Hewitts new hascan
  23. 23. Network Results Timelinebrowserscope.org/timeline?category=summary
  24. 24. Acid3 Results Timelinebrowserscope.org/timeline?category=acid3
  25. 25. Summary Results Timelinebrowserscope.org/timeline?category=summary
  26. 26. User Tests
  27. 27. Modernizr 2.0.4 Results Timeline
  28. 28. HTML5 Ready - current desktopmodernizr2 results - browserscope.org/evolution?v=top-d
  29. 29. HTML5 Ready - edge desktopmodernizr2 results - browserscope.org/evolution?v=top-d-e
  30. 30. HTML5 Ready - mobilemodernizr2 results - browserscope.org/evolution?v=top-m
  31. 31. under the hood● App Engine● runtime median engine driven by task queues● memcache, lots of it● closure-library/compiler● gdata python lib + gviz table component● cross domain iframe posting● jQuery
  32. 32. data points ● 2M tests run ● 100M discrete data points ● 1308 browsers (600K discrete user agent strings) ● 54K User tests ● 21GB of data ● ~ 10QPS ● High traffic integrations: ○ jsperf.com/popular ○ www.modernizr.comUser Test demo: browserscope-test.appspot.com/Google UI demo: groupmenuselect.appspot.com/results
  33. 33. roadmap ● code linting based on DOM property support ● mobile / OS distinction ● more data vizlinks:browserscope.orgcode.google.com/p/browserscopegroups.google.com/group/browserscope
  34. 34. Thank You Free Software!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×