0
A Node.JS Bag of Goodies for Analysing Web                  Traffic      Philip Tellis / philip@lognormal.com              ...
Leave feeback on this talk at joind.in/5967    ConFoo / 2012-03-02   A Node.JS Bag of Goodies for Analysing Web Traffic   2
$ finger philip       Philip Tellis       philip@lognormal.com       @bluesmoon       geek - paranoid - speedfreak       co...
Log-Normal             We do real user web performance analysis                ConFoo / 2012-03-02   A Node.JS Bag of Good...
Log-Normal   This talk covers some of the Node.JS modules we use to do this                              analysis         ...
Examples for this talk         http://bluesmoon.github.com/talks/node-modules/                ConFoo / 2012-03-02   A Node...
–    Node.JS & npmConFoo / 2012-03-02   A Node.JS Bag of Goodies for Analysing Web Traffic   7
0.1 The 2 minute Node.JS Tutorial     node     > m = require(’module’)     > console.log(m.foo)   We’ll be using node v0.6...
0.1 The 2 minute Node.JS Tutorial     node script.js              ConFoo / 2012-03-02   A Node.JS Bag of Goodies for Analy...
0.1 The 2 minute Node.JS Tutorial     echo "console.log(’hello, world!’)" | node   Notice that in JavaScript, semicolons a...
0.2 Installing modules     npm install module   Installs into a local node_modules directory                 ConFoo / 2012...
0.2 Installing modules globally     sudo npm install -g module   Installs into the /usr/lib/node_modules directory        ...
0.2 npm help    npm help npm               ConFoo / 2012-03-02   A Node.JS Bag of Goodies for Analysing Web Traffic   13
0.3 npmjs.org                   search.npmjs.org            ConFoo / 2012-03-02   A Node.JS Bag of Goodies for Analysing W...
What’s the time?                                     13:37?               ConFoo / 2012-03-02    A Node.JS Bag of Goodies ...
1          HTTP LogsConFoo / 2012-03-02   A Node.JS Bag of Goodies for Analysing Web Traffic   16
1.1 UserAgent Parsing    npm install ua-parser    var parser=require(’ua-parser’);    var ua = parser.parse(ua_string);   ...
1.2 IP Addresses    var net = require(’net’);    net.isIP(ip);           // returns 0, 4 or 6  The net module is part of n...
1.3 GeoLocation    npm install geoip-lite    var geo = require(’geip-lite’);    var loc = geo.lookup(ip);    // country, r...
1.3 geoip-lite’s hidden function     var geo = require(’geip-lite’);     geo.cmp(ip1, ip2);     // -1, 0 or 1   Used inter...
1.4 Date formatting     npm install prettydate     var strftime = require(’prettydate’).strftime;     var dstr = strftime(...
1.5 Querystring Parsing     var qs = require(’querystring’);     qs.parse(’name=Larry&name=Moe&name=Curly’);     // { name...
1.6 Creating Hashes    var crypto = require(’crypto’);    var hash = crypto.createHash(’sha1’);    hash.update(data);    v...
1.6 Creating HMACs    var crypto = require(’crypto’);    var hmac = crypto.createHmac(’sha1’, key);    hmac.update(data); ...
What’s the time?                             It should be                                 14:05                           ...
2Statistical AnalysisConFoo / 2012-03-02   A Node.JS Bag of Goodies for Analysing Web Traffic   26
2.1 faststats     npm install fast-stats     var Stats = require(’fast-stats’).Stats;     var s = new Stats().push(1, 2, 3...
2.2 gauss    npm install gauss    var gauss = require(’gauss’);    var set = new gauss.Vector(5, 1, 3, 2, 21);    console....
2.3 statsd    npm install statsd node-statsd      github.com/etsy/statsd      Easy to set up      Requires graphite to plo...
2.3 statsd  var StatsD = require(’node-statsd’).StatsD  c = new StatsD(’example.org’,8125)  c.timing(’node_test.some_servi...
2.4 cube  Disclaimer: I’ve never used this one before    npm install cube      square.github.com/cube/      github.com/squ...
What’s the time?                                     14:20?               ConFoo / 2012-03-02    A Node.JS Bag of Goodies ...
Help me out   Still trying to figure out the best way to debug Node.JS memory                      usage. Ideas? Let me kno...
Questions?ConFoo / 2012-03-02   A Node.JS Bag of Goodies for Analysing Web Traffic   34
Leave feeback on this talk at joind.in/5967    ConFoo / 2012-03-02   A Node.JS Bag of Goodies for Analysing Web Traffic   35
Contact me      Philip Tellis      philip@lognormal.com      @bluesmoon      geek - paranoid - speedfreak      co-founder ...
Upcoming SlideShare
Loading in...5
×

A Node.JS bag of goodies for analyzing Web Traffic

5,470

Published on

0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,470
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
33
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Transcript of "A Node.JS bag of goodies for analyzing Web Traffic"

  1. 1. A Node.JS Bag of Goodies for Analysing Web Traffic Philip Tellis / philip@lognormal.com ConFoo / 2012-03-02 ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 1
  2. 2. Leave feeback on this talk at joind.in/5967 ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 2
  3. 3. $ finger philip Philip Tellis philip@lognormal.com @bluesmoon geek - paranoid - speedfreak co-founder @ Log-Normal http://bluesmoon.info/ ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 3
  4. 4. Log-Normal We do real user web performance analysis ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 4
  5. 5. Log-Normal This talk covers some of the Node.JS modules we use to do this analysis ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 5
  6. 6. Examples for this talk http://bluesmoon.github.com/talks/node-modules/ ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 6
  7. 7. – Node.JS & npmConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 7
  8. 8. 0.1 The 2 minute Node.JS Tutorial node > m = require(’module’) > console.log(m.foo) We’ll be using node v0.6.x ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 8
  9. 9. 0.1 The 2 minute Node.JS Tutorial node script.js ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 9
  10. 10. 0.1 The 2 minute Node.JS Tutorial echo "console.log(’hello, world!’)" | node Notice that in JavaScript, semicolons are optional ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 10
  11. 11. 0.2 Installing modules npm install module Installs into a local node_modules directory ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 11
  12. 12. 0.2 Installing modules globally sudo npm install -g module Installs into the /usr/lib/node_modules directory ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 12
  13. 13. 0.2 npm help npm help npm ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 13
  14. 14. 0.3 npmjs.org search.npmjs.org ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 14
  15. 15. What’s the time? 13:37? ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 15
  16. 16. 1 HTTP LogsConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 16
  17. 17. 1.1 UserAgent Parsing npm install ua-parser var parser=require(’ua-parser’); var ua = parser.parse(ua_string); // family, major, minor, patch, os Extracted from Google’s BrowserScope Project ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 17
  18. 18. 1.2 IP Addresses var net = require(’net’); net.isIP(ip); // returns 0, 4 or 6 The net module is part of node ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 18
  19. 19. 1.3 GeoLocation npm install geoip-lite var geo = require(’geip-lite’); var loc = geo.lookup(ip); // country, region, city, ll Uses MaxMind’s GeoIP database. Very fast lookups, IPv4 & v6 ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 19
  20. 20. 1.3 geoip-lite’s hidden function var geo = require(’geip-lite’); geo.cmp(ip1, ip2); // -1, 0 or 1 Used internally to do a binary search on the IP database ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 20
  21. 21. 1.4 Date formatting npm install prettydate var strftime = require(’prettydate’).strftime; var dstr = strftime(new Date, "%c"); Also accepts a locale as a third parameter. ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 21
  22. 22. 1.5 Querystring Parsing var qs = require(’querystring’); qs.parse(’name=Larry&name=Moe&name=Curly’); // { name: [ ’Larry’, ’Moe’, ’Curly’ ] } ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 22
  23. 23. 1.6 Creating Hashes var crypto = require(’crypto’); var hash = crypto.createHash(’sha1’); hash.update(data); var digest = hash.digest(’hex’); ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 23
  24. 24. 1.6 Creating HMACs var crypto = require(’crypto’); var hmac = crypto.createHmac(’sha1’, key); hmac.update(data); var digest = hmac.digest(’base64’); ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 24
  25. 25. What’s the time? It should be 14:05 I hope ;) ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 25
  26. 26. 2Statistical AnalysisConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 26
  27. 27. 2.1 faststats npm install fast-stats var Stats = require(’fast-stats’).Stats; var s = new Stats().push(1, 2, 3, 10, 8, 4, 3); console.log(s.amean().toFixed(2)); Caveat: I haven’t pushed out a new version in a while ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 27
  28. 28. 2.2 gauss npm install gauss var gauss = require(’gauss’); var set = new gauss.Vector(5, 1, 3, 2, 21); console.log(set.mean()); ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 28
  29. 29. 2.3 statsd npm install statsd node-statsd github.com/etsy/statsd Easy to set up Requires graphite to plot charts Brought to you by Etsy ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 29
  30. 30. 2.3 statsd var StatsD = require(’node-statsd’).StatsD c = new StatsD(’example.org’,8125) c.timing(’node_test.some_service.task.time’, 500) ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 30
  31. 31. 2.4 cube Disclaimer: I’ve never used this one before npm install cube square.github.com/cube/ github.com/square/cube/wiki ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 31
  32. 32. What’s the time? 14:20? ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 32
  33. 33. Help me out Still trying to figure out the best way to debug Node.JS memory usage. Ideas? Let me know. ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 33
  34. 34. Questions?ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 34
  35. 35. Leave feeback on this talk at joind.in/5967 ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 35
  36. 36. Contact me Philip Tellis philip@lognormal.com @bluesmoon geek - paranoid - speedfreak co-founder @ Log-Normal http://bluesmoon.info/ slideshare.net/bluesmoon joind.in/5967 ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 36
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×