A Node.JS bag of goodies for analyzing Web Traffic
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
5,491
On Slideshare
5,478
From Embeds
13
Number of Embeds
5

Actions

Shares
Downloads
28
Comments
0
Likes
5

Embeds 13

http://localhost 4
http://us-w1.rockmelt.com 3
http://duckduckgo.com 3
https://twimg0-a.akamaihd.net 2
http://nodeslide.herokuapp.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. A Node.JS Bag of Goodies for Analysing Web Traffic Philip Tellis / philip@lognormal.com ConFoo / 2012-03-02 ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 1
  • 2. Leave feeback on this talk at joind.in/5967 ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 2
  • 3. $ finger philip Philip Tellis philip@lognormal.com @bluesmoon geek - paranoid - speedfreak co-founder @ Log-Normal http://bluesmoon.info/ ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 3
  • 4. Log-Normal We do real user web performance analysis ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 4
  • 5. Log-Normal This talk covers some of the Node.JS modules we use to do this analysis ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 5
  • 6. Examples for this talk http://bluesmoon.github.com/talks/node-modules/ ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 6
  • 7. – Node.JS & npmConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 7
  • 8. 0.1 The 2 minute Node.JS Tutorial node > m = require(’module’) > console.log(m.foo) We’ll be using node v0.6.x ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 8
  • 9. 0.1 The 2 minute Node.JS Tutorial node script.js ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 9
  • 10. 0.1 The 2 minute Node.JS Tutorial echo "console.log(’hello, world!’)" | node Notice that in JavaScript, semicolons are optional ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 10
  • 11. 0.2 Installing modules npm install module Installs into a local node_modules directory ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 11
  • 12. 0.2 Installing modules globally sudo npm install -g module Installs into the /usr/lib/node_modules directory ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 12
  • 13. 0.2 npm help npm help npm ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 13
  • 14. 0.3 npmjs.org search.npmjs.org ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 14
  • 15. What’s the time? 13:37? ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 15
  • 16. 1 HTTP LogsConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 16
  • 17. 1.1 UserAgent Parsing npm install ua-parser var parser=require(’ua-parser’); var ua = parser.parse(ua_string); // family, major, minor, patch, os Extracted from Google’s BrowserScope Project ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 17
  • 18. 1.2 IP Addresses var net = require(’net’); net.isIP(ip); // returns 0, 4 or 6 The net module is part of node ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 18
  • 19. 1.3 GeoLocation npm install geoip-lite var geo = require(’geip-lite’); var loc = geo.lookup(ip); // country, region, city, ll Uses MaxMind’s GeoIP database. Very fast lookups, IPv4 & v6 ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 19
  • 20. 1.3 geoip-lite’s hidden function var geo = require(’geip-lite’); geo.cmp(ip1, ip2); // -1, 0 or 1 Used internally to do a binary search on the IP database ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 20
  • 21. 1.4 Date formatting npm install prettydate var strftime = require(’prettydate’).strftime; var dstr = strftime(new Date, "%c"); Also accepts a locale as a third parameter. ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 21
  • 22. 1.5 Querystring Parsing var qs = require(’querystring’); qs.parse(’name=Larry&name=Moe&name=Curly’); // { name: [ ’Larry’, ’Moe’, ’Curly’ ] } ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 22
  • 23. 1.6 Creating Hashes var crypto = require(’crypto’); var hash = crypto.createHash(’sha1’); hash.update(data); var digest = hash.digest(’hex’); ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 23
  • 24. 1.6 Creating HMACs var crypto = require(’crypto’); var hmac = crypto.createHmac(’sha1’, key); hmac.update(data); var digest = hmac.digest(’base64’); ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 24
  • 25. What’s the time? It should be 14:05 I hope ;) ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 25
  • 26. 2Statistical AnalysisConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 26
  • 27. 2.1 faststats npm install fast-stats var Stats = require(’fast-stats’).Stats; var s = new Stats().push(1, 2, 3, 10, 8, 4, 3); console.log(s.amean().toFixed(2)); Caveat: I haven’t pushed out a new version in a while ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 27
  • 28. 2.2 gauss npm install gauss var gauss = require(’gauss’); var set = new gauss.Vector(5, 1, 3, 2, 21); console.log(set.mean()); ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 28
  • 29. 2.3 statsd npm install statsd node-statsd github.com/etsy/statsd Easy to set up Requires graphite to plot charts Brought to you by Etsy ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 29
  • 30. 2.3 statsd var StatsD = require(’node-statsd’).StatsD c = new StatsD(’example.org’,8125) c.timing(’node_test.some_service.task.time’, 500) ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 30
  • 31. 2.4 cube Disclaimer: I’ve never used this one before npm install cube square.github.com/cube/ github.com/square/cube/wiki ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 31
  • 32. What’s the time? 14:20? ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 32
  • 33. Help me out Still trying to figure out the best way to debug Node.JS memory usage. Ideas? Let me know. ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 33
  • 34. Questions?ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 34
  • 35. Leave feeback on this talk at joind.in/5967 ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 35
  • 36. Contact me Philip Tellis philip@lognormal.com @bluesmoon geek - paranoid - speedfreak co-founder @ Log-Normal http://bluesmoon.info/ slideshare.net/bluesmoon joind.in/5967 ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 36