A Node.JS bag of goodies for analyzing Web Traffic
Upcoming SlideShare
Loading in...5
×
 

A Node.JS bag of goodies for analyzing Web Traffic

on

  • 5,034 views

 

Statistics

Views

Total Views
5,034
Views on SlideShare
5,021
Embed Views
13

Actions

Likes
5
Downloads
28
Comments
0

5 Embeds 13

http://localhost 4
http://us-w1.rockmelt.com 3
http://duckduckgo.com 3
https://twimg0-a.akamaihd.net 2
http://nodeslide.herokuapp.com 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    A Node.JS bag of goodies for analyzing Web Traffic A Node.JS bag of goodies for analyzing Web Traffic Presentation Transcript

    • A Node.JS Bag of Goodies for Analysing Web Traffic Philip Tellis / philip@lognormal.com ConFoo / 2012-03-02 ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 1
    • Leave feeback on this talk at joind.in/5967 ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 2
    • $ finger philip Philip Tellis philip@lognormal.com @bluesmoon geek - paranoid - speedfreak co-founder @ Log-Normal http://bluesmoon.info/ ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 3
    • Log-Normal We do real user web performance analysis ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 4
    • Log-Normal This talk covers some of the Node.JS modules we use to do this analysis ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 5
    • Examples for this talk http://bluesmoon.github.com/talks/node-modules/ ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 6
    • – Node.JS & npmConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 7
    • 0.1 The 2 minute Node.JS Tutorial node > m = require(’module’) > console.log(m.foo) We’ll be using node v0.6.x ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 8
    • 0.1 The 2 minute Node.JS Tutorial node script.js ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 9
    • 0.1 The 2 minute Node.JS Tutorial echo "console.log(’hello, world!’)" | node Notice that in JavaScript, semicolons are optional ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 10
    • 0.2 Installing modules npm install module Installs into a local node_modules directory ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 11
    • 0.2 Installing modules globally sudo npm install -g module Installs into the /usr/lib/node_modules directory ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 12
    • 0.2 npm help npm help npm ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 13
    • 0.3 npmjs.org search.npmjs.org ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 14
    • What’s the time? 13:37? ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 15
    • 1 HTTP LogsConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 16
    • 1.1 UserAgent Parsing npm install ua-parser var parser=require(’ua-parser’); var ua = parser.parse(ua_string); // family, major, minor, patch, os Extracted from Google’s BrowserScope Project ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 17
    • 1.2 IP Addresses var net = require(’net’); net.isIP(ip); // returns 0, 4 or 6 The net module is part of node ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 18
    • 1.3 GeoLocation npm install geoip-lite var geo = require(’geip-lite’); var loc = geo.lookup(ip); // country, region, city, ll Uses MaxMind’s GeoIP database. Very fast lookups, IPv4 & v6 ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 19
    • 1.3 geoip-lite’s hidden function var geo = require(’geip-lite’); geo.cmp(ip1, ip2); // -1, 0 or 1 Used internally to do a binary search on the IP database ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 20
    • 1.4 Date formatting npm install prettydate var strftime = require(’prettydate’).strftime; var dstr = strftime(new Date, "%c"); Also accepts a locale as a third parameter. ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 21
    • 1.5 Querystring Parsing var qs = require(’querystring’); qs.parse(’name=Larry&name=Moe&name=Curly’); // { name: [ ’Larry’, ’Moe’, ’Curly’ ] } ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 22
    • 1.6 Creating Hashes var crypto = require(’crypto’); var hash = crypto.createHash(’sha1’); hash.update(data); var digest = hash.digest(’hex’); ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 23
    • 1.6 Creating HMACs var crypto = require(’crypto’); var hmac = crypto.createHmac(’sha1’, key); hmac.update(data); var digest = hmac.digest(’base64’); ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 24
    • What’s the time? It should be 14:05 I hope ;) ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 25
    • 2Statistical AnalysisConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 26
    • 2.1 faststats npm install fast-stats var Stats = require(’fast-stats’).Stats; var s = new Stats().push(1, 2, 3, 10, 8, 4, 3); console.log(s.amean().toFixed(2)); Caveat: I haven’t pushed out a new version in a while ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 27
    • 2.2 gauss npm install gauss var gauss = require(’gauss’); var set = new gauss.Vector(5, 1, 3, 2, 21); console.log(set.mean()); ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 28
    • 2.3 statsd npm install statsd node-statsd github.com/etsy/statsd Easy to set up Requires graphite to plot charts Brought to you by Etsy ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 29
    • 2.3 statsd var StatsD = require(’node-statsd’).StatsD c = new StatsD(’example.org’,8125) c.timing(’node_test.some_service.task.time’, 500) ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 30
    • 2.4 cube Disclaimer: I’ve never used this one before npm install cube square.github.com/cube/ github.com/square/cube/wiki ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 31
    • What’s the time? 14:20? ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 32
    • Help me out Still trying to figure out the best way to debug Node.JS memory usage. Ideas? Let me know. ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 33
    • Questions?ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 34
    • Leave feeback on this talk at joind.in/5967 ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 35
    • Contact me Philip Tellis philip@lognormal.com @bluesmoon geek - paranoid - speedfreak co-founder @ Log-Normal http://bluesmoon.info/ slideshare.net/bluesmoon joind.in/5967 ConFoo / 2012-03-02 A Node.JS Bag of Goodies for Analysing Web Traffic 36