To Infinity & Beyond: Protocols & sequences in Node - Part 2
Upcoming SlideShare
Loading in...5
×
 

To Infinity & Beyond: Protocols & sequences in Node - Part 2

on

  • 502 views

 

Statistics

Views

Total Views
502
Views on SlideShare
490
Embed Views
12

Actions

Likes
0
Downloads
0
Comments
0

2 Embeds 12

http://lanyrd.com 6
https://twitter.com 6

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

To Infinity & Beyond: Protocols & sequences in Node - Part 2 To Infinity & Beyond: Protocols & sequences in Node - Part 2 Presentation Transcript

  • To Infinity & Beyond! Protocols & Lazy Sequences in Node Part Deux – Sh*t Just Got RealBahul Neel Upadhyaya (@bahulneel)BraveNewTalenthttp://github.com/bahulneelhttp://www.bravenewtalent.com
  • cosy.lang Library Protocols● Protocols ● ISeq● Sequences Lazy & Async ● ISync● Argument length dispatch ● IStream● Tail recursion ● IPromise● Object Identity● Object Metadata npm install cosy-lang
  • An Example (TF-IDF)“Tf–idf, term frequency–inverse document frequency,is a numerical statistic which reflects how important a word is to adocument in a collection or corpus. It is often used as a weightingfactor in information retrieval and text mining.” - Wikipedia
  • Wordsfunction stripWord(word) { return word.replace(/[^-a-zA-Z_0-9]+/, ).toLowerCase();}function isWord(word) { return /^[-a-zA-Z_0-9]+$/.exec(word)}function words(string) { return vec(filter(isWord, map(stripWord, string.split(/ /))));}
  • Term Frequenciesfunction tf(words) { var max = 0, counts, word; function countFeq(counts, word) { var newCounts = clone(counts); if (undefined === typeof newCounts[word]) newCounts[word] = 0; newCounts[word] += 1; if (newCounts[word] > max) max = newCounts[word]; return newCounts; } counts = reduce(countFeq, {}, words); if (max) { for (word in counts) { if (counts.hasOwnProperty(word)) counts[word] /= max; } } return counts;}
  • Inverse Document Frequencyidf = fn$({ 1: function (terms) { return idf({}, 1, terms); }, 3: function (freq, docCount, terms) { if (null === first(terms)) return null; function calcIdf(terms) { var docFreq, invDocFreq = {}, word; docFreq = merge(freq, first(terms)); for (word in docFreq) { if (docFreq.hasOwnProperty(word)) invDocFreq[word] = docCount/(1+docFreq[word]); } return cons(invDocFreq, idf(docFreq, docCount + 1, rest(terms))); } return lazy(terms, calcIdf); }});
  • TF-IDFfunction tfIdf(documents) { var theWords, terms, freq; terms = map(tf, map(words, documents)); freq = idf(terms); function calcTfIdf(tf, idf) { var word, tfIdf = {}; for (word in tf) { if (tf.hasOwnProperty(word)) tfIdf[word] = tf[word] * idf[word]; } return tfIdf; } return map(calcTfIdf, terms, freq);}
  • Making Sequences Asyncronous Source Sink● Takes an ISeq & ISync as ● Takes an IStream as its its argument argument● Extends IStream ● Extends ISeq & ISync● Registers a tick callback ● First returns stream.skip using the ISync interface until stream emits● Emits first element when ● Calls tick callback when callback is called stream emits
  • Socket IO - Serverlang.protocol.extend(lang.stream.IStream, socketServer.Socket, ["tap", function (socket, fn) { socket.on("message", function (data) { fn(JSON.parse(data)); }); }], ["emit", function (socket, val) { socket.send(JSON.stringify(val)); }] ); function server(port, callback) { var io = socketServer.listen(port); io.sockets.on(connection, callback); }
  • Socker IO - Server(function (lang, tfIdf, server) { server(1234, function (socket) { lang.stream.pipe(tfIdf(socket), socket); });})(require(cosy-lang), require(./lib/tf-idf), require(./lib/socket-server).server);
  • SocketIO - Clientlang.protocol.extend(lang.stream.IStream, socketClient.SocketNamespace, ["tap", function (socket, fn) { socket.on("message", function (data) { fn(JSON.parse(data)); }); }], ["emit", function (socket, val) { socket.send(JSON.stringify(val)); }] ); function client(addr, callback) { var io = socketClient.connect(addr); io.on(connect, function () { callback(io); }); }
  • Socket IO - Clientclient("http://localhost:1234", function (socket) { lang.stream.tap(socket, function (val) { console.log(td-idf, val); }); lang.stream.pipe(documents, socket);});
  • Demo #!
  • Future work● Queues● Persistent Data Structures● Performance● Graphs● Persistence
  • Links● Cosy getcosy.org github.com/organizations/getcosy● Demo github.com/bahulneel/cosy-lang-demo● Me @bahulneel github.com/bahulneel
  • FinQuestions