Functional Programming with Streams in node.js
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


Functional Programming with Streams in node.js



My talk at HTML5DevConf && FluentConf covering how I combined some basic functional programming techniques with node.js streams to abstract a streaming VFS zipping layer that can also (theoretically / ...

My talk at HTML5DevConf && FluentConf covering how I combined some basic functional programming techniques with node.js streams to abstract a streaming VFS zipping layer that can also (theoretically / eventually) run in the browser.



Total Views
Views on SlideShare
Embed Views



8 Embeds 6,170 6147 8 5 4 3 1 1 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Functional Programming with Streams in node.js Presentation Transcript

  • 1. Functional Programming with Streams in node.js
  • 2. Who Am I?  HP webOS Framework Team  @CrabDude (Twitter, Github, etc…)  Organizer BayNode & Dallas.node  Author:   trycatch – async try/catch in node.js   stepup – step node.js control flow + trycatch   tiki – in-browser package manager
  • 3. The Problem VFS VFS VFShermesClient hermesClient hermesClient hermesBuild 3rd Party API hermes hermesDeploy Ares (browser)
  • 4. The Goal  Don’t buffer any data  Don’t write to disk  Minimize latency  Allow reuse of functionality (e.g., readdir)  Functionality may be local or remote (e.g., getFolder)
  • 5. Why node.js?“Nodes goal is to provide an easy way to build scalable network programs.” –
  • 6. node.js === JavaScriptJavaScript is the future of the web, and code reuse is its unfair advantage.
  • 7. Basics  EventEmitter  Stream  Traditional streams (aka non-node.js lazy streams)
  • 8. EventEmitter: Example
  • 9. EventEmitter: Extend
  • 10. Streams“Streams are to time as arrays are to space.” – Jed Schmidt @
  • 11. Readable Streams  Events: data, end, error, close  Methods: pause, resume, end, destroy
  • 12. Writable Streams  Events: drain, error, close, pause, resume  Methods: write, end, destroy
  • 13. readable.pipe(writeable)Readable streams can be piped to writable streams (and vice-versa!)  on(‘data’) => write()  on(‘end’) => end()  on(‘drain’) => resume()  on(‘close’) => destroy()  on(‘error’) => on(‘error’)  on(‘pause’) => pause()
  • 14. readable.pipe(writeable): Example
  • 15. Lazy StreamsStreams in functional programming…
  • 16. Lazy Streams: Higher Order FunctionsA higher-order function is a function that does at least one of the following:  Take one or more functions as input  output a function
  • 17. Lazy Streams: Lists
  • 18. Got All That?
  • 19. Hermes: Revisited FTP Dropbox FilesystemhermesFtp hermesDropbox hermesBox hermesFilesystem hermesBuild 3rd Party API hermesDeploy Ares (browser)
  • 20. Still Got All That?
  • 21. We need…We need to recursively zip a directory…  Let’s start with abstract implementation non-specific functional versions of…   Filter paths from readdir   Filter directories from paths   Filter files from paths   Recursively return all file paths in a directory
  • 22. We need…  More “abstract implementation non-specific functional versions” (errr…) of…   ls (aka readdir)   getFile (aka readFile)   getFolder (stream of gets)
  • 23. We need…  Yet More AINSFV of...   Zip a stream of files   Allow node.js streams to be piped to lazy streams   pipe(): lazy stream => node.js stream   pump(): node.js stream => lazy stream
  • 24. Almost there!
  • 25. Now we can…Zip a folder up using an arbitrary…  getFile / readFile  ls / readdir… And stream the response! Yeeeeehaw!
  • 26. Sweeeeeet.
  • 27. Now What?Since all the implementation specific functions are passed in, we can…
  • 28. Port this to the browser!
  • 29. Say whaaat?node-browserify by @Substack does a lot of this already   Stream, Buffer, Path, etc…We need…   Complete buffer implementation (browser has TypedArrays & ArrayBuffer)   fs (browser has FileSystem API)   zlib…   inflate(), deflate(), gzip(), gunzip(), unzip()
  • 30. Stay Tuned.I’m currently porting the fs and buffer modules to the browser, and eventually zlib.
  • 31. We’re HIRING! webOS Framework TeamIf you think this is pretty cool… Or you think I’M pretty cool…Stop by the Enyo booth or talk to me afterwards.
  • 32. Thank you. adam.crabtree@palm.com Programming in JavaScript: javascript.html Track6%201500-1600.mp4