Your SlideShare is downloading. ×
node.js + Riak
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

node.js + Riak


Published on

Javascript is becoming a compelling platform - from browser to database. Node and Riak are two big players in this space.

Javascript is becoming a compelling platform - from browser to database. Node and Riak are two big players in this space.

Published in: Technology

1 Comment
No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. An introduction to node.js & Riak Francisco Treacy May 31st, 2010 @ ams.rb Amsterdam
  • 2. `whoami` Building a web-based collaborative e-reading platform J2EE veteran Trumpetless wannabe trumpet player @frank06 on the interwebs
  • 3. Let’s look back, shall we? How did we learn to program?
  • 4. Does this sound familiar? printf("What is your name?"); gets(string); printf("Your name is: %s", string); is how we learn how to program
  • 5. So then we go build our first webapp... <?php $query = "SELECT astronaut FROM spacecraft"; $result = mysql_query($query); while($result) { ... } ?> (with the first language we come across)
  • 6. Let’s imagine a bunch of people trying to access your website (run that code) at the same time
  • 7. It’s like hosting a party and provide only one toilet © hopechurchokc / flickr
  • 8. The cost of I/O L1 cache: 3 cycles L2 cache: 14 cycles RAM: 250 cycles Disk: 41,000,000 cycles Network: 240,000,000 cycles
  • 9. The cost of I/O L1 L2 RAM Disk Network
  • 10. The cost of I/O In other words, reaching RAM is like going from here to the Red Light District. Accessing the network is like going to the moon.
  • 11. So... what is your program still doing while MySQL goes fetch Neil Armstrong? $result = mysql_query($query);
  • 12. But we have threads! (to do other stuff while we wait)
  • 13. Programming with threads Race conditions Coarse-grained locks – more blocking Fine-grained locks – more complexity Risk for deadlocks Hard to reason about and therefore to get right Context switching overhead
  • 14. Programming with threads
  • 15. A look at Apache and Nginx Memory vs concurrent connections
  • 16. A look at Apache and Nginx Apache uses 1 thread per connection Nginx doesn’t use threads it runs an event loop small memory allocation per connection 1 thread per connection is limiting for massive concurrency
  • 17. Event loops Especially relevant when dealing with I/O All code runs in a single thread No need for multithreading – no locks!
  • 18. Event loops window.onload = function() { alert("Apollo 11 landed!") } Rings a bell?
  • 19. Javascript! Fortunately, Google has been developing a brand new Javascript VM for Chrome... It’s robust, blazingly fast, and open-source: V8
  • 20. The node way A set of bindings to Google’s V8 Javascript VM A purely evented, non-blocking infrastructure that makes it super simple to build highly concurrent programs Ability to handle thousands of concurrent connections with minimal overhead on a single process
  • 21. The node way Why should I care? I love Ruby I can use EventMachine for async And, after all, my Rails app doesn’t have youporn’s traffic
  • 22. The node way “Libraries like eventmachine will never be truly intuitive to use, because event- driven I/O is enough of a fundamental shift that it requires deep language integration. Javascript, it turns out, is a fundamentally event-driven language because of its origins in the browser” Adam Wiggins – Heroku
  • 23. The node way "Threads should be used by experts only" Javascript is perfect for event loops with first class function objects and closures It is arguably the most popular programming language Full Javascript stack?
  • 24. Full Javascript stack Client-side $.ajax({ url: '/api/feedme', success: function(data) { $('.result').html(data); } });
  • 25. Full Javascript stack Client-side – even better var socket = new io.Socket('localhost'); socket.connect(); socket.send('some data'); socket.addEvent('message', function(data){ $('.result').html(data); }); WebSockets through Socket.IO-node
  • 26. Full Javascript stack Server-side – node.js http.createServer(function (request, response) { response.writeHead(200, {'Content-Type': 'text/ html'}); response.end('I am back!'); }).listen(8000);
  • 27. Full Javascript stack Database ?
  • 28. Introducing Riak Content-agnostic key/value store REST API embraces HTTP Javascript Map/Reduce Distributed, “assume that failures will happen” Linearly scalable
  • 29. Introducing Riak What is scalable anyway? (other than a buzzword) € throughput – cost predictability Both up and down; less headaches for operations and development
  • 30. Brief Riak overview'astronauts', 'neil', { name: "Neil Armstrong", retired: true, daysInSpace: 8, missions: ['Apollo 11', 'Gemini 8'] })() riak-js – Available for node.js and jQuery
  • 31. Brief Riak overview var map = function(v, keydata, args) { var result = [], doc = Riak.mapValuesJson(v)[0] doc.missions.forEach(function(mission) { if (mission.match(new RegExp (args.mission))) result.push(doc) }) return result; } Map/Reduce jobs can be written in Javascript and submitted via the HTTP interface
  • 32. Brief Riak overview db.mapReduce({ inputs: "astronauts", query: [{ map: { source: map, arg: { mission: "Apollo" } } }] })(function(results) { // ... }); Bring the computation to the data – map phases are executed in parallel Aggregation happens in one node
  • 33. Brief Riak overview db.get('astronauts', 'neil', {r: 2})( function(neil) { $('.result').html(neil) }) Tunable N/R/W values to tweak CAP behaviour Eventual consistency: Brief sacrifices of consistency in failure conditions “Choose your own fault tolerance/performance tradeoff”
  • 34. Brief Riak overview CAP theorem Consistency: Reads and writes reflect a globally consistent system state Availability: System is available for reads and writes Partition tolerance: System can handle the failure of individual parts
  • 35. Brief Riak overview CAP theorem No real-world data store can serve completely consistent data while being 100% available and handling disconnected networks
  • 36. Internet NGINX Rails Rails Rails Rails Node
  • 37. Collaborative platform for studying Tens of thousands of books alongside with user generated content Highlighting, note-taking, sharing Web-based, use it anywhere: laptop, phone, iPad HTML5 + Javascript + node.js + Riak (and Rails!) Expect a beta release in October
  • 38. Riak is developer- and ops-friendly: it scales down to your laptop as easily as up to a cluster –  especially during exams period! Allows us to store multimedia assets just as JSON Lucene-like search coming soon Node is used for Ajax calls and WebSocket Rails for the rest (it’s convenient and mature)
  • 39. There are caveats though Cutting-edge technologies are not bug-free Riak still has some rough edges (some in terms of performance) node is approaching its first stable version Async JS code can get “boomerang-shaped”
  • 40. There are caveats though Boomerang-shaped code, doc, content)(function(response, meta) { db.get(bucket, doc)(function(response2) { assert.equal(response2, content); db.remove(bucket, doc)(function() { db.get(bucket, doc)(null, function(r, meta3) { assert.equal(404, meta3.statusCode); db.get(bucket, other)(function() { // ... }) }); }); }); });
  • 41. Step or flow-js Address boomerang-shaped code Step( function readSelf() { fs.readFile(__filename, this); }, function capitalize(err, text) { if (err) { throw err; } return text.toUpperCase(); }, function showIt(err, newText) { sys.puts(newText); } );
  • 42. CoffeeScript is a new language inspired by Javascript and Ruby grade: (student) -> if student.excellent_work "A+" else if student.okay_stuff if student.tried_hard then "B" else "B-" else "C" eldest: if 24 > 21 then "Liz" else "Ike"
  • 43. CoffeeScript compiles down to Javascript var eldest, grade; grade = function(student) { if (student.excellent_work) { return "A+"; } else if (student.okay_stuff) { if (student.tried_hard) { return "B"; } else { return "B-"; } } else { return "C"; } }; eldest = 24 > 21 ? "Liz" : "Ike"; And can be run on node!
  • 44. There’s no doubt about it. Javascript as a platform is serious stuff. Full JS stacks will become more and more popular.
  • 45. Maybe it’s about time we started teaching this? puts("Enter your spacecraft:") gets(function(s) { puts("You’re flying your " + s) })
  • 46. Go fetch a beer – see you in a bit for a demo and questions!
  • 47. event-loops-and-nodejs