Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Advanced Node.JS Meetup

1,461 views

Published on

Writing NodeJS applications is an easy task for JavaScript developers. However, getting what is happening under the hood in NodeJS may be intimidating, but understanding it is vital for web developers.

Indeed, when you try to learn NodeJS, most tutorials are about the NodeJS ecosystem like Express, Socket.IO, PassportJS. It is really rare to see some tutorials about the NodeJS runtime itself.

By this meetup, I want to spot the light on some advanced NodeJS topics so as to help developers answering questions an experienced NodeJS developer is expected to answer. Understanding these topics is essential to make you a much more desirable developer. I want to explore several topics including the famous event-loop along with NodeJS Module Patterns and how dependencies actually work in NodeJS.

I hope that this meetup would help you to be more comfortable understanding advanced code written in NodeJS.

Published in: Technology
  • Be the first to comment

Advanced Node.JS Meetup

  1. 1. Advanced Node.js Abdulkader BENCHI
  2. 2. The French Leader in Open Source Software
  3. 3. French leader in Open Source Software
  4. 4. An alternative to proprietary software
  5. 5. Let us start!
  6. 6. expect(NodeJS)to.shallowDeepEqual(JS)
  7. 7. Node's Architecture: V8 ● The two most important players in Node's architecture are V8 and libuv. Node's default VM is, and will continue to be, V8, although V8 is one option out of many ● One other strong option for a Node's VM is the Chakra engine by Microsoft. The Chakra engine is what powers the Microsoft Edge web browser ● JavaScript features that are available in Node are really the JavaScript features supported by the V8 engine shipped with Node
  8. 8. Node's Architecture: V8 ● When Node has a callback to invoke, it simply passes the control into the V8 engine. When V8 is done with the code in the callback, the control is passed back to Node. ● V8 is single-threaded, Node cannot execute several JavaScript code, no matter how many callbacks have been registered ● Node will wait until V8 can handle more operations.
  9. 9. expect(NodeJS)to.onlyProvide(V8)
  10. 10. Node's Architecture: libuv ● Node is more than a wrapper for V8, it provides APIs for working with operating system files, binary data, networking and much more. ● Node also handles the waiting for asynchronous events for us using libuv. ● Libuv is a C library developed for Node. ● It's used to abstract the non-blocking I/O operations to a consistent interface across many operating systems. ● Libuv is also what provides Node with the event-loop
  11. 11. Concurrency Model and Event Loop
  12. 12. I/O ● Slow I/O operations are handled with events and callbacks so that they don't block the main single-threaded execution runtime. ● I/O is used to label a communication between a process in a computer CPU and anything external to that CPU, including memory, disk, network, and even another process.
  13. 13. Handling Slow I/O
  14. 14. The Call Stack ● The V8 Call stack which is simply a list of functions. ● FILO data structure ○ const f1 = () => { f2(); }; ○ const f2 = () => { f3(); }; ○ const f3 = () => { f4(); }; ○ const f4 = () => { f4(); }; ● When start dealing with slow operations, the fact that we have a single thread becomes a problem, because these slow operations will block the execution.
  15. 15. The Call Stack const add = (a, b) => a + b; const double = a => add(a, a); const printDouble = a => { const output = double(a); console.log(output); }; printDouble(9); printDouble(9) double(9) add(9, 9) console.log(18)
  16. 16. The Call Stack const slowAdd = (a, b) => { for(let i=0; i<999999999; i++) {}; return a + b; }; const a = slowAdd(3, 3); const b = slowAdd(4, 4); console.log(a); console.log(b); slowAdd(3, 3)slowAdd(4, 4)console.log(6)console.log(8)
  17. 17. How Callbacks Actually Work const slowAdd = (a, b) => { setTimeout(() => { console.log(a+b); }, 5000); }; slowAdd(3, 3); slowAdd(4, 4); slowAdd(3, 3) setTimeout() slowAdd(4, 4) setTimeout() console.log(6)console.log(8)
  18. 18. What?!
  19. 19. How Callbacks Actually Work const slowAdd = (a, b) => { setTimeout(() => { console.log(a+b); }, 5000); }; slowAdd(3, 3); slowAdd(4, 4); slowAdd(3, 3) setTimeout() slowAdd(4, 4) setTimeout() Call stack Node Timer cb1 Timer cb2
  20. 20. How Callbacks Actually Work const slowAdd = (a, b) => { setTimeout(() => { console.log(a+b); }, 5000); }; slowAdd(3, 3); slowAdd(4, 4); console.log(6)console.log(8) Call stack Node Queue Timer cb1 Timer cb2 cb1 cb2
  21. 21. The Event loop ● Both the Stack and the Heap are part of the run-time engine V8, not Node itself. ● Node adds APIs like timers, emitters, and wrappers around OS operations. It also provides the event Queue and the event loop using the libuv library. ● Event loop = a simple loop that works between the event queue and the call stack.
  22. 22. YOLO
  23. 23. JavaScript module system
  24. 24. Look familiar? module bundlers vs. module loaders Webpack vs. Browserify AMD vs. CommonJS
  25. 25. Explain what modules plz Good authors divide their books into chapters and sections Good programmers divide their programs into modules.
  26. 26. Module pattern Module pattern is used to mimic the concept of classes, so that we can store both public and private methods and variables inside a single object  There are several ways to accomplish the module pattern Anonymous closure: our anonymous function has its own evaluation environment or “closure”, and then we immediately evaluate it. This lets us hide variables from the parent (global) namespace. (function () { // We keep these variables private inside this closure scope var myGrades = [93, 95, 88, 0, 55, 91]; var average = function() { var total = myGrades.reduce(function(accumulator, item) { return accumulator + item}, 0); return 'Your average grade is ' + total / myGrades.length + '.'; } var failing = function(){ var failingGrades = myGrades.filter(function(item) { return item < 70;}); return 'You failed ' + failingGrades.length + ' times.'; } console.log(failing()); }()); // ‘You failed 2 times.’
  27. 27. Have a look again Remember: in JavaScript, functions are the only way to create new scope. Note that the parenthesis around the anonymous function are required, because statements that begin with the keyword function are always considered to be function declarations Remember, you can’t have unnamed function declarations in JavaScript The surrounding parentheses create a function expression instead. (function () { // We keep these variables private inside this closure scope var myGrades = [93, 95, 88, 0, 55, 91]; var average = function() { var total = myGrades.reduce(function(accumulator, item) { return accumulator + item}, 0); return 'Your average grade is ' + total / myGrades.length + '.'; } var failing = function(){ var failingGrades = myGrades.filter(function(item) { return item < 70;}); return 'You failed ' + failingGrades.length + ' times.'; } console.log(failing()); }()); // ‘You failed 2 times.’
  28. 28. Why do I need a module pattern As a developer, you need to know the right dependency order to load your files in. For instance, let’s say you’re using Backbone in your project, so you include the script tag for Backbone’s source code in your file. However, since Backbone has a hard dependency on Underscore.js, the script tag for the Backbone file can’t be placed before the Underscore.js file. As a developer, managing dependencies and getting these things right can sometimes be a headache.
  29. 29. Why do I need a module pattern Namespace collisions: For example, ● what if two of your modules have the same name? ● Or what if you have two versions of a module, and you need both?
  30. 30. CommonJS A CommonJS module is essentially a reusable piece of JavaScript which exports specific objects, making them available for other modules to require in their programs. If you’ve programmed in Node.js, you’ll be very familiar with this format. In CommonJS, we use the module.exports object to expose modules, and require to import them.
  31. 31. What to notice CommonJS takes a server-first approach and synchronously loads modules. This matters because if we have three other modules we need to require, it’ll load them one by one. Now, that works great on the server but, unfortunately, makes it harder to use when writing JavaScript for the browser.
  32. 32. AMD CommonJS is all well and good, but what if we want to load modules asynchronously? The answer is called Asynchronous Module Definition (AMD) define(['myModule', 'myOtherModule'], function(myModule, myOtherModule) { console.log(myModule.hello()); }); Define function takes an array of each of the module’s dependencies. These dependencies are loaded in the background (in a non-blocking manner), and once loaded define calls the callback function it was given.
  33. 33. What to notice Unlike CommonJS, AMD takes a browser-first approach alongside asynchronous behavior to get the job done. Another benefit of AMD is that your modules can be objects, functions, constructors, strings, JSON and many other types, while CommonJS only supports objects as modules. That being said, AMD isn’t compatible with io, filesystem, and other server-oriented features available via CommonJS
  34. 34. UMD For projects that require you to support both AMD and CommonJS features, there’s yet another format: Universal Module Definition (UMD). UMD essentially creates a way to use either of the two, while also supporting the global variable definition. UMD modules are capable of working on both client and server.
  35. 35. Back to NodeJS Back to CommonJS
  36. 36. require(‘something’) Node uses two core modules for managing module dependencies: ● require ● module How do I access them: ● Are they global? ● no need to require('require') ● no need to require('module')
  37. 37. Resolving: To find the absolute path of the file. Loading: To determine the type of the file content. Wrapping: To give the file its private scope. This is what makes both the require and module objects local to every file we require. Evaluating: This is what the VM eventually does with the loaded code. Caching: So that when we require this file again, we don’t go over all the steps another time.
  38. 38. Resolving a local path Every module object gets an id property to identify it. This id is usually the full path to the file, but in a REPL session it’s simply <repl>. Node modules have a one-to-one relation with files on the file-system. We require a module by loading the content of a file into memory. ~/learn-node $ node > module Module { id: '<repl>', exports: {}, parent: undefined, filename: null, loaded: false, children: [], paths: [ ... ] }
  39. 39. require('find-me'); Node will look for find-me.js in all the paths specified by module.paths The paths list is basically a list of node_modules directories under every directory from the current directory to the root directory. If Node can’t find find-me.js in any of these paths, it will throw a “cannot find module error.” ~/learn-node $ node > module.paths [ '/Users/kader/learn-node/repl/node_modules', '/Users/kader/learn-node/node_modules', '/Users/kader/node_modules', '/Users/node_modules', '/node_modules', '/Users/kader/.node_modules', '/Users/kader/.node_libraries']
  40. 40. require.resolve If you want to only resolve the module and not execute it, you can use the require.resolve function This behaves exactly the same as the main require function, but does not load the file. > require.resolve('find-me'); '/Users/kader/learn-node/node_modules/find-me/start.js' > require.resolve('not-there'); Error: Cannot find module 'not-there' at Function.Module._resolveFilename (module.js:470:15) at Function.resolve (internal/module.js:27:19) at repl:1:9 at ContextifyScript.Script.runInThisContext (vm.js:23:33) at REPLServer.defaultEval (repl.js:336:29) at bound (domain.js:280:14) at REPLServer.runBound [as eval] (domain.js:293:12) at REPLServer.onLine (repl.js:533:10) at emitOne (events.js:101:20) at REPLServer.emit (events.js:191:7)
  41. 41. Parent-child relation between files // learn-node/lib/util.js console.log('In util', module); //learn-node/index.js console.log('In index', module); require('./lib/util');" ~/learn-node $ node index.js In index Module { id: '.', exports: {}, parent: null, filename: '/Users/kader/learn-node/index.js', loaded: false, children: [], paths: [ ... ] } In util Module { id: '/Users/kader/learn-node/lib/util.js', exports: {}, parent: Module { id: '.', exports: {}, parent: null, filename: '/Users/kader/learn-node/index.js', loaded: false, children: [ [Circular] ], paths: [...] }, filename: '/Users/kader/learn-node/lib/util.js', loaded: false, children: [], paths: [...] }
  42. 42. Parent-child relation Main index module (id: '.') is now listed as the parent for the lib/util module. However, the lib/util module was not listed as a child of the index module. Instead, we have the [Circular] value there because this is a circular reference. !infinite loop ~/learn-node $ node index.js In index Module { id: '.', exports: {}, parent: null, filename: '/Users/kader/learn-node/index.js', loaded: false, children: [], paths: [ ... ] } In util Module { id: '/Users/kader/learn-node/lib/util.js', exports: {}, parent: Module { id: '.', exports: {}, parent: null, filename: '/Users/kader/learn-node/index.js', loaded: false, children: [ [Circular] ], paths: [...] }, filename: '/Users/kader/learn-node/lib/util.js', loaded: false, children: [], paths: [...] }
  43. 43. exports, module.exports Notice the exports property So let us try this: // Add this to lib/util.js exports.id = 'lib/util'; // Add this to index.js exports.id = 'index'; ~/learn-node $ node index.js In index Module { id: '.', exports: {}, parent: null, filename: '/Users/kader/learn-node/index.js', loaded: false, children: [], paths: [ ... ] } In util Module { id: '/Users/kader/learn-node/lib/util.js', exports: {}, parent: Module { id: '.', exports: {}, parent: null, filename: '/Users/kader/learn-node/index.js', loaded: false, children: [ [Circular] ], paths: [...] }, filename: '/Users/kader/learn-node/lib/util.js', loaded: false, children: [], paths: [...] } ~/learn-node $ node index.js In index Module { id: '.', exports: { id: 'index' }, parent: null, filename: '/Users/kader/learn-node/index.js', loaded: false, children: [], paths: [ ... ] } In util Module { id: '/Users/kader/learn-node/lib/util.js', exports: { id: 'lib/util' }, parent: Module { id: '.', exports: { id: 'index' } parent: null, filename: '/Users/kader/learn-node/index.js', loaded: false, children: [ [Circular] ], paths: [...] }, filename: '/Users/kader/learn-node/lib/util.js', loaded: false, children: [], paths: [...] }
  44. 44. exports, module.exports Exports variable inside each module is just a reference to module.exports. When we reassign the exports variable, that reference is lost and we would be introducing a new variable instead of changing the module.exports object. The module.exports object in every module is what the require function returns when we require that module. ~/learn-node $ node index.js In index Module { id: '.', exports: { id: 'index' }, parent: null, filename: '/Users/kader/learn-node/index.js', loaded: false, children: [], paths: [ ... ] } In util Module { id: '/Users/kader/learn-node/lib/util.js', exports: { id: 'lib/util' }, parent: Module { id: '.', exports: { id: 'index' } parent: null, filename: '/Users/kader/learn-node/index.js', loaded: false, children: [ [Circular] ], paths: [...] }, filename: '/Users/kader/learn-node/lib/util.js', loaded: false, children: [], paths: [...] }
  45. 45. loading Loaded = false The module module uses the loaded attribute to track which modules have been loaded (true value) and which modules are still being loaded (false value). We can, for example, see the index.js module fully loaded if we print its module object on the next cycle of the event loop using a setImmediate call The exports object becomes complete when Node finishes loading the module (and labels it so). ~/learn-node $ node index.js In index Module { id: '.', exports: {}, parent: null, filename: '/Users/kader/learn-node/index.js', loaded: false, children: [], paths: [ ... ] } In util Module { id: '/Users/kader/learn-node/lib/util.js', exports: {}, parent: Module { id: '.', exports: {}, parent: null, filename: '/Users/kader/learn-node/index.js', loaded: false, children: [ [Circular] ], paths: [...] }, filename: '/Users/kader/learn-node/lib/util.js', loaded: false, children: [], paths: [...] }
  46. 46. loading The exports object becomes complete when Node finishes loading the module (and labels it so). The whole process of requiring/loading a module is synchronous. This also means that we cannot change the exports object asynchronously. fs.readFile('/etc/passwd', (err, data) => { if (err) throw err; exports.data = data; // Will not work. });
  47. 47. Wrapping In a browser, when we declare a variable in a script like this: var answer = 42; That answer variable will be globally available in all scripts after the script that defined it. This is not the case in Node, So how come variables in Node are magically scoped?
  48. 48. Wrapping The answer is simple. Before compiling a module, Node wraps the module code in a function The wrapper function can be inspected using the wrapper property of the module module. ~ $ node > require('module').wrapper [ '(function (exports, require, module, __filename, __dirname) { ', 'n});' ] >
  49. 49. Wrapping Node does not execute any code you write in a file directly. It executes this wrapper function which will have your code in its body. This wrapper function has 5 arguments: exports, require, module, __filename, and __dirname. This is what makes them appear to look global when in fact they are specific to each module. ~ $ node > require('module').wrapper [ '(function (exports, require, module, __filename, __dirname) { ', 'n});' ] >
  50. 50. Wrapping exports is defined as a reference to module.exports prior to that. What happens is roughly equivalent to: function (require, module, __filename, __dirname) { let exports = module.exports; // Your Code… return module.exports; }
  51. 51. The require object It’s an object that acts mainly as a function that takes a module name or path and returns the module.exports object. We can simply override the require object with our own logic if we want to. Now, every require('something') call in the script will just return the mocked object. require = function() { return { mocked: true }; }
  52. 52. The require object The require object also has properties of its own. require.main which can be helpful to determine if the script is being required or run directly. require('module').do(); require.main !== module node module require.main === module if (require.main === module) { // The file is being executed directly (not with require) }
  53. 53. What is the output? function yolo() { console.log('yolo'); } console.log('Hello there') module.exports = { yolo } require('./test').yolo(); require('./test').yolo();
  54. 54. Caching All modules will be cached Node caches the first call and does not load the file on the second call. We can see this cache by printing require.cache after the first require. We can simply delete a property from that require.cache object to invalidate that cache. function yolo() { console.log('yolo'); } console.log('Hello there') module.exports = { yolo } require('./test').yolo(); delete require.cache[require.resolve('./test')] require('./test').yolo();
  55. 55. Join us https://job.linagora.com
  56. 56. Follow us!
  57. 57. Next Meetup : Kaldi Au programme : Intelligence Artificielle et Machine Learning, Deep Learning, Reconnaissance Automatique de la Parole (RAP), le challenge de la parole spontanée, variabilité du vocabulaire, du locuteur et de l'environnement, Kaldi : de la théorie à la pratique, Deep Learning en pratique : application à la RAP, échanges autour d'un cocktail !
  58. 58. Happy to discuss with you

×