Node js with steroids
•Download as PPTX, PDF•
2 likes•579 views
Make better Node.js Applications with Native Addons Video from Node Interactive 2017 https://youtu.be/KuFSDDggbOs
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Node js with steroids
Similar to Node js with steroids (20)
More from Nicola Del Gobbo
Recently uploaded
Recently uploaded (20)
Node js with steroids
- 1. Node.js With Steroids Make Better Node.js Application with Native Add-Ons Nicola Del Gobbo, Developer, Packly
- 2. Who is Nicola Del Gobbo? Developer Contribute to
- 3. What is Node.js Native Addon? In a very simple way Native Addons could be considered as C / C++ code called from JavaScript BRIDGE Native environment JavaScript environment
- 4. From Node.js documentation Node.js Addons are dynamically-linked shared objects, written in C++, that can be loaded into Node.js using the require() function, and used just as if they were an ordinary Node.js module. They are used primarily to provide an interface between JavaScript running in Node.js and C/C++ libraries.
- 5. How is it possible? All the magic is behind the Node.js architecture
- 6. Why should you implement new Native Add-on? Performance In general C / C++ code performs better than JavaScript code, but it’s really true for CPU bounds operations • Image processing (in average 6 times faster) • Video processing • CRC cyclic redundancy check (in average 125 times faster) • Compression • Scientific calculus • Algorithms that execute CPU heavy tasks
- 7. Why should you implement new Native Add-on? Integrate legacy application You have the source code of an old C / C++ application and want to expose something of its functionalities through the new Node.js application
- 8. Why should you implement new Native Add-on? You don’t find what fits your specific needs on npm Sometimes completely reimplementing the module in JavaScript is not the right solution Think at libraries like: • ImageMagick • Ghostscript • FFmpeg • TensorFlow You have to concentrate on business logic of your application and not on developing a single module
- 9. Why should you implement new Native Add-on? Desktop application You want to use the local underlying CPU or GPU resources to execute the most complex tasks Es. Electron or NW.js • Better performance on the local side • Better performance on the server
- 10. Why should you implement new Native Add-on? Better error handling Even if you will use an old C / C++ library you will get an error code that explains the error that just happened. In new C / C++ library you have the exception You don’t have to parse some string to identify if there was an error and what kind of it
- 11. Problems Fragmentation API 0.8 - 0.10.x - 0.12.x - 1.x - 2.x - 3.x - 4.x - 5.x - 6.x - 7.x - 8.x The API to implement Native Add-Ons have been changed across different version of Node.js Most of the changes were on V8 API and ObjectWrap API For more info http://v8docs.nodesource.com/
- 12. Problems Fragmentation API Need an adapter to stay compatible across different version of Node.js • NAN - Native Abstraction for Node.js • API compatibility • Strong bonded with V8 API • You have to recompile your native add-ons switching to different version of Node.js
- 13. Problems Write portable C / C++ code At one point your native code must compile on different: ARCHITECTURE PLATFORM COMPILER
- 14. Problems Documentation • C / C++ libraries that you are integrating are not well documented • There are good references but not so much practical guide focusing on concepts about Native Add-Ons
- 15. How to organize your native Addon project src C/C++ code and maybe dependencies binding.gyp building configurations lib JavaScript code package.json
- 16. Example: the JavaScript part
- 17. Example: the C / C++ part
- 18. Example: the binding.gyp Sometimes finding the right settings for binding.gyp is not easy Refer to GYP documentation https://gyp.gsrc.io/docs/UserDocumentation.md More useful examples to take inspiration from other developers https://github.com/nodejs/node-gyp/wiki/”binding.gyp”-files-out-in-the-wild
- 19. Which kind of Addon should you implement? TIME: Amount of processing time spent on the C++ code DATA: Amount of data flowing between C++ and JavaScript
- 20. Async example
- 21. Buffer example
- 22. ObjectWrap API • ObjectWrap is a way to expose your C++ code to JavaScript • You have to extend ObjectWrap class that includes the plumbing to connect JavaScript code to a C++ object • Classes extending ObjectWrap can be instantiated from JavaScript using the new operator, and their methods can be directly invoked from JavaScript • Unfortunately, the wrap part really refers to a way to group methods and state • It’s your responsibility write custom code to bridge each of your C++ class methods.
- 24. Lessons learned from my experience • Create simple interface between C / C++ and JavaScript • Insert complexity on the JavaScript side • Think carefully at JavaScript interface • Use ObjectWrap API to return C / C++ object to JavaScript • Validate the types on the native side • Expose only the functionalities you need • Don’t block the event loop, stay asynchronous: this is a must if you will use the Addons on a distributed system • Use Buffer to pass big quantity of data • If you have many parameters use JSON to pass and parse data • Consider the using of protobuf
- 25. Present and future • The largest number of native addons is written using NAN • NAN helps stay compatible with old versions of Node.js • N-API will be a game changer on the native addon development • ABI compatibility • New ES6 types • Isolated from V8 API (VM agnostic) • C / C++ • Conversion tool that helps to migrate from NAN • Great developer experience for mantainers
- 26. Implement high performance key value database Binding to Vedis an embeddable datastore C library https://github.com/NickNaso/conf-ni- 2017
- 27. Thanks! nicoladelgobbo@gmail.com @NickNaso on Twitter All examples and materials are here https://github.com/NickNaso/conf-ni- 2017
Editor's Notes
- Good afternoon everyone. Thank you to attend my talk, today I’m very honored to be here and share my ideas with all of you.
- My name is Nicola Del Gobbo and I’m developer at Packly. In my every day work I take care of building Packly’s backend systems. In my spare time I try to give my contribution to all frameworks, libraries and modules that I use in my work and obviously I like creating new one. Today the talk is focused on Node.js native addon start with its definition.
- In a very simple way Native Addons could be considered as C / C++ code called from JavaScript. They are a bridge between our Application programming language JavaScript and Native environment that is completely written in C / C++. This allow us to call C / C++ functions and methods directly from JavaScript.
- If we take a look at Node.js documentation under Addons section Native Addons are defined as: dynamically-linked shared objects, written in C++, that can be loaded into Node.js using the require() function, and used just as if they were an ordinary Node.js module Their main purpose is to provide an interface between JavaScript running in Node.js and C / C++ libraries. This increase the developer experience because if you are using a Native Addon you don’t care of this, but you just use it as any other pure JavaScript module.
- How is it possible? All the magic is behind the Node.js architecture. Node.js is built on some C / C++ libraries the most important for us as native addon developers are V8 and libuv. V8 is a JavaScript engine that take care to compile JavaScript code, execute it and give us great performance. Through the V8 APIs we can create function, object and do almost everything that we do on the JavaScript side. Libuv is a high performance evented I/O library and has the responsibility to handle the event loop (our main thread) and all the asynchronous tasks.
- Why should you implement new Native Add-On? The first reason is for performance In general C / C++ code performs better then JavaScript code, but it’s really true for CPU bounds operations. Think for example at: Image processing | Video processing | Compression | Scientific calculus | Cyclic redundancy check and all algorithms that at one point execute some CPU heavy tasks. In all these cases you can gain more in performance.
- Sometimes you want just integrate a legacy application So you have the source code of an old C / C++ application and want to expose something of its functionalities through new Node.js application
- Even if npm is a very large repo where you can find almost everything sometimes you don’t find what fits your specific needs and in this case you have to take a choice. Completely reimplementing the module in JavaScript, but in my opinion it’s an insane idea to reimplement from the ground solutions like ImageMagick | Ghostscript | Ffmpeg and TensorFlow because you will lose all your time on develop a module and not to concentrate on business logic of your application. In these cases the
- When I start to develop a new native addon I use the boilerplate depicted on the slide. Here the most important elements are the SRC and LIB folders. SRC folder contains the C / C++ code and the native dependencies while the LIB folder contains the JavaScript code There are also the usual package.json file and the binging.gyp file that contains all the building configurations for the addon. At the end our addon will be composed by two parts one written in C++ and another written in JavaScript and usually the JavaScript part will use the native code to expose some features
- In the example we can see the JavaScript part of bcrypt addon. For who of you that don’t know it: Bcrypt is a module that help us to hash passwords. In the reported portion of code it import the addon and call its function: gen_salt_sync. To understand how to create this function we have to take a look at C / C++ part of the module
- What happen here? The macro NODE_MODULE creates code that will register a module named ”bcypt_lib” and in addition it will ensure that a function init is called when the module is required. In the init we export our function or object. In the reported example we export gen_salt_sync and you can find the code executed by this function in the method GenerateSaltSync. As first thing we validate the input data, then transfer the JavaScript input to C++ data structure. C++ code perform the requested computation and before returning transfer the output data to JavaScript context. This is the most straightforward integration pattern that you can realize to pass data between JavaScript and C / C++ . The main reason for using this approach is simplicity and because you can integrate the C++ without modification. In addition we have a complete decoupling of the JavaScript and C++ code.
- In our binding.gyp we need to set the target name that will match the module name. The reported example is almost simple sometimes finding the right settings for binding.gyp is not easy so I suggest to take a look at GYP documentation or at nodey-gyp wiki where there are examples of existing binding.gyp and use them as source of inspiration to solve your problems with these kind of configurations.
- We can categorize the different use cases for addons considering the amount of time we’ll spend in the C++ code and the amount of data flowing between C++ and JavaScript. If you are in the left quadrant maybe your native addon can be synchronous and this means that the native code will be executed directly on the event loop when called. In the right quadrants for sure, you have to implement the addon using the asynchronous pattern. The difference between the top and bottom quadrants is often overlooked, but it is equally important. Sometimes the cost of copying all data between JvaScript to C++ and vice versa usually kills the performance benefits you obtained from C++. In this scenarios we should consider to use the Buffer to pass the data because it uses the memory outside the V8 heap, this means that it’s not subject to the multi-threading rules of V8 and we can interact with it in place from a C++.
- In an asynchronous addon function, the calling JavaScript code returns immediately. The calling code passes a callback function to the addon, and the addon does its work in a separate worker thread. This avoids locking up the Node.js event loop, as the addon function does not block.
- Store a reference to this constructor so it can be called in New if New is called without constructor syntax