Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Real World Lessons on the Pain Points of Node.JS Application

549 views

Published on

Delivered at Confoo 2017, Montreal CA, 8th March 2017

Published in: Technology
  • Login to see the comments

  • Be the first to like this

Real World Lessons on the Pain Points of Node.JS Application

  1. 1. Real World Lessons on the Pain Points of Node.js Applications @Ben_Hall Ben@BenHall.me.uk OcelotUproar.com / Katacoda.com
  2. 2. @Ben_Hall / Blog.BenHall.me.uk Ocelot Uproar WHOAMI?
  3. 3. Learn via Interactive Browser-Based Labs Katacoda.com
  4. 4. Agenda • Creating strong foundation – Node v6/v7, NPM, Security • Error Handling • Async / Promises • Deploying / Scaling • Performance • Debugging
  5. 5. Provide an overview of our node.js experiences and the tasks we wish we did earlier!
  6. 6. Strong Foundations
  7. 7. Upgrading from Node v0.10.38 to v6.1.0
  8. 8. #nodestats for March: Node 6.x hits 48% share, Node 7.x breaks 15%, 0.10 and 0.12 drop below 10% share for the first time: https://mobile.twitter.com/i/web/status/838922927735132160
  9. 9. Previous Node Versioning • Even === Stable – 0.10, 0.12, 4, 6 • Odd === Beta – 0.9, 0.11, 5
  10. 10. Node Release Stages • CURRENT: new features (and bug fixes and security patches) • ACTIVE LTS: bug fixes (and security patches) • MAINTENANCE: only security patches
  11. 11. Docker to test deployment • Didn’t need to install anything on host > docker run -it -v $(pwd):/src -p 3000 node:6 root@container:> npm install root@container:> npm start
  12. 12. Default to Underscore.js ?
  13. 13. Fixing NPM
  14. 14. Lock down NPM dependencies because no-one respects SemVer
  15. 15. AngularJs 1.2 => 1.3 "dependencies": { "angular": "^1.2.16” }
  16. 16. Angular 1.2 => 1.3 > angular.element(document) [#document] > angular.element(document) TypeError: undefined is not a function
  17. 17. Lock Down Dependencies Randomly breaking builds and deployments will occur otherwise
  18. 18. $ npm shrinkwrap Lock down dependencies to what’s running locally
  19. 19. Hard code versions in package.json "dependencies": { "angular": “1.2.23” }
  20. 20. $ npm outdated
  21. 21. .npmrc • https://docs.npmjs.com/misc/config > cat .npmrc save=true save-exact=true
  22. 22. npm install -g
  23. 23. Replaced Glup, Grunt with Make
  24. 24. templates: handlebars views/templates/*.hbs -f public/js/templates.js > make templates
  25. 25. Security
  26. 26. Child Process Exec child_process.exec(req.query.url, function (err, data) { console.log(data); }); https://localhost:49155/api/openUrlInDefaultBrowser?url=c:/windows/sy stem32/calc.exe Thanks TrendMicro Antivirus on Windows! https://code.google.com/p/google-security-research/issues/detail?id=693
  27. 27. Cross-Site Request Forgery var csrf = require('csurf'); var csrfProtection = csrf({ cookie: true }); var parseForm = bodyParser.urlencoded({ extended: false }); app.get('/form', csrfProtection, function(req, res) { res.render('send', { csrfToken: req.csrfToken() }); }); app.post('/process', parseForm, csrfProtection, function(req, res) { res.send('data is being processed'); }); https://blog.risingstack.com/node-js-security-checklist/
  28. 28. Rate Limiting var ratelimit = require('koa-ratelimit'); var ipBasedRatelimit = ratelimit({ db: redis.createClient(), duration: 60000, max: 10, id: function (context) { return context.ip; } }); app.post('/login', ipBasedRatelimit, handleLogin); https://blog.risingstack.com/node-js-security-checklist/
  29. 29. Security Audit NPM Packages > npm install nsp > nsp check (+) 18 vulnerabilities found
  30. 30. https://nodesecurity.io/ • Root Path Disclosure (2x) • Regular Expression Denial of Service (10x) • Incorrect Handling of Non-Boolean Comparisons During Minification • Denial-of-Service Extended Event Loop Blocking • Denial-of-Service Memory Exhaustion • Symlink Arbitrary File Overwrite • Remote Memory Disclosure (2x)
  31. 31. NPM Credentials Leaks • https://github.com/ChALkeR/notes/blob/mast er/Do-not-underestimate-credentials- leaks.md
  32. 32. https://blog.acolyer.org/2017/03/07/thou-shalt-not-depend-on-me-analysing-the-use-of- outdated-javascript-libraries-on-the-web/
  33. 33. Create Strong Foundations
  34. 34. Error Handling
  35. 35. Wasn’t great from the start
  36. 36. Try {} Catch {}
  37. 37. Try {} Catch {}
  38. 38. Domains haven’t really worked
  39. 39. https://raw.githubusercontent.com/strongloop/zone/master/showcase/curl/curl-zone.js Zones? No. Not really
  40. 40. Returning String as Error
  41. 41. Strongly typed errors
  42. 42. Async Flow Control
  43. 43. Promises Promises… Promises… Never break your promises. Personally, I never make promises.
  44. 44. http://domenic.me/2012/10/14/youre-missing-the-point-of-promises/
  45. 45. Callbacks So good it’s got it’s own website callbackhell.com
  46. 46. “The goal isn’t about removing levels of indentation but rather writing modular code that is easy to reason about” Strongloop Blog http://strongloop.com/strongblog/node-js-callback-hell-promises-generators/
  47. 47. Loops + Async Callbacks
  48. 48. Loops + Async Callbacks
  49. 49. “You can't get into callback hell if you don't go there.” Isaac Schlueter
  50. 50. Node 7 – Async Await class Demo { async greeting() { const h = await this.world(); return h; } world() { return Promise.resolve('hello world'); } } const retval = await demo.greeting();
  51. 51. Node 7 – Async Await Errors async getPersonFullNameWithTryCatch() { try { let response = await fetch('./data/person2.json'); } catch(e) { console.log('there was an error'); console.log(e); } }
  52. 52. DEPLOY!
  53. 53. Single Threaded
  54. 54. CPU Intensive? Array Filtering / Sorting / Processing
  55. 55. Maybe not use Node? Golang has a great community Where Node was a few years ago?
  56. 56. Have more threads
  57. 57. Solved Problem?
  58. 58. var cluster = require('cluster'); var http = require('http'); var numCPUs = require('os').cpus().length; if (cluster.isMaster) { // Fork workers. for (var i = 0; i < numCPUs; i++) { cluster.fork(); } cluster.on('exit', function(worker, code, signal) { console.log('worker ' + worker.process.pid + ' died'); }); } else { // Workers can share any TCP connection // In this case it is an HTTP server http.createServer(function(req, res) { res.writeHead(200); res.end("hello worldn"); }).listen(8000); }
  59. 59. > NODE_DEBUG=cluster node server.js 23521,Master Worker 23524 online 23521,Master Worker 23526 online 23521,Master Worker 23523 online 23521,Master Worker 23528 online
  60. 60. Deploying with Docker
  61. 61. Container https://www.docker.com/whatisdocker/ Container
  62. 62. Deploying Node App via Docker > cat Dockerfile FROM node:6-onbuild EXPOSE 3000 > docker build –t my-node-app . > docker run –p 3000:3000 my-node-app
  63. 63. 59,040 environments/day/server
  64. 64. 112,320 environments/day/server
  65. 65. > docker run -d -p 80:80 -v /var/run/docker.sock:/tmp/docker.sock:ro jwilder/nginx-proxy > docker run --name web -e VIRTUAL_HOST=www.katacoda.com my-node-app
  66. 66. Load Balancer Nginx Proxy Node Node Node Node Node Node Nginx Proxy Node Node Node Node Node Node Nginx Proxy Node Node Node Node Node Node
  67. 67. Health Endpoints router.get('/_specialfunctions/_check', function(req, res) { async.parallel([ check_docker, check_starter, check_redis, check_pg ], function(err, results) { if(err) { console.log("Health check failed", err); res.status(500); return res.json({healthy: false, details: err}); } res.json({healthy: true}); }) });
  68. 68. var check_docker = function(cb) { docker.ping(function(err) { handle_error('docker', err, cb);}); }; var check_redis = function(cb) { redis.status(function(err, connected) { if(err === null && connected === "ready") { cb(); } else { handle_error('redis', {msg: 'Not Connected', err: err}, cb); } }) }; var check_pg = function(cb) { pg.status(function(err) { handle_error('postgres', err, cb);}); };
  69. 69. Careful! socket.io and global state
  70. 70. Sticky Sessions Compiled Nginx + OSS Modules
  71. 71. Global State as a Service Microservices FTW!!
  72. 72. Code Performance Still Matters
  73. 73. Performance
  74. 74. > npm install v8-profiler const profiler = require('v8-profiler') const fs = require('fs') var profilerRunning = false function toggleProfiling () { if (profilerRunning) { const profile = profiler.stopProfiling() console.log('stopped profiling') profile.export() .pipe(fs.createWriteStream('./myapp-'+Date.now()+'.cpuprofile')) .once('error', profiler.deleteAllProfiles) .once('finish', profiler.deleteAllProfiles) profilerRunning = false return } profiler.startProfiling() profilerRunning = true console.log('started profiling') } process.on('SIGUSR2', toggleProfiling) > kill -SIGUSR2 <pid>
  75. 75. JetBrains WebStorm
  76. 76. OpenTracing - Zipkin
  77. 77. Prometheus + Grafana
  78. 78. Debugging
  79. 79. require(‘debug’);
  80. 80. Webstorm
  81. 81. VS Code
  82. 82. Profiling Node Applications March 10, 2017 @ 09:00 Fontaine E
  83. 83. Real-Time Monitoring with Grafana, StatsD and InfluxDB March 10, 2017 @ 15:00
  84. 84. Testing?!
  85. 85. Summary • Upgrade to Node.js v6 • Start looking at ES6 / Node 7 • Security • Manage your errors • Forgot making promises • Scale using Docker
  86. 86. Thank you! @Ben_Hall Ben@BenHall.me.uk www.Katacoda.com

×