Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

LNUG: Having Your Node.js Cake and Eating It Too

1,283 views

Published on

Presentation given at LNUG Meeting on 27 September 2017

Published in: Software
  • Be the first to comment

LNUG: Having Your Node.js Cake and Eating It Too

  1. 1. Copyright © 2017 M/Gateway Developments Ltd Having Your Node.js Cake and Eating It Too Rob Tweed M/Gateway Developments Ltd Twitter: @rtweed http://qewdjs.com
  2. 2. Copyright © 2017 M/Gateway Developments Ltd A bit of background • Royal Marsden Hospital: 1980s • Touche Ross Management Consultants (now Deloitte): early 1990s – NHS-wide Networking Project • Independent consultant & software developer since 1993 – Specialising in Web and associated technologies, particularly in healthcare
  3. 3. Copyright © 2017 M/Gateway Developments Ltd A bit of background
  4. 4. Copyright © 2017 M/Gateway Developments Ltd A bit of background • Node.js: – Since August 2010 (v0.2 had just come out!) – Gave one of the talks at LNUG inaugural meeting
  5. 5. Copyright © 2017 M/Gateway Developments Ltd And So, 6 Years Later
  6. 6. Copyright © 2017 M/Gateway Developments Ltd Node.js Architecture • Everything executes in a single process – You might have 1000's of users doing stuff concurrently • They're all sharing the same process!
  7. 7. Copyright © 2017 M/Gateway Developments Ltd Node.js Architecture • Non-blocking I/O – No user activity must block the process, or everyone grinds to a halt • Event-driven, asynchronous logic – Fire off the request, but don't wait for the results • Carry on with the next task – When results come back, handle them at the next opportunity
  8. 8. Copyright © 2017 M/Gateway Developments Ltd Node.js = server-side JavaScript • Ryan Dahl wasn't actually a big JavaScript fan • However, he realised that it was a convenient language for such an environment, as it's designed for exactly the same kind of way of working in the browser • Google's V8 JavaScript engine was Open Sourced, so he used it to provide the language for Node.js – (which is actually mostly written in C++)
  9. 9. Copyright © 2017 M/Gateway Developments Ltd Asynchronous Syntax in Other Languages? • Node.js isn't unique in supporting asynchronous logic and non-blocking I/O • Available also in most other modern languages – Java – Python, etc • Allows efficient use of resources when making parallel requests for resources – Files, external web services etc
  10. 10. Copyright © 2017 M/Gateway Developments Ltd Asynchronous Syntax in Node.js • Node.js is unique in that you have no option but to use Asynchronous logic
  11. 11. Copyright © 2017 M/Gateway Developments Ltd Drawbacks of Node.js • In every other language, zealots will encourage you to use it for everything! • But even Node's greatest exponents will tell you not to use it for certain things
  12. 12. Copyright © 2017 M/Gateway Developments Ltd Drawbacks of Node.js • Don't use Node.js if you have: – CPU-intensive logic – Complex database manipulation • Particularly relational database handling
  13. 13. Copyright © 2017 M/Gateway Developments Ltd Drawbacks of Node.js • Don't use Node.js if you have: – CPU-intensive logic • Tie up the CPU and everyone else using the process will be blocked – Complex database manipulation • Particularly relational database handling
  14. 14. Copyright © 2017 M/Gateway Developments Ltd Drawbacks of Node.js • Don't use Node.js if you have: – CPU-intensive logic – Complex database manipulation • Particularly relational database handling – Due to the limitations of asynchronous logic, eg you can't chain functions » Limits creation of very high-level database abstractions
  15. 15. Copyright © 2017 M/Gateway Developments Ltd Main criticism of newbies to Node.js • Asynchronous logic
  16. 16. Copyright © 2017 M/Gateway Developments Ltd Async is the New Sync? • Node.js version 8 now supports Async/Await which greatly improves the syntax needed to handle asynchronous logic – Avoids "callback hell" – Very like synchronous logic
  17. 17. Copyright © 2017 M/Gateway Developments Ltd Async/Await Doesn't solve: • Node.js concurrency – Still need to avoid CPU-intensive code • High-level database abstractions – Proper chaining of functions requires synchronous logic
  18. 18. Copyright © 2017 M/Gateway Developments Ltd Usual Solutions • Use a third-party queue to offload CPU- intensive work to some other environment – RabbitMQ – ZeroMQ • Use another language such as Rails for database-intensive work
  19. 19. Copyright © 2017 M/Gateway Developments Ltd The down-sides • Heterogeneous environment • Multiple skill-sets • Multiple moving parts Node.js RabbitMQ Rails Java Database
  20. 20. Copyright © 2017 M/Gateway Developments Ltd What does Ryan Dahl Think?
  21. 21. Copyright © 2017 M/Gateway Developments Ltd What does Ryan Dahl Think? "..within a single process we could handle many requests by being completely asynchronous. I believed strongly in this idea at the time, but over the past couple of years, I think that’s probably not the be-all and end-all idea for programming."
  22. 22. Copyright © 2017 M/Gateway Developments Ltd What does Ryan Dahl Think? "..when I first started hearing about Go, which was around 2012, they had really easy to use abstractions, that make "blocking I/O", because it’s all in green threads at the interface between Go and the operating system. I think it is actually all non-blocking I/O."
  23. 23. Copyright © 2017 M/Gateway Developments Ltd What does Ryan Dahl Think? "I think Node is not the best system to build a massive server web. I would definitely use Go for that. And honestly, that’s basically the reason why I left Node. It was the realization that: oh, actually, this is not the best server side system ever."
  24. 24. Copyright © 2017 M/Gateway Developments Ltd What do I think? I want to have my Node.js Cake and Eat it!
  25. 25. Copyright © 2017 M/Gateway Developments Ltd I want my Node.js Cake & Eat it • I like JavaScript • I want just one language for everything – 1 skill set – 1 set of moving parts • No extra technologies, thank you – Keep it simple • And I want to be able to use Node.js for all situations
  26. 26. Copyright © 2017 M/Gateway Developments Ltd The Problem is Concurrency • All concurrent users in Node.js share the same process
  27. 27. Copyright © 2017 M/Gateway Developments Ltd The Problem is Concurrency • All concurrent users in Node.js share the same process • As it happens, Amazon Web Services accidentally created a solution – And nobody (including AWS) seems to have realised the consequences of what they've done
  28. 28. Copyright © 2017 M/Gateway Developments Ltd AWS Lambda • "Function As A Service" • AKA "Serverless"
  29. 29. Copyright © 2017 M/Gateway Developments Ltd AWS Lambda • "Function As A Service" • AKA "Serverless" • You upload functions • AWS will execute them – You don't worry about how or on what physical machine(s) • You pay per invocation of your function(s)
  30. 30. Copyright © 2017 M/Gateway Developments Ltd AWS Lambda • The first technology they supported was Node.js
  31. 31. Copyright © 2017 M/Gateway Developments Ltd AWS Concurrency? • Your function will be invoked in a private computation container of some sort • Your function has that container all to itself for the duration of its execution • No competition with other concurrent users
  32. 32. Copyright © 2017 M/Gateway Developments Ltd Lambda Node.js Examples • AWS examples show use of asynchronous APIs • Node.js users of Lambda use asynchronous APIs and Async/Await
  33. 33. Copyright © 2017 M/Gateway Developments Ltd Lambda Node.js Examples • AWS examples show use of asynchronous APIs • Node.js users of Lambda use asynchronous APIs and Async/Await • But unless your Lambda function really needs to be asynchronous, why use asynchronous logic?
  34. 34. Copyright © 2017 M/Gateway Developments Ltd Asynchronous Syntax in Other Languages? • Available also in most other modern languages – Java – Python, etc • Allows efficient use of resources when making parallel requests for resources – Files, external web services etc • But no programmer in those languages would use async logic if they didn't have to
  35. 35. Copyright © 2017 M/Gateway Developments Ltd Async is the New Sync? • Why would Node.js developers use asynchronous logic in Lambda functions if they don't have to? – Partly because that's what you do, right?
  36. 36. Copyright © 2017 M/Gateway Developments Ltd Async is the New Sync? • Why would Node.js developers use asynchronous logic in Lambda functions if they don't have to? – Partly because that's what you do, right? – Partly because almost no synchronous APIs exist, particularly for: • Database integration • Web/REST service access
  37. 37. Copyright © 2017 M/Gateway Developments Ltd Such APIs are possible
  38. 38. Copyright © 2017 M/Gateway Developments Ltd Such APIs are possible
  39. 39. Copyright © 2017 M/Gateway Developments Ltd Any other way… • To have your Node.js cake and eat it? – Not everyone will want to use Lambda • Would it be possible to create a locally- available environment where my Node.js code runs in an isolated container where concurrency isn't an issue?
  40. 40. Copyright © 2017 M/Gateway Developments Ltd
  41. 41. Copyright © 2017 M/Gateway Developments Ltd What Is QEWD? • Essentially it's a multi-purpose Node.js- based run-time Platform
  42. 42. Copyright © 2017 M/Gateway Developments Ltd What Is QEWD? • Essentially it's a multi-purpose Node.js- based run-time Platform • Creates an isolated run-time container for your message/request handler functions, allowing: – CPU-intensive work – Database abstractions using synchronous logic
  43. 43. Copyright © 2017 M/Gateway Developments Ltd QEWD's Architecture • Master Process • Pool of Worker Processes
  44. 44. Copyright © 2017 M/Gateway Developments Ltd QEWD's Architecture • Master Process – Handles and queues all incoming requests from client • HTTP/REST requests via Express or Koa.js • WebSocket requests via socket.io – Returns responses to client
  45. 45. Copyright © 2017 M/Gateway Developments Ltd QEWD's Architecture • Pool of Persistent Worker Processes – Where all the processing occurs – A single queued request is dispatched to an available worker process – Each Worker process handles a single request at a time
  46. 46. Copyright © 2017 M/Gateway Developments Ltd QEWD's Architecture • The queue / dispatcher / worker-process pool management part of QEWD is handled by a module named ewd-qoper8
  47. 47. Copyright © 2017 M/Gateway Developments Ltd Master Node.js Process Queue Queue processor/ dispatcher Incoming Requests QEWD Architecture Every incoming request is passed from Express and placed in a queue No further processing of requests occurs in the master process Express or Koa.js socket.io HTTP REST WebSocket
  48. 48. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Queue dispatcher is invoked whenever a request is added to the queue
  49. 49. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Node.js Worker Process Master Node.js Process Queue Queue processor/ dispatcher Worker process started if none available
  50. 50. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Node.js Worker Process Master Node.js Process Queue Queue processor/ dispatcher Redis/Cache/GT.M QEWD & application-specific Modules loaded Custom Worker Module Custom Worker Module
  51. 51. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Node.js Worker Process Master Node.js Process Queue Queue processor/ dispatcher Custom Worker Module Node.js Worker Process Custom Worker Module Request passed to worker Redis/Cache/GT.M
  52. 52. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Node.js Worker Process Master Node.js Process Queue Queue processor/ dispatcher Custom Worker Module Worker flagged as Unavailable Node.js Worker Process Custom Worker Module Begin processing message Redis/Cache/GT.M
  53. 53. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Unavailable / processing Another incoming request Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache/GT.M
  54. 54. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Node.js Worker Process Master Node.js Process Queue Queue processor/ dispatcher Custom Worker Module Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module If worker pool size not exceeded, another worker is started and request passed to it Redis/Cache/GT.M Redis/Cache/GT.M
  55. 55. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache/GT.M If entire Worker Pool is busy: Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache/GT.M Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache/GT.M
  56. 56. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher If entire Worker Pool is busy: New requests remain in queue Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache/GT.M Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache/GT.M Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache/GT.M
  57. 57. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache/GT.M As soon as a worker is available again, a queued message can be passed to it Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache/GT.M Available Node.js Worker Process Custom Worker Module Redis/Cache/GT.M
  58. 58. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Finished Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache/GT.M A user's handler function signals completion using the function: finished(responseObject); This returns the response object to the master process
  59. 59. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Finished Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module And the response is returned to the client that sent the original request (via Express/Koa/socket.io) Redis/Cache/GT.M
  60. 60. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Available Node.js Worker Process Custom Worker Module The finished() function also automatically returns the worker process back to the available pool So it can now handle the next queued request Redis/Cache/GT.M
  61. 61. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Available Node.js Worker Process Custom Worker Module Worker processes, once started, are persistent No start-up / tear-down cost Redis/Cache/GT.M
  62. 62. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Node.js concurrency is handled by the master process. 100% asynchronous logic The master process does almost nothing The perfect Node.js application: no CPU-intensive or long- running tasks, so very high-performance Multiple Concurrent Incoming requests
  63. 63. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Node.js concurrency is handled by the master process. 100% asynchronous logic The master process does almost nothing The perfect Node.js application: no CPU-intensive or long- running tasks, so very high-performance All the actual work happens in the isolated worker processes Multiple Concurrent Incoming requests
  64. 64. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Available Node.js Worker Process Custom Worker Module Worker processes only handle a single request at a time Completely isolated run-time environment for handler functions No need for concerns about Node.js concurrency, so synchronous APIs can be used Redis/Cache/GT.M
  65. 65. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Available Node.js Worker Process Custom Worker Module Redis/Cache/GT.M Long-running or CPU-intensive logic has no direct impact on other worker processes
  66. 66. Copyright © 2017 M/Gateway Developments Ltd Why not just use child_process? • To avoid the very high performance cost of starting and tearing down each child process – At least 30ms to start a child_process • Each QEWD worker is a child_process – But keeps running, and is re-used
  67. 67. Copyright © 2017 M/Gateway Developments Ltd Why not just use Cluster? • Cluster simply spreads the concurrent requests across a pool of persistent child_processes – If one of them is CPU intensive, all other requests in that cluster are held up • QEWD forces each child_process to handle a single request only – Creates an isolated run-time environment for each request
  68. 68. Copyright © 2017 M/Gateway Developments Ltd Why not just use RabbitMQ or ZeroMQ? • I just want to just use a single technology: Node.js / JavaScript • Initial tests showed no performance benefit in using ZeroMQ
  69. 69. Copyright © 2017 M/Gateway Developments Ltd Performance? • ewd-qoper8 module in isolation: – Handles the queue/master/worker architecture • Raspberry Pi 3 Model B – 4 core CPU – Readily-available commodity item – Low-cost • Easily-replicable benchmark
  70. 70. Copyright © 2017 M/Gateway Developments Ltd Performance • ewd-qoper8 benchmark script provided: – How many workers? 3 – How many messages? 500,000 – Create a steady state of messages added to the queue as they're being processed: • Add a batch of 622 messages at a time • Wait 100ms between each batch – Messages are sent to workers which echo them straight back
  71. 71. Copyright © 2017 M/Gateway Developments Ltd Performance • 5,800 messages/second sustained throughput • Limiting factor is master process hitting 100% CPU • Workers only 30% CPU, so plenty of capacity to do real work at this rate
  72. 72. Copyright © 2017 M/Gateway Developments Ltd What can QEWD be Used For? • Full-stack platform for browser & Native mobile applications – WebSocket or Ajax messaging • Seamlessly swap between transports • All automated and abstracted out of the way – With security also automated for you too
  73. 73. Copyright © 2017 M/Gateway Developments Ltd What can QEWD be Used For? • API server for REST applications • Optional session management – Automated token-based authentication – Server-side (persistent JSON) session storage • Or automated JSON Web Token based: – Authentication – Client-side session storage
  74. 74. Copyright © 2017 M/Gateway Developments Ltd What can QEWD be Used For? • MicroService Platform – With each MicroService supported by its own shared or dedicated QEWD server – Using JWTs • Shared secret on each QEWD server – Using socket.io to provide high-performance, persistent connections between QEWD servers • Secured over HTTPS if required
  75. 75. Copyright © 2017 M/Gateway Developments Ltd QEWD MicroService Fabric ewd-qoper8 queue Express Node.js socket.io Cache GT.M, YottaDB Redis Node.js Worker Process Cache GT.M, YottaDB Redis Node.js Worker Process Cache GT.M, YottaDB Redis Node.js Worker Process ewd-qoper8 queue Express Node.js socket.io Cache GT.M, YottaDB Redis Node.js Worker Process Cache GT.M, YottaDB Redis Node.js Worker Process Cache GT.M, YottaDB Redis Node.js Worker Process ewd-qoper8 queue Express Node.js socket.io Cache GT.M, YottaDB Redis Node.js Worker Process Cache GT.M, YottaDB Redis Node.js Worker Process Cache GT.M, YottaDB Redis Node.js Worker Process User authentication Demographics Pharmacy ewd-qoper8 queue Express Node.js socket.io Cache GT.M, YottaDB Redis Node.js Worker Process Cache GT.M, YottaDB Redis Node.js Worker Process Cache GT.M, YottaDB Redis Node.js Worker Process Client Orchestration HTTPS WebSocket Connections
  76. 76. Copyright © 2017 M/Gateway Developments Ltd What can QEWD be Used For? • Federation platform – Providing a REST or WebSocket-based middle tier to multiple REST-based back-end servers • of any type, not just QEWD servers – One request in • Routed to all back-end systems and responses combined • Performing a "dance" between federated servers
  77. 77. Copyright © 2017 M/Gateway Developments Ltd Federated Access to OpenEHR Browser QEWD GT.M or Redis ewd-qoper8 queue qewd-ripple Module Express OpenEHR Server AQL over HTTP(S) Worker PulseTile UI OpenEHR Server
  78. 78. Copyright © 2017 M/Gateway Developments Ltd What can QEWD be Used For? • Interface to particular databases known as Global Storage databases – GT.M / YottaDB – Cache / IRIS – Redis (ewd-globals-redis module) • All of which are abstracted as: – Persistent JavaScript Objects – Fine-grained Document Database • Accessible at any level down to individual name/value pair anywhere in a JSON object
  79. 79. Copyright © 2017 M/Gateway Developments Ltd QEWD.js • Resilient / Audit mode – When enabled, permanent record kept in the database of: • Queued requests • Processing status • Response(s) – If QEWD restarted, database is examined for queued, unprocessed requests • Automatically re-queued – Can specify retention period of database records
  80. 80. Copyright © 2017 M/Gateway Developments Ltd QEWD.js • Supports all browser-side JavaScript frameworks • Some cool 3rd -party tooling and support available for React and Vue.js
  81. 81. Copyright © 2017 M/Gateway Developments Ltd Have Your Node.js Cake And Eat It Too http://qewdjs.com

×