Recommended
More Related Content
Similar to Nodeconf AR 18 Decomposition of the Main thread in Node.js to increase throughput
Similar to Nodeconf AR 18 Decomposition of the Main thread in Node.js to increase throughput (20)
Recently uploaded
Recently uploaded (20)
Nodeconf AR 18 Decomposition of the Main thread in Node.js to increase throughput
- 1. 1 DECOMPOSITION OF THE MAIN THREAD IN NODE.JS TO INCREASE THROUGHPUT
- 2. HELLO!I am Nikolay Matvienko JS Developer at Grid Dynamics You can find me at twitter.com/matvi3nko github.com/matvi3nko 2
- 3. 3@matvi3nko NODE.JS IN ENTERPRISE ● IS WIDELY USED ● HIGHLOAD ON BLACK FRIDAY ● COMPETITION IN THE BUSINESS ○ GROWTH OF PERFORMANCE REQUIREMENTS ○ GROWTH OF FUNCTIONALITY/FEATURES
- 4. “WHAT IS FAST TODAY WILL BE SLOW TOMORROW AS DEMANDS CAN ONLY GROW” 44
- 6. MUST-HAVE PART MONITORING LOGGING FRAMEWORKS ... MONITORING LOGGING FRAMEWORKS ... MONITORING LOGGING FRAMEWORKS ... MONITORING LOGGING FRAMEWORKS ... MONITORING LOGGING FRAMEWORKS ... 6 CODE MONOLITH MICROSERVICES @matvi3nko CODE CODE CODECODE 100K lines of code 100 lines of code each MUST-HAVE PART CODE CODE
- 7. BLOCKED EVENT LOOP 1. HTTP 2. HTTP 3. DB 4. NATIVE MODULE ... CALLBACK QUEUE HTTP response DB response MODULE response HTTP response EVENT LOOP INCOMING REQUESTS USERS THAT ARE WAITING FOR RESPONSE 7@matvi3nko
- 10. SCALING APP worker 1. MULTIPLE PROCESSES - CLUSTER module - PM2 2. MULTIPLE SERVERS WEB SERVER / REVERSE PROXY - PHUSION PASSENGER - NGINX APP worker APP worker APP worker CLUSTER WEB SERVER / REVERSE PROXY APP worker APP worker FREE CORE FREE CORE CLUSTER SERVER 10@matvi3nko
- 11. LOAD BALANCING CLUSTER WEB SERVER / REVERSE PROXY APP worker APP worker FREE CORE FREE CORE CLUSTER 1. 2. PROCESS MANAGER CLUSTER SERVER1 SERVERS1,2 11@matvi3nko
- 12. ms RESPONSE TIME CRITICAL BASELINE USER LEFT SITE 12@matvi3nko
- 13. MAIN THREAD PROFILLER GARBAGE COLLECTION METRICS COLLECTION FRAMEWORK Server-Side RENDERING DISPERSION 13 COMPUTATIONS LOGGING @matvi3nko DE/SERIALIZATION
- 14. JS LATENCY 14 @matvi3nko EXECUTION EACH REQUEST USERS PAY FOR LOGGING, GC AND METRICS COLLECTIONS … WITH THEIR TIME.
- 16. “THE WORLD IS MINE” © GARBAGE COLLECTOR 1959 16@matvi3nko
- 17. GARBAGE COLLECTION STOP THE SERVER STOP THE WORLD MAIN THREAD APP CODE EXECUTION + JS execution MAIN THREAD OLD SPACE NEW SPACE MARK EVACUATE APP CODE EXECUTION 17@matvi3nko
- 18. LESS PERFORMANCE IMPACT OLD SPACE INCREMENTAL COLLECTION NEW SPACE MARK EVACUATE APP CODE EXECUTION 18 MAIN THREAD @matvi3nko
- 19. MAIN THREAD APP CODE EXECUTION + JS execution GC DECOMPOSITION THREAD THREAD THREAD 1. PARALLEL MARK-SWEEP 2. PARALLEL SCAVENGER 3. PARALLEL MARK-EVACUATE X2 execution -30% mark overhead -70% evacuate overhead ALGORITHMS of ORINOCO GC: New V8 threads: MAIN THREAD APP CODE EXECUTION + JS execution MAIN THREAD APP CODE EXECUTION + JS execution MAIN THREAD APP CODE EXECUTION 19@matvi3nko
- 20. “THE WORLD IS MINE NOW” © JS COMPUTING OPERATION IN NODE.JS 2009 … 50 years later 20@matvi3nko
- 22. MULTY IN NODE.JS THREADING 22@matvi3nko WEBWORKER THREADS / NAPA.JS / ALIOS
- 23. WORKER THREADS 23@matvi3nko ● RESOURCES ARE LIMITED ○ IoT, modules ● TASK IS DIFFICULT TO SEPARATE FROM THE APPLICATION CONTEXT ● PARSING, HASHING, HTML MINIFICATION, ...
- 24. FRAMEWORK DEMO APP ARCHITECTURE IO TASK APP CONFIGU RATION LOGIC CPU TASK APM AGENT LOGGER RENDERING INSTANCE COUNT CHOOSE 24@matvi3nko
- 25. MAIN THREAD PROFILLER CURRENT THROUGHPUT: 190 REQ/S 25 COMPUTATIONS @matvi3nko DECOMPOSING
- 26. CPU-BOUND TASKS MAIN THREAD APP CODE EXECUTION CPU-bound processing 26@matvi3nko CALLBACK QUEUE LOGIC IN CB REDIS response REDIS response HTTP response EVENT LOOP INCOMING REQUESTS
- 28. MAIN THREAD WORKER THREAD NEW WORKER THREAD THREAD POOL THREAD POOL POOL of workers ACTIVE workers WAITING tasks Applicationprocess 28@matvi3nko MESSAGECHANNEL
- 30. CPU TASKS PARALLELIZATION RESULT 30 3XMORE REQ/S WITH PARALLEL JS COMPUTATION @matvi3nko 190 605
- 31. MAIN THREAD PROFILLER CURRENT THROUGHPUT: 605 REQ/S 31 DE/SERIALIZATION @matvi3nko DE/SERIALIZATION
- 34. IN NOT NATIVE ENV 34@matvi3nko SERIALIZATION & JSON PARSE Х6 FASTERPROTOBUF *Average values in not JavaScript environment.
- 35. JSON IN NATIVE ENV 35@matvi3nko + OPTIMIZED BY V8 TEAM - DEPENDS ON THE TYPE OF DATA
- 36. NUMBERS 36@matvi3nko SERIALIZATION JSON JSON WITH SCHEMA PARSE Х2 FASTER Х2 FASTER +30% PROTOBUF *Average values in JavaScript environment. See libraries in resources. IN NODE.JS
- 37. JSON STRINGS 37@matvi3nko SERIALIZATION JSON WITH SCHEMA Х2 PARSE X2.3 PROTOBUF Х3 FASTER Х2 FASTER *Average values in JavaScript environment. See libraries in resources. IN NODE.JS
- 38. “THAT JSON IS NOT COMPLETELY AWFUL” Eddie Broc 38@matvi3nko
- 39. JSON.PARSE CHANGE 39 RESULT +27% MORE REQ/S WITH FAST-JSON-STRINGIFY & JITSON/ TURBO-JSON-PARSE @matvi3nko 768 605 JSON.parse JSON with Schema
- 40. MAIN THREAD PROFILLER FRAMEWORK 40 CURRENT THROUGHPUT: 768 REQ/S @matvi3nko FRAMEWORK
- 41. HAPI, EXPRESS, RESTIFY… FRAMEWORK Node.js HTTP SERVER OVERHEAD ~ 1.5 – 2X SLOWLY than http.createServer https://github.com/fastify/fast-json-stringify 41@matvi3nko
- 42. FRAMEWORK OPTIMIZATIONS Node.js HTTP SERVER 1. Express https://github.com/expressjs/express ROUTER JSON Stringify X5 faster https://github.com/delve dor/router-benchmark X2-3 faster https://github.com/fastify/ fast-json-stringify 2. Fastify https://github.com/fastify/fastify 42@matvi3nko
- 43. FRAMEWORK CHANGE 43 RESULT +20% MORE REQ/S WITH FASTIFY @matvi3nko 922 768
- 44. MAIN THREAD PROFILLER LOGGING 44 LOG CURRENT THROUGHPUT: 922 REQ/S @matvi3nko
- 45. MAIN THREAD APP CODE EXECUTION WRITE LOG LOGGING WRITE LOG FILES: sync on Windows and POSIX TERMINALS:: async on Windows, sync on POSIX PIPES, SOCKETS: sync on Windows, async on POSIX 45@matvi3nko process.stdout and .stderr FORMAT MESSAGE SERIALIZE MESSAGE HANDLE TRANSPORT LOGIC
- 46. MAIN THREAD APP CODE EXECUTION SEND MSG OFF-PROCESS LOGGER TRANSPORT 1. Pino 2. Roarr LOGGERS: SEND MSG 2nd Node.js app LOGGER TRANSPORT SEND LOG SEND LOG process.stdout 46@matvi3nko
- 48. OFF-PROCESS LOGGING RESULT 48 +17% MORE REQ/S WITH OFF-PROCESS LOGGER TRANSPORT @matvi3nko 922 1078
- 49. MAIN THREAD PROFILLER APPLICATION PERFORMANCE MONITORING 49 APM CURRENT THROUGHPUT: 1078 REQ/S @matvi3nko
- 50. MAIN THREAD APP CODE EXECUTION APM AGENT APPLICATION PERFORMANCE MONITORING 1. NewRelic 2. Dynatrace 3. OpenTracing 4. node-measured APM vendors/agents: METRICS COLLECTION AGGREGATION TRANSPORT 50@matvi3nko
- 51. WEB SERVER WORKER WEB SERVER WORKER WEB SERVER WORKER WEB SERVER WORKER Cluster/Load balancer APM Time series DB Analytics & Monitoring Dashboard APM APM APM SaaS 51 IN-PROCESS APM AGENT THE APM AGENT PROBLEMS ARE APPLICATION PROBLEMS @matvi3nko
- 52. WEB SERVER APP WEB SERVER APP WEB SERVER APP WEB SERVER APP .send() Message queue APM PROCESS .on() Message Queue Time series DB Analytics & Monitoring Dashboard 52 OFF-PROCESS APM AGENT @matvi3nko
- 53. OFF-PROCESS MONITORING RESULT 53 +25% MORE REQ/S WITH OFF-PROCESS METRIC AGENT @matvi3nko 1078 1347
- 54. MAIN THREAD PROFILLER 54 SSR CURRENT THROUGHPUT: 1347 REQ/S @matvi3nko SERVER SIDE RENDERING
- 55. SERVER-SIDE RENDERING CALLBACK QUEUE RENDERING REDIS response REDIS response HTTP response EVENT LOOP MAIN THREAD APP CODE EXECUTION RENDERING 55 INCOMING REQUESTS @matvi3nko
- 57. STREAMING SERVER-SIDE RENDERING MAIN THREAD APP CODE EXECUTION R E N D E R ... Network HTML chunks Asynchronous execution in STREAM with REACT 16 renderStream.pipe(res, { end: 'false' }); renderStream.on('end', () => {response.end('</div></body></html>'); }); 57@matvi3nko
- 58. MICRO FRONTENDS MAIN THREAD OF Monolith WEB UI app 1 2 3 Renders full page to HTML string Backend/Aggregator Mirco service Monolith WEB UI Mirco service Mirco service WEB UI 1 WEB UI 2 WEB UI 3 Renders full page 58@matvi3nko
- 59. RENDERING WORKER Node.js PARALLEL RENDERING WITH WORKERS MAIN APP / MICROSERVICE Node.js Combines streams of Different page parts 59@matvi3nko RENDERING WORKER Node.js renderToNodeStream() renderToStaticNodeStream() DYNAMIC CONTENT STATIC CONTENT
- 60. 2X THROUGHPUT * Average value. REACT 16 60@matvi3nko
- 61. FROM 190 REQ/S TO 1350 61@matvi3nko
- 62. DECOMPOSED MAIN THREAD MAIN THREAD APPLICATION 1 LOG 2 APM 3 CPU V8 WORKER THREAD MAIN THREAD of LOGGER process V8 THREADPOOL LIBUV THREADPOOL MAIN THREAD of APM process MAIN THREAD of CPU Tasks WORKER 4 GC 5 IO Use for creation of New Node.js modules 62@matvi3nko
- 63. 63
- 64. ORDER 64@matvi3nko CPU BOUND TASKS RENDERING SERIALIZATION …. FROM LONGEST OPERATION
- 68. REFERENCES Long-running Background Process in Node.js https://vimeo.com/229536743 Background tasks in Node.js https://www.youtube.com/watch?v=NNTsHzER31I&t=2207s https://blog.evantahler.com/background-tasks-in-node-js-a-survey-with-redis-971d3575d9d2 Streaming Server-Side Rendering and Caching https://zeit.co/blog/streaming-server-rendering-at-spectrum https://github.com/zalando/tailor Microservices on UI https://www.youtube.com/watch?v=3l9IP9j5n1o https://www.youtube.com/watch?v=E6_UyQPmiSg&t=2997s 68
- 69. REFERENCES New Garbage Collection with threads https://v8project.blogspot.ru/2017/11/ https://v8project.blogspot.com/2016/04/jank-busters-part-two-orinoco.html Pino https://github.com/pinojs/pino New Worker API in Node.js discussion https://github.com/nodejs/worker/issues/4 IPC Communication Performance https://60devs.com/performance-of-inter-process-communications-in-nodejs.html List of Parallel JS Projects https://github.com/SyntheticSemantics/List-of-Parallel-JS-Projects 69
- 70. WEBWORKER THREADS WORKER THREAD 1 V8 INSTANCE WebWorker Threads https://www.npmjs.com/package/webworker-threads WORKER THREAD V8 INSTANCE WORKER THREAD V8 INSTANCE WORKER THREAD 2 V8 INSTANCE EVENT LOOP MODULES LIBUV THREAD POOL for IO Node.js 70@matvi3nko
- 71. MICROSOFT NAPA.JS EVENT LOOP MODULES LIBUV THREAD POOL for IO Node.js ZONE 1 JS WORKERS THREAD POOL WORKER 1 V8 Instance WORKER 2 V8 Instance WORKER 3 V8 Instance ZONE 2 JS WORKERS THREAD POOL WORKER 1 V8 Instance WORKER 2 V8 Instance WORKER 3 V8 Instance Napa.js https://github.com/Microsoft/napajs MEMORY USAGE 6.7 MB vs 8 MB STARTUP TIME 50 ms vs 70 ms MESSAGE PASSING 2x vs IPC 71@matvi3nko
- 72. ALIBABA ALIOS EVENT LOOP MODULES LIBUV THREAD POOL for IO Node.js THREAD 1 NODE.JS INSTANCE EVENT LOOP V8 INSTANCE MODULES ALiOS-node.js https://github.com/alibaba/AliOS-nodejs THREAD 2 NODE.JS INSTANCE EVENT LOOP V8 INSTANCE MODULES MEMORY USAGE 2.5 MB vs 8 MB STARTUP TIME 13 ms vs 70 ms SHARED GLOBAL MEMORY 72@matvi3nko