The document discusses the challenges of handling CPU-intensive operations in Node.js, highlighting the differences between blocking and non-blocking I/O operations as well as the role of the event loop. It introduces techniques like clustering and worker threads to leverage multi-core systems and perform parallel computations. The document concludes by suggesting that while Node.js is effective for I/O-bound tasks, it may not be the best choice for heavy computational tasks.

1. NODE, CAN YOU EVEN IN CPU
INTENSIVE OPERATIONS?

2. RAFAŁ OSTROWSKI
RAFAL.OSTROWSKI@TSH.IO
NODE.JS DEVELOPER

3. WHAT'S THE PLAN?
▸ SINGLE THREADED?
▸ NON BLOCKING I/O OPERATIONS
▸ PERFORMANCE TRAPS
▸ CPU-INTENSIVE OPERATIONS WITHOUT KILLING
OUR APPS

4. ▸ HTTP REQUESTS
▸ DISK READS/WRITES
▸ DB
I/O OPERATIONS
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?

5. BLOCKING VS NON-
BLOCKING I/O

6. ▸ NEXT OPERATIONS HAVE TO WAIT
▸ APPLICATION DOES NOTHING WHILE WAITING
FOR THE RESULT
▸ TOP-DOWN - SYNCHRONOUS
▸ TIME = FUNCTION'S CALL + OPERATION’S TIME
▸ EXAMPLE: readFileSync
BLOCKING I/O
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?

7. BLOCKING I/O EXAMPLE

8. ▸ NEXT OPERATIONS DON'T HAVE TO WAIT
▸ ASYNC
▸ CALLBACKS!
▸ TIME = FUNCTION'S CALL + CALLBACK'S
HANDLING
▸ EXAMPLE: readFile
NON-BLOCKING I/O
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?

9. NON-BLOCKING I/O EXAMPLE

10. EVENT LOOP

11. “THE EVENT LOOP IS WHAT ALLOWS NODE.JS TO PERFORM NON-BLOCKING
I/O OPERATIONS,
DESPITE THE FACT THAT JAVASCRIPT IS SINGLE-THREADED,
BY OFFLOADING OPERATIONS TO THE SYSTEM KERNEL WHENEVER POSSIBLE.”

12. ▸ SIMPLIFIED EXPLANATION!
▸ IMPLEMENTED IN LIBUV
▸ EVENT LOOP IS, WELL, AN EVENT LOOP
▸ STARTED WHEN NODE PROCESS STARTS
▸ ENDS WHEN THERE'S NOTHING MORE TO
PROCESS
EVENT LOOP
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?

14. ▸ SYNCHRONOUS CODE IS EASY
▸ ASYNC NOT SO MUCH - ESPECIALLY IN A SINGLE
THREADED ENVIRONMENT
▸ CALLBACKS MUST BE HANDLED SOMEHOW
▸ CAN'T HAPPEN ANYTIME
▸ WE NEED A MECHANISM THAT WILL QUEUE THESE
CALLBACKS
EVENT LOOP, WHAT FOR?
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?

15. LET'S RUN SOME
BENCHMARKS

16. BENCHMARKS
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
▸ API
▸ /userBlocking
▸ /userNonBlocking
▸ USER READ TIME = 75ms
▸ APACHE BENCHMARK: -c 10 -t 5

19. LIFE'S GOOD
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
▸ API RUNNING HAPPILY ON PROD
▸ NON-BLOCKING I/O ONLY
▸ AWESOME PERFORMANCE OUT OF THE BOX
▸ $$$$

20. NEW REQUIREMENTS

22. NEW REQUIREMENTS
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
▸ NEW ENDPOINT FOR SOME HYPER COMPLICATED
CALCULATIONS
▸ CONSUMED BY OTHER TEAM
▸ BATCH JOB
▸ OUR ALGORITHM WILL PROCESS SOME DATA
▸ WE'LL EXPOSE THE ENDPOINT IN OUR EXISTING
API

23. DEPLOY

27. LET’S FIND (AND HOPEFULLY
FIX) THE PROBLEM

28. LET’S FIND THE PROBLEM
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
▸ HIT CPU-CONSUMING ENDPOINT IN A LOOP
▸ RUN THE SAME BENCHMARK AS BEFORE (ab -c 10
-t 5)

31. CLUSTER MODE

32. "A SINGLE INSTANCE OF NODE.JS RUNS IN A SINGLE THREAD.
TO TAKE ADVANTAGE OF MULTI-CORE SYSTEMS, THE USER
WILL SOMETIMES WANT TO LAUNCH A CLUSTER OF NODE.JS
PROCESSES TO HANDLE THE LOAD."

33. ▸ WE'LL HARNESS OUR CPU'S FULL POWER
▸ LOAD BALANCER OUT OF THE BOX
▸ LET'S IMPLEMENT IT!
CLUSTER MODE
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?

34. const cluster = require('cluster');
const numCPUs = require('os').cpus().length;
if (cluster.isMaster) {
console.log(`Master ${process.pid} is running`);
for (let i = 0; i < numCPUs; i++) {
cluster.fork();
}
cluster.on('exit', (worker, code, signal) => {
console.log(`worker ${worker.process.pid} died`);
});
} else {
// run http server
console.log(`Process ${process.pid} started`);
}

35. const cluster = require('cluster');
const numCPUs = require('os').cpus().length;
if (cluster.isMaster) {
console.log(`Master ${process.pid} is running`);
for (let i = 0; i < numCPUs; i++) {
cluster.fork();
}
cluster.on('exit', (worker, code, signal) => {
console.log(`worker ${worker.process.pid} died`);
});
} else {
// run http server
console.log(`Process ${process.pid} started`);
}

36. const cluster = require('cluster');
const numCPUs = require('os').cpus().length;
if (cluster.isMaster) {
console.log(`Master ${process.pid} is running`);
for (let i = 0; i < numCPUs; i++) {
cluster.fork();
}
cluster.on('exit', (worker, code, signal) => {
console.log(`worker ${worker.process.pid} died`);
});
} else {
// run http server
console.log(`Process ${process.pid} started`);
}

37. const cluster = require('cluster');
const numCPUs = require('os').cpus().length;
if (cluster.isMaster) {
console.log(`Master ${process.pid} is running`);
for (let i = 0; i < numCPUs; i++) {
cluster.fork();
}
cluster.on('exit', (worker, code, signal) => {
console.log(`worker ${worker.process.pid} died`);
});
} else {
// run http server
console.log(`Process ${process.pid} started`);
}

38. LET'S RUN SOME
BENCHMARKS

41. DEPLOY

45. LET’S FIND (AND FIX) THE
PROBLEM

46. LET’S FIND THE PROBLEM (AGAIN)
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
▸ HIT CPU-CONSUMING ENDPOINT IN A LOOP, BUT
IN PARALLEL THIS TIME (5 SCRIPTS
SIMULTANEOUSLY)
▸ RUN THE SAME BENCHMARK AS BEFORE (ab -c 10
-t 5) on /userNonBlocking

49. WORKER THREADS

50. "WORKERS (THREADS) ARE USEFUL FOR PERFORMING CPU-INTENSIVE JAVASCRIPT
OPERATIONS.
THEY WILL NOT HELP MUCH WITH I/O-INTENSIVE WORK.
NODE.JS’S BUILT-IN ASYNCHRONOUS I/O OPERATIONS ARE MORE EFFICIENT THAN
WORKERS CAN BE."

51. WORKER THREADS
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
▸ ADDED IN NODE 10.5
▸ ALLOWS TO EXECUTE JAVASCRIPT IN PARALLEL
▸ STABILITY: EXPERIMENTAL
▸ FLAG --experimental-worker NEEDED FOR NODE <
11.7
▸ 'worker_threads' MODULE

52. LET'S IMPLEMENT IT!

53. const {Worker, isMainThread, parentPort} = require('worker_threads')
if (isMainThread) {
module.exports = async function timeConsumingOperationOnThreads(raw) {
return new Promise((resolve, reject) => {
const worker = new Worker(__filename, {
workerData: raw
});
worker.on('message', resolve)
worker.on('error', reject)
worker.on('exit', (code) => {
if (code !== 0) {
reject(new Error(`Worker stopped with exit code ${code}`))
}
});
});
};
} else {
const result = runSuperComplicatedAlgorithm()
parentPort.postMessage({ result })
}

54. const {Worker, isMainThread, parentPort} = require('worker_threads')
if (isMainThread) {
module.exports = async function timeConsumingOperationOnThreads(raw) {
return new Promise((resolve, reject) => {
const worker = new Worker(__filename, {
workerData: raw
});
worker.on('message', resolve)
worker.on('error', reject)
worker.on('exit', (code) => {
if (code !== 0) {
reject(new Error(`Worker stopped with exit code ${code}`))
}
});
});
};
} else {
const result = runSuperComplicatedAlgorithm()
parentPort.postMessage({ result })
}

55. const {Worker, isMainThread, parentPort} = require('worker_threads')
if (isMainThread) {
module.exports = async function timeConsumingOperationOnThreads(raw) {
return new Promise((resolve, reject) => {
const worker = new Worker(__filename, {
workerData: raw
});
worker.on('message', resolve)
worker.on('error', reject)
worker.on('exit', (code) => {
if (code !== 0) {
reject(new Error(`Worker stopped with exit code ${code}`))
}
});
});
};
} else {
const result = runSuperComplicatedAlgorithm()
parentPort.postMessage({ result })
}

56. const {Worker, isMainThread, parentPort} = require('worker_threads')
if (isMainThread) {
module.exports = async function timeConsumingOperationOnThreads(raw) {
return new Promise((resolve, reject) => {
const worker = new Worker(__filename, {
workerData: raw
});
worker.on('message', resolve)
worker.on('error', reject)
worker.on('exit', (code) => {
if (code !== 0) {
reject(new Error(`Worker stopped with exit code ${code}`))
}
});
});
};
} else {
const result = runSuperComplicatedAlgorithm()
parentPort.postMessage({ result })
}

57. LET'S RUN SOME
BENCHMARKS

60. DEPLOY

63. ALTERNATIVES

64. PARTITIONING
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
▸ COMPLETE PART OF THE WORK => PUSH BACK TO
EVENT LOOP
▸ NOT APPLICABLE TO ALL ALGORITHMS
▸ WILL TAKE LONGER TIME
▸ MIGHT NOT BE POSSIBLE WHEN USING EXTERNAL
LIBRARIES
▸ HARDER TO REASON ABOUT

65. WITHOUT PARTITIONING:

66. WITH PARTITIONING:

67. PROCESS.FORK
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
▸ IMPOSSIBLE TO SHARE MEMORY BETWEEN
PROCESSES
▸ EXPENSIVE IN TERMS OF RESOURCES
▸ NOT AS LIGHTWEIGHT AS THREADS
▸ WHY?

69. PROCESS.FORK CONT.
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
▸ NEW WORKER THREAD * 500 = ~2000 ms
▸ NEW PROCESS * 500 = ~12300 ms

70. CHANGE YOUR ARCHITECTURE
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
▸ SEPARATE API
▸ MOVE TIME CONSUMING OPERATION TO A
SEPARATE SERVICE WITH A QUEUE IN FRONT OF IT
▸ THEY WILL STILL SUFFER FROM LOW
PERFORMANCE, JUST WON’T AFFECT OTHER
ENDPOINTS

71. “(...) HOWEVER, IF YOUR SERVER RELIES HEAVILY ON COMPLEX
CALCULATIONS, YOU SHOULD THINK ABOUT WHETHER NODE IS REALLY A GOOD
FIT.
NODE EXCELS FOR I/O-BOUND WORK, BUT FOR EXPENSIVE COMPUTATION IT
MIGHT NOT BE THE BEST OPTION.”

72. ▸ FAVOR NON-BLOCKING (ASYNC) I/O OVER
BLOCKING OPERATION
▸ VERTICAL SCALING (CLUSTER MODE) MIGHT NOT
SOLVE YOUR PROBLEMS WITH CPU-INTENSIVE
OPERATIONS...
▸ ...BUT WORKER THREADS CAN (JUST REMEMBER -
THEY'RE STILL AN EXPERIMENTAL FEATURE!)
▸ NODE.JS EXCELS IN I/O BOUND AND MIGHT NOT
BE THE BEST CHOICE FOR CPU-HEAVY
OPERATIONS
SUMMARY
NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?

73. THANKS