The document discusses memory limitations in Node.js and provides insights on common issues such as memory leaks and excessive data loading that can lead to 'out of memory' errors. It includes technical details such as JavaScript stack traces, memory allocation statistics, and methods for testing memory limits. Additionally, it offers recommendations to mitigate memory issues, including monitoring usage and avoiding loading entire datasets.

1. Node.js memory
limitations
How to blow up your app in 100 easy steps

2. Out of memory!
<--- Last few GCs --->
[43108:00000272EA114EF0] 2209 ms: Mark-sweep 1391.0 (1399.7) -> 1390.9 (1396.7) MB, 224.2 / 0.0 ms
last resort GC in old space requested
[43108:00000272EA114EF0] 2460 ms: Mark-sweep 1390.9 (1396.7) -> 1390.9 (1396.7) MB, 251.0 / 0.0 ms
last resort GC in old space requested
==== JS stack trace =========================================
Security context: 00000189F87A5EE1 <JSObject>
1: alloc [C:DropboxProjectsJSDWGithubnodejs-memory-testindex.js:~9]
[pc=000000A10D205D61](this=0000006F6330BF99 <JSGlobal Object>,size=10240000)
3: allocToMax [C:DropboxProjectsJSDWGithubnodejs-memory-testindex.js:41]
[bytecode=00000142282CFFB1 offset=160](this=0000006F6330BF99 <JSGlobal Object>)
5: /* anonymous */ [C:DropboxProjectsJSDWGithubnodejs-memory-testind...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node_module_register
2: v8::internal::FatalProcessOutOfMemory
3: v8::internal::FatalProcessOutOfMemory
4: v8::internal::Factory::NewUninitializedFixedArray

3. Data Wrangling with JavaScript

4. Why?

5. Why?
● Loading too much data
● Memory leak

6. Node.js memory architecture
● Code
● Stack
● Heap

7. Node.js heap
● New space
● Old space

8. Allocate unto death
while (true) {
Allocate some memory
const mu = process.memoryUsage();
Print memory allocated
}

9. Node.js memory architecture
Allocated since start 1.26 GB
Allocated since start 1.27 GB
Allocated since start 1.28 GB
Allocated since start 1.29 GB
Allocated since start 1.3 GB
Allocated since start 1.31 GB
Allocated since start 1.32 GB
Allocated since start 1.33 GB
Allocated since start 1.34 GB
Allocated since start 1.35 GB
Allocated since start 1.35 GB

10. Live demo
Who wants to see me blow up Node.js?
(if time)

11. The dirty hack
node --max-old-space-size=6000 index.js

12. Memory to burn
Allocated since start 5.74 GB
Allocated since start 5.75 GB
Allocated since start 5.76 GB
Allocated since start 5.77 GB
Allocated since start 5.78 GB
Allocated since start 5.79 GB
Allocated since start 5.8 GB
Allocated since start 5.81 GB
Allocated since start 5.82 GB
Allocated since start 5.83 GB
Allocated since start 5.84 GB
Allocated since start 5.85 GB

13. How far?
node --max-old-space-size=??? index.js

14. Fitting more in memory
● Don’t load your entire data set
● Avoid memory leaks
● Monitor memory usage
● Separate processes

15. Resources / Q&A
Memory limit testing code:
https://github.com/Data-Wrangling-with-JavaScript/nodejs-memory-test
Interesting articles:
https://blog.codeship.com/understanding-garbage-collection-in-node-js/
http://jayconrod.com/posts/55/a-tour-of-v8-garbage-collection
https://blog.risingstack.com/finding-a-memory-leak-in-node-js/