Talk about Node.js Streams from the online Geekle Global Summit 2020

1. Taking Node.js
to the ex-stream
stream to scale
Gadi Raymond

2. About myself
Married to Hila, Father of Ori (and Angie)
Senior full-stack developer @
Love to speak and write about technology
Follow me on @gadiray

3. Agenda
● My big data challenge
● Intro to Node.js Streams
● My challenge solution using Streams
● Streams tips
● Async Iterators and Async Generators

4. Don’t be afraid of Streams

5. LOVE Streams

6. Architecture Overview
Athena
Reports Consumers
Reports Queue
EMAIL
S3
CSV

7. Architecture Overview
Presto
Reports Consumers
Reports Queue
EMAIL
S3

8. In memory solution

9. What are Streams?
● Interfaces for reading and writing data
● Streams can be piped together
● Node.js Streams extends EventEmitter

10. Types of Streams
● Readable
● Writable
● Duplex
● Transform

11. Node.js Streams solution

12. Let’s Compare!

13. Can we create a Stream?

14. Readable Stream Example

15. Transform Stream Example

16. Pipe Together
HELLO GEEKLE

17. Streams Tips

18. Error handling

19. Error handling

20. Error handling

21. Error handling

22. Error handling

23. Back pressure

24. Back pressure

25. ObjectMode option

26. ObjectMode option

27. highWaterMark option

28. Generator Function
Generator
Usage
Creates Iterable Object

29. Readable Stream From Generator Function

30. Async Generator Function
Async Generator
Usage
Creates Async Iterable Object

31. Streams with Async Iterators
Readable Streams are Async Iterable Object

32. Async Generator and Async Iterator

33. Pipeline API Supports Generators
NEW TO NODE 14

34. Summary
● Take Node to the ex-Stream!
● Streams API is easy
● Follow the best practices

35. gadi.r@walkme.com
https://www.linkedin.com/in/gadi-raymond
https://twitter.com/gadiray
https://github.com/gadi-ray