Andrii Shumada ''Use cases of Node.js Streams''
•Download as PPTX, PDF•
1 like•1,053 views
OdessaJS'2017
Recommended
More Related Content
Similar to Andrii Shumada ''Use cases of Node.js Streams''
Similar to Andrii Shumada ''Use cases of Node.js Streams'' (20)
More from OdessaJS Conf
More from OdessaJS Conf (20)
Recently uploaded
Recently uploaded (20)
Andrii Shumada ''Use cases of Node.js Streams''
- 1. Use cases of Node.js Streams Andrii Shumada Sr. Dev at Debitoor/Ciklum Tw: @eagleeye_s Gh: @eagleeye
- 2. About me 7 years in commercial development 5 years in JS Judge at DEV_CHALLENGE Program expert at js fwdays Own projects: Tank.ly hvz.kiev.ua 2
- 4. Content < 1.4 GB Node.js service for sharing porn files Read file in variable Send to http response
- 5. How to share porn in GOOD quality with JavaScript?
- 6. Read chunk Write chunk Read chunk Write chunk Read chunk Write chunk
- 8. Event: 'data' Event: 'end' Event: 'error' readable.read([size]) readable.pipe() Class: stream.Readable Data source Data chunks https://nodejs.org/dist/latest-v6.x/docs/api/stream.html
- 9. Event: 'close' Event: 'drain' Event: 'error' writable.end() writable.write() Class: stream.Writable https://nodejs.org/dist/latest-v6.x/docs/api/stream.html
- 10. Readable streams: 1) Files
- 11. Readable streams: 2) Databases http://mongodb.github.io/node-mongodb-native/2.2/api/Cursor.html#stream
- 12. Readable streams: 2) Database cursors http://mongodb.github.io/node-mongodb-native/2.2/api/Cursor.html#stream
- 13. Readable streams: 3) http request https://www.npmjs.com/package/request#streaming
- 14. Sources of readable streams: 4) websocket/your own source
- 15. 5) process.stdin
- 16. Writable streams: - fs.createWritableStream(filename) - http.res - Database (@stream-to-mongo-db) - process.stdout
- 17. Transform streams - @image-resizer-stream - @png-optimizer-stream - @zip-stream - @csv.parse(), @csv.transform(), @csv.stringify()
- 18. Usecases
- 19. Send data without streams Read data Transform data time Send dataMemory consumption
- 20. Send data with streams time Read data Transform data Send data Memory consumption
- 21. Proxy servers Client Proxy request req.pipe(proxyRequest) proxyResponse.pipe(res) Another service
- 22. Send file between browsers without storing First browser sends data Proxy backend Second browser reads data Memory consumption time
- 23. Parsing CSV Files Read CSV line by line Parse one line Analyze/transform Write to db/http client time
- 24. Generating CSV Files Database stream Format data Write to CSV line by line One document at time <- transform stream
- 26. Serving files Read file chunks Write to response time
- 27. Multifetch endpoints Suppose you have an API - GET api/users, api/users/:id - GET api/customers, api/customers:id - GET api/countries etc GET api/fetch?users=api/users&customer=api/customers/23&countries=api/countries should return {“users”: [..], “customer”: {..}, “countries”: [..] }
- 30. {“users”: [..], “customer”: {..}, “countries”: [..] } { “users” : json1 , “products” : json2 , … } write(‘{“users”:’); request(‘/api/users’).pipe(res) write(‘,“products”:’); request(‘/api/customer).pipe(res) res.end(‘}’);
- 31. Microservices Gateway Serve static files Reports service REST API Render PDF service File upload service S3
- 32. App, that proxies all request in debitoor No memory consumption!
- 33. App, that proxies all request in debitoor No CPU consumption!
- 34. content < 1.4 GB How not to use streams
- 35. Conclusion: - Streams save your memory - Streams speed up data transfer - Streams are not blocking your main thread - Use streams every time you are working with data