Do you have that Spark in your ECTL? by Piotr Sych Scalac
•
0 likes•60 views
Sometimes there is a need for a simple, high-level implementation of ECTL in Spark with a lightning-fast introduction to Spark concepts. study based on a real story. I wish to present an idea of dealing with multiple input formats and structures in a unified approach using Spark. As a result to use it in your Data Science oriented environment as a preparation step for further analyses and experiments. Such a way of handling non-production data inputs particularly helped with providing data quality analyses and conducting experiments on production-like sized datasets without the need of "testing on production" mess.
Recommended
Recommended
More Related Content
More from Scalac
More from Scalac (9)
Recently uploaded
Recently uploaded (20)
Do you have that Spark in your ECTL? by Piotr Sych Scalac
- 1. Do you have that Spark in your ECTL ?
- 2. The Story
- 3. The Story ENG TEAM DS TEAM ML ALGO
- 8. ECTL
- 15. Usage
- 17. > Spark
- 19. > Spark > Freedom > Modularity
- 20. Closing Remarks
- 22. Closing Remarks > Solution > ECTL
- 23. Closing Remarks > Solution > ECTL > Spark
- 24. THANK YOU