Downloaded 10 times
Uploaded byRajiv Shah
Using Spark Part Time
The document is a presentation by Rajiv Shah discussing the status and roadmap of Spark, highlighting its strengths and challenges. Key points include the need for better algorithms and documentation, as well as tips for effectively utilizing Spark with data engineers and data scientists. Shah also provides examples of applications like Spark Streaming and anomaly detection.
More Related Content
Similar to Using Spark Part Time
![[Sneak Preview] Apache Spark: Preparing for the next wave of Reactive Big Data](https://cdn.slidesharecdn.com/ss_thumbnails/coll-report-typesafe-apache-spark-slide-share-150127023731-conversion-gate01-thumbnail.jpg?width=640&height=640&fit=bounds)
Using Spark Part Time
- 1. PART TIME SPARKUSER Rajiv Shah www.rajivshah.com Chicago Spark Users Meetup Nov 5, 2015
- 2. ROADMAP • Status ofspark • My take • Examples
- 3.
- 5.
- 6. Cloudera Blog Poston Sparkling Water http://blog.cloudera.com/blog/2015/10/how-to-build-a-machine-learning-app-using-sparkling-water-and-a
- 7.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 17.
- 19. Not for smallor big data
- 24.
- 25. Spark makes theimpossible, possible
- 26.
- 27. COOL THINGS ABOUT SPARK •Scales up • Streaming • Enterprise worthy • It looks like it will play nice
- 28. SUGGESTIONS • Get dataengineers that will work with your data scientists • If you can’t take advantage of spark’s strengths, don't use it
- 29. EXAMPLES • Spark streaming- Streaming Kmeans clustering • Anomaly Detection using H2O • Recommenders
Editor's Notes
- #2 Perspective on spark - not a typical technical talk Should all leave with a better appreciation of how spark fits into data science at this point in time
- #5 IBM’s announcement - hiring of spark users donate system ML thousands of its own researchers on spark training its leaders in using spark ML educate more than one million data scientists and data engineers on Spark. 630,000 PMP
- #6 http://conferences.oreilly.com/strata/big-data-conference-ca-2015/public/schedule/full/public
- #7 http://blog.cloudera.com/blog/2015/10/how-to-build-a-machine-learning-app-using-sparkling-water-and-apache-spark http://h2o.ai/blog/2015/04/deep-learning-public-safety/ http://www.kdnuggets.com/2015/04/deep-learning-fight-crime.html every cliche spark, h2o, deep learning, visualizations, especially geographic we are missing a D3/javascript and we even have weather - IBM’s annomucment, weather in every model worst kind of data science analysis for the sake of analysis also, look at the difference between code in the post and the actual code