HadoopSummit2015_SelfEvolvingModels
•
2 likes•583 views
As a speaker at San Jose Hadoop Summit 2015, presented the principles for Self Evolving Models for Dynamic System Accuracy.The theme of the topic is streaming and machine learning. [moved to my rekhajoshm official slideshare account; with side effects of loss of stats]
Recommended
Recommended
More Related Content
What's hot
What's hot (11)
Viewers also liked
Similar to HadoopSummit2015_SelfEvolvingModels
Similar to HadoopSummit2015_SelfEvolvingModels (20)
Recently uploaded
Recently uploaded (20)
HadoopSummit2015_SelfEvolvingModels
- 1. Self Evolving Model For Dynamic System Accuracy Hadoop Streaming and Machine Learning Rekha Joshi Intuit Data Engineering & Analytics
- 2. Who Am I? Staff Engineer At Intuit Inc. working on Finance domain, on Big Data Ecosystem Worked at Yahoo! on Advertising domain, with Big Data Ecosystem
- 3. A Case For Intelligence
- 4. Model me Something like a Brain..
- 5. What does it take to be as Smart as Human? Natural Language Processing Classification Clustering Regression Sentiment Analysis Decision Tree Predict Extract Meaning out of Data Detect Intent Optimize
- 6. If We are So Smart, Why a Machine? We fail with: •Scale •Focus •Variants •Emotion •Reliability
- 7. A Matter Of Trust
- 8. And We hear Beaches Calling..
- 9. Entrusting Machine To Do The Best Job!
- 10. A Case For Self Evolving Model
- 11. Are Static Models Doing The Best Job?
- 12. • The Best Job Done? • Does not simulate a brain • Not Scalable • Long Arduous Process • Rethinks feature space • Does not tell the truth! Drawbacks of Static Models
- 13. Why should we care? Self Evolving Model For Dynamic Accuracy
- 14. We live in a Data Age Volume Variety Velocity From all sides You
- 15. We live in a Fast Paced World
- 16. We live in a Complex World Our Ecosystems are characterized as: •Complex •Global •Mobile •Pervasive •Interdependent •Multi variants
- 17. Thankfully,We are in Superpower Computing Age!
- 18. Expectations and Constraints System Constraints ExpectationsExpectations Low Latency High Throughput Real Time High Availability Faster Accurate Dynamic Context Scalable
- 19. Hadoop Streaming and Machine Learning Glad They Meet, and Glad They work well together!
- 20. Self Evolving Model For Dynamic Accuracy
- 21. Self Evolving Models? Eliza – Your Free Psychotherapist? DeepBlue – The Grandmaster?
- 22. • Can work at scale • Can sense • Can judge what is important, relevant • Can discover connections • Can retain what it learnt previously • Can build upon what it learnt • Can learn to learn • Can be reused Self Evolving Model For Dynamic Accuracy
- 23. Self Evolving Model For Dynamic Accuracy Can work at scale: Data and Platform Fulfillment
- 24. Self Evolving Model For Dynamic Accuracy Can sense: Does not work in isolation, but as part of a ecosystem.
- 25. Self Evolving Model For Dynamic Accuracy Can judge what is important, relevant: Overriding the Curse of Dimensionality
- 26. Self Evolving Model For Dynamic Accuracy Can discover connections: Disambiguate Correctly for Nearness
- 27. Self Evolving Model For Dynamic Accuracy Can retain what it learnt previously: Can learn to learn: Pulse on the ecosystem, domain variations Selective Garbage Collection
- 28. Self Evolving Model For Dynamic Accuracy Can build upon what it learnt: Can learn to learn: Improves its rate of overall favorable outcome Human Inference Model
- 29. Self Evolving Model For Dynamic Accuracy Can be reused
- 30. Can work at scale Can sense Can judge what is important, relevant Can discover connections Can retain what it learnt previously Can build upon what it learnt Can learn to learn Can be reused Self Evolving Model For Dynamic Accuracy
- 31. At Intuit
- 32. Intuit and Our Mission
- 33. Examples of Data Use Cases at Intuit • Real Time and Batch Scalable Responsive Platform • Path of Least Resistance To Users • Personalization And Optimization Engines • Prediction Models
- 35. System Components Computational •Messaging – Kafka •Streaming - Samza, Storm, Spark •Big Data - Hadoop Yarn HDFS/AWS S3 •NoSQL - Cassandra Algorithmic •Algorithmic modules
- 36. Tax - Self Evolving Model For Dynamic Accuracy
- 37. We Must Commit! Of progressing towards self evolving model for dynamic system accuracy, and validating the trust of our people.