H2O Overview with Amy Wang at useR! Aalborg
•
2 likes•2,318 views
H2O Overview with Amy Wang at useR! Aalborg 2015, including overview of architecture.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to H2O Overview with Amy Wang at useR! Aalborg
Similar to H2O Overview with Amy Wang at useR! Aalborg (20)
More from Sri Ambati
More from Sri Ambati (20)
Recently uploaded
Recently uploaded (20)
H2O Overview with Amy Wang at useR! Aalborg
- 2. H2O.ai Machine Intelligence Company Overview Company Product • Team: 35. Founded in 2012, Mountain View, CA • Stanford Math & Systems Engineers • Open Source Leader in Machine & Deep learning • Ease of Use and Smarter Applications • R, Python, Spark & Hadoop Interfaces • Expanding Predictions to Mass Analyst markets
- 3. H2O.ai Machine Intelligence Execu3ve Team Sri Satish Ambati CEO & Co-‐founder Cliff Click CTO & Co-‐founder Tom Kraljevic VP of Engineering Board of Directors Jishnu Bhattacharjee // Nexus Ventures Ash Bhardwaj // Flextronics Scientific Advisory Council Trevor Hastie Stephen Boyd Rob Tibshirani DataStax Sun, Java Hotspot Abrizio, Intel Arno Candel Chief Architect Physicist, Deep Learning
- 5. H2O.ai Machine Intelligence Scien3fic Advisory Council Dr. Trevor Hastie Dr. Rob Tibshirani Dr. Stephen Boyd • PhD in Statistics, Stanford University • John A. Overdeck Professor of Mathematics, Stanford University • Co-‐author, The Elements of Statistical Learning: Prediction, Inference and Data Mining • Co-‐author with John Chambers, Statistical Models in S • Co-‐author, Generalized Additive Models • 108,404 citations (﴾via Google Scholar)﴿ • PhD in Statistics, Stanford University • Professor of Statistics and Health Research and Policy, Stanford University • COPPS Presidents’ Award recipient • Co-‐author, The Elements of Statistical Learning: Prediction, Inference and Data Mining • Author, Regression Shrinkage and Selection via the Lasso • Co-‐author, An Introduction to the Bootstrap • PhD in Electrical Engineering and Computer Science, UC Berkeley • Professor of Electrical Engineering and Computer Science, Stanford University • Co-‐author, Convex Optimization • Co-‐author, Linear Matrix Inequalities in System and Control Theory • Co-‐author, Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers
- 6. H2O.ai Machine Intelligence Accuracy with Speed and Scale
- 7. H2O.ai Machine Intelligence Accuracy with Speed and Scale HDFS S3 SQL NoSQL Classifica3on Regression Feature Engineering In-‐Memory Map Reduce/Fork Join Columnar Compression Deep Learning PCA, GLM, Cox Random Forest / GBM Ensembles Fast Modeling Engine Streaming Nano Fast Java Scoring Engines Matrix Factoriza3on Clustering Munging
- 8. H2O.ai Machine Intelligence Ensembles Deep Neural Networks Algorithms on H2O • Generalized Linear Models : Binomial, Gaussian, Gamma, Poisson and Tweedie • Cox Proportional Hazards Models • Naïve Bayes • Distributed Random Forest : Classification or regression models • Gradient Boosting Machine : Produces an ensemble of decision trees with increasing refined approximations • Deep learning : Create multi-‐layer feed forward neural networks starting with an input layer followed by multiple layers of nonlinear transformations Supervised Learning Sta3s3cal Analysis
- 9. H2O.ai Machine Intelligence Dimensionality Reduc3on Anomaly Detec3on Algorithms on H2O • K-‐means : Partitions observations into k clusters/ groups of the same spatial size • Principal Component Analysis : Linearly transforms correlated variables to independent components • Autoencoders: Find outliers using a nonlinear dimensionality reduction using deep learning Unsupervised Learning Clustering
- 10. H2O.ai Machine Intelligence Reading Data from HDFS into H2O with R STEP 1 R user h2o_df = h2o.importFile(﴾“hdfs://path/to/data.csv”)﴿
- 11. H2O.ai Machine Intelligence Reading Data from HDFS into H2O with R H2O H2O H2O data.csv HTTP REST API request to H2O has HDFS path H2O Cluster Initiate distributed ingest HDFS Request data from HDFS STEP 2 2.2 2.3 2.4 R h2o.importFile(﴾)﴿ 2.1 R function call
- 12. H2O.ai Machine Intelligence Reading Data from HDFS into H2O with R H2O H2O H2O R HDFS STEP 3 Cluster IP Cluster Port Pointer to Data Return pointer to data in REST API JSON Response HDFS provides data 3.3 3.4 3.1 h2o_df object created in R data.csv h2o_df H2O Frame 3.2 Distributed H2O Frame in DKV H2O Cluster
- 13. H2O.ai Machine Intelligence R Script Star3ng H2O GLM HTTP REST/JSON .h2o.startModelJob() POST /3/ModelBuilders/glm h2o.glm() R script Standard R process TCP/IP HTTP REST/JSON /3/ModelBuilders/glm endpoint Job GLM algorithm GLM tasks Fork/Join framework K/V store framework H2O process Network layer REST layer H2O -‐ algos H2O -‐ core User process H2O process Legend
- 14. H2O.ai Machine Intelligence R Script Retrieving H2O GLM Result HTTP REST/JSON h2o.getModel() GET /3/Models/glm_model_id h2o.glm() R script Standard R process TCP/IP HTTP REST/JSON /3/Models endpoint Fork/Join framework K/V store framework H2O process Network layer REST layer H2O -‐ algos H2O -‐ core User process H2O process Legend
Editor's Notes
- Data never goes into or through R R tells H2O what to do R has a pointer to the big data that lives in H2O Operations on the R object are intercepted using operator overleading and forwarded to H2O via the REST API H2OFrame object in R is a proxy for the big data in H2O R: df= H2O.imporrtFile(“hdfs://path/to/data.csv”) h2o>_df is an H2OFrame object in R
- Data never goes into or through R R tells H2O what to do R has a pointer to the big data that lives in H2O Operations on the R object are intercepted using operator overleading and forwarded to H2O via the REST API H2OFrame object in R is a proxy for the big data in H2O R: df= H2O.imporrtFile(“hdfs://path/to/data.csv”) h2o>_df is an H2OFrame object in R