This document summarizes a presentation about how to win data science competitions using deep learning with H2O. It discusses H2O's architecture and capabilities for deep learning. It then demonstrates live modeling on Kaggle competitions, providing step-by-step explanations of building and evaluating deep learning models on three different datasets - an African soil properties prediction challenge, a display advertising challenge, and a Higgs boson machine learning challenge. It concludes with tips and tricks for deep learning with H2O and an invitation to the H2O World conference.
How To Interview a Data Scientist
Daniel Tunkelang
Presented at the O'Reilly Strata 2013 Conference
Video: https://www.youtube.com/watch?v=gUTuESHKbXI
Interviewing data scientists is hard. The tech press sporadically publishes “best” interview questions that are cringe-worthy.
At LinkedIn, we put a heavy emphasis on the ability to think through the problems we work on. For example, if someone claims expertise in machine learning, we ask them to apply it to one of our recommendation problems. And, when we test coding and algorithmic problem solving, we do it with real problems that we’ve faced in the course of our day jobs. In general, we try as hard as possible to make the interview process representative of actual work.
In this session, I’ll offer general principles and concrete examples of how to interview data scientists. I’ll also touch on the challenges of sourcing and closing top candidates.
Winning data science competitions, presented by Owen ZhangVivian S. Zhang
<featured> Meetup event hosted by NYC Open Data Meetup, NYC Data Science Academy. Speaker: Owen Zhang, Event Info: http://www.meetup.com/NYC-Open-Data/events/219370251/
Learning to rank (LTR) for information retrieval (IR) involves the application of machine learning models to rank artifacts, such as webpages, in response to user's need, which may be expressed as a query. LTR models typically employ training data, such as human relevance labels and click data, to discriminatively train towards an IR objective. The focus of this lecture will be on the fundamentals of neural networks and their applications to learning to rank.
How To Interview a Data Scientist
Daniel Tunkelang
Presented at the O'Reilly Strata 2013 Conference
Video: https://www.youtube.com/watch?v=gUTuESHKbXI
Interviewing data scientists is hard. The tech press sporadically publishes “best” interview questions that are cringe-worthy.
At LinkedIn, we put a heavy emphasis on the ability to think through the problems we work on. For example, if someone claims expertise in machine learning, we ask them to apply it to one of our recommendation problems. And, when we test coding and algorithmic problem solving, we do it with real problems that we’ve faced in the course of our day jobs. In general, we try as hard as possible to make the interview process representative of actual work.
In this session, I’ll offer general principles and concrete examples of how to interview data scientists. I’ll also touch on the challenges of sourcing and closing top candidates.
Winning data science competitions, presented by Owen ZhangVivian S. Zhang
<featured> Meetup event hosted by NYC Open Data Meetup, NYC Data Science Academy. Speaker: Owen Zhang, Event Info: http://www.meetup.com/NYC-Open-Data/events/219370251/
Learning to rank (LTR) for information retrieval (IR) involves the application of machine learning models to rank artifacts, such as webpages, in response to user's need, which may be expressed as a query. LTR models typically employ training data, such as human relevance labels and click data, to discriminatively train towards an IR objective. The focus of this lecture will be on the fundamentals of neural networks and their applications to learning to rank.
Feature Engineering in H2O Driverless AI - Dmitry Larko - H2O AI World London...Sri Ambati
This talk was recorded in London on October 30th, 2018 and can be viewed here: https://youtu.be/d6UMEmeXB6o
In his talk Dmitry is going to cover common feature engineering techniques used to build robust machine learning models as well as some not widely known/used approaches.
Bio: Senior Data Scientist at H2O.ai, Dmitry Larko also is a former #25 Kaggle Grandmaster and loves to use his machine learning and data science skills in Kaggle Competitions and predictive analytics software development. He has more than 15 years of experience in information technology. Post his masters in computer information systems from Krasnoyarsk State Technical University (KSTU), he started his career in data warehousing and business intelligence and gradually moved to big data and data science. He holds a lot of experience in predictive analytics in a wide array of domains and tasks. Prior to H2O.ai, Dmitry held the position of SAP BW Developer at Chevron, Data Scientist at EPAM, and that of Lead Software Engineer with the Russian Federation.
사내 스터디용으로 공부하며 만든 발표 자료입니다. 부족한 부분이 있을 수도 있으니 알려주시면 정정하도록 하겠습니다.
*슬라이드 6에 나오는 classical CNN architecture(뒤에도 계속 나옴)에서 ReLU - Pool - ReLu에서 뒤에 나오는 ReLU는 잘못된 표현입니다. ReLU - Pool에서 ReLU 계산을 또 하는 건 redundant 하기 때문입니다(Kyung Mo Kweon 피드백 감사합니다)
Government GraphSummit: Leveraging Graphs for AI and MLNeo4j
Phani Dathar, Ph.D., Data Science Solution Architect, Neo4j
Relationships are highly predictive of behavior. Graph technology abstracts connections in our data so businesses can apply relationships and network structures to make better predictions. Hear about the journey from graph analytics and machine learning to graph-enhanced AI. We’ll also cover how enterprises are using graph data science in areas such as fraud, targeted marketing, healthcare, and recommendations.
빅데이터, 클라우드, IoT, 머신러닝. 왜 이렇게 많은 것들이 나타날까?Yongho Ha
클라우드라는 말이 들리더니, 어느새 빅데이터가 유행했습니다. 데이터가 중요하다는 것을 겨우 받아들일까 하는 판국에, 이제는 IoT라던가 머신러닝이 중요하다고 합니다. 이 많은 유행들은 그냥 일시적인 걸까요? 아니면 동시에 나타나게된 이유가 있는 걸까요? 이것들 뒤에 큰 흐름이 있지는 않을까요? 있다면 그것은 어디에서 시작되고 있을까요? numberworks.io
오늘 밤부터 쓰는 google analytics (구글 애널리틱스, GA) Yongho Ha
http://ga.yonghosee.com 에서 진행하는 구글 어날리틱스(google analytics) 에 대한 강의 슬라이드 입니다. 이 슬라이드는 샘플이지만, 초반부는 실재 강의 교재 그대로 입니다. 이것 자체로도 여러분이 GA를 이해하는데 좀 도움이 된다면 기쁘겠습니다^^ 감사합니다.
Analytics for large-scale time series and event dataAnodot
Time series and event data form the basis for real-time insights about the performance of businesses such as ecommerce, the IoT, and web services, but gaining these insights involves designing a learning system that scales to millions and billions of data streams. In this presentation, Ira Cohen, Anodot cofounder and chief data scientist, outlines such a system that performs real-time machine learning and analytics on streams at massive scale.
H2O Open Source Deep Learning, Arno Candel 03-20-14Sri Ambati
More information in our Deep Learning webinar: http://www.slideshare.net/0xdata/h2-o-deeplearningarnocandel052114
Latest slide deck: http://www.slideshare.net/0xdata/h2o-distributed-deep-learning-by-arno-candel-071614
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
San Francisco Hadoop User Group Meetup Deep LearningSri Ambati
Hadoop User Group, San Francisco, Dec 10 2014.
Video: http://new.livestream.com/accounts/10932136/events/3649553 (starting at 48 minutes)
Deep Learning has been dominating recent machine learning competitions with better predictions. Unlike the neural networks of the past, modern Deep Learning methods have cracked the code for training stability and generalization. Deep Learning is not only the leader in image and speech recognition tasks, but is also emerging as the algorithm of choice for highest predictive performance in traditional business analytics. This talk introduces Deep Learning and implementation concepts in the open-source H2O in-memory prediction engine. Designed for the solution of business-critical problems on distributed compute clusters, it offers advanced features such as adaptive learning rate, dropout regularization, parameter tuning and a fully-featured R interface. World record performance on the classic MNIST dataset, best-in-class accuracy for a high-dimensional eBay text classification problem and other relevant datasets showcase the power of this game-changing technology. A whole new ecosystem of Intelligent Applications is emerging with Deep Learning at its core.
Bio:
Prior to joining 0xdata as Physicist & Hacker, Arno was a founding Senior MTS at Skytree where he designed and implemented high-performance machine learning algorithms. He has over a decade of experience in HPC with C++/MPI and had access to the world’s largest supercomputers as a Staff Scientist at SLAC National Accelerator Laboratory where he participated in US DOE scientific computing initiatives. While at SLAC, he authored the first curvilinear finite-element simulation code for space-charge dominated relativistic free electrons and scaled it to thousands of compute nodes. He also led a collaboration with CERN to model the electromagnetic performance of CLIC, a ginormous e+e- collider and potential successor of LHC. Arno has authored dozens of scientific papers and was a sought-after academic conference speaker. He holds a PhD and Masters summa cum laude in Physics from ETH Zurich. Arno was named 2014 Big Data All-Star by Fortune Magazine.
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
Feature Engineering in H2O Driverless AI - Dmitry Larko - H2O AI World London...Sri Ambati
This talk was recorded in London on October 30th, 2018 and can be viewed here: https://youtu.be/d6UMEmeXB6o
In his talk Dmitry is going to cover common feature engineering techniques used to build robust machine learning models as well as some not widely known/used approaches.
Bio: Senior Data Scientist at H2O.ai, Dmitry Larko also is a former #25 Kaggle Grandmaster and loves to use his machine learning and data science skills in Kaggle Competitions and predictive analytics software development. He has more than 15 years of experience in information technology. Post his masters in computer information systems from Krasnoyarsk State Technical University (KSTU), he started his career in data warehousing and business intelligence and gradually moved to big data and data science. He holds a lot of experience in predictive analytics in a wide array of domains and tasks. Prior to H2O.ai, Dmitry held the position of SAP BW Developer at Chevron, Data Scientist at EPAM, and that of Lead Software Engineer with the Russian Federation.
사내 스터디용으로 공부하며 만든 발표 자료입니다. 부족한 부분이 있을 수도 있으니 알려주시면 정정하도록 하겠습니다.
*슬라이드 6에 나오는 classical CNN architecture(뒤에도 계속 나옴)에서 ReLU - Pool - ReLu에서 뒤에 나오는 ReLU는 잘못된 표현입니다. ReLU - Pool에서 ReLU 계산을 또 하는 건 redundant 하기 때문입니다(Kyung Mo Kweon 피드백 감사합니다)
Government GraphSummit: Leveraging Graphs for AI and MLNeo4j
Phani Dathar, Ph.D., Data Science Solution Architect, Neo4j
Relationships are highly predictive of behavior. Graph technology abstracts connections in our data so businesses can apply relationships and network structures to make better predictions. Hear about the journey from graph analytics and machine learning to graph-enhanced AI. We’ll also cover how enterprises are using graph data science in areas such as fraud, targeted marketing, healthcare, and recommendations.
빅데이터, 클라우드, IoT, 머신러닝. 왜 이렇게 많은 것들이 나타날까?Yongho Ha
클라우드라는 말이 들리더니, 어느새 빅데이터가 유행했습니다. 데이터가 중요하다는 것을 겨우 받아들일까 하는 판국에, 이제는 IoT라던가 머신러닝이 중요하다고 합니다. 이 많은 유행들은 그냥 일시적인 걸까요? 아니면 동시에 나타나게된 이유가 있는 걸까요? 이것들 뒤에 큰 흐름이 있지는 않을까요? 있다면 그것은 어디에서 시작되고 있을까요? numberworks.io
오늘 밤부터 쓰는 google analytics (구글 애널리틱스, GA) Yongho Ha
http://ga.yonghosee.com 에서 진행하는 구글 어날리틱스(google analytics) 에 대한 강의 슬라이드 입니다. 이 슬라이드는 샘플이지만, 초반부는 실재 강의 교재 그대로 입니다. 이것 자체로도 여러분이 GA를 이해하는데 좀 도움이 된다면 기쁘겠습니다^^ 감사합니다.
Analytics for large-scale time series and event dataAnodot
Time series and event data form the basis for real-time insights about the performance of businesses such as ecommerce, the IoT, and web services, but gaining these insights involves designing a learning system that scales to millions and billions of data streams. In this presentation, Ira Cohen, Anodot cofounder and chief data scientist, outlines such a system that performs real-time machine learning and analytics on streams at massive scale.
H2O Open Source Deep Learning, Arno Candel 03-20-14Sri Ambati
More information in our Deep Learning webinar: http://www.slideshare.net/0xdata/h2-o-deeplearningarnocandel052114
Latest slide deck: http://www.slideshare.net/0xdata/h2o-distributed-deep-learning-by-arno-candel-071614
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
San Francisco Hadoop User Group Meetup Deep LearningSri Ambati
Hadoop User Group, San Francisco, Dec 10 2014.
Video: http://new.livestream.com/accounts/10932136/events/3649553 (starting at 48 minutes)
Deep Learning has been dominating recent machine learning competitions with better predictions. Unlike the neural networks of the past, modern Deep Learning methods have cracked the code for training stability and generalization. Deep Learning is not only the leader in image and speech recognition tasks, but is also emerging as the algorithm of choice for highest predictive performance in traditional business analytics. This talk introduces Deep Learning and implementation concepts in the open-source H2O in-memory prediction engine. Designed for the solution of business-critical problems on distributed compute clusters, it offers advanced features such as adaptive learning rate, dropout regularization, parameter tuning and a fully-featured R interface. World record performance on the classic MNIST dataset, best-in-class accuracy for a high-dimensional eBay text classification problem and other relevant datasets showcase the power of this game-changing technology. A whole new ecosystem of Intelligent Applications is emerging with Deep Learning at its core.
Bio:
Prior to joining 0xdata as Physicist & Hacker, Arno was a founding Senior MTS at Skytree where he designed and implemented high-performance machine learning algorithms. He has over a decade of experience in HPC with C++/MPI and had access to the world’s largest supercomputers as a Staff Scientist at SLAC National Accelerator Laboratory where he participated in US DOE scientific computing initiatives. While at SLAC, he authored the first curvilinear finite-element simulation code for space-charge dominated relativistic free electrons and scaled it to thousands of compute nodes. He also led a collaboration with CERN to model the electromagnetic performance of CLIC, a ginormous e+e- collider and potential successor of LHC. Arno has authored dozens of scientific papers and was a sought-after academic conference speaker. He holds a PhD and Masters summa cum laude in Physics from ETH Zurich. Arno was named 2014 Big Data All-Star by Fortune Magazine.
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
H2O.ai's Distributed Deep Learning by Arno Candel 04/03/14Sri Ambati
Deep Learning is a new area of Machine Learning research, which has been introduced with the objective of moving Machine Learning closer to one of its original goals: Artificial Intelligence.
http://docs.0xdata.com/datascience/deeplearning.html
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
MLconf - Distributed Deep Learning for Classification and Regression Problems...Sri Ambati
Video recording (no audio?): http://new.livestream.com/accounts/7874891/events/3565981/videos/68114143 from 32:00 to 54:30
Deep Learning has been dominating recent machine learning competitions with better predictions. Unlike the neural networks of the past, modern Deep Learning methods have cracked the code for training stability and generalization. Deep Learning is not only the leader in image and speech recognition tasks, but is also emerging as the algorithm of choice for highest predictive performance in traditional business analytics. This talk introduces Deep Learning and implementation concepts in the open-source H2O in-memory prediction engine. Designed for the solution of business-critical problems on distributed compute clusters, it offers advanced features such as adaptive learning rate, dropout regularization, parameter tuning and a fully-featured R interface. World record performance on the classic MNIST dataset, best-in-class accuracy for a high-dimensional eBay text classification problem and other relevant datasets showcase the power of this game-changing technology. A whole new ecosystem of Intelligent Applications is emerging with Deep Learning at its core.
Bio:
Prior to joining 0xdata as Physicist & Hacker, Arno was a founding Senior MTS at Skytree where he designed and implemented high-performance machine learning algorithms. He has over a decade of experience in HPC with C++/MPI and had access to the world’s largest supercomputers as a Staff Scientist at SLAC National Accelerator Laboratory where he participated in US DOE scientific computing initiatives. While at SLAC, he authored the first curvilinear finite-element simulation code for space-charge dominated relativistic free electrons and scaled it to thousands of compute nodes. He also led a collaboration with CERN to model the electromagnetic performance of CLIC, a ginormous e+e- collider and potential successor of LHC. Arno has authored dozens of scientific papers and was a sought-after academic conference speaker. He holds a PhD and Masters summa cum laude in Physics from ETH Zurich. Arno was named 2014 Big Data All-Star by Fortune Magazine.
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
Suggestions:
1) For best quality, download the PDF before viewing.
2) Open at least two windows: One for the Youtube video, one for the screencast (link below), and optionally one for the slides themselves.
3) The Youtube video is shown on the first page of the slide deck, for slides, just skip to page 2.
Screencast: http://youtu.be/VoL7JKJmr2I
Video recording: http://youtu.be/CJRvb8zxRdE (Thanks to Al Friedrich!)
In this talk, we take Deep Learning to task with real world data puzzles to solve.
Data:
- Higgs binary classification dataset (10M rows, 29 cols)
- MNIST 10-class dataset
- Weather categorical dataset
- eBay text classification dataset (8500 cols, 500k rows, 467 classes)
- ECG heartbeat anomaly detection
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
Scalable Data Science and Deep Learning with H2O
In this session, we introduce the H2O data science platform. We will explain its scalable in-memory architecture and design principles and focus on the implementation of distributed deep learning in H2O. Advanced features such as adaptive learning rates, various forms of regularization, automatic data transformations, checkpointing, grid-search, cross-validation and auto-tuning turn multi-layer neural networks of the past into powerful, easy-to-use predictive analytics tools accessible to everyone. We will present a broad range of use cases and live demos that include world-record deep learning models, anomaly detection tools and approaches for Kaggle data science competitions. We also demonstrate the applicability of H2O in enterprise environments for real-world customer production use cases.
By the end of the hands-on-session, attendees will have learned to perform end-to-end data science workflows with H2O using both the easy-to-use web interface and the flexible R interface. We will cover data ingest, basic feature engineering, feature selection, hyperparameter optimization with N-fold cross-validation, multi-model scoring and taking models into production. We will train supervised and unsupervised methods on realistic datasets. With best-of-breed machine learning algorithms such as elastic net, random forest, gradient boosting and deep learning, you will be able to create your own smart applications.
A local installation of RStudio is recommended for this session.
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
H2O Deep Learning through Examples, Silicon Valley Big Data Science Meetup, Mountain View, 2/12/15
http://www.meetup.com/Silicon-Valley-Big-Data-Science/events/219790984/?a=md1_grp&rv=md1&_af_eid=219790984&_af=event
Live Stream: http://new.livestream.com/accounts/10932136/events/3806139
Note: Make sure to download the slides to get the high-resolution version!
Also, you can find the webinar recording here (please also download for better quality): https://www.dropbox.com/s/72qi6wjzi61gs3q/H2ODeepLearningArnoCandel052114.mov
Come hear how Deep Learning in H2O is unlocking never before seen performance for prediction!
H2O is google-scale open source machine learning engine for R & Big Data. Enterprises can now use all of their data without sampling and build intelligent applications. This live webinar introduces Distributed Deep Learning concepts, implementation and results from recent developments. Real world classification & regression use cases from eBay text dataset, MNIST handwritten digits and Cancer datasets will present the power of this game changing technology.
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
Deep Learning in the Wild with Arno CandelSri Ambati
"Deep Learning in the Wild" Meetup at H2O, Mountain View
Livestream: http://t.co/o7p2hYcWgy (includes part 2 with Alex Tellez)
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
H2O Distributed Deep Learning by Arno Candel 071614Sri Ambati
Deep Learning R Vignette Documentation: https://github.com/0xdata/h2o/tree/master/docs/deeplearning/
Deep Learning has been dominating recent machine learning competitions with better predictions. Unlike the neural networks of the past, modern Deep Learning methods have cracked the code for training stability and generalization. Deep Learning is not only the leader in image and speech recognition tasks, but is also emerging as the algorithm of choice in traditional business analytics.
This talk introduces Deep Learning and implementation concepts in the open-source H2O in-memory prediction engine. Designed for the solution of enterprise-scale problems on distributed compute clusters, it offers advanced features such as adaptive learning rate, dropout regularization and optimization for class imbalance. World record performance on the classic MNIST dataset, best-in-class accuracy for eBay text classification and others showcase the power of this game changing technology. A whole new ecosystem of Intelligent Applications is emerging with Deep Learning at its core.
About the Speaker: Arno Candel
Prior to joining 0xdata as Physicist & Hacker, Arno was a founding Senior MTS at Skytree where he designed and implemented high-performance machine learning algorithms. He has over a decade of experience in HPC with C++/MPI and had access to the world's largest supercomputers as a Staff Scientist at SLAC National Accelerator Laboratory where he participated in US DOE scientific computing initiatives. While at SLAC, he authored the first curvilinear finite-element simulation code for space-charge dominated relativistic free electrons and scaled it to thousands of compute nodes.
He also led a collaboration with CERN to model the electromagnetic performance of CLIC, a ginormous e+e- collider and potential successor of LHC. Arno has authored dozens of scientific papers and was a sought-after academic conference speaker. He holds a PhD and Masters summa cum laude in Physics from ETH Zurich.
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
In this deck, Huihuo Zheng from Argonne National Laboratory presents: Data Parallel Deep Learning.
"The Argonne Training Program on Extreme-Scale Computing (ATPESC) provides intensive, two weeks of training on the key skills, approaches, and tools to design, implement, and execute computational science and engineering applications on current high-end computing systems and the leadership-class computing systems of the future."
Watch the video: https://wp.me/p3RLHQ-lsl
Learn more: https://extremecomputingtraining.anl.gov/archive/atpesc-2019/agenda-2019/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Separating Hype from Reality in Deep Learning with Sameer FarooquiDatabricks
Deep Learning is all the rage these days, but where does the reality of what Deep Learning can do end and the media hype begin? In this talk, I will dispel common myths about Deep Learning that are not necessarily true and help you decide whether you should practically use Deep Learning in your software stack.
I’ll begin with a technical overview of common neural network architectures like CNNs, RNNs, GANs and their common use cases like computer vision, language understanding or unsupervised machine learning. Then I’ll separate the hype from reality around questions like:
• When should you prefer traditional ML systems like scikit learn or Spark.ML instead of Deep Learning?
• Do you no longer need to do careful feature extraction and standardization if using Deep Learning?
• Do you really need terabytes of data when training neural networks or can you ‘steal’ pre-trained lower layers from public models by using transfer learning?
• How do you decide which activation function (like ReLU, leaky ReLU, ELU, etc) or optimizer (like Momentum, AdaGrad, RMSProp, Adam, etc) to use in your neural network?
• Should you randomly initialize the weights in your network or use more advanced strategies like Xavier or He initialization?
• How easy is it to overfit/overtrain a neural network and what are the common techniques to ovoid overfitting (like l1/l2 regularization, dropout and early stopping)?
Caffe (Convolutional Architecture for Fast Feature Embedding) is a deep learning framework made with expression, speed, and modularity in mind. It is developed by the Berkeley Vision and Learning Center (BVLC) and by community contributors.
Caffe’s expressive architecture encourages application and innovation. Models and optimization are defined by configuration without hard-coding. Switch between CPU and GPU by setting a single flag to train on a GPU machine then deploy to commodity clusters or mobile devices.Caffe’s extensible code fosters active development. In Caffe’s first year, it has been forked by over 1,000 developers and had many significant changes contributed back. Thanks to these contributors the framework tracks the state-of-the-art in both code and models.Speed makes Caffe perfect for research experiments and industry deployment. Caffe can processover 60M images per day with a single NVIDIA K40 GPU*. That’s 1 ms/image for inference and 4 ms/image for learning. We believe that Caffe is the fastest convnet implementation available.Caffe already powers academic research projects, startup prototypes, and even large-scale industrial applications in vision, speech, and multimedia. Join our community of brewers on the caffe-users group and Github.
This tutorial is designed to equip researchers and developers with the tools and know-how needed to incorporate deep learning into their work. Both the ideas and implementation of state-of-the-art deep learning models will be presented. While deep learning and deep features have recently achieved strong results in many tasks, a common framework and shared models are needed to advance further research and applications and reduce the barrier to entry. To this end we present the Caffe framework, public reference models, and working examples for deep learning. Join our tour from the 1989 LeNet for digit recognition to today’s top ILSVRC14 vision models. Follow along with do-it-yourself code notebooks. While focusing on vision, general techniques are covered.
Presentation given on Monday 10 September at the ROOT Users' Workshop 2018 in Sarajevo. Progress update on the Automated Parallel Computation of Collaborative Statistical Models project, a collaboration between the Netherlands eScience Center and Nikhef.
We present an update on our recent efforts to further parallelize RooFit. We have performed extensive benchmarks and identified at least three bottlenecks that will benefit from parallelization. To tackle these and possible future bottlenecks, we designed a parallelization layer that allows us to parallelize existing classes with minimal effort, but with high performance and retaining as much of the existing class's interface as possible. The high-level parallelization model is a task-stealing approach. The implementation is currently based on the bi-directional memory mapped pipe (BidirMMapPipe), but could in the future be replaced by other modes of communication between processes.
Similar to How to win data science competitions with Deep Learning (20)
Building LLM Solutions using Open Source and Closed Source Solutions in Coher...Sri Ambati
Sandeep Singh, Head of Applied AI Computer Vision, Beans.ai
H2O Open Source GenAI World SF 2023
In the modern era of machine learning, leveraging both open-source and closed-source solutions has become paramount for achieving cutting-edge results. This talk delves into the intricacies of seamlessly integrating open-source Large Language Model (LLM) solutions like Vicuna, Falcon, and Llama with industry giants such as ChatGPT and Google's Palm. As the demand for fine-tuned and specialized datasets grows, it is imperative to understand the synergy between these tools. Attendees will gain insights into best practices for building and enriching datasets tailored for fine-tuning tasks, ensuring that their LLM projects are both robust and efficient. Through real-world examples and hands-on demonstrations, this talk will equip attendees with the knowledge to harness the power of both open and closed-source tools in a coherent and effective manner.
Patrick Hall, Professor, AI Risk Management, The George Washington University
H2O Open Source GenAI World SF 2023
Language models are incredible engineering breakthroughs but require auditing and risk management before productization. These systems raise concerns about toxicity, transparency and reproducibility, intellectual property licensing and ownership, disinformation and misinformation, supply chains, and more. How can your organization leverage these new tools without taking on undue or unknown risks? While language models and associated risk management are in their infancy, a small number of best practices in governance and risk are starting to emerge. If you have a language model use case in mind, want to understand your risks, and do something about them, this presentation is for you!
Dr. Alexy Khrabrov, Open Source Science Community Director, IBM
H2O Open Source GenAI World SF 2023
In this talk, Dr. Alexy Khrabrov, recently elected Chair of the new Generative AI Commons at Linux Foundation for AI & Data, outlines the OSS AI landscape, challenges, and opportunities. With new models and frameworks being unveiled weekly, one thing remains constant: community building and validation of all aspects of AI is key to reliable and responsible AI we can use for business and society needs. Industrial AI is one key area where such community validation can prove invaluable.
Michelle Tanco, Head of Product, H2O.ai
H2O Open Source GenAI World SF 2023
Learn how the makers at H2O.ai are building internal tools to solve real use cases using H2O Wave and h2oGPT. We will walk through an end-to-end use case and discuss how to incorporate business rules and generated content to rapidly develop custom AI apps using only Python APIs.
Applied Gen AI for the Finance Vertical Sri Ambati
Megan Kurka, Vice President, Customer Data Scientist, H2O.ai
H2O Open Source GenAI World SF 2023
Discover the transformative power of Applied Gen AI. Learn how the H2O team builds customized applications and workflows that integrate capabilities of Gen AI and AutoML specifically designed to address and enhance financial use cases. Explore real world examples, learn best practices, and witness firsthand how our innovative solutions are reshaping the landscape of finance technology.
Practitioner's Guide to LLMs: Exploring Use Cases and a Glimpse Beyond Curren...Sri Ambati
Pascal Pfeiffer, Principal Data Scientist, H2O.ai
H2O Open Source GenAI World SF 2023
This talk dives into the expansive ecosystem of Large Language Models (LLMs), offering practitioners an insightful guide to various relevant applications, from natural language understanding to creative content generation. While exploring use cases across different industries, it also honestly addresses the current limitations of LLMs and anticipates future advancements.
Introducción al Aprendizaje Automatico con H2O-3 (1)Sri Ambati
En esta reunión virtual, damos una introducción a la plataforma de aprendizaje automático de código abierto número 1, H2O-3 y te mostramos cómo puedes usarla para desarrollar modelos para resolver diferentes casos de uso.
From Rapid Prototypes to an end-to-end Model Deployment: an AI Hedge Fund Use...Sri Ambati
Numerai is an open, crowd-sourced hedge fund powered by predictions from data scientists around the world. In return, participants are rewarded with weekly payouts in crypto.
In this talk, Joe will give an overview of the Numerai tournament based on his own experience. He will then explain how he automates the time-consuming tasks such as testing different modelling strategies, scoring new datasets, submitting predictions to Numerai as well as monitoring model performance with H2O Driverless AI and R.
AI Foundations Course Module 1 - Shifting to the Next Step in Your AI Transfo...Sri Ambati
In this session, you will learn about what you should do after you’ve taken an AI transformation baseline. Over the span of this session, we will discuss the next steps in moving toward AI readiness through alignment of talent and tools to drive successful adoption and continuous use within an organization.
To find additional videos on AI courses, earn badges, join the courses at H2O.ai Learning Center: https://training.h2o.ai/products/ai-foundations-course
To find the Youtube video about this presentation: https://youtu.be/K1Cl3x3rd8g
Speaker:
Chemere Davis (H2O.ai - Senior Data Scientist Training Specialist)
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Subhajit Sahu
Abstract — Levelwise PageRank is an alternative method of PageRank computation which decomposes the input graph into a directed acyclic block-graph of strongly connected components, and processes them in topological order, one level at a time. This enables calculation for ranks in a distributed fashion without per-iteration communication, unlike the standard method where all vertices are processed in each iteration. It however comes with a precondition of the absence of dead ends in the input graph. Here, the native non-distributed performance of Levelwise PageRank was compared against Monolithic PageRank on a CPU as well as a GPU. To ensure a fair comparison, Monolithic PageRank was also performed on a graph where vertices were split by components. Results indicate that Levelwise PageRank is about as fast as Monolithic PageRank on the CPU, but quite a bit slower on the GPU. Slowdown on the GPU is likely caused by a large submission of small workloads, and expected to be non-issue when the computation is performed on massive graphs.
Chatty Kathy - UNC Bootcamp Final Project Presentation - Final Version - 5.23...John Andrews
SlideShare Description for "Chatty Kathy - UNC Bootcamp Final Project Presentation"
Title: Chatty Kathy: Enhancing Physical Activity Among Older Adults
Description:
Discover how Chatty Kathy, an innovative project developed at the UNC Bootcamp, aims to tackle the challenge of low physical activity among older adults. Our AI-driven solution uses peer interaction to boost and sustain exercise levels, significantly improving health outcomes. This presentation covers our problem statement, the rationale behind Chatty Kathy, synthetic data and persona creation, model performance metrics, a visual demonstration of the project, and potential future developments. Join us for an insightful Q&A session to explore the potential of this groundbreaking project.
Project Team: Jay Requarth, Jana Avery, John Andrews, Dr. Dick Davis II, Nee Buntoum, Nam Yeongjin & Mat Nicholas
Show drafts
volume_up
Empowering the Data Analytics Ecosystem: A Laser Focus on Value
The data analytics ecosystem thrives when every component functions at its peak, unlocking the true potential of data. Here's a laser focus on key areas for an empowered ecosystem:
1. Democratize Access, Not Data:
Granular Access Controls: Provide users with self-service tools tailored to their specific needs, preventing data overload and misuse.
Data Catalogs: Implement robust data catalogs for easy discovery and understanding of available data sources.
2. Foster Collaboration with Clear Roles:
Data Mesh Architecture: Break down data silos by creating a distributed data ownership model with clear ownership and responsibilities.
Collaborative Workspaces: Utilize interactive platforms where data scientists, analysts, and domain experts can work seamlessly together.
3. Leverage Advanced Analytics Strategically:
AI-powered Automation: Automate repetitive tasks like data cleaning and feature engineering, freeing up data talent for higher-level analysis.
Right-Tool Selection: Strategically choose the most effective advanced analytics techniques (e.g., AI, ML) based on specific business problems.
4. Prioritize Data Quality with Automation:
Automated Data Validation: Implement automated data quality checks to identify and rectify errors at the source, minimizing downstream issues.
Data Lineage Tracking: Track the flow of data throughout the ecosystem, ensuring transparency and facilitating root cause analysis for errors.
5. Cultivate a Data-Driven Mindset:
Metrics-Driven Performance Management: Align KPIs and performance metrics with data-driven insights to ensure actionable decision making.
Data Storytelling Workshops: Equip stakeholders with the skills to translate complex data findings into compelling narratives that drive action.
Benefits of a Precise Ecosystem:
Sharpened Focus: Precise access and clear roles ensure everyone works with the most relevant data, maximizing efficiency.
Actionable Insights: Strategic analytics and automated quality checks lead to more reliable and actionable data insights.
Continuous Improvement: Data-driven performance management fosters a culture of learning and continuous improvement.
Sustainable Growth: Empowered by data, organizations can make informed decisions to drive sustainable growth and innovation.
By focusing on these precise actions, organizations can create an empowered data analytics ecosystem that delivers real value by driving data-driven decisions and maximizing the return on their data investment.
As Europe's leading economic powerhouse and the fourth-largest hashtag#economy globally, Germany stands at the forefront of innovation and industrial might. Renowned for its precision engineering and high-tech sectors, Germany's economic structure is heavily supported by a robust service industry, accounting for approximately 68% of its GDP. This economic clout and strategic geopolitical stance position Germany as a focal point in the global cyber threat landscape.
In the face of escalating global tensions, particularly those emanating from geopolitical disputes with nations like hashtag#Russia and hashtag#China, hashtag#Germany has witnessed a significant uptick in targeted cyber operations. Our analysis indicates a marked increase in hashtag#cyberattack sophistication aimed at critical infrastructure and key industrial sectors. These attacks range from ransomware campaigns to hashtag#AdvancedPersistentThreats (hashtag#APTs), threatening national security and business integrity.
🔑 Key findings include:
🔍 Increased frequency and complexity of cyber threats.
🔍 Escalation of state-sponsored and criminally motivated cyber operations.
🔍 Active dark web exchanges of malicious tools and tactics.
Our comprehensive report delves into these challenges, using a blend of open-source and proprietary data collection techniques. By monitoring activity on critical networks and analyzing attack patterns, our team provides a detailed overview of the threats facing German entities.
This report aims to equip stakeholders across public and private sectors with the knowledge to enhance their defensive strategies, reduce exposure to cyber risks, and reinforce Germany's resilience against cyber threats.
How to win data science competitions with Deep Learning
1. How to win data science
competitions with Deep Learning
@ArnoCandel
Silicon Valley Big Data Science Meetup
0xdata Campus, Mountain View, Oct 9 2014
Join us at H2O World 2014 | November 18th and 19th | Computer History Museum, Mountain View
!
Register now at http://h2oworld2014.eventbrite.com
3. Who am I?
@ArnoCandel
PhD in Computational Physics, 2005
from ETH Zurich Switzerland
!
6 years at SLAC - Accelerator Physics Modeling
2 years at Skytree, Inc - Machine Learning
10 months at 0xdata/H2O - Machine Learning
!
15 years in HPC/Supercomputing/Modeling
!
Named “2014 Big Data All-Star” by Fortune Magazine
9. “We're thrilled to have H2O bring their machine
learning know-how to Apache Spark in the form
of Sparkling Water, and look forward to more
future collaboration.”
–Ion Stoica (CEO, Databricks)
10. Live Modeling #1
Kaggle
African Soil Properties
Prediction Challenge
12. Special thanks to
Jo-fai Chow for pointing
us to this challenge and
for his awesome blog!
Let’s follow this tutorial!
13. Jo-fai’s blog continued
launch H2O from R (on your laptop)
single node mode (here)
multi-threading enabled
import (zipped) data
14. Jo-fai’s blog continued
train/validation split
5 different regression targets
3594 features
Train Deep Learning model
check validation error
predict on test set
save to CSV
15. Jo-fai made it to the very top with H2O Deep Learning!
(He probably used ensembles: stacking/blending, etc.)
My best ranking was 11th with a single H2O Deep Learning model (per target)
!
BUT: Too early to tell: Leaderboard scores are computed on ~100 rows.
16. It’s time for Live Modeling!
Challenges:
• no domain knowledge -> need to find/read papers
• small data -> need to apply regularization to avoid overfitting
• need to explore data and do feature engineering, if applicable
!
We will do the following steps:
• launch H2O from R
• import Kaggle data
• train separate Deep Learning model for 5 targets
• compute cross-validation error (N-fold)
• tune model parameters (grid search)
• do simple feature engineering (dimensionality reduction)
• build an ensemble model
• create a Kaggle submission
23. Real-world dirty data:
Missing values, categorical values
with >10k factor levels, etc.
45.8M rows
41 columns
binary classifier
24. H2O Deep Learning expands categorical factor levels (on the
fly, but still):
41 features turn into N=47,411 input values for neural network
fully categorical expansion:
training is slow (also more communication)
25. Use hashing trick to reduce dimensionality N:
!
naive: input[key_to_idx(key)] = 1 //one-hot encoding
hash: input[hash_fun(key) % M] += 1, where M << N
E.g, M=1000 —> much faster training, rapid convergence!
Note:
Hashing trick is also used by winners of Kaggle
challenge, and the 4-th ranked submission uses an
Ensemble of Deep Learning models with some
feature engineering (drop rare categories etc)
27. Higgs Particle Discovery
Large Hadron Collider: Largest experiment of mankind!
$13+ billion, 16.8 miles long, 120 MegaWatts, -456F, 1PB/day, etc.
Higgs boson discovery (July ’12) led to 2013 Nobel prize!
Higgs
vs
Background
Images courtesy CERN / LHC
I missed the deadline for Kaggle by 1 day! But still want to get good numbers!
28. Higgs: Binary Classification Problem
Conventional methods of choice for physicists:
- Boosted Decision Trees
- Neural networks with 1 hidden layer
BUT: Must first add derived high-level features (physics formulae)
Metric: AUC = Area under the ROC curve (range: 0.5…1, higher is better)
Algorithm low-level H2O AUC all features H2O AUC
Generalized Linear Model 0.596 0.684
add
derived
Random Forest 0.764 0.840
Gradient Boosted Trees 0.753 0.839
features
Neural Net 1 hidden layer 0.760 0.830
HIGGS UCI Dataset:
21 low-level features AND
7 high-level derived features
Train: 10M rows, Test: 500k rows
29. Higgs: Can Deep Learning Do Better?
Algorithm low-level H2O AUC all features H2O AUC
Generalized Linear Model 0.596 0.684
Random Forest 0.764 0.840
Gradient Boosted Trees 0.753 0.839
Neural Net 1 hidden layer 0.760 0.830
Deep Learning ? ?
<Your guess goes here>
reference paper results: baseline 0.733
Let’s build a H2O Deep Learning model and find out!
Let’s build a H2O Deep Learning model and
find out!
30. H2O Steam: Scoring Platform
http://server:port/steam/index.html
Higgs Dataset Demo on 10-node cluster
Let’s score all our H2O models and compare them!
31. Scoring Higgs Models in H2O Steam
Live Demo on 10-node cluster:
<10 minutes runtime for all algos!
Better than LHC baseline of AUC=0.73!
33. Higgs Particle Detection with H2O
HIGGS UCI Dataset:
21 low-level features AND
7 high-level derived features
Train: 10M rows, Test: 500k rows
*Nature paper: http://arxiv.org/pdf/1402.4735v2.pdf
Algorithm Paper’s
l-l AUC
low-level
H2O AUC
all features
H2O AUC
Parameters (not heavily tuned),
H2O running on 10 nodes
Generalized Linear Model - 0.596 0.684 default, binomial
Random Forest - 0.764 0.840 50 trees, max depth 50
Gradient Boosted Trees 0.73 0.753 0.839 50 trees, max depth 15
Neural Net 1 layer 0.733 0.760 0.830 1x300 Rectifier, 100 epochs
Deep Learning 3 hidden layers 0.836 0.850 - 3x1000 Rectifier, L2=1e-5, 40 epochs
Deep Learning 4 hidden layers 0.868 0.869 - 4x500 Rectifier, L1=L2=1e-5, 300 epochs
Deep Learning 6 hidden layers 0.880 running - 6x500 Rectifier, L1=L2=1e-5
Deep Learning on low-level features alone beats everything else!
H2O prelim. results compare well with paper’s results* (TMVA & Theano)
34. What methods led the Higgs
Boson Kaggle contest?
Winning 3 submissions:
1. Bag of 70 dropout (50%) deep neural networks
(600,600,600), channel-out activation function, l1=5e-6 and
l2=5e-5 (for first weight matrix only). Each input neurons is
connected to only 10 input values (sampled with replacement
before training).
2. A blend of boosted decision tree ensembles constructed using
Regularized Greedy Forest
3. Ensemble of 108 rather small deep neural networks (30,50),
(30,50,25), (30,50,50,25)
Ensemble Deep Learning models are winning!
Check out h2o/R/ensemble by Erin LeDell!
35. Deep Learning Tips & Tricks !
General:
More layers for more complex functions (exp. more non-linearity).
More neurons per layer to detect finer structure in data (“memorizing”).
Validate model performance on holdout data.
Add some regularization for less overfitting (lower validation set error).
Ensembles typically perform better than individual models.
!
Specifically:
Do a grid search to get a feel for convergence, then continue training.
Try Tanh/Rectifier, try max_w2=10…50, L1=1e-5..1e-3 and/or L2=1e-5…1e-3
Try Dropout (input: up to 20%, hidden: up to 50%) with test/validation set. Input dropout is
recommended for noisy high-dimensional input.
Distributed: More training samples per iteration: faster, but less accuracy?
With ADADELTA: Try epsilon = 1e-4,1e-6,1e-8,1e-10, rho = 0.9,0.95,0.99
Without ADADELTA: Try rate = 1e-4…1e-2, rate_annealing = 1e-5…1e-9,
momentum_start = 0.5…0.9, momentum_stable = 0.99,
momentum_ramp = 1/rate_annealing.
Try balance_classes = true for datasets with large class imbalance.
Enable force_load_balance for small datasets.
Enable replicate_training_data if each node can hold all the data.
36. You can participate!
- Image Recognition: Convolutional & Pooling Layers PUB-644
- Faster Training: GPGPU support PUB-1013
- NLP: Recurrent Neural Networks PUB-1052
- Unsupervised Pre-Training: Stacked Auto-Encoders PUB-1014
- Benchmark vs other Deep Learning packages
- Investigate other optimization algorithms
- Win Kaggle competitions with H2O!
37. More info
http://h2o.ai
http://h2o.ai/blog/
http://bit.ly/deeplearningh2o
http://www.slideshare.net/0xdata/
https://www.youtube.com/user/0xdata
https://github.com/h2oai/h2o
https://github.com/h2oai/h2o-dev
@h2oai
@ArnoCandel
Join us at H2O World 2014!!!
Join us at H2O World 2014 | November 18th and 19th | Computer History Museum, Mountain View
!
Register now at http://h2oworld2014.eventbrite.com