- The document presents research on predicting online purchases for Homesite Group Inc. using conversion prediction modeling.
- Various machine learning algorithms were tested on Homesite's dataset to predict whether an insurance quote seeker would purchase a policy, including naive Bayes, k-nearest neighbors, logistic regression, and boosting.
- The models were tested on different train-test splits of Homesite's dataset containing 260,000 training and 173,000 test records. The logistic regression model achieved the highest accuracy of around 81% at predicting conversions.
Tuning the Untunable - Insights on Deep Learning OptimizationSigOpt
Patrick Hayes originally gave this talk at ODSC West in 2018. During this talk, Patrick discusses a couple key barriers to deep learning optimization and how SigOpt solves them. First, Patrick discusses the problem of lengthy training cycles and how novel techniques like multitask optimization are designed to use partial information to solve this challenge. Second, Patrick discusses automated cluster management and how solving this problem makes it much easier to manage training cycles for these models.
Comparative Recommender System Evaluation: Benchmarking Recommendation Frame...Alan Said
Video available here http://www.youtube.com/watch?v=1jHxGCl8RXc
Recommender systems research is often based on comparisons of predictive accuracy: the better the evaluation scores, the better the recommender.
However, it is difficult to compare results from different recommender systems due to the many options in design and implementation of an evaluation strategy.
Additionally, algorithmic implementations can diverge from the standard formulation due to manual tuning and modifications that work better in some situations.
In this work we compare common recommendation algorithms as implemented in three popular recommendation frameworks.
To provide a fair comparison, we have complete control of the evaluation dimensions being benchmarked: dataset, data splitting, evaluation strategies, and metrics.
We also include results using the internal evaluation mechanisms of these frameworks.
Our analysis points to large differences in recommendation accuracy across frameworks and strategies, i.e. the same baselines may perform orders of magnitude better or worse across frameworks.
Our results show the necessity of clear guidelines when reporting evaluation of recommender systems to ensure reproducibility and comparison of results.
Understanding how high powered ML models arrive at their predictions is an important aspect of Machine Learning, and SHAP is a powerful tool that enables practitioners to understand how different features combine to help a model arrive at a prediction.
This slidedeck is from a presentation given at pydata global on the theoretical foundations of SHAP as well as how to use its library. Link to the presentation can be found here: https://pydata.org/global2021/schedule/presentation/3/behind-the-black-box-how-to-understand-any-ml-model-using-shap/
In this video I’m going to show you how SigOpt can help you amplify your machine learning and AI models by optimally tuning them using our black-box optimization platform.
Video: https://youtu.be/EjGrRxXWg8o
The SigOpt platform provides an ensemble of state-of-the-art Bayesian and Global optimization algorithms via a simple Software-as-a-Service API.
Kaggle Higgs Boson Machine Learning ChallengeBernard Ong
What It Took to Score the Top 2% on the Higgs Boson Machine Learning Challenge. A journey into advanced machine learning models ensembles stacking methods.
Sample Codes: https://github.com/davegautam/dotnetconfsamplecodes
Presentation on How you can get started with ML.NET. If you are existing .NET Stack Developer and Wanna use the same technology into Machine Learning, this slide focuses on how you can use ML.NET for Machine Learning.
Tuning the Untunable - Insights on Deep Learning OptimizationSigOpt
Patrick Hayes originally gave this talk at ODSC West in 2018. During this talk, Patrick discusses a couple key barriers to deep learning optimization and how SigOpt solves them. First, Patrick discusses the problem of lengthy training cycles and how novel techniques like multitask optimization are designed to use partial information to solve this challenge. Second, Patrick discusses automated cluster management and how solving this problem makes it much easier to manage training cycles for these models.
Comparative Recommender System Evaluation: Benchmarking Recommendation Frame...Alan Said
Video available here http://www.youtube.com/watch?v=1jHxGCl8RXc
Recommender systems research is often based on comparisons of predictive accuracy: the better the evaluation scores, the better the recommender.
However, it is difficult to compare results from different recommender systems due to the many options in design and implementation of an evaluation strategy.
Additionally, algorithmic implementations can diverge from the standard formulation due to manual tuning and modifications that work better in some situations.
In this work we compare common recommendation algorithms as implemented in three popular recommendation frameworks.
To provide a fair comparison, we have complete control of the evaluation dimensions being benchmarked: dataset, data splitting, evaluation strategies, and metrics.
We also include results using the internal evaluation mechanisms of these frameworks.
Our analysis points to large differences in recommendation accuracy across frameworks and strategies, i.e. the same baselines may perform orders of magnitude better or worse across frameworks.
Our results show the necessity of clear guidelines when reporting evaluation of recommender systems to ensure reproducibility and comparison of results.
Understanding how high powered ML models arrive at their predictions is an important aspect of Machine Learning, and SHAP is a powerful tool that enables practitioners to understand how different features combine to help a model arrive at a prediction.
This slidedeck is from a presentation given at pydata global on the theoretical foundations of SHAP as well as how to use its library. Link to the presentation can be found here: https://pydata.org/global2021/schedule/presentation/3/behind-the-black-box-how-to-understand-any-ml-model-using-shap/
In this video I’m going to show you how SigOpt can help you amplify your machine learning and AI models by optimally tuning them using our black-box optimization platform.
Video: https://youtu.be/EjGrRxXWg8o
The SigOpt platform provides an ensemble of state-of-the-art Bayesian and Global optimization algorithms via a simple Software-as-a-Service API.
Kaggle Higgs Boson Machine Learning ChallengeBernard Ong
What It Took to Score the Top 2% on the Higgs Boson Machine Learning Challenge. A journey into advanced machine learning models ensembles stacking methods.
Sample Codes: https://github.com/davegautam/dotnetconfsamplecodes
Presentation on How you can get started with ML.NET. If you are existing .NET Stack Developer and Wanna use the same technology into Machine Learning, this slide focuses on how you can use ML.NET for Machine Learning.
Guiding through a typical Machine Learning PipelineMichael Gerke
Many People are talking about AI and Machine Learning. Here's a quick guideline how to manage ML Projects and what to consider in order to implement machine learning use cases.
Introduction to Mahout and Machine LearningVarad Meru
This presentation gives an introduction to Apache Mahout and Machine Learning. It presents some of the important Machine Learning algorithms implemented in Mahout. Machine Learning is a vast subject; this presentation is only a introductory guide to Mahout and does not go into lower-level implementation details.
Multi-modal sources for predictive modeling using deep learningSanghamitra Deb
Using Vision Language models : Is it possible to prompt them similar to LLMs? when to use out of the box and when to pre-train? General multi-modal models --- deeplearning. Machine learning metrics, feature engineering and setting up an ML problem.
Scalable Automatic Machine Learning in H2OSri Ambati
Abstract:
In recent years, the demand for machine learning experts has outpaced the supply, despite the surge of people entering the field. To address this gap, there have been big strides in the development of user-friendly machine learning software that can be used by non-experts. Although H2O and other tools have made it easier for practitioners to train and deploy machine learning models at scale, there is still a fair bit of knowledge and background in data science that is required to produce high-performing machine learning models. Deep Neural Networks in particular, are notoriously difficult for a non-expert to tune properly.
In this presentation, we provide an overview of the the field of "Automatic Machine Learning" and introduce the new AutoML functionality in H2O. H2O's AutoML provides an easy-to-use interface which automates the process of training a large, comprehensive selection of candidate models and a stacked ensemble model which, in most cases, will be the top performing model in the AutoML Leaderboard.
H2O AutoML is available in all the H2O interfaces including the h2o R package, Python module and the Flow web GUI. We will also provide simple code examples to get you started using AutoML.
Erin’s Bio:
Erin is a Statistician and Machine Learning Scientist at H2O.ai. She is the main author of H2O Ensemble. Before joining H2O, she was the Principal Data Scientist at Wise.io and Marvin Mobile Security (acquired by Veracode in 2012) and the founder of DataScientific, Inc. Erin received her Ph.D. in Biostatistics with a Designated Emphasis in Computational Science and Engineering from University of California, Berkeley. Her research focuses on ensemble machine learning, learning from imbalanced binary-outcome data, influence curve based variance estimation and statistical computing. She also holds a B.S. and M.A. in Mathematics.
How Machine Learning Helps Organizations to Work More Efficiently?Tuan Yang
Data is increasing day by day and so is the cost of data storage and handling. However, by understanding the concepts of machine learning one can easily handle the excessive data and can process it in an affordable manner.
The process includes making models by using several kinds of algorithms. If the model is created precisely for certain task, then the organizations have a very wide chance of making use of profitable opportunities and avoiding the risks lurking behind the scenes.
Learn more about:
» Understanding Machine Learning Objectives.
» Data dimensions in Machine Learning.
» Fundamentals of Algorithms and Mapping from Input/Output.
» Parametric and Non-parametric Machine Learning Algorithms.
» Supervised, Unsupervised and Semi-Supervised Learning.
» Estimating Over-fitting and Under-fitting.
» Use Cases.
Strata San Jose 2016: Scalable Ensemble Learning with H2OSri Ambati
Erin LeDell's presentation on Scalable Ensemble Learning with H2O at Strata + Hadoop World San Jose, 03.29.16
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
Building High Available and Scalable Machine Learning ApplicationsYalçın Yenigün
The slide contains some high level information about some machine learning algorithms, cross validation and feature extraction techniques. It also contains high level techniques about high available and scalable ML products.
In this presentation I review various data science techniques and discuss their usefulness to pricing actuaries working in general insurance.
This presentation was originally given at the TIGI webinar in 2020.
https://www.actuaries.org.uk/learn-develop/attend-event/tigi-2020-technical-issues-general-insurance
Advanced Hyperparameter Optimization for Deep Learning with MLflowDatabricks
Building on the "Best Practices for Hyperparameter Tuning with MLflow" talk, we will present advanced topics in HPO for deep learning, including early stopping, multi-metric optimization, and robust optimization. We will then discuss implementations using open source tools. Finally, we will discuss how we can leverage MLflow with these tools and techniques to analyze the performance of our models.
This is an introductory workshop for machine learning. Introduced machine learning tasks such as supervised learning, unsupervised learning and reinforcement learning.
Smart like a Fox: How clever students trick dumb programming assignment asses...Nane Kratzke
This case study reports on two first-semester programming courses with more than 190 students. Both courses made use of automated assessments. We observed how students trick these systems by analysing the version history of suspect submissions. By analysing more than 3300 submissions, we revealed four astonishingly simple tricks (overfitting, evasion) and cheat-patterns (redirection, and injection) that students used to trick automated programming assignment assessment systems (APAAS). Although not the main focus of this study, it discusses and proposes corresponding counter-measures where appropriate.
Nevertheless, the primary intent of this paper is to raise problem awareness and to identify and systematise observable problem patterns in a more formal approach. The identified immaturity of existing APAAS solutions might have implications for courses that rely deeply on automation like MOOCs. Therefore, we conclude to look at APAAS solutions much more from a security point of view (code injection). Moreover, we identify the need to evolve existing unit testing frameworks into more evaluation-oriented teaching solutions that provide better trick and cheat detection capabilities and differentiated grading support.
The Power of Auto ML and How Does it WorkIvo Andreev
Automated ML is an approach to minimize the need of data science effort by enabling domain experts to build ML models without having deep knowledge of algorithms, mathematics or programming skills. The mechanism works by allowing end-users to simply provide data and the system automatically does the rest by determining approach to perform particular ML task. At first this may sound discouraging to those aiming to the “sexiest job of the 21st century” - the data scientists. However, Auto ML should be considered as democratization of ML, rather that automatic data science.
In this session we will talk about how Auto ML works, how is it implemented by Microsoft and how it could improve the productivity of even professional data scientists.
Credit scoring has been used to categorize customers based on various characteristics to evaluate their credit worthiness. Increasingly, machine learning techniques are being deployed for customer segmentation, classification and scoring. In this talk, we will discuss various machine learning techniques that can be used for credit risk applications. Through a case study built in R, we will illustrate the nuances of working with practical data sets which includes categorical and numerical data, different techniques that can be used to evaluate and explore customer profiles, visualizing high dimensional data sets and machine learning techniques for customer segmentation.
More Related Content
Similar to Presentation - Predicting Online Purchases Using Conversion Prediction Modeling 8.19.2016
Guiding through a typical Machine Learning PipelineMichael Gerke
Many People are talking about AI and Machine Learning. Here's a quick guideline how to manage ML Projects and what to consider in order to implement machine learning use cases.
Introduction to Mahout and Machine LearningVarad Meru
This presentation gives an introduction to Apache Mahout and Machine Learning. It presents some of the important Machine Learning algorithms implemented in Mahout. Machine Learning is a vast subject; this presentation is only a introductory guide to Mahout and does not go into lower-level implementation details.
Multi-modal sources for predictive modeling using deep learningSanghamitra Deb
Using Vision Language models : Is it possible to prompt them similar to LLMs? when to use out of the box and when to pre-train? General multi-modal models --- deeplearning. Machine learning metrics, feature engineering and setting up an ML problem.
Scalable Automatic Machine Learning in H2OSri Ambati
Abstract:
In recent years, the demand for machine learning experts has outpaced the supply, despite the surge of people entering the field. To address this gap, there have been big strides in the development of user-friendly machine learning software that can be used by non-experts. Although H2O and other tools have made it easier for practitioners to train and deploy machine learning models at scale, there is still a fair bit of knowledge and background in data science that is required to produce high-performing machine learning models. Deep Neural Networks in particular, are notoriously difficult for a non-expert to tune properly.
In this presentation, we provide an overview of the the field of "Automatic Machine Learning" and introduce the new AutoML functionality in H2O. H2O's AutoML provides an easy-to-use interface which automates the process of training a large, comprehensive selection of candidate models and a stacked ensemble model which, in most cases, will be the top performing model in the AutoML Leaderboard.
H2O AutoML is available in all the H2O interfaces including the h2o R package, Python module and the Flow web GUI. We will also provide simple code examples to get you started using AutoML.
Erin’s Bio:
Erin is a Statistician and Machine Learning Scientist at H2O.ai. She is the main author of H2O Ensemble. Before joining H2O, she was the Principal Data Scientist at Wise.io and Marvin Mobile Security (acquired by Veracode in 2012) and the founder of DataScientific, Inc. Erin received her Ph.D. in Biostatistics with a Designated Emphasis in Computational Science and Engineering from University of California, Berkeley. Her research focuses on ensemble machine learning, learning from imbalanced binary-outcome data, influence curve based variance estimation and statistical computing. She also holds a B.S. and M.A. in Mathematics.
How Machine Learning Helps Organizations to Work More Efficiently?Tuan Yang
Data is increasing day by day and so is the cost of data storage and handling. However, by understanding the concepts of machine learning one can easily handle the excessive data and can process it in an affordable manner.
The process includes making models by using several kinds of algorithms. If the model is created precisely for certain task, then the organizations have a very wide chance of making use of profitable opportunities and avoiding the risks lurking behind the scenes.
Learn more about:
» Understanding Machine Learning Objectives.
» Data dimensions in Machine Learning.
» Fundamentals of Algorithms and Mapping from Input/Output.
» Parametric and Non-parametric Machine Learning Algorithms.
» Supervised, Unsupervised and Semi-Supervised Learning.
» Estimating Over-fitting and Under-fitting.
» Use Cases.
Strata San Jose 2016: Scalable Ensemble Learning with H2OSri Ambati
Erin LeDell's presentation on Scalable Ensemble Learning with H2O at Strata + Hadoop World San Jose, 03.29.16
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
Building High Available and Scalable Machine Learning ApplicationsYalçın Yenigün
The slide contains some high level information about some machine learning algorithms, cross validation and feature extraction techniques. It also contains high level techniques about high available and scalable ML products.
In this presentation I review various data science techniques and discuss their usefulness to pricing actuaries working in general insurance.
This presentation was originally given at the TIGI webinar in 2020.
https://www.actuaries.org.uk/learn-develop/attend-event/tigi-2020-technical-issues-general-insurance
Advanced Hyperparameter Optimization for Deep Learning with MLflowDatabricks
Building on the "Best Practices for Hyperparameter Tuning with MLflow" talk, we will present advanced topics in HPO for deep learning, including early stopping, multi-metric optimization, and robust optimization. We will then discuss implementations using open source tools. Finally, we will discuss how we can leverage MLflow with these tools and techniques to analyze the performance of our models.
This is an introductory workshop for machine learning. Introduced machine learning tasks such as supervised learning, unsupervised learning and reinforcement learning.
Smart like a Fox: How clever students trick dumb programming assignment asses...Nane Kratzke
This case study reports on two first-semester programming courses with more than 190 students. Both courses made use of automated assessments. We observed how students trick these systems by analysing the version history of suspect submissions. By analysing more than 3300 submissions, we revealed four astonishingly simple tricks (overfitting, evasion) and cheat-patterns (redirection, and injection) that students used to trick automated programming assignment assessment systems (APAAS). Although not the main focus of this study, it discusses and proposes corresponding counter-measures where appropriate.
Nevertheless, the primary intent of this paper is to raise problem awareness and to identify and systematise observable problem patterns in a more formal approach. The identified immaturity of existing APAAS solutions might have implications for courses that rely deeply on automation like MOOCs. Therefore, we conclude to look at APAAS solutions much more from a security point of view (code injection). Moreover, we identify the need to evolve existing unit testing frameworks into more evaluation-oriented teaching solutions that provide better trick and cheat detection capabilities and differentiated grading support.
The Power of Auto ML and How Does it WorkIvo Andreev
Automated ML is an approach to minimize the need of data science effort by enabling domain experts to build ML models without having deep knowledge of algorithms, mathematics or programming skills. The mechanism works by allowing end-users to simply provide data and the system automatically does the rest by determining approach to perform particular ML task. At first this may sound discouraging to those aiming to the “sexiest job of the 21st century” - the data scientists. However, Auto ML should be considered as democratization of ML, rather that automatic data science.
In this session we will talk about how Auto ML works, how is it implemented by Microsoft and how it could improve the productivity of even professional data scientists.
Credit scoring has been used to categorize customers based on various characteristics to evaluate their credit worthiness. Increasingly, machine learning techniques are being deployed for customer segmentation, classification and scoring. In this talk, we will discuss various machine learning techniques that can be used for credit risk applications. Through a case study built in R, we will illustrate the nuances of working with practical data sets which includes categorical and numerical data, different techniques that can be used to evaluate and explore customer profiles, visualizing high dimensional data sets and machine learning techniques for customer segmentation.
Similar to Presentation - Predicting Online Purchases Using Conversion Prediction Modeling 8.19.2016 (20)
Presentation - Predicting Online Purchases Using Conversion Prediction Modeling 8.19.2016
1. Rakesh Gupta1, Chris Sneed1,Vipul Tyagi1
1College of Computing and Technology,
Lipscomb University, Nashville, TN, USA
Predicting Online Purchases Using
Conversion Prediction Modeling
1
2. Executive Summary
• Homesite Group Inc. sponsored a Kaggle* competition to
understand how they could better predict what price will
entice it’s quote seekers to purchase a home insurance policy.
• The outcome of this research will be important to the field of
retail sales, with special importance to online sales
• The benefits of this implementation for Homesite are more
sales from its leads through effective product pricing.
• In this presentation, our team will demonsrate the process we
followed to create the model and our results in predicting the
data
*https://www.kaggle.com/c/homesite-quote-conversion
3. *U.S. Census Bureau News. Quarterly Retail E-Commerce Sales for 1st Quarter 2016. (May, 2016).
*
4. Sales Lead
Articles
History
Predictive
Models
Sales and Lead
Cycle Research
Sales Pricing Models
Classification
Algorithms
Naïve Bayes
Neural Networks
Binary Logistic
Regression
AdaBoost
Patents
Sales Lead
Prioritization
Lead
Conversion
Predicting Online Purchases – A Comparison of
Machine Learning Approaches
Dynamic Pricing
Sales Lead
Conversion
Weighted KNN
Gradient Boosting
Decision Trees
CART
C5.0
CHAID
Support Vector
Machines
7. Data Source Analysis
• Data from Homesite was relatively clean to begin with
• The dataset had 299 predictor variables and one target variable:
“QuoteConversion” Flag.
– Target variable has the values : 0 or 1
• Data collected had a train dataset of 260K records and test dataset
of 173K records
• During analysis, we removed the variable “QuoteDate” and the
following variables:
Summary Statistics
Variable Name GeographicField10A GeographicField10B PersonalField84 PropertField29 PropertyField6
Min -1 -1 1 0 0
1st Quartile -1 25 2 0 0
Median -1 25 2 0 0
Mean -1 25 1.99 0 0
3rd Quartile -1 25 2 0 0
Max -1 25 8 10 0
NAs 207020 334630
8.
9. Data Cleansing & Preparation
• Categorical variables conversion to numeric
– 27 variables converted
• 293 predictor variables in the full training set
• Multiple split ratios of train/test
– 90/10
– 80/20
– 67/33
• Randomized sample
• Multiple iterations
11. Naïve Bayes*
• Naive Bayes is a simple technique for constructing classifiers.
• Models that assign class labels to problem instances, represented as
vectors of feature values.
• All naive Bayes classifiers assume that the value of a particular
feature is independent of the value of any other feature, given the
class variable.
• The method of maximum likelihood is applied for parameter
estimation for naive Bayes models.
• Despite the naive design and apparently oversimplified assumptions,
naive Bayes classifiers have worked quite well in many complex real-
world situations.
• An advantage of naive Bayes is that it only requires a small amount of
training data to estimate the parameters necessary for classification.
• Our team used Gaussian Naïve Bayes as it is good for continuous data
*Naïve Bayes classifier. (n.d.). In Wikipedia. Retrieved from
https://en.wikipedia.org/wiki/Naive_Bayes_classifier
12. Logistic Regression*
•Binary logistic regression
as our target variable is 0
or 1
•Predicts probabilities of
dependent variable
*Logistic Regression. (n.d.). In Wikipedia. Retrieved from
https://en.wikipedia.org/wiki/Logistic_regression
13. kNN*
• An object is classified by a majority vote of its neighbors, assigning it to
the “nearest” neighbor
• The nearer neighbors contribute more to the average than the distant
ones
• Sensitive to the local structure of the data
*k-nearest neighbors algorithm. (n.d.). In Wikipedia. Retrieved from
https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm
14. Boosting*
•Boosting is a general method for improving
the accuracy of any given learning algorithm
•Works by combining rough and less than
accurate rules of thumb
• Produce a classifier with a
low generalization error
• Increase weights on
incorrectly classified
examples, forcing the base
learner to focus it’s attention
on them
*Schapire, Robert E. and Freund, Yoav. Boosting: Foundations and Algorithms. Massachusetts Institute
of Technology, Cambridge, MA. 2012
15. Trials & Tribulations
• Neural
Networks?
• CSV
Vector?
Mahout
• Output of
model
• Learning
curve
RapidMiner
• Complicated
to fit model
SVM
• VIF
Functions
• Corrgrams*
Multicollinearity
Analysis
*Package ‘corrgram’ Retrieved from https://cran.r-project.org/web/packages/corrgram/corrgram.pdf
17. Results - Accuracy Matrices
“No Models are perfect, but some are better than others…”
Technology Classifier
Naïve
Bayes KNN
Logistic
Regression
HS Test File
0’s
1’s
Python
Split Ratio
90/10 81% 78.34% 81.33%
0’s = 168422
1’s = 5414
Python
Split Ratio
80/20 78.47% 81.15%
0’s = 165859
1’s = 7977
Python
Split Ratio
67/33 78.64% 81.13%
0’s = 165870
1’s = 7966
R
Split Ratio
80/20 71%
R
Split Ratio
80/20
0’s =
124,544
1’s = 49,292
18. Conclusion & Discussion
• Boosting helped identify the 6 variables that provided the
most value
• We know we can predict a sale from a lead about 80% of the
time given Homesite’s data set
• We reduced the number of predictor values from 292 to 6!
• This allows Homesite to focus on these data points.
• Following the 80/20 Pareto principle – From these 6
predictors we get 80% of the benefit without wasting time on
the other factors that don’t carry as much weight.
• Simple, fast market strategy that will provide immediate
benefits in terms of increased sales and revenue for Homesite
19. Future Works
• Continue work on additional data cleaning to
improve accuracy of the model from 81% to 97%
• Investigate the use of the remaining
classification models to see if we achieve better
results
• Design and build a process to provide real-time
prediction as new quotes are sent out by
HomeSite.
• Complete ANOVA analysis to determine strength
of logistic regression model