Valencian Summer School 2015
Day 2
Lecture 15
Machine Learning - Black Art
Charles Parker (Alston Trading)
https://bigml.com/events/valencian-summer-school-in-machine-learning-2015
Valencian Summer School 2015
Day 2
Lecture 11
The Future of Machine Learning
José David Martín-Guerrero (IDAL, UV)
https://bigml.com/events/valencian-summer-school-in-machine-learning-2015
Fairly Measuring Fairness In Machine LearningHJ van Veen
We look at a case and two research papers on measuring discrimination in machine learning models for extending credit. Presentation given as part of the Sao Paulo Machine Learning Meetup, theme "Ethics in Data Science".
If you are curious what is ML all about, this is a gentle introduction to Machine Learning and Deep Learning. This includes questions such as why ML/Data Analytics/Deep Learning ? Intuitive Understanding o how they work and some models in detail. At last I share some useful resources to get started.
Valencian Summer School 2015
Day 2
Lecture 11
The Future of Machine Learning
José David Martín-Guerrero (IDAL, UV)
https://bigml.com/events/valencian-summer-school-in-machine-learning-2015
Fairly Measuring Fairness In Machine LearningHJ van Veen
We look at a case and two research papers on measuring discrimination in machine learning models for extending credit. Presentation given as part of the Sao Paulo Machine Learning Meetup, theme "Ethics in Data Science".
If you are curious what is ML all about, this is a gentle introduction to Machine Learning and Deep Learning. This includes questions such as why ML/Data Analytics/Deep Learning ? Intuitive Understanding o how they work and some models in detail. At last I share some useful resources to get started.
Lecture 2 Basic Concepts in Machine Learning for Language TechnologyMarina Santini
Definition of Machine Learning
Type of Machine Learning:
Classification
Regression
Supervised Learning
Unsupervised Learning
Reinforcement Learning
Supervised Learning:
Supervised Classification
Training set
Hypothesis class
Empirical error
Margin
Noise
Inductive bias
Generalization
Model assessment
Cross-Validation
Classification in NLP
Types of Classification
In this Lunch & Learn session, Chirag Jain gives us a friendly & gentle introduction to Machine Learning & walks through High-Level Learning frameworks using Linear Classifiers.
Lecture 01: Machine Learning for Language Technology - IntroductionMarina Santini
What Is Machine Learning? Machine learning is programming computers to optimize a performance criterion using example data or past experience. We have a model defined up to some parameters, and learning is the execution of a computer program to optimize the parameters of the model using the training data or past experience. The model may be predictive to make predictions in the future, or descriptive to gain knowledge from data, or both. Machine learning uses the theory of statistics in building mathematical models, because the core task is making inference from a sample. (Alpaydin, 2010)
In this lecture, we discuss supervised learning starting from the simplest case. We introduce the concepts of: Margin, Noise, and Bias.
VSSML16 LR2. Summary Day 2
Valencian Summer School in Machine Learning 2016
Day 2 VSSML16
Summary Day 2
Mercè Martin (BigML)
https://bigml.com/events/valencian-summer-school-in-machine-learning-2016
An introductory course on building ML applications with primary focus on supervised learning. Covers the typical ML application cycle - Problem formulation, data definitions, offline modeling, platform design. Also, includes key tenets for building applications.
Note: This is an old slide deck. The content on building internal ML platforms is a bit outdated and slides on the model choices do not include deep learning models.
Machine Learning and Real-World ApplicationsMachinePulse
This presentation was created by Ajay, Machine Learning Scientist at MachinePulse, to present at a Meetup on Jan. 30, 2015. These slides provide an overview of widely used machine learning algorithms. The slides conclude with examples of real world applications.
Ajay Ramaseshan, is a Machine Learning Scientist at MachinePulse. He holds a Bachelors degree in Computer Science from NITK, Suratkhal and a Master in Machine Learning and Data Mining from Aalto University School of Science, Finland. He has extensive experience in the machine learning domain and has dealt with various real world problems.
Provides a brief overview of what machine learning is, how it works (theory), how to prepare data for a machine learning problem, an example case study, and additional resources.
The term Machine Learning was coined by Arthur Samuel in 1959, an american pioneer in the field of computer gaming and artificial intelligence and stated that “ it gives computers the ability to learn without being explicitly programmed” And in 1997, Tom Mitchell gave a “ well-Posed” mathematical and relational definition that “ A Computer Program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E”.
Machine learning is needed for tasks that are too complex for humans to code directly. So instead, we provide a large amount of data to a machine learning algorithm and let the algorithm work it out by exploring that data and searching for a model that will achieve what the programmers have set it out to achieve.
Machine Learning in NutShell. Machine learning is essentially a subfield of artificial intelligence (AI). In a nutshell, the goal of machine learning is to learn from data and make accurate outcome predictions, without being explicitly programmed.
Five Things I Learned While Building Anomaly Detection Tools - Toufic Boubez ...tboubez
This is my presentation from LISA 2014 in Seattle on November 14, 2014.
Most IT Ops teams only keep an eye on a small fraction of the metrics they collect because analyzing this haystack of data and extracting signal from the noise is not easy and generates too many false positives.
In this talk I will show some of the types of anomalies commonly found in dynamic data center environments and discuss the top 5 things I learned while building algorithms to find them. You will see how various Gaussian based techniques work (and why they don’t!), and we will go into some non-parametric methods that you can use to great advantage.
Lecture 2 Basic Concepts in Machine Learning for Language TechnologyMarina Santini
Definition of Machine Learning
Type of Machine Learning:
Classification
Regression
Supervised Learning
Unsupervised Learning
Reinforcement Learning
Supervised Learning:
Supervised Classification
Training set
Hypothesis class
Empirical error
Margin
Noise
Inductive bias
Generalization
Model assessment
Cross-Validation
Classification in NLP
Types of Classification
In this Lunch & Learn session, Chirag Jain gives us a friendly & gentle introduction to Machine Learning & walks through High-Level Learning frameworks using Linear Classifiers.
Lecture 01: Machine Learning for Language Technology - IntroductionMarina Santini
What Is Machine Learning? Machine learning is programming computers to optimize a performance criterion using example data or past experience. We have a model defined up to some parameters, and learning is the execution of a computer program to optimize the parameters of the model using the training data or past experience. The model may be predictive to make predictions in the future, or descriptive to gain knowledge from data, or both. Machine learning uses the theory of statistics in building mathematical models, because the core task is making inference from a sample. (Alpaydin, 2010)
In this lecture, we discuss supervised learning starting from the simplest case. We introduce the concepts of: Margin, Noise, and Bias.
VSSML16 LR2. Summary Day 2
Valencian Summer School in Machine Learning 2016
Day 2 VSSML16
Summary Day 2
Mercè Martin (BigML)
https://bigml.com/events/valencian-summer-school-in-machine-learning-2016
An introductory course on building ML applications with primary focus on supervised learning. Covers the typical ML application cycle - Problem formulation, data definitions, offline modeling, platform design. Also, includes key tenets for building applications.
Note: This is an old slide deck. The content on building internal ML platforms is a bit outdated and slides on the model choices do not include deep learning models.
Machine Learning and Real-World ApplicationsMachinePulse
This presentation was created by Ajay, Machine Learning Scientist at MachinePulse, to present at a Meetup on Jan. 30, 2015. These slides provide an overview of widely used machine learning algorithms. The slides conclude with examples of real world applications.
Ajay Ramaseshan, is a Machine Learning Scientist at MachinePulse. He holds a Bachelors degree in Computer Science from NITK, Suratkhal and a Master in Machine Learning and Data Mining from Aalto University School of Science, Finland. He has extensive experience in the machine learning domain and has dealt with various real world problems.
Provides a brief overview of what machine learning is, how it works (theory), how to prepare data for a machine learning problem, an example case study, and additional resources.
The term Machine Learning was coined by Arthur Samuel in 1959, an american pioneer in the field of computer gaming and artificial intelligence and stated that “ it gives computers the ability to learn without being explicitly programmed” And in 1997, Tom Mitchell gave a “ well-Posed” mathematical and relational definition that “ A Computer Program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E”.
Machine learning is needed for tasks that are too complex for humans to code directly. So instead, we provide a large amount of data to a machine learning algorithm and let the algorithm work it out by exploring that data and searching for a model that will achieve what the programmers have set it out to achieve.
Machine Learning in NutShell. Machine learning is essentially a subfield of artificial intelligence (AI). In a nutshell, the goal of machine learning is to learn from data and make accurate outcome predictions, without being explicitly programmed.
Five Things I Learned While Building Anomaly Detection Tools - Toufic Boubez ...tboubez
This is my presentation from LISA 2014 in Seattle on November 14, 2014.
Most IT Ops teams only keep an eye on a small fraction of the metrics they collect because analyzing this haystack of data and extracting signal from the noise is not easy and generates too many false positives.
In this talk I will show some of the types of anomalies commonly found in dynamic data center environments and discuss the top 5 things I learned while building algorithms to find them. You will see how various Gaussian based techniques work (and why they don’t!), and we will go into some non-parametric methods that you can use to great advantage.
Influx/Days 2017 San Francisco | Baron SchwartzInfluxData
WHAT GOOD IS ANOMALY DETECTION?
Static thresholds on metrics have been falling out of fashion for a while, and for good reason. Modern tooling lets you analyze and monitor a lot more data points than you used to be able to, resulting in lots more noise. The hope is that anomaly detection answers some of this, by replacing static thresholds (anomalies) with dynamic ones. But it doesn’t work as well as most people think it will. In this talk I’ll explain how anomaly detection works, so you can understand why it isn’t a good general-purpose solution, and which specific cases it’s good at.
DutchMLSchool. Logistic Regression, Deepnets, Time SeriesBigML, Inc
DutchMLSchool. Logistic Regression, Deepnets, and Time Series (Supervised Learning II) - Main Conference: Introduction to Machine Learning.
DutchMLSchool: 1st edition of the Machine Learning Summer School in The Netherlands.
This talk suggests how we might make sense of the tools landscape of the near future, where the pressure to modernise processes and automate is greatest, and what a new test process supported by tools might look like.
Takeaways:
- We need to take machine learning in testing seriously, but it won’t be taking our jobs just yet
- We don’t need more test automation tools; today we need tools that capture tester knowledge
- Tools that that learn and think can’t work for testers until we solve the knowledge capture challenge.
View On-Demand Webinar: https://youtu.be/EzyUdJFuzlE
A talk given by Eugene Dubossarsky on predictive analytics at the Big Data Analytics meetup in Sydney this month. The talk is available at http://www.youtube.com/watch?v=aG16YSFgtLY
AI Models For Fun and Profit by Walmart Director of Artificial IntelligenceProduct School
Product Management Event at #ProductCon NY on how to create AI models for fun and for profit by Jason Nichols, Director of Artificial Intelligence at Walmart Intelligent Research Lab.
Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#Secur...Alex Pinto
We could all have predicted this with our magical Big Data analytics platforms, but it seems that Machine Learning is the new hotness in Information Security. A great number of startups with ‘cy’ and ‘threat’ in their names that claim that their product will defend or detect more effectively than their neighbour's product "because math". And it should be easy to fool people without a PhD or two that math just works.
Indeed, math is powerful and large scale machine learning is an important cornerstone of much of the systems that we use today. However, not all algorithms and techniques are born equal. Machine Learning is a most powerful tool box, but not every tool can be applied to every problem and that’s where the pitfalls lie.
This presentation will describe the different techniques available for data analysis and machine learning for information security, and discuss their strengths and caveats. The Ghost of Marketing Past will also show how similar the unfulfilled promises of deterministic and exploratory analysis were, and how to avoid making the same mistakes again.
Finally, the presentation will describe the techniques and feature sets that were developed by the presenter on the past year as a part of his ongoing research project on the subject, in particular present some interesting results obtained since the last presentation on DefCon 21, and some ideas that could improve the application of machine learning for use in information security, especially in its use as a helper for security analysts in incident detection and response.
How Machine Learning works, the relationship between machine learning and other fields (AI, Data Science, Statistics, Big Data, and Data Mining).
Examples of ML (Regression, Classification)
Mathematics of ML
November 15th 2018 denver cu seminar (drew miller) ai robotics cryptocurrency...Drew Miller
A multiple-topic seminar focused on emerging technologies such as emotional AI, cognition in robotics, problems to solve in cryptocurrency, multi-dimensional blockchains and end-to-end project development in software systems. Topics were limited to approximately one hour of lecture. Various questions and comments resulted in clarification to the slides prior to their upload here.
A fascinating View of the Artificial Intelligence Journey.
Ramón López de Mántaras, Ph.D.
Technical and Business Perspectives on the Current and Future Impact of Machine Learning - MLVLC
October 20, 2015
Real-world Stories and Long-term Risks and Opportunities.
Tom Dietterich, Ph.D.
Technical and Business Perspectives on the Current and Future Impact of Machine Learning - MLVLC
October 20, 2015
Valencian Summer School 2015
Day 1
Lecture 9
Real World Machine Learning - Cooking Predictions
Andrés González (CleverTask)
https://bigml.com/events/valencian-summer-school-in-machine-learning-2015
Valencian Summer School 2015
Day 1
Lecture 7
A developers’ overview of the world of predictive APIs
Louis Dorard (PAPIs.io)
https://bigml.com/events/valencian-summer-school-in-machine-learning-2015
Valencian Summer School 2015
Day 1
Lecture 5
Data Transformation and Feature Engineering
Charles Parker (Alston Trading)
https://bigml.com/events/valencian-summer-school-in-machine-learning-2015
Valencian Summer School 2015
Day 1
Lecture 3
Ensembles of Decision Trees
Gonzalo Martínez (UAM)
https://bigml.com/events/valencian-summer-school-in-machine-learning-2015
Valencian Summer School 2015
Day 1
Lecture 3
Decision Trees
Gonzalo Martínez (UAM)
https://bigml.com/events/valencian-summer-school-in-machine-learning-2015
Valencian Summer School 2015
Day 1
Lecture 1
State of the Art in Machine Learning
Poul Petersen (BigML)
https://bigml.com/events/valencian-summer-school-in-machine-learning-2015
As Europe's leading economic powerhouse and the fourth-largest hashtag#economy globally, Germany stands at the forefront of innovation and industrial might. Renowned for its precision engineering and high-tech sectors, Germany's economic structure is heavily supported by a robust service industry, accounting for approximately 68% of its GDP. This economic clout and strategic geopolitical stance position Germany as a focal point in the global cyber threat landscape.
In the face of escalating global tensions, particularly those emanating from geopolitical disputes with nations like hashtag#Russia and hashtag#China, hashtag#Germany has witnessed a significant uptick in targeted cyber operations. Our analysis indicates a marked increase in hashtag#cyberattack sophistication aimed at critical infrastructure and key industrial sectors. These attacks range from ransomware campaigns to hashtag#AdvancedPersistentThreats (hashtag#APTs), threatening national security and business integrity.
🔑 Key findings include:
🔍 Increased frequency and complexity of cyber threats.
🔍 Escalation of state-sponsored and criminally motivated cyber operations.
🔍 Active dark web exchanges of malicious tools and tactics.
Our comprehensive report delves into these challenges, using a blend of open-source and proprietary data collection techniques. By monitoring activity on critical networks and analyzing attack patterns, our team provides a detailed overview of the threats facing German entities.
This report aims to equip stakeholders across public and private sectors with the knowledge to enhance their defensive strategies, reduce exposure to cyber risks, and reinforce Germany's resilience against cyber threats.
Explore our comprehensive data analysis project presentation on predicting product ad campaign performance. Learn how data-driven insights can optimize your marketing strategies and enhance campaign effectiveness. Perfect for professionals and students looking to understand the power of data analysis in advertising. for more details visit: https://bostoninstituteofanalytics.org/data-science-and-artificial-intelligence/
2. Machine Learning is Hard!
• By now, you know kind of a lot
• Different types of models
• Feature engineering
• Ways to evaluate
• But you’ll still fail!
• Out in the real world, there’s a
whole bunch of things that will kill
your project
• FYI - A lot of these talks are stolen
2
3. Join Me!
• On a journey into the Machine Learning House of
Horrors!
• Mwa ha ha!
3
4. 5
• The Horror of The Huge Hypothesis Space
• The Perils of The Poorly Picked Loss Function
• The Creeping Creature Called Cross Validation
• The Dread of the Drifting Domain
• The Repugnance of Reliance on Research Results
The Machine Learning House of Horrors!
5. Choosing A Hypothesis Space
• By “hypothesis space” we
mean the possible classifiers
you could build with an
algorithm given the data
• This is the choice you make
when you pick a learning
algorithm
• You have one job!
• Is there any way to make it
easier?
6
6. Theory to The Rescue!
• Probably Approximately Correct
• We’d like our model to have error less than epsilon
• We’d like that to happen at least some percentage of the time
• If the error is epsilon, the percentage is sigma, the number of
training examples is m, and the hypothesis space size is d:
7
7. The Triple Trade-Off
• There is a triple-trade off between the error, the size
of the hypothesis space, and the amount of training
data you have
8
Error
Hypothesis Space Training Data
8. What About Huge Data?
• I’m clever, so I’ll use non-
parametric methods (Decision
tree, k-NN, kernelized SVMs)
• As data scales, curious things
tend to happen
• Simpler models become more
desirable as they’re faster to fit.
• You can increase model
complexity by adding features
(maybe word counts)
• Big data often trumps modeling!
9
9. 10
• The Horror of The Huge Hypothesis Space
• The Perils of The Poorly Picked Loss Function
• The Creeping Creature Called Cross Validation
• The Dread of the Drifting Domain
• The Repugnance of Reliance on Research Results
The Machine Learning House of Horrors!
10. A Dirty Little Secret About ML Algorithms
• They don’t care what you want
• Decision Trees:
• SVM:
• LR:
• LDA:
11
11. Real-world Losses
• Real losses are nothing like this
• False positive in disease
diagnosis
• False positive in face
detection
• False positive in thumbprint
identification
• Some aren’t even instance-
based
• Path dependencies
• Game playing
12
12. Specializing Your Loss
• One solution is to let developers apply their own loss
• This is the approach of SVM light:
http://svmlight.joachims.org/
It’s been around for a while
• Losses other than Mutual Information can be plugged into the appropriate
place in splitting code
• Models trained via gradient descent can obviously be customized (Python’s
Theano is interesting for this)
• In the case of multi-example loss function, we have SEARN in Vowpal Wabbit
https://github.com/JohnLangford/vowpal_wabbit
13
13. Other Hackery
• Sometimes, the solution is just to hack
around the actual prediction
• Have several levels (cascade) of
classifiers in e.g., medical diagnosis, text
recognition
• Apply logic to explicitly avoid high loss
cases (e.g., when buying/selling equities)
• Changing the problem setting
• Will you be doing queries? Use ranking
or metric learning
• “I want to do crazy thing x with
classifiers”, chances are it’s already been
done and you can read about it.
14
14. 15
• The Horror of The Huge Hypothesis Space
• The Perils of The Poorly Picked Loss Function
• The Creeping Creature Called Cross Validation
• The Dread of the Drifting Domain
• The Repugnance of Reliance on Research Results
The Machine Learning House of Horrors!
15. When Validation Attacks!
• Cross validation
• n-Fold - Hold out one fold for
testing, train on n - 1 folds
• Great way to measure
performance, right?
• It’s all about information leakage
• via instances
• via features
16
16. Case Study #1: Law of Averages
• Estimate sporting event
outcomes
• Use previous games to
estimate points scored for
each team (via windowing
transform)
• Choose winner based on
predicted score
• What if you’re off by one on
the window?
17
17. Case Study #2: Photo Dating
• Take scanned photos from
30 different users (on
average 200 per user) and
create a model to assign a
date taken (plus or minus
five years)
• Perform 10-cross
validation
• Accuracy is 85%. Can
you trust it?
18
18. Case Study #3: Moments In Time
• You have a buy/sell
opportunity every five
seconds
• The signals you use to
evaluate the opportunity
are aggregates of market
activity over the last five
minutes
• How careful must you be
with cross-validation?
19
19. 20
• The Horror of The Huge Hypothesis Space
• The Perils of The Poorly Picked Loss Function
• The Creeping Creature Called Cross Validation
• The Dread of the Drifting Domain
• The Repugnance of Reliance on Research Results
The Machine Learning House of Horrors!
20. Breaking Machine Learning
• You’ve got this great model!
Congratulations!
• Suddenly it stops working.
Why?
• You might be in a domain
that tends to change over
time (document classification,
sales prediction)
• You might be experiencing
adverse selection (market
data predictions, spam)
21
21. Concept Drift
• This is called non-stationarity in either the prior or the conditional
distributions
• Could be a couple of different things
• If the prior p(input) is changing, it’s covariate shift
• If the conditional p(output | input) is changing, it’s concept drift
• No rule that it can’t be both
• http://blog.bigml.com/2013/03/12/machine-learning-from-
streaming-data-two-problems-two-solutions-two-concerns-and-
two-lessons/
22
22. Take Action!
• First: Look for symptoms
• Getting a lot of errors
• The distribution of predicted values changes
• Drift detection algorithms (that I know about) have the same basic flavor:
• Buffer some data in memory
• If recent data is “different” from past data, retrain, update or give up
• Some resources - A nice survey paper and an open source package:
23
http://www.win.tue.nl/~mpechen/publications/pubs/Gama_ACMCS_AdaptationCD_accepted.pdf
http://moa.cms.waikato.ac.nz/
23. The Benefits of Archeology
• Why might you train on old
data, even if it’s not relevant?
• Verification of your research
process
• You’d do the same thing
last year. Did it work?
• Gives you a good idea of
how much drift you should
expect
24
24. 25
• The Horror of The Huge Hypothesis Space
• The Perils of The Poorly Picked Loss Function
• The Creeping Creature Called Cross Validation
• The Dread of the Drifting Domain
• The Repugnance of Reliance on Research Results
The Machine Learning House of Horrors!
25. Publish or Perish
• Academic papers are a certain type of
result
• Show incremental improvement in
accuracy or generality
• Prove something about your
algorithm
• This latter is hard to come by as results
get more realistic
• Machine learning proofs assume data
is “i.i.d”, but this is obviously false.
• Real world data sucks, and dealing
with that significantly changes the
dataset
26
26. Usefulness of Results
• Theoretical Results
• Most of the time bounds do not apply (error, sample
complexity, convergence)
• Sometimes they don’t even make any sense
• Beware of putting too much faith in a single person or single
person’s work
• Usefulness generally occurs only in the aggregate
• And sometimes not even then (researchers are people, too)
27
27. Machine Learning Isn’t About Machine Learning
• Why doesn’t it work like in the
paper?
• Remember, the paper is carefully
controlled in a way your application
is not.
• Performance is rarely driven by
machine learning
• It’s driven by camera
microphones
• It’s driven by Mario Draghi
28
28. So, Don’t Bother With It?
• Of course not!
• What’s the alternative?
• “All our science, measured
against reality, is primitive
and childlike — and yet it is
the most precious thing we
have” - Albert Einstein
• Use academia as your
starting point, but don’t
think it will get you out of
the work
29
29. Some Themes
• The major points of this talk:
• Machine learning is hard to get right
• The algorithms won’t do what you want
• Good results are probably spurious
• Even if they aren’t, it won’t last
• Reading the research won’t help
• Wait, no!
• Have an attitude of skeptical optimism (or optimal skepticism?)
30