Machine learning is a branch of artificial intelligence concerned with algorithms that allow computers to learn from data without being explicitly programmed. The document discusses the history and evolution of machine learning from early work in the 1950s to recent advances. It covers different types of learning including supervised, unsupervised, semi-supervised, and reinforcement learning. Various machine learning algorithms and techniques are presented, such as decision trees, clustering, dimensionality reduction, and ensemble methods. Issues in machine learning like problem representation and theoretical limits are also addressed.

1. MACHINE LEARNING
Priyadharshini .R
Mtech-(IT)
Reg no:2016246017 1

2. Evolution of Machine Learning
2

3. What is machine learning?
A branch of artificial intelligence,
concerned with the design and development
of algorithms that allow computers to evolve
behaviors based on empirical data.
As intelligence requires knowledge, it is
necessary for the computers to acquire
knowledge.
Definition of Machine learning:
Machine learning focuses on the
development of computer programs that can
access data and use it learn for themselves.
3

4. History of Machine Learning
• 1950s
– Samuel’s checker player
– Selfridge’s Pandemonium
• 1960s:
– Neural networks: Perceptron
– Pattern recognition
– Learning in the limit theory
– Minsky and Papert prove limitations of Perceptron
• 1970s:
– Symbolic concept induction
– Winston’s arch learner
– Expert systems and the knowledge acquisition
bottleneck
– Quinlan’s ID3
4

5. History of Machine Learning (cont.)
• 1980s:
– Advanced decision tree and rule learning
– Explanation-based Learning (EBL)
– Learning and planning and problem solving
– Utility problem
– Analogy
– Cognitive architectures
– Resurgence of neural networks (connectionism,
backpropagation)
– Valiant’s Learning Theory
– Focus on experimental methodology
5

6. 1990s:
-Data mining
-Adaptive software agents and web applications
-Text learning
-Reinforcement learning (RL)
-Inductive Logic Programming (ILP)
-Ensembles: Bagging, Boosting, and Stacking
-Bayes Net learning
History of Machine Learning (cont.)
6

7. History of Machine Learning (cont.)
• 2000s
– Support vector machines
– Kernel methods
– Graphical models
– Statistical relational learning
– Transfer learning
– Sequence labeling
– Collective classification and structured outputs
– Computer Systems Applications
• Compilers
• Debugging
• Graphics
• Security (intrusion, virus, and worm detection)
7

8. Traditional Programming
Machine Learning
Computer
Data
Program
Output
Computer
Data
Output
Program
8

9. Types of Learning
• Supervised (inductive) learning
– Training data includes desired outputs
• Unsupervised learning
– Training data does not include desired outputs
• Semi-supervised learning
– Training data includes a few desired outputs
• Reinforcement learning
– Rewards from sequence of actions
9

10. Many online software packages & datasets
• onlineData sets
• UC Respirtory
• http://www.kdnuggets.com/datasets/index.html
• Software (much related to data mining)
• JMIR Open Source
• Weka
• Shogun
• RapidMiner
• ODM
• Orange
• CMU
• Several researchers put their software online
10

11. Training and testing
Training set
(observed)
Universal
set
(unobserved)
Testing set
(unobserved)
Data acquisition Practical usage
11

12. 12
Learning (training): Learn a model using the training data
Testing: Test the model using unseen test data to assess the
model accuracy
Training and testing

13. Optimization
Combinatorial optimization
– E.g.: Greedy search
Convex optimization
– E.g.: Gradient descent
Constrained optimization
– E.g.: Linear programming
13

14.  The success of machine learning system also depends on the
algorithms.
 The algorithms control the search to find and build the
knowledge structures.
 The learning algorithms should extract useful information
from training examples.
Algorithms
14

15.  Supervised learning
– Prediction
– Classification (discrete labels), Regression (real values)
 Unsupervised learning
– Clustering
– Probability
– estimation
– Finding association (in features)
– Dimension reduction
 Semi-supervised learning
 Reinforcement learning
– Decision making (robot, chess machine)
Algorithms
15

16. Algorithms
Supervised learning Unsupervised learning
Semi-supervised learning 16

17.  Supervised learning categories and techniques
 Linear classifier (numerical functions)
 Parametric (Probabilistic functions)
• Naïve Bayes, Gaussian discriminant analysis (GDA), Hidden
Markov models (HMM), Probabilistic graphical models
 Non-parametric (Instance-based functions)
• K-nearest neighbors, Kernel regression, Kernel density estimation,
Local regression
 Non-metric (Symbolic functions)
• Classification and regression tree (CART), decision tree
 Aggregation
• Bagging (bootstrap + aggregation), Adaboost, Random forest
Learning techniques
17

18.  Unsupervised learning categories and techniques
Clustering
• K-means clustering
• Spectral clustering
Density Estimation
• Gaussian mixture model (GMM)
• Graphical models
Dimensionality reduction
• Principal component analysis (PCA)
• Factor analysis
Learning techniques
18

19. Issues in Machine Learning
• Problem representation / feature extraction
• Intention/independent learning
• Integrating learning with systems
• What are the theoretical limits of learnability
• Transfer learning
• Continuous learning
19

20. Measuring Performance
• Generalization accuracy
• Solution correctness
• Solution quality (length, efficiency)
• Speed of performance
20

21. Working Applications of ML
• Electrical power control
• Chemical process control
• Character recognition
• Face recognition
• DNA classification
• Credit card fraud detection
• Cancer cell detection
21

22. Current Machine Learning Research
• Representation
– data sequences
– spatial/temporal data
– probabilistic relational models
– …
• Approaches
– ensemble methods
– cost-sensitive learning
– active learning
– semi-supervised learning
– collective classification
– …
22

23. We have a simple overview of some techniques and
algorithms in machine learning. Furthermore, there are more and
more techniques apply machine learning as a solution. In the
future, machine learning will play an important role in our daily
life.
Conclusion
23

24. [1] W. L. Chao, J. J. Ding, “Integrated Machine
Learning Algorithms for Human Age
Estimation”, NTU, 2011.
Reference
24