The document discusses machine learning concepts and programming with scikit-learn. It introduces the machine learning process of getting data, pre-processing, partitioning for training and testing, creating a classifier, training and evaluating the model. As an example, it loads the Iris dataset and plots sepal length vs width with labels. It also uses PCA for dimensionality reduction to better classify the Iris data in 3 dimensions.

1. 1
Yossi Cohen
Machine Learning
with
Scikit-learn

2. 2
INTRO TO ML PROGRAMMING

3. 3
ML Programming
1. Get Data
Get labels for supervised learning
1. Create a classifier
2. Train the classifier
3. Predict test data
4. Evaluate predictor accuracy
*Configure and improve by repeating 2-5

4. 4
The ML Process
Filter
Outliers
Regression
Classify
Validate
configure
Model
Partition

5. 5
Get Data & Labels
• Sources
–Open data sources
–Collect on your own
• Verify data validity and correctness
• Wrangle data
–make it readable by computer
–Filter it
• Remove Outliers
PANDAS Python library could assist in pre-
processing & data manipulation before ML
http://pandas.pydata.org/

6. 6
Pre-Processing
Change formatting
Remove redundant data
Filter Data (take partial data)
Remove Outliers
Label
Split for testing (10/90, 20/80)

7. 7
Data Partitioning
• Data and labels
–{[data], [labels]}
–{[3,7, 76, 11, 22, 37, 56,2],[T, T, F, T, F, F, F, T]}
–Data: [Age, Do you love Nutella?]
• Partitioning will create
–{[train data], [train labels],[test data], [test labels]}
–We usually split the data on a ration of 9:1
–There is a tradeoff between the effectiveness of
the test and the learning we could provide to the
classifier
• We will look at a partitioning function later

8. 8
Learn (The “Smart Part”)
Classification
If the output is discrete to a limited amount of
classes (groups)
Regression
If the output is continues

9. 9
Learn Programming

10. 10
Create Classifier
For most SUPERVISED LEARNING
algorithms this would be
C = ClassifyAlg(Params)
Its up to us (ML guys) to set the best
params
How?
1. We could develop a hunch for it
2. Perform an exhaustive search

11. 11
Train the classifier
We assigned
C = ClassifyAlg(Params)
This is a general algorithm with some
initalizer and configurations.
In this stage we train it using:
C.fit(Data, Labels)

12. 12
Predict
After we have a trained Algorithm
classifier C
Prdeicted_Labels = C.predict(Data)

13. 13
Predictor Evaluation
We are not done yet
There is a need to evaluate the predictor
accuracy in comparison to other predictors
and to the system requirements
We will learn several methods for this

14. 14
ENVIRONMENT

15. 15
The Environment
• There are many existing environments and
tools we could use
–Matlab with Machine learning toolbox
–Apache Mahout
–Python with Scikit-learn
• Additional tools
–Hadoop / Map-Reduce to accelerate and
parallelize large data set processing
–Amazon ML tools
–NVIDIA Tools

16. 16
Scikit-learn
• Installation Instructions in
http://scikit-learn.org/stable/install.html#install-official-release
• Depends on two other libraries
• numpy and scipy
• Easiest way to install on windows:
• Install WinPython
http://sourceforge.net/projects/winpython/files/WinPython_2.7/2.7.9.4/
–Lets install this together
For Linux / Mac computers just install the 3
libs separately using PIP

17. 17
THE DATA

18. 18
Data sets
There are many data sets to work on
One of them is the Iris data classification
into three groups. It has an interesting story
you could google later
Well work on the iris
data

19. 19
Lab A – Plot the Iris data
Plot septal length vs septal width with labels
ONLY
How? Google Iris data and the scikit learn
environment
Try to understand the second part of the
program with the PCA

20. 20
Iris Data
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from sklearn import datasets
iris = datasets.load_iris()
X = iris.data[:, :2] # we only take the first two features.
Y = iris.target
x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5
y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5

21. 21
Plot Iris Data
plt.figure(2, figsize=(8, 6))
plt.clf()
plt.scatter(X[:, 0], X[:, 1],
c=Y, cmap=plt.cm.Paired)
plt.xlabel('Sepal length')
plt.ylabel('Sepal width')
plt.xlim(x_min, x_max)
plt.ylim(y_min, y_max)
plt.xticks(())
plt.yticks(())

22. 22
Add PCA for better classification
fig = plt.figure(1, figsize=(8, 6))
ax = Axes3D(fig, elev=-150, azim=110)
X_reduced = PCA(n_components=3).fit_transform(iris.data)
ax.scatter(X_reduced[:, 0], X_reduced[:, 1], X_reduced[:, 2], c=Y,
cmap=plt.cm.Paired)
ax.set_title("First three PCA directions")
ax.set_xlabel("1st eigenvector")
ax.w_xaxis.set_ticklabels([])
ax.set_ylabel("2nd eigenvector")
ax.w_yaxis.set_ticklabels([])
ax.set_zlabel("3rd eigenvector")
ax.w_zaxis.set_ticklabels([])
plt.show()

23. 23
Iris Data Classified

24. 24

25. 25
Thank you!
More About me:
Yossi CohenYossi Cohen
yossicohen19@gmail.comyossicohen19@gmail.com
+972-545-313092+972-545-313092
 Video compression and computer vision enthusiast & lecturer
 Surfer