Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
MACHINE
LEARNING WITH
ACCORD
FRAMEWORK
A N D R I U S D A P Š E V I Č I U S
Machine Learning
MACHINE LEARNING
• Decision tree
• Artificial neural networks
• Clustering
• Association rule discovery
• Genetic algorith...
MACHINE LEARNING
• Decision tree
• Artificial neural networks
• Clustering
• Association rule discovery
• Genetic algorith...
MACHINE LEARNING
• Decision tree
• Artificial neural networks
• Clustering
ACCORD FRAMEWORK
•Machine learning made in a minute
• GNU LGPL
• Entirely C#
• Runs on ONE machine
DECISION TREE
𝑥 ∙ 𝑦
𝑥 ∙ 𝑦 = 0
𝑥 ∙ 𝑦 = 0
𝑥 ∙ 𝑦 = 1
CAN WE PLAY TENNIS
Outlook Temperature Humidity Wind PlayTennis
Sunny Hot High Weak No
Sunny Hot High Strong No
Overcast H...
CAN WE PLAY TENNIS
Outlook Temperature Humidity Wind PlayTennis
Sunny Hot High Weak No
Sunny Hot High Strong No
Overcast H...
CAN WE PLAY TENNIS
Outlook PlayTennis
Sunny No
Sunny No
Overcast Yes
Rain Yes
Rain Yes
Rain No
Overcast Yes
Sunny No
Sunny...
CAN WE PLAY TENNIS
Outlook PlayTennis
Overcast Yes
Overcast Yes
Overcast Yes
Overcast Yes
Rain Yes
Rain Yes
Rain Yes
Rain ...
CAN WE PLAY TENNIS
Outlook PlayTennis
Overcast Yes
Overcast Yes
Overcast Yes
Overcast Yes
Rain Yes
Rain Yes
Rain Yes
Rain ...
CAN WE PLAY TENNIS
Outlook PlayTennis
Overcast Yes
Overcast Yes
Overcast Yes
Overcast Yes
Rain Yes
Rain Yes
Rain Yes
Rain ...
CAN WE PLAY TENNIS
Outlook PlayTennis
Overcast Yes
Overcast Yes
Overcast Yes
Overcast Yes
Rain Yes
Rain Yes
Rain Yes
Rain ...
CAN WE PLAY TENNIS
Outlook PlayTennis
Overcast Yes
Overcast Yes
Overcast Yes
Overcast Yes
Rain Yes
Rain Yes
Rain Yes
Rain ...
CAN WE PLAY TENNIS
Outlook PlayTennis
Overcast Yes
Overcast Yes
Overcast Yes
Overcast Yes
Rain Yes
Rain Yes
Rain Yes
Rain ...
CAN WE PLAY TENNIS
Outlook Temperature Humidity Wind PlayTennis
Sunny Cool Normal Weak Yes
Sunny Mild Normal Strong Yes
Su...
CAN WE PLAY TENNIS
Outlook Temperature Humidity Wind PlayTennis
Sunny Cool Normal Weak Yes
Sunny Mild Normal Strong Yes
Su...
CAN WE PLAY TENNIS
𝑂𝑢𝑡𝑙𝑜𝑜𝑘?
𝑅𝑎𝑖𝑛
𝑌𝑒𝑠
𝑊𝑖𝑛𝑑?
𝑌𝑒𝑠
𝑁𝑜
𝑌𝑒𝑠
𝑁𝑜
𝐻𝑢𝑚𝑖𝑑𝑖𝑡𝑦?
CREATING TREE
DecisionVariable[] attributes =
{
new DecisionVariable("Outlook", 3),
new DecisionVariable("Temperature", 3)...
CONSTRUCTING TRAINER
ID3Learning id3learning = new ID3Learning(tree);
public class ID3Learning
{
public double Run(int[][]...
NORMALIZING DATA
Codification codebook = ...;
DataTable symbols = codebook.Apply(data);
int[][] inputs =
symbols.ToArray<i...
NORMALIZING DATA
Outlook Temperature Humidity Wind PlayTennis
Sunny Hot High Weak No
Sunny Hot High Strong No
Overcast Hot...
COMBINING EVERYTHING
id3learning.Run(inputs, outputs);
var question = codebook.Translate("Sunny",
"Hot", "High", "Strong")...
ID3 ALGORITHM WITH ACCORD
tree.ToRules():
0 =: (Outlook == 0) && (Humidity == 0)
0 =: (Outlook == 2) && (Wind == 1)
1 =: (...
USES
𝑂𝑢𝑡𝑙𝑜𝑜𝑘?
𝑅𝑎𝑖𝑛
𝑌𝑒𝑠
𝑊𝑖𝑛𝑑?
𝑌𝑒𝑠
𝑁𝑜
𝑌𝑒𝑠
𝑁𝑜
𝐻𝑢𝑚𝑖𝑑𝑖𝑡𝑦?
• Image
classification
• Face recognition
• Visualisation
• Algorithm...
CLUSTERING
KMEANS
var k = new KMeans(2);
int[] clusters = k.Compute(inputs);
MEANS SHIFT
var k = new MeanShift(2, new UniformKernel(), 0.2);
int[] clusters = k.Compute(inputs);
USE CASES
Market segmentation
Customer analysis
NEURAL NETWORKS
MAGIC
NEURAL NETWORKS
𝑋 ∗ 𝑤1
𝑌 ∗ 𝑤2
NEURAL NETWORKS
𝑌 ∗ 𝑤3
𝑥1 ∗ 𝑤1
𝑥2 ∗ 𝑤2
NEURAL NETWORKS
𝑌1 ∗ 𝑤
𝑥1 ∗ 𝑤1
𝑥2 ∗ 𝑤2
𝑥2 ∗ 𝑤2
𝑌2 ∗ 𝑤
Mixed National Institute of Standards and
Technology database
NEURAL NETWORK
var network = new ActivationNetwork(new SigmoidFunction(), 28 * 28, 10);
NEURAL NETWORK
var network = new ActivationNetwork(new SigmoidFunction(), 28 * 28, 10);
NEURAL NETWORK
var network = new ActivationNetwork(new SigmoidFunction(), 28 * 28, 10);
𝑥
𝑥
𝑦
𝑦
NEURAL NETWORK
var network = new ActivationNetwork(new SigmoidFunction(), 28 * 28, 10);
new NguyenWidrow(network).Randomiz...
NEURAL NETWORK
+
−
NEURAL NETWORK
public abstract class Network
{
...
double[] Compute(double[] input);
}
NEURAL NETWORKS
𝑌1 ∗ 𝑤
𝑥1 ∗ 𝑤1
𝑥2 ∗ 𝑤2
𝑥2 ∗ 𝑤2
𝑌2 ∗ 𝑤
Self driving cars
Spam filters
OVERFITTING
OVERFITTING
30-
20%
70-80%
DIFFICULTIES
Everything is double[]
Data preparation
WHAT’S NEXT
tensorflow
Upcoming SlideShare
Loading in …5
×

Machine Learning with Accord Framework

1,008 views

Published on

Introduction to machine learning algorithms: decision trees; artificial neural networks; clustering.

Published in: Data & Analytics
  • Be the first to comment

Machine Learning with Accord Framework

  1. 1. MACHINE LEARNING WITH ACCORD FRAMEWORK A N D R I U S D A P Š E V I Č I U S
  2. 2. Machine Learning
  3. 3. MACHINE LEARNING • Decision tree • Artificial neural networks • Clustering • Association rule discovery • Genetic algorithms • etc...
  4. 4. MACHINE LEARNING • Decision tree • Artificial neural networks • Clustering • Association rule discovery • Genetic algorithms • etc...
  5. 5. MACHINE LEARNING • Decision tree • Artificial neural networks • Clustering
  6. 6. ACCORD FRAMEWORK •Machine learning made in a minute • GNU LGPL • Entirely C# • Runs on ONE machine
  7. 7. DECISION TREE 𝑥 ∙ 𝑦 𝑥 ∙ 𝑦 = 0 𝑥 ∙ 𝑦 = 0 𝑥 ∙ 𝑦 = 1
  8. 8. CAN WE PLAY TENNIS Outlook Temperature Humidity Wind PlayTennis Sunny Hot High Weak No Sunny Hot High Strong No Overcast Hot High Weak Yes Rain Mild High Weak Yes Rain Cool Normal Weak Yes Rain Cool Normal Strong No Overcast Cool Normal Strong Yes Sunny Mild High Weak No Sunny Cool Normal Weak Yes Rain Mild Normal Weak Yes Sunny Mild Normal Strong Yes Overcast Mild High Strong Yes Overcast Hot Normal Weak Yes Rain Mild High Strong No Dataset from Mitchell, T. M. Machine Learning. McGraw-Hill, 1997. pp. 59- 60.
  9. 9. CAN WE PLAY TENNIS Outlook Temperature Humidity Wind PlayTennis Sunny Hot High Weak No Sunny Hot High Strong No Overcast Hot High Weak Yes Rain Mild High Weak Yes Rain Cool Normal Weak Yes Rain Cool Normal Strong No Overcast Cool Normal Strong Yes Sunny Mild High Weak No Sunny Cool Normal Weak Yes Rain Mild Normal Weak Yes Sunny Mild Normal Strong Yes Overcast Mild High Strong Yes Overcast Hot Normal Weak Yes Rain Mild High Strong No
  10. 10. CAN WE PLAY TENNIS Outlook PlayTennis Sunny No Sunny No Overcast Yes Rain Yes Rain Yes Rain No Overcast Yes Sunny No Sunny Yes Rain Yes Sunny Yes Overcast Yes Overcast Yes Rain No
  11. 11. CAN WE PLAY TENNIS Outlook PlayTennis Overcast Yes Overcast Yes Overcast Yes Overcast Yes Rain Yes Rain Yes Rain Yes Rain No Rain No Sunny Yes Sunny Yes Sunny No Sunny No Sunny No
  12. 12. CAN WE PLAY TENNIS Outlook PlayTennis Overcast Yes Overcast Yes Overcast Yes Overcast Yes Rain Yes Rain Yes Rain Yes Rain No Rain No Sunny Yes Sunny Yes Sunny No Sunny No Sunny No
  13. 13. CAN WE PLAY TENNIS Outlook PlayTennis Overcast Yes Overcast Yes Overcast Yes Overcast Yes Rain Yes Rain Yes Rain Yes Rain No Rain No Sunny Yes Sunny Yes Sunny No Sunny No Sunny No 𝑒𝑛𝑡𝑟𝑜𝑝𝑦 𝐻(𝑥) = −𝑝 𝑦𝑒𝑠 log2(𝑝 𝑦𝑒𝑠) −𝑝 𝑛𝑜 log2(𝑝 𝑛𝑜)
  14. 14. CAN WE PLAY TENNIS Outlook PlayTennis Overcast Yes Overcast Yes Overcast Yes Overcast Yes Rain Yes Rain Yes Rain Yes Rain No Rain No Sunny Yes Sunny Yes Sunny No Sunny No Sunny No 𝑒𝑛𝑡𝑟𝑜𝑝𝑦 𝑂𝑣𝑒𝑟𝑐𝑎𝑠𝑡 = −1 log2 1 − 0 log2 0 = 0 𝑒𝑛𝑡𝑟𝑜𝑝𝑦 𝑅𝑎𝑖𝑛 = −0.6 log2 0.6 − 0.4 log2 0.4 = 0.97 𝑒𝑛𝑡𝑟𝑜𝑝𝑦𝑆𝑢𝑛𝑛𝑦 = −0.4 log2 0.4 − 0.6 log2 0.6 = 0.97
  15. 15. CAN WE PLAY TENNIS Outlook PlayTennis Overcast Yes Overcast Yes Overcast Yes Overcast Yes Rain Yes Rain Yes Rain Yes Rain No Rain No Sunny Yes Sunny Yes Sunny No Sunny No Sunny No 𝑒𝑛𝑡𝑟𝑜𝑝𝑦 𝑂𝑣𝑒𝑟𝑐𝑎𝑠𝑡 = −1 log2 1 − 0 log2 0 = 0 𝑒𝑛𝑡𝑟𝑜𝑝𝑦 𝑅𝑎𝑖𝑛 = −0.6 log2 0.6 − 0.4 log2 0.4 = 0.97 𝑒𝑛𝑡𝑟𝑜𝑝𝑦𝑆𝑢𝑛𝑛𝑦 = −0.4 log2 0.4 − 0.6 log2 0.6 = 0.97
  16. 16. CAN WE PLAY TENNIS Outlook PlayTennis Overcast Yes Overcast Yes Overcast Yes Overcast Yes Rain Yes Rain Yes Rain Yes Rain No Rain No Sunny Yes Sunny Yes Sunny No Sunny No Sunny No 𝑒𝑛𝑡𝑟𝑜𝑝𝑦 𝐵𝑒𝑓𝑜𝑟𝑒 = 0.94 𝑒𝑛𝑡𝑟𝑜𝑝𝑦 𝑎𝑓𝑡𝑒𝑟 = 0.7 𝑖𝑛𝑓𝑜𝑟𝑚𝑎𝑡𝑖𝑜𝑛 𝑔𝑎𝑖𝑛 = 0.94 − 0.7 = 0.24
  17. 17. CAN WE PLAY TENNIS Outlook Temperature Humidity Wind PlayTennis Sunny Cool Normal Weak Yes Sunny Mild Normal Strong Yes Sunny Hot High Weak No Sunny Hot High Strong No 𝑂𝑢𝑡𝑙𝑜𝑜𝑘? 𝑅𝑎𝑖𝑛 𝑌𝑒𝑠 Outlook Temperature Humidity Wind PlayTennis Rain Mild High Weak Yes Rain Cool Normal Weak Yes Rain Mild Normal Weak Yes Rain Cool Normal Strong No
  18. 18. CAN WE PLAY TENNIS Outlook Temperature Humidity Wind PlayTennis Sunny Cool Normal Weak Yes Sunny Mild Normal Strong Yes Sunny Hot High Weak No Sunny Hot High Strong No 𝑂𝑢𝑡𝑙𝑜𝑜𝑘? 𝑅𝑎𝑖𝑛 𝑌𝑒𝑠 𝑊𝑖𝑛𝑑? 𝑌𝑒𝑠 𝑁𝑜
  19. 19. CAN WE PLAY TENNIS 𝑂𝑢𝑡𝑙𝑜𝑜𝑘? 𝑅𝑎𝑖𝑛 𝑌𝑒𝑠 𝑊𝑖𝑛𝑑? 𝑌𝑒𝑠 𝑁𝑜 𝑌𝑒𝑠 𝑁𝑜 𝐻𝑢𝑚𝑖𝑑𝑖𝑡𝑦?
  20. 20. CREATING TREE DecisionVariable[] attributes = { new DecisionVariable("Outlook", 3), new DecisionVariable("Temperature", 3), new DecisionVariable("Humidity", 2), new DecisionVariable("Wind", 2) }; int classCount = 2; DecisionTree tree = new DecisionTree(attributes, classCount);
  21. 21. CONSTRUCTING TRAINER ID3Learning id3learning = new ID3Learning(tree); public class ID3Learning { public double Run(int[][] inputs, int[] outputs); }
  22. 22. NORMALIZING DATA Codification codebook = ...; DataTable symbols = codebook.Apply(data); int[][] inputs = symbols.ToArray<int>("Outlook","Temperature“, . . .); int[] outputs = symbols.ToArray<int>("PlayTennis");
  23. 23. NORMALIZING DATA Outlook Temperature Humidity Wind PlayTennis Sunny Hot High Weak No Sunny Hot High Strong No Overcast Hot High Weak Yes Rain Mild High Weak Yes Rain Cool Normal Weak Yes Rain Cool Normal Strong No Overcast Cool Normal Strong Yes Sunny Mild High Weak No Sunny Cool Normal Weak Yes Rain Mild Normal Weak Yes Sunny Mild Normal Strong Yes Overcast Mild High Strong Yes Overcast Hot Normal Weak Yes Rain Mild High Strong No 0, 1, 2, 3 0, 0, 0, 0 0, 0, 0, 1 1, 0, 0, 0 2, 1, 0, 0 2, 2, 1, 0 2, 2, 1, 1 1, 2, 1, 1 0, 1, 0, 0 0, 2, 1, 0 2, 1, 1, 0 0, 1, 1, 1 1, 1, 0, 1 1, 0, 1, 0 2, 1, 0, 1
  24. 24. COMBINING EVERYTHING id3learning.Run(inputs, outputs); var question = codebook.Translate("Sunny", "Hot", "High", "Strong") int answer = tree.Compute(question); string willPlayTenis = codebook.Translate("PlayTennis", answer);
  25. 25. ID3 ALGORITHM WITH ACCORD tree.ToRules(): 0 =: (Outlook == 0) && (Humidity == 0) 0 =: (Outlook == 2) && (Wind == 1) 1 =: (Outlook == 1) 1 =: (Outlook == 0) && (Humidity == 1) 1 =: (Outlook == 2) && (Wind == 0) tree.Save("./tree"); DecisionTree.Load("./tree"); tree.ToCode("Tree");
  26. 26. USES 𝑂𝑢𝑡𝑙𝑜𝑜𝑘? 𝑅𝑎𝑖𝑛 𝑌𝑒𝑠 𝑊𝑖𝑛𝑑? 𝑌𝑒𝑠 𝑁𝑜 𝑌𝑒𝑠 𝑁𝑜 𝐻𝑢𝑚𝑖𝑑𝑖𝑡𝑦? • Image classification • Face recognition • Visualisation • Algorithm refinement
  27. 27. CLUSTERING
  28. 28. KMEANS var k = new KMeans(2); int[] clusters = k.Compute(inputs);
  29. 29. MEANS SHIFT var k = new MeanShift(2, new UniformKernel(), 0.2); int[] clusters = k.Compute(inputs);
  30. 30. USE CASES Market segmentation Customer analysis
  31. 31. NEURAL NETWORKS MAGIC
  32. 32. NEURAL NETWORKS 𝑋 ∗ 𝑤1 𝑌 ∗ 𝑤2
  33. 33. NEURAL NETWORKS 𝑌 ∗ 𝑤3 𝑥1 ∗ 𝑤1 𝑥2 ∗ 𝑤2
  34. 34. NEURAL NETWORKS 𝑌1 ∗ 𝑤 𝑥1 ∗ 𝑤1 𝑥2 ∗ 𝑤2 𝑥2 ∗ 𝑤2 𝑌2 ∗ 𝑤
  35. 35. Mixed National Institute of Standards and Technology database
  36. 36. NEURAL NETWORK var network = new ActivationNetwork(new SigmoidFunction(), 28 * 28, 10);
  37. 37. NEURAL NETWORK var network = new ActivationNetwork(new SigmoidFunction(), 28 * 28, 10);
  38. 38. NEURAL NETWORK var network = new ActivationNetwork(new SigmoidFunction(), 28 * 28, 10); 𝑥 𝑥 𝑦 𝑦
  39. 39. NEURAL NETWORK var network = new ActivationNetwork(new SigmoidFunction(), 28 * 28, 10); new NguyenWidrow(network).Randomize(); var teacher = new ResilientBackpropagationLearning(network); double[] [][] inputs = ... double[] [][] outputs = ... for (var i = 0; i < 20; i++) { teacher.RunEpoch(inputs[i], outputs[i]); }
  40. 40. NEURAL NETWORK + −
  41. 41. NEURAL NETWORK public abstract class Network { ... double[] Compute(double[] input); }
  42. 42. NEURAL NETWORKS 𝑌1 ∗ 𝑤 𝑥1 ∗ 𝑤1 𝑥2 ∗ 𝑤2 𝑥2 ∗ 𝑤2 𝑌2 ∗ 𝑤 Self driving cars Spam filters
  43. 43. OVERFITTING
  44. 44. OVERFITTING 30- 20% 70-80%
  45. 45. DIFFICULTIES Everything is double[] Data preparation
  46. 46. WHAT’S NEXT tensorflow

×