Recommended
More Related Content
Similar to Machine Learning with Accord Framework
Similar to Machine Learning with Accord Framework (11)
Recently uploaded
Recently uploaded (20)
Machine Learning with Accord Framework
- 1. MACHINE LEARNING WITH ACCORD FRAMEWORK A N D R I U S D A P Š E V I Č I U S
- 3. MACHINE LEARNING • Decision tree • Artificial neural networks • Clustering • Association rule discovery • Genetic algorithms • etc...
- 4. MACHINE LEARNING • Decision tree • Artificial neural networks • Clustering • Association rule discovery • Genetic algorithms • etc...
- 5. MACHINE LEARNING • Decision tree • Artificial neural networks • Clustering
- 6. ACCORD FRAMEWORK •Machine learning made in a minute • GNU LGPL • Entirely C# • Runs on ONE machine
- 7. DECISION TREE 𝑥 ∙ 𝑦 𝑥 ∙ 𝑦 = 0 𝑥 ∙ 𝑦 = 0 𝑥 ∙ 𝑦 = 1
- 8. CAN WE PLAY TENNIS Outlook Temperature Humidity Wind PlayTennis Sunny Hot High Weak No Sunny Hot High Strong No Overcast Hot High Weak Yes Rain Mild High Weak Yes Rain Cool Normal Weak Yes Rain Cool Normal Strong No Overcast Cool Normal Strong Yes Sunny Mild High Weak No Sunny Cool Normal Weak Yes Rain Mild Normal Weak Yes Sunny Mild Normal Strong Yes Overcast Mild High Strong Yes Overcast Hot Normal Weak Yes Rain Mild High Strong No Dataset from Mitchell, T. M. Machine Learning. McGraw-Hill, 1997. pp. 59- 60.
- 9. CAN WE PLAY TENNIS Outlook Temperature Humidity Wind PlayTennis Sunny Hot High Weak No Sunny Hot High Strong No Overcast Hot High Weak Yes Rain Mild High Weak Yes Rain Cool Normal Weak Yes Rain Cool Normal Strong No Overcast Cool Normal Strong Yes Sunny Mild High Weak No Sunny Cool Normal Weak Yes Rain Mild Normal Weak Yes Sunny Mild Normal Strong Yes Overcast Mild High Strong Yes Overcast Hot Normal Weak Yes Rain Mild High Strong No
- 10. CAN WE PLAY TENNIS Outlook PlayTennis Sunny No Sunny No Overcast Yes Rain Yes Rain Yes Rain No Overcast Yes Sunny No Sunny Yes Rain Yes Sunny Yes Overcast Yes Overcast Yes Rain No
- 11. CAN WE PLAY TENNIS Outlook PlayTennis Overcast Yes Overcast Yes Overcast Yes Overcast Yes Rain Yes Rain Yes Rain Yes Rain No Rain No Sunny Yes Sunny Yes Sunny No Sunny No Sunny No
- 12. CAN WE PLAY TENNIS Outlook PlayTennis Overcast Yes Overcast Yes Overcast Yes Overcast Yes Rain Yes Rain Yes Rain Yes Rain No Rain No Sunny Yes Sunny Yes Sunny No Sunny No Sunny No
- 13. CAN WE PLAY TENNIS Outlook PlayTennis Overcast Yes Overcast Yes Overcast Yes Overcast Yes Rain Yes Rain Yes Rain Yes Rain No Rain No Sunny Yes Sunny Yes Sunny No Sunny No Sunny No 𝑒𝑛𝑡𝑟𝑜𝑝𝑦 𝐻(𝑥) = −𝑝 𝑦𝑒𝑠 log2(𝑝 𝑦𝑒𝑠) −𝑝 𝑛𝑜 log2(𝑝 𝑛𝑜)
- 14. CAN WE PLAY TENNIS Outlook PlayTennis Overcast Yes Overcast Yes Overcast Yes Overcast Yes Rain Yes Rain Yes Rain Yes Rain No Rain No Sunny Yes Sunny Yes Sunny No Sunny No Sunny No 𝑒𝑛𝑡𝑟𝑜𝑝𝑦 𝑂𝑣𝑒𝑟𝑐𝑎𝑠𝑡 = −1 log2 1 − 0 log2 0 = 0 𝑒𝑛𝑡𝑟𝑜𝑝𝑦 𝑅𝑎𝑖𝑛 = −0.6 log2 0.6 − 0.4 log2 0.4 = 0.97 𝑒𝑛𝑡𝑟𝑜𝑝𝑦𝑆𝑢𝑛𝑛𝑦 = −0.4 log2 0.4 − 0.6 log2 0.6 = 0.97
- 15. CAN WE PLAY TENNIS Outlook PlayTennis Overcast Yes Overcast Yes Overcast Yes Overcast Yes Rain Yes Rain Yes Rain Yes Rain No Rain No Sunny Yes Sunny Yes Sunny No Sunny No Sunny No 𝑒𝑛𝑡𝑟𝑜𝑝𝑦 𝑂𝑣𝑒𝑟𝑐𝑎𝑠𝑡 = −1 log2 1 − 0 log2 0 = 0 𝑒𝑛𝑡𝑟𝑜𝑝𝑦 𝑅𝑎𝑖𝑛 = −0.6 log2 0.6 − 0.4 log2 0.4 = 0.97 𝑒𝑛𝑡𝑟𝑜𝑝𝑦𝑆𝑢𝑛𝑛𝑦 = −0.4 log2 0.4 − 0.6 log2 0.6 = 0.97
- 16. CAN WE PLAY TENNIS Outlook PlayTennis Overcast Yes Overcast Yes Overcast Yes Overcast Yes Rain Yes Rain Yes Rain Yes Rain No Rain No Sunny Yes Sunny Yes Sunny No Sunny No Sunny No 𝑒𝑛𝑡𝑟𝑜𝑝𝑦 𝐵𝑒𝑓𝑜𝑟𝑒 = 0.94 𝑒𝑛𝑡𝑟𝑜𝑝𝑦 𝑎𝑓𝑡𝑒𝑟 = 0.7 𝑖𝑛𝑓𝑜𝑟𝑚𝑎𝑡𝑖𝑜𝑛 𝑔𝑎𝑖𝑛 = 0.94 − 0.7 = 0.24
- 17. CAN WE PLAY TENNIS Outlook Temperature Humidity Wind PlayTennis Sunny Cool Normal Weak Yes Sunny Mild Normal Strong Yes Sunny Hot High Weak No Sunny Hot High Strong No 𝑂𝑢𝑡𝑙𝑜𝑜𝑘? 𝑅𝑎𝑖𝑛 𝑌𝑒𝑠 Outlook Temperature Humidity Wind PlayTennis Rain Mild High Weak Yes Rain Cool Normal Weak Yes Rain Mild Normal Weak Yes Rain Cool Normal Strong No
- 18. CAN WE PLAY TENNIS Outlook Temperature Humidity Wind PlayTennis Sunny Cool Normal Weak Yes Sunny Mild Normal Strong Yes Sunny Hot High Weak No Sunny Hot High Strong No 𝑂𝑢𝑡𝑙𝑜𝑜𝑘? 𝑅𝑎𝑖𝑛 𝑌𝑒𝑠 𝑊𝑖𝑛𝑑? 𝑌𝑒𝑠 𝑁𝑜
- 19. CAN WE PLAY TENNIS 𝑂𝑢𝑡𝑙𝑜𝑜𝑘? 𝑅𝑎𝑖𝑛 𝑌𝑒𝑠 𝑊𝑖𝑛𝑑? 𝑌𝑒𝑠 𝑁𝑜 𝑌𝑒𝑠 𝑁𝑜 𝐻𝑢𝑚𝑖𝑑𝑖𝑡𝑦?
- 20. CREATING TREE DecisionVariable[] attributes = { new DecisionVariable("Outlook", 3), new DecisionVariable("Temperature", 3), new DecisionVariable("Humidity", 2), new DecisionVariable("Wind", 2) }; int classCount = 2; DecisionTree tree = new DecisionTree(attributes, classCount);
- 21. CONSTRUCTING TRAINER ID3Learning id3learning = new ID3Learning(tree); public class ID3Learning { public double Run(int[][] inputs, int[] outputs); }
- 22. NORMALIZING DATA Codification codebook = ...; DataTable symbols = codebook.Apply(data); int[][] inputs = symbols.ToArray<int>("Outlook","Temperature“, . . .); int[] outputs = symbols.ToArray<int>("PlayTennis");
- 23. NORMALIZING DATA Outlook Temperature Humidity Wind PlayTennis Sunny Hot High Weak No Sunny Hot High Strong No Overcast Hot High Weak Yes Rain Mild High Weak Yes Rain Cool Normal Weak Yes Rain Cool Normal Strong No Overcast Cool Normal Strong Yes Sunny Mild High Weak No Sunny Cool Normal Weak Yes Rain Mild Normal Weak Yes Sunny Mild Normal Strong Yes Overcast Mild High Strong Yes Overcast Hot Normal Weak Yes Rain Mild High Strong No 0, 1, 2, 3 0, 0, 0, 0 0, 0, 0, 1 1, 0, 0, 0 2, 1, 0, 0 2, 2, 1, 0 2, 2, 1, 1 1, 2, 1, 1 0, 1, 0, 0 0, 2, 1, 0 2, 1, 1, 0 0, 1, 1, 1 1, 1, 0, 1 1, 0, 1, 0 2, 1, 0, 1
- 24. COMBINING EVERYTHING id3learning.Run(inputs, outputs); var question = codebook.Translate("Sunny", "Hot", "High", "Strong") int answer = tree.Compute(question); string willPlayTenis = codebook.Translate("PlayTennis", answer);
- 25. ID3 ALGORITHM WITH ACCORD tree.ToRules(): 0 =: (Outlook == 0) && (Humidity == 0) 0 =: (Outlook == 2) && (Wind == 1) 1 =: (Outlook == 1) 1 =: (Outlook == 0) && (Humidity == 1) 1 =: (Outlook == 2) && (Wind == 0) tree.Save("./tree"); DecisionTree.Load("./tree"); tree.ToCode("Tree");
- 26. USES 𝑂𝑢𝑡𝑙𝑜𝑜𝑘? 𝑅𝑎𝑖𝑛 𝑌𝑒𝑠 𝑊𝑖𝑛𝑑? 𝑌𝑒𝑠 𝑁𝑜 𝑌𝑒𝑠 𝑁𝑜 𝐻𝑢𝑚𝑖𝑑𝑖𝑡𝑦? • Image classification • Face recognition • Visualisation • Algorithm refinement
- 27. CLUSTERING
- 28. KMEANS var k = new KMeans(2); int[] clusters = k.Compute(inputs);
- 29. MEANS SHIFT var k = new MeanShift(2, new UniformKernel(), 0.2); int[] clusters = k.Compute(inputs);
- 30. USE CASES Market segmentation Customer analysis
- 32. NEURAL NETWORKS 𝑋 ∗ 𝑤1 𝑌 ∗ 𝑤2
- 33. NEURAL NETWORKS 𝑌 ∗ 𝑤3 𝑥1 ∗ 𝑤1 𝑥2 ∗ 𝑤2
- 34. NEURAL NETWORKS 𝑌1 ∗ 𝑤 𝑥1 ∗ 𝑤1 𝑥2 ∗ 𝑤2 𝑥2 ∗ 𝑤2 𝑌2 ∗ 𝑤
- 35. Mixed National Institute of Standards and Technology database
- 36. NEURAL NETWORK var network = new ActivationNetwork(new SigmoidFunction(), 28 * 28, 10);
- 37. NEURAL NETWORK var network = new ActivationNetwork(new SigmoidFunction(), 28 * 28, 10);
- 38. NEURAL NETWORK var network = new ActivationNetwork(new SigmoidFunction(), 28 * 28, 10); 𝑥 𝑥 𝑦 𝑦
- 39. NEURAL NETWORK var network = new ActivationNetwork(new SigmoidFunction(), 28 * 28, 10); new NguyenWidrow(network).Randomize(); var teacher = new ResilientBackpropagationLearning(network); double[] [][] inputs = ... double[] [][] outputs = ... for (var i = 0; i < 20; i++) { teacher.RunEpoch(inputs[i], outputs[i]); }
- 41. NEURAL NETWORK public abstract class Network { ... double[] Compute(double[] input); }
- 42. NEURAL NETWORKS 𝑌1 ∗ 𝑤 𝑥1 ∗ 𝑤1 𝑥2 ∗ 𝑤2 𝑥2 ∗ 𝑤2 𝑌2 ∗ 𝑤 Self driving cars Spam filters
- 43. OVERFITTING
- 45. DIFFICULTIES Everything is double[] Data preparation