13: Practical Artificial Intelligence & Machine Learning (Arturo Servin)


Published on

Published in: Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

13: Practical Artificial Intelligence & Machine Learning (Arturo Servin)

  1. 1. Practical Artificial Intelligence and Machine Learning <ul><li>arturo.servin_at_gmail.com </li></ul><ul><li>http://arturo.servin.googlepages.com/ </li></ul>
  2. 2. About this presentation <ul><li>Some theory on AI and ML </li></ul><ul><li>Some practical ideas and simple how to </li></ul><ul><li>What's out there using AI </li></ul><ul><li>Resources, Kits and Data </li></ul>
  3. 3. Artificial Intelligence <ul><li>Machine Learning </li></ul><ul><li>Natural Language Processing </li></ul><ul><li>Knowledge representation </li></ul><ul><li>Plannning </li></ul><ul><li>Multi-Agent Systems </li></ul><ul><li>and some other stuff depending of the author of the book </li></ul>
  4. 4. Machine Learning <ul><li>A program is learning when it executes a task T and acquires experience E and the measured performance P of T improves with experience E (T. Mitchell, Machine Learning, 1997) </li></ul>
  5. 5. Machine Learning Flavours <ul><li>Supervised Learning </li></ul><ul><ul><li>Programs learn a concept/hypothesis by means of labeled examples </li></ul></ul><ul><ul><li>Examples: Artificial Neural Networks, Bayesian Methods, Decision Trees </li></ul></ul><ul><li>Unsupervised Learning </li></ul><ul><ul><li>Programs learn to categorise unlabelled examples </li></ul></ul><ul><ul><li>Examples: Non-negative matrix factorization and self-organising maps </li></ul></ul>
  6. 6. More flavours <ul><li>Reinforcement Learning </li></ul><ul><ul><li>Programs learn interacting with the environment, the execution of actions and observing the feedback in the form of + or – rewards </li></ul></ul><ul><ul><li>Examples: SARSA, Q-Learning </li></ul></ul>
  7. 7. Training Examples <ul><li>Continuous </li></ul><ul><li>Discrete </li></ul><ul><li>Inputs know as Vectors or Features </li></ul><ul><li>Example in Wine Classification: Alcohol level, Malic acid, Ash, Alcalinity of ash, etc. </li></ul>
  8. 8. <ul><li>Linear and Non-linear feature relations </li></ul><ul><li>source: Oracle Data Mining Concepts </li></ul>
  9. 9. More complex feature relations
  10. 10. Decision Trees <ul><li>Easy to understand and to interpret </li></ul><ul><li>Hierarchical structure </li></ul><ul><li>They use Entropy and Gini impurity to create groups </li></ul><ul><li>Disadvantage: It's an off-line method </li></ul><ul><li>Examples: ID3, C4.5 </li></ul>Source: http://www2.cs.uregina.ca/~dbd/cs831/notes/ml/dtrees/c4.5
  11. 11. Decision Trees, example <ul><li>Create .names and .data files with training data </li></ul><ul><li>Generate tree and rules (c4.5 -f <file> and c4.5rules -f <file>) </li></ul><ul><li>Outlook Temperature Humidity Windy Play or Don't Play </li></ul><ul><li>Sunny 80 90 true Don't Play </li></ul><ul><li>Overcast 83 78 false Play </li></ul><ul><li>Rain 70 96 false Play </li></ul><ul><li>Categorize new data (consult,consultr). Use GPS/Geocoding, Google Maps and Yahoo Weather APIs to enhance </li></ul><ul><li>aservin@turin:~/Projects/C45: consult -f golf </li></ul><ul><li>C4.5 [release 8] decision tree interpreter Sat Jan 17 00:05:16 2009 </li></ul><ul><li>------------------------------------------ </li></ul><ul><li>outlook: sunny </li></ul><ul><li>humidity: 80 </li></ul><ul><li>Decision: </li></ul><ul><li>Don't Play CF = 1.00 [ 0.63 - 1.00 ] </li></ul>
  12. 12. Bayesian Classifiers <ul><li>Bayes Theorem: P(h|D) = P(D|h) P(h) P(D) </li></ul><ul><li>P(h) = prior probability of hypothesis h </li></ul><ul><li>P(D) = prior probability of training data D </li></ul><ul><li>P(h|D) = probability of h given D </li></ul><ul><li>P(D|h) = probability of D given h </li></ul><ul><li>Naive Bayes Classifier, Fisher Classifier </li></ul><ul><li>Commonly used in SPAM filters </li></ul>
  13. 13. Classifying your RSS feeds <ul><li>Use the unofficial Google Reader API http://blog.gpowered.net/2007/08/google-reader-api-functions.html </li></ul><ul><li>Some Python Code (Programming Collective Intelligence, Chapter 6) </li></ul><ul><li>Tag interesting and non-interesting items </li></ul><ul><li>Train using Naive-bayes or Fisher classsifier </li></ul><ul><li>>> cl.train('Google changes favicon','bad') </li></ul><ul><li>>> cl.train('SearchWiki: make search your own','good') </li></ul><ul><li>New items are tagged as interesting or not </li></ul><ul><li>>> cl.classify('Ignite Leeds Today') </li></ul><ul><li>Good </li></ul><ul><li>You can re-train online </li></ul><ul><li>Add more features, try with e-mail </li></ul>
  14. 14. Finding Similarity <ul><li>Euclidean Distance, Pearson Correlation Score, Manhattan </li></ul><ul><li>Document Clustering </li></ul><ul><li>Price Prediction </li></ul><ul><li>Item similarity </li></ul><ul><li>k-Nearest Neighbors, k-means, Hierarchical Clustering, Support-Vector Machines, Kernel Methods </li></ul>
  15. 15. Similar items Source: http://home.dei.polimi.it/matteucc/Clustering/tutorial_html
  16. 16. Artificial Neural Networks <ul><li>Mathematical/Computational model based on biological neural networks </li></ul><ul><li>Many types. The most common use Backpropagation algorithm for training and Feedforward algorithm to get results/training </li></ul>
  17. 17. Artificial Neural Networks <ul><li>Input is high-dimensional, discrete or real-valued (e.g. raw sensor input) </li></ul><ul><li>Output is discrete or real valued </li></ul><ul><li>Output is a vector of values </li></ul><ul><li>Perceptron, linear </li></ul><ul><li>Sigmoid, non-linear and multi-layer </li></ul>
  18. 18. Example, finding the best price <ul><li>Create training data using Amazon/Ebay API </li></ul><ul><li>Laptop prices. Use price, screen size as features </li></ul><ul><li>Use a ANN, i.e. Fast Artificial Neural Network (FANN) </li></ul><ul><li>struct fann *ann = fann_create_standard(num_layers, num_input, num_neurons_hidden, num_output); #C++ </li></ul><ul><li>ann = fann.create(connection_rate, (num_input, num_neurons_hidden, num_output)) #Python </li></ul><ul><li>$ann = fann_create(array(2, 4, 1),1.0,0.7); // PHP </li></ul><ul><li>You can also try k-Nearest Neighbours </li></ul><ul><li>Try it! </li></ul>
  19. 19. Resources 1 <ul><li>Books </li></ul><ul><ul><li>Practical Artificial Intelligence Programming in Java, Mark Watson http://www.markwatson.com/opencontent/ (There is a Ruby one as well) </li></ul></ul><ul><ul><li>Programming Collective Intelligence, Toby Segaran; O'Reilly </li></ul></ul><ul><ul><li>Artificial Intelligence: A Modern Approach, S. Russell, P. Norvig, J. Canny; Prentice Hall, </li></ul></ul><ul><ul><li>Machine Learning, Tom Mitchell; MIT Press </li></ul></ul><ul><li>Online Stuff </li></ul><ul><ul><li>ML course in Stanford http://www.stanford.edu/class/cs229/materials.html </li></ul></ul><ul><ul><li>Statistical ML http://bengio .abracadoudou.com/lectures/ </li></ul></ul>
  20. 20. Resources 2 <ul><li>Code </li></ul><ul><ul><li>FANN http://leenissen.dk/fann/index.php </li></ul></ul><ul><ul><li>NLP http://opennlp.sourceforge.net/ </li></ul></ul><ul><ul><li>C4.5 http://www2.cs.uregina.ca/~dbd/cs831/notes/ml/dtrees/c4.5/tutorial.html </li></ul></ul><ul><ul><li>ML and Java http://www.developer.com/java/other/article.php/10936_1559871_1 </li></ul></ul><ul><li>Data </li></ul><ul><ul><li>UC Irvine Machine Learning Repository http://archive.ics.uci.edu/ml/ </li></ul></ul><ul><ul><li>Amazon Public Datasets http://aws.amazon.com/publicdatasets/ </li></ul></ul>
  21. 21. More info <ul><li>For questions, projects and job offers: </li></ul><ul><ul><li>arturo.servin (at) gmail.com </li></ul></ul><ul><ul><li>http://twitter.com/the_real_r2d2 </li></ul></ul><ul><ul><li>http://arturo.servin.googlepages.com/ </li></ul></ul>