Upcoming SlideShare
×

# 课堂讲义(最后更新:2009-9-25)

408 views

Published on

0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total views
408
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
6
0
Likes
0
Embeds 0
No embeds

No notes for slide
• Learn some patterns!
• 8*8, 12 pieces each side. The pieces are usually made of wood and are flat and cylindrical. Two kinds of pieces: king and men. The back side always initiates the game. Move: simple move, jump; Kings: When a piece becomes “kinged”? How the game ends: A player wins by capturing all of the opposing player&apos;s pieces, or by leaving the opposing player with no legal moves, or a player&apos;s piece jumps into the kings row.
• Active learning by querying teacher is much more data-efficient than random observation.
• Explain intuitively!
• Successor: after two moves. How to compute Successor ( b )?
• Meaning under Bayesian learning setting
• LMS: viewed as stochastic gradient-descent search
• *: This slide is from a presentation by Jaime Carbonell et al. in April 2007.
• See “The Discipline of Machine Learning” by Tom Mitchell 舆情分析
• Tell a story about Bioinformatics
• ### 课堂讲义(最后更新:2009-9-25)

1. 1. 机器学习 陈昱 北京大学计算机科学技术研究所 信息安全工程研究中心
2. 2. 课程基本信息 <ul><li>主讲老师：陈昱 </li></ul><ul><li>[email_address] </li></ul><ul><li>Tel ： 82529680 </li></ul><ul><li>助教： 王洪艳 </li></ul><ul><li>[email_address] </li></ul><ul><li>课程网页： http://www.icst.pku.edu.cn/course/mlearning/index.htm </li></ul>
3. 3. Ch1 Introduction <ul><li>What is machine learning (ML)? </li></ul><ul><li>Design a learning system: an example </li></ul><ul><li>ML applications </li></ul><ul><li>Miscellaneous issues </li></ul>
4. 4. Ch1 Introduction <ul><li>What is machine learning (ML)? </li></ul><ul><li>Design a learning system: an example </li></ul><ul><li>ML applications </li></ul><ul><li>Miscellaneous issues </li></ul>
5. 5. A Brief History of Machine Learning <ul><li>ML as a scientific discipline was born in mid-seventies of last century. </li></ul><ul><li>The first ML workshop was held in 1980 at CMU, with some two dozen participants and photocopied preprints. </li></ul><ul><li>The first ML publication “Machine Learning” started in 1986. </li></ul>
6. 6. Some Early Seminal Works <ul><li>Perceptron model proposed by Rosenblatt (1958), so-called “connectionist” approach ， a seminal work in neural work. </li></ul><ul><li>A system that learns to play checkers (Samuel, 1959 & 1963) </li></ul><ul><li>META-DENTRAL program, which generates rules that explains mass spectroscopy data used by expert system DENTRAL (Buchanan, 1978), an example of symbolic learning. </li></ul>
7. 7. What is Machine Learning? <ul><li>The central problem it studies: </li></ul><ul><li>How can we build computer systems that automatically improve with experience, and what are the laws that govern all learning processes? </li></ul><ul><li>We state a learning problem as: a machine learns with respect to (w.r.t.) a particular task T , performance metric P , and type of experience E . </li></ul>
8. 8. What is Machine Learning (2) <ul><li>More precisely, a computer program is said to learn from experience E , w.r.t. to some class of tasks T and performance measure P , if its performance at tasks in T , as measured by P , improves with experience E . </li></ul>
9. 9. Alternative Views <ul><li>Machine learning as an attempt to automate parts of the scientific method (Wikipedia) </li></ul><ul><ul><li>Scientific method refers to a body of techniques for investigating phenomena, acquiring new knowledge, or correcting and integrating previous knowledge. </li></ul></ul><ul><li>Machine learning as computational approaches to learning </li></ul>
10. 10. Example of Learning Problem <ul><li>Handwriting Recognition </li></ul><ul><li>Task T : recognizing and classifying handwritten words within images </li></ul><ul><li>Performance measure P : percent of words correctly classified </li></ul><ul><li>Training experience E : a database of handwritten words with given classification </li></ul>
11. 11. Place within Computer Science <ul><li>Think about a niche within the space of all software applications where ML plays a special role </li></ul><ul><ul><li>Software applications that we can’t program by hand (too complicated) </li></ul></ul><ul><ul><li>Self customizing programs </li></ul></ul>
12. 12. Relation with other Disciplines <ul><li>Human and animal learning (Psychology, Neuroscience …) </li></ul><ul><li>Biology, economics, control theory (adaptiveness, optimization) </li></ul>Computer Science Machine Learning Statistics
13. 13. Ch1 Introduction <ul><li>What is machine learning (ML)? </li></ul><ul><li>Design a learning system: an example </li></ul><ul><li>ML applications </li></ul><ul><li>Miscellaneous issues </li></ul>
14. 14. Design a Learning System <ul><li>Consider the example of learning how to play checkers </li></ul><ul><ul><li>T : playing checkers </li></ul></ul><ul><ul><li>P : the percent of games it wins in the world tournament </li></ul></ul><ul><ul><li>E ? </li></ul></ul>
15. 15. starting position of a checkers game, from Wikipedia
16. 16. a checkers board state, from http://www.5025488.net/bbs/thread-49430-1-1.html
17. 17. Choose the Training Experien ce <ul><li>Type of feedback provided by training examples (to improve P ) </li></ul><ul><ul><li>Direct: individual checkers board states plus the correct move for each state </li></ul></ul><ul><ul><li>Indirect: move sequences plus final outcome for each game </li></ul></ul><ul><ul><ul><li>Need to assign each move a credit/punish for the final outcome </li></ul></ul></ul>
18. 18. Choose the Training Experience <ul><li>Type of feedback provided by training examples (to improve P ) </li></ul><ul><ul><li>Direct: individual checkers board states plus the correct move for each state </li></ul></ul><ul><ul><li>Indirect: move sequences plus final outcome for each game </li></ul></ul><ul><ul><ul><li>Need to assign each move a credit/punish for the final outcome </li></ul></ul></ul>Easy for learning!
19. 19. Choose the Training Experience (2) <ul><li>How much the learner can control training examples? </li></ul><ul><ul><li>Completely rely on a teacher to select board states and provide correct move for each state, </li></ul></ul><ul><ul><li>have complete control over board states and final game outcome (indirect feedback), as in the case of playing against itself, or </li></ul></ul><ul><ul><li>propose confusing board states to a teacher and ask for correct move. </li></ul></ul>
20. 20. Choose the Training Experience (3) <ul><li>How well the training examples resemble to the cases in which the final performance P is measured? </li></ul><ul><ul><li>Theoretical assumption vs. reality </li></ul></ul><ul><ul><li>Related topics: </li></ul></ul><ul><ul><ul><li>Concept drifting </li></ul></ul></ul><ul><ul><ul><li>Incremental learning </li></ul></ul></ul><ul><ul><ul><li>Transfer learning </li></ul></ul></ul>
21. 21. Chose the Training Experience (3) <ul><li>How well the training examples resemble to the cases in which the final performance P is measured? </li></ul><ul><ul><li>Theoretical assumption vs. reality </li></ul></ul><ul><ul><li>Related topics: </li></ul></ul><ul><ul><ul><li>Concept drifting </li></ul></ul></ul><ul><ul><ul><li>Incremental learning </li></ul></ul></ul><ul><ul><ul><li>Transfer learning (current research hotspot!) </li></ul></ul></ul>
22. 22. Update Summary <ul><li>A checkers learning problem </li></ul><ul><ul><li>T : playing checkers </li></ul></ul><ul><ul><li>P : the percent of games it wins in the world tournament </li></ul></ul><ul><ul><li>E : games played against itself </li></ul></ul>
23. 23. Remaining Issues <ul><li>What knowledge to be learned? </li></ul><ul><li>How to represent this knowledge? </li></ul><ul><li>What algorithm used to learn the knowledge (learning mechanism)? </li></ul>
24. 24. Remaining Issues <ul><li>What knowledge to be learned? </li></ul><ul><li>How to represent this knowledge? </li></ul><ul><li>What algorithm used to learn the knowledge (learning mechanism)? </li></ul>
25. 25. Choose the Target Function <ul><li>Think of a checker learning program as an optimization problem: At every board state the program chooses the best move among all the legal moves. </li></ul><ul><li>Reformulate what to be learned as a function ChooseMove : B -> M , or a better representation, V : B -> R </li></ul>real number set
26. 26. How to Define Target Function V ? <ul><li>If b is a final board state, then it is simple: </li></ul><ul><ul><li>If b is won, V ( b )=100 (or other big number) </li></ul></ul><ul><ul><li>If b is lost, V ( b )=-100 </li></ul></ul><ul><ul><li>If b is draw, V ( b )=0 </li></ul></ul>
27. 27. How to Define Target Function V ?(2) <ul><li>Otherwise, it is tough! We might define V( b )= V ( b’ ), where b’ is the best final state that can be achieved starting from b and playing optimally until end of the program. </li></ul><ul><li>However, such definition is not operational! </li></ul>
28. 28. Remaining Issues <ul><li>What knowledge to be learned? </li></ul><ul><li>How to represent this knowledge? </li></ul><ul><li>What algorithm used to learn the knowledge (learning mechanism)? </li></ul>
29. 29. Choose a Representation of V <ul><li>A tradeoff between the expressiveness of V and demand for training data </li></ul><ul><li>Let us consider a simple representation Ṽ of V : a linear combination of following board states: </li></ul><ul><ul><li>x 1 : #black pieces on the board </li></ul></ul><ul><ul><li>x 2 : #red pieces </li></ul></ul><ul><ul><li>x 3 : #black kings </li></ul></ul><ul><ul><li>x 4 : #red kings </li></ul></ul><ul><ul><li>x 5 : #black pieces threatened by red (i.e. which can be captured on red’s next move) </li></ul></ul><ul><ul><li>x 6 : #red pieces threatened by black (i.e. which can be captured on black’s next move) </li></ul></ul>
30. 30. A Simple Representation of V
31. 31. Remaining Issues <ul><li>What knowledge to be learned? </li></ul><ul><li>How to represent this knowledge? </li></ul><ul><li>What algorithm used to learn the knowledge (learning mechanism)? </li></ul>
32. 32. Choose an Approximation Algorithm <ul><li>Choose a set of training examples ( b , V train ( b )) </li></ul><ul><li>Estimate V train ( b ) </li></ul><ul><ul><li>For some board state, it is obvious, e.g. V train ( b )=100 if x 2 =0 (assuming the learning program plays black). </li></ul></ul><ul><ul><li>Only indirect training examples are available. One common approach is via iteration, such as </li></ul></ul><ul><ul><li>V train ( b ) ←Ṽ( Successor ( b )). </li></ul></ul>
33. 33. Adjust the Weights <ul><li>A common approach to obtain the weights is by minimizing the sum of square of error </li></ul>
34. 34. An Algorithm for Finding Weights <ul><li>Least mean square (LMS) weight update rule: </li></ul><ul><li>For each training example ( b , V train ( b )) </li></ul><ul><ul><li>Use the current weights to calculate Ṽ( b ). </li></ul></ul><ul><ul><li>For each weight w i , update it as </li></ul></ul><ul><ul><li> </li></ul></ul>
35. 35. Summary of the Whole Design Process
36. 36. Issues in Machine Learning <ul><li>What algorithms exist for learning general target functions from training examples? Convergence of algorithms given sufficient examples? Which algorithms work best for which kind of target functions? </li></ul><ul><li>How does number of examples influence accuracy of learned functions? How dose character of hypothesis space impact accuracy? </li></ul><ul><li>How can prior knowledge of learner help? </li></ul>
37. 37. Issues in Machine Learning (2) <ul><li>What specific functions should the learner attempt to learn? Can this process be automated? </li></ul><ul><li>How can the learner automatically alter its representation to improve its ability to represent and learn the target function? </li></ul>
38. 38. Ch1 Introduction <ul><li>What is machine learning (ML)? </li></ul><ul><li>Design a learning system: an example </li></ul><ul><li>ML applications </li></ul><ul><li>Miscellaneous issues </li></ul>
39. 39. Machine Learning* <ul><li>Speech Recognition </li></ul><ul><li>Automated Control learning </li></ul><ul><li>Reinforcement learning </li></ul><ul><li>Predictive modeling </li></ul><ul><li>Pattern discovery </li></ul><ul><li>Hidden Markov models </li></ul><ul><li>Convex optimization </li></ul><ul><li>Explanation-based learning </li></ul><ul><li>.... </li></ul><ul><li>Extracting facts from text </li></ul><ul><li>Object recognition </li></ul><ul><li>Data Mining </li></ul>
40. 40. Example: Self-Learning Robot iCub <ul><li>iCub is a humanoid robot the size of a 3.5 year old child. It has been developing for 5-years under the project RobotCub, funded by European Commission for studying human cognition. </li></ul><ul><li>RobotCub is an open source project. </li></ul>
41. 41. Application Successes <ul><li>Speech recognition </li></ul><ul><ul><li>Two training stages: speaker-independent and speaker-dependent </li></ul></ul><ul><li>Computer vision </li></ul><ul><ul><li>Face recognition, sorting letters contain hand-written addresses by US postal office </li></ul></ul><ul><li>Bio-surveillance </li></ul><ul><ul><li>Detecting and tracking outbreak of disease </li></ul></ul><ul><li>Robot control </li></ul><ul><ul><li>Robots drive autonomously </li></ul></ul>
42. 42. Ch1 Introduction <ul><li>What is machine learning (ML)? </li></ul><ul><li>Design a learning system: an example </li></ul><ul><li>ML applications </li></ul><ul><li>Miscellaneous issues </li></ul>
43. 43. Research on ML <ul><li>Current research questions </li></ul><ul><li>Long-term questions </li></ul><ul><ul><li>For the above two items, see “The Discipline of Machine Learning” by Tom Mitchell for a sample of questions. </li></ul></ul><ul><ul><li>Machine learning for tough problems: relevant novelty detection, structural learning, active learning.* </li></ul></ul>*: from a slide by Jaime Carbonell et al. in April 2007.
44. 44. Ethical Questions <ul><li>When and where to apply ML technology? </li></ul><ul><ul><li>For example, when collecting data for security or law enforcement, or for marketing purpose, what about our privacy? </li></ul></ul><ul><ul><ul><li>Privacy-preserving data mining. Borrow something from Secure Multiparty Computing (SMC)? </li></ul></ul></ul>
45. 45. Major Conference and Journal <ul><li>International Conference on Machine Learning (ICML) </li></ul><ul><li>Conference on Neural Information Processing Systems (NIPS) </li></ul><ul><li>Annual Conference on Learning Theory (COLT) </li></ul><ul><li>Journal of Machine Learning Research (JMLR) </li></ul><ul><li>Machine Learning </li></ul>
46. 46. Some Interesting Ref <ul><li>Pattern Recognition in industry, by Phiroz Bhagat, Elsevier, 2005. </li></ul><ul><li>UCI Machine Learning Repository </li></ul><ul><li>“machine learning” item on Wikipedia </li></ul>
47. 47. HW <ul><li>Read “The Discipline of Machine Learning” by Tom Mitchell </li></ul><ul><li>1.2 (10pt, due Sept 22) </li></ul><ul><li>Bonus problem: pick up one challenge from Jaime’s paper written in 1992, and write a detailed update progress report. (10pt) </li></ul>