Download to read offline
ML-Chapter_one.pptx
- 1. 1 ML-Chapter One Introduction toMachine Learning Belay E., Asst. Prof. e-mail: belayenyew@gmail.com Mobile: 0946235206 University of Gondar College of Informatics Department of Information Technology
- 2. 2 The Way Forward TopicsAreas covered Introduction to Machine Learning Definition of Machine Learning, a brief history of Machine Learning, Fundamentals of Machine Learning, Application of Machine Learning Data Preprocessing Data Cleaning, data integration, Data reduction, Data transformation, and data discretization Supervised Learning Algorithms Concept of Supervised Learning, Decision Trees, NaĂŻve Bayesian Classification, the k-Nearest Neighbors Classifiers, Ensemble, Linear Discriminant Analysis, Support Vector Machine, Time-Series Forecasting, Sequential Pattern Analysis Evaluation techniques Metrics, cross-validation, statistics, Addressing the multiple comparisons problem. Unsupervised Learning Algorithms Concept of unsupervised learning, k-Means Clustering, Hierarchical Clustering, Gaussian Mixture Model, Hidden Markov Model, Principal Component Analysis Reinforcement Learning Introduction to Reinforcement Learning, Markov Decision Process, Monte Carlo Methods for Prediction & Control Deep learning Concept of deep learning, Regularization , convolutional neural networks, recurrent neural networks etc.
- 3. 3 Reference 1. Ethem ALPAYDIN.Introduction to Machine Learning, Third Edition, 2014 2. Mohssen Mohammed, Muhammad Badruddin Khan, Eihab Bashier, Mohammed Bashier Machine Learning: Algorithms and Applications,2017 3. Ian Goodfellow Yoshua Bengio Aaron Courville, Deep Learning, 2016 Evaluation: -Assignment-10% -Article Review-10% -project-15% -Test -20% -Final Exam-45%
- 4. 4 Tools and Dataset Tools: â˘Python ⢠R/Rstudio ⢠MatLab Dataset: ⢠UCI Machine Learning Repository: http://archive.ics.uci.edu/ml/ ⢠www.Kaggle.com
- 5. 5 Introduction Definition of MachineLearning(ML): ✠It is the field of study that gives a computer the ability to learn without being explicitly programmed (Arthur Samuel, 1959) ✠A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E (Tom Mitchell, 1999). ⪠Task T: classifying handwritten digits from images ⪠Performance measure P : percentage of digits classified correctly ⪠Training experience E: dataset of digits given classifications ✠Learning is Generalization: the ability to perform a task in a situation which has never been encountered before
- 6. 6 Introduction.. Why Machine Learning? âśNecessity: many things we want to do cannot be done by âprogrammingâ. âś Recent progress in algorithms and theory âś Growing flood of online data âś Computational power is available âś Growth of Industry etc.. When Do We Need Machine Learning? âś Tasks That Are Too Complex to Program: ⪠Tasks beyond Human Capabilities-analysis of very large and complex data sets like medical archives, weather prediction and electronic commerce etc. ⪠Develop systems that are too difficult or impossible to construct manually âś Adaptivity: ⪠Programmed tools is rigidity ⪠Develop systems that can automatically adapt and customize themselves to the needs of the individual user through experience ⪠Machine learning tools âadapts to their input data â adaptive to changes in the environment they interact with.
- 7. 7 Introduction.. AI vs MLvs DL Artificial Intelligence(AI) Machine Learning(ML) Deep Learning (DL) A technique which enables a machine to mimic human behaviors A subset of AI technique which uses statistical methods to enable machine to improve with experiences A subset of ML which makes the computation of multilayers neural network feasible
- 8. 8 Introduction.. Basic Paradigm ofML âś Observe set of examples: Training data âś Infer something about process that generated data âś Use inference to make predictions about previously unseen data: test data.
- 9. 9 Introduction.. Brief History ofMachine Learning âśFrom the 1950s to now, machine learning has significantly developed. THE TURING TEST 1950 âś Alan Turing created the âTuring Testâ to determine whether or not a computer was capable of real intelligence. âś Turing explored the idea of how to determine whether machines can think THE FIRST COMPUTER PROGRAM 1952 âś Arthur Samuel created the first implementation of machine learning, the game of checkers. âś In winning strategies, and incorporating those strategies into the game.
- 10. 10 Introduction.. Brief History ofMachine Learning.. NEURAL NETWORKS FOR COMPUTERS 1957 âś Frank Rosenblatt designed the first neural network which is called perceptron for computers in 1957, which was meant to simulate the thought process of a human brain. âNEAREST NEIGHBORâ ALGORITHM 1967 âś The ânearest neighborâ algorithm was written in 1967, allowing computers to begin recognizing basic patterns. This could be used as a mapping route for traveling salesmen. EXPLANATION BASED LEARNING 1981 âś EBL, or Explanation Based Learning, was created in 1981 by Gerald Dejong. This concept allowed a computer to analyze training data and create a general rule it can follow by discarding unimportant data.
- 11. 11 Introduction.. Brief History ofMachine Learning.. NetTalk program 1985 ď Terry Sejnowski invented the NetTalk program that could learn to like a baby does during the process of language acquisition. ď The artificial neural network aimed to reconstruct a simplified the complexity of learning human-level cognitive tasks
- 12. 12 Introduction.. Brief History ofMachine Learning.. Machine Learning in 1990s ď the work in machine learning shifted from the knowledge-driven driven approach. ď Scientists and researchers created programs for computers that amounts of data and draw conclusions from the results. ď This led to the development of the IBM Deep Blue computer, worldâs chess champion Garry Kasparov in 1997
- 13. 13 Introduction.. Brief History ofMachine Learning Deep Learning 2006 âś This is the year when the term âdeep learningâ was coined by âś He used the term to explain a brand-new type of algorithms that and distinguish objects or text in images or videos
- 14. 14 Introduction.. Brief History ofMachine Learning GOOGLE AND FACEBOOK UTILIZE MACHINE LEARNING 2014 âś In 2014, Google and Facebook made machine learning the pivotal technology of their businesses. MACHINE LEARNING AND CUSTOMER CARE 2015 âś In 2015, Interactions acquired AT&Tâs Watson and the AT&T speech and language research team. Combined with their award winning Adaptive Understanding⢠technology, Interactions delivers unprecedented accuracy in understanding that helps enterprises revolutionize their customer care experience.
- 15. 15 Introduction.. Brief History ofMachine Learning MACHINE LEARNING AND SOCIAL MEDIA 2017 âś Acquired by Interactions in 2017, Digital Roots provides companies with AI- based social media. âś Its technology allows brands to quickly filter, respond, and interact with followers on social media.
- 16. 16 Introduction.. The three differenttypes of machine learning
- 17. 17 Introduction.. Supervised Learning(SL) ✠Themain goal in SL is to learn a model from labeled training data that allows us to make predictions about unseen or future data. ⪠Here, the term supervised refers to a set of samples where the desired output signals (labels) are already known. ✠SL is where you have input variables(x) and an output variable(y) and you use an algorithm to learn the mapping function from the input to the output. ✠It inference a function from labeled training data consisting of a set of training examples. Y=f(X) Where y is a target and x is input values
- 18. 18 Introduction.. Supervised Learning(SL) ✠Example,we can predict the market value of a used car by analyzing other cars and the relationship between car attributes (X) such as year of make, car brand, mileage, etc., and the selling price of the car (y). ✠SL can be: ⪠Classification: e-mail as Spam or Non-Spam, Bank customers as Fraud or default, Patient as Diabetic or Non- diabetic etc. ⪠Regression/continuous value: predict electric power consumption of a city in Kilo-watt, Predict house price based on square meter, Predict sale amount based on advertisement etc.
- 19. 19 Introduction.. Unsupervised Learning âś InSL, we know the right answer beforehand when we train our model, and in unsupervised learning, however, we are dealing with unlabeled data or data of unknown structure âś Using unsupervised learning techniques, we are able to explore the structure of our data to extract meaningful information without the guidance of a known outcome variable. Finding subgroups with clustering: âś Clustering is an exploratory data analysis technique that allows us to organize a pile of information into meaningful subgroups (clusters) without having any prior knowledge of their group memberships âś Each cluster defines a group of objects that share a certain degree of similarity but are more dissimilar to objects in other clusters, which is why clustering is also sometimes called "unsupervised classificationâ
- 20. 20 Introduction.. Unsupervised Learning.. ✠Clusteringis a great technique for structuring information and deriving meaningful relationships among data. ⪠For example, it allows marketers to discover customer groups based on their interests in order to develop distinct marketing programs. ✠The figure below illustrates how clustering can be applied to organizing unlabeled data into three distinct groups based on the similarity of their features x1 and x2 :
- 21. 21 Introduction.. Reinforcement Learning(RL): âś Reinforcementis the process of learning from rewards while performing a series of actions. ⪠the goal is to develop a system (agent) that improves its performance based on interactions with the environment âś In reinforcement learning, we do not tell the learner or agent, for example, a (robot), which action to take but merely assign a reward to each action and/or the overall outcome. ⢠Agent: takes actions ⢠Environment: the world in which agent exist and operate ⢠Action: a move the agent can make in the environment ⢠Action Space: the set of possible actions an agent can make in the environment. ⢠Reward: feedback that measures the success or failure of agentâs actions
- 22. 22 Introduction.. Reinforcement learning(RL).. ✠RLis learning by interacting with a space or an environment ✠An RL learns from the consequences of its actions, rather than from being taught explicit. ⪠It selects its actions on basis of its past experience and also by new choices ✠The machine learning program should be able to assess the goodness of policies and learn from past good action sequences to be able to generate a policy i.e. RL ✠Example: Game playing ⪠A popular example of RL is a chess engine. ⪠Here, the agent decides upon a series of moves depending on the state of the board (the environment), and the reward can be defined as win or lose at the end of the game ⪠a single move by itself is not that important ⪠It is the sequence of right moves that is good. ⪠A move is good if it is part of a good game playing policy
- 23. 23 Introduction.. Example of MachineLearning Application âś Learning Associations: Basket analysis/association discovery ⪠In the case of retail-ďŹnding associations between products bought by customers: If people who buy X typically also buy Y ⪠Going over our data and calculate that P(chips/beer) = 0.7. ⪠The rule can be defined: 70 percent of customers who buy beer also buy chips. âś Classification: Credit scoring ⪠The bank calculates the risk of giving the amount of credit for a customer based on customer information. ⪠Customer Information:-income, savings, collaterals, profession, age, past ďŹnancial history, and so forth ⪠record of past loan- the loan was paid back or not ⪠The aim is to infer a general rule coding the association between a customerâs attributes and his risk. A classiďŹcation rule learned may be of the form: Discriminant rule: IF income > θ1 AND savings > θ2 THEN low-risk ELSE high-risk
- 24. 24 Introduction.. Application of MachineLearning.. âś Pattern recognition: recognizing character codes from their images ⪠Handwrittenâfor example, to read zip codes on envelopes or amounts on checks. ⪠People have diďŹerent handwriting styles; characters may be written small or large, slanted, with a pen or pencil, and there are many possible images corresponding to the same character
- 25. 25 Introduction.. Example of MachineLearning Application.. âś Face recognition ⪠The input is an image, the classes are people to be recognized, and the learning program should learn to associate the face images to identities. ⪠Difficulty in lighting, glasses may hide the eyes and eyebrows, and a beard may hide the chin. âś In medical diagnosis ⪠The inputs are the relevant information we have about the patient and the classes are the illnesses. ⪠The inputs contain the patientâs age, gender, past medical history, and current symptoms âś Biometrics is recognition ⪠Recognition or authentication of people using their physiological and/or behavioral characteristics ⪠Examples of physiological characteristics are images of the face, ďŹngerprint, iris, and palm; examples of behavioral characteristics are dynamics of signature, voice, gait, and key stroke
- 26. 26 Introduction.. Example of MachineLearning Application.. âś Regression: Predict the price of a used car ⪠Inputs are the car attributesâbrand, year, engine capacity, mileage, and other informationâthat we believe aďŹect a carâs worth. ⪠The output is the price of the car. Such problems where the output is a number are regression problems âś Document clustering ⪠the aim is to group similar documents ⪠For example, news reports can be subdivided as those related to politics, sports, fashion, arts, and so on ⪠Documents are then grouped depending on the number of shared words.
- 27.
- 28. 28 Introduction Machine Learning Workflow.. DataPreparation ✠The preprocessing of the data is one of the most crucial steps in any machine learning application ✠Data collected from the real world is transformed to a clean dataset. ✠Raw data may contain missing values, inconsistent values, duplicate instances etc. ✠So, raw data cannot be directly used for building a model. ✠Different methods of cleaning the dataset are- ⪠Handling missing values ⪠Removing duplicate instances from the dataset. ⪠Normalizing the data in the dataset. ⪠Extract meaningful features ⪠Dimensionality reduction
- 29. 29 Introduction Machine Learning Workflow.. ChoosingLearning Algorithm- âś The best performing learning algorithm is researched. âś It depends upon the type of problem that needs to solved and the type of data we have. âś If the problem is to classify and the data is labeled, classification algorithms are used. âś If the problem is to perform a regression task and the data is labeled, regression algorithms are used. âś If the problem is to create clusters and the data is unlabeled, clustering algorithms are used.
- 30. 30 Introduction Machine Learning Workflow.. TrainingModel- âś The model is trained to improve its ability. âś The dataset is divided into training dataset and testing dataset. âś The training and testing split is order of 80/20 or 70/30. âś It also depends upon the size of the dataset. âś Training dataset is used for training purpose. âś Testing dataset is used for the testing purpose. âś Training dataset is fed to the learning algorithm. âś The learning algorithm finds a mapping between the input and the output and generates the model.
- 31. 31 Introduction Machine Learning Workflow.. EvaluatingModel- âś The model is evaluated to test if the model is any good. âś The model is evaluated using the kept-aside testing dataset. âś It allows to test the model against data that has never been used before for training. âś Metrics such as accuracy, precision, recall etc. are used to test the performance. âś If the model does not perform well, the model is re-built using different hyper parameters. âś The accuracy may be further improved by tuning the hyper parameters.
- 32. 32 Introduction Machine Learning Workflow.. Predictions âśThe build system is finally used o do something useful in the real world âś Here, the true value of machine learning is realized
- 33. 33 Introduction Limitation of MachineLearning ✠Are not useful while working with high dimensional data, that is where we have large number of inputs and outputs. ✠Cannot solve crucial AI problem like NLP, image recognition etc. ✠One of the big challenges with traditional machine learning model is a process called feature extraction ⪠Such as object recognition, handwriting recognition huge challenges .
- 34. 34 Understand and installthe following python packages âś NumPy âś SciPy âś scikit-learn âś Matplotlib âś pandas