Chapter 1 : Introduction to KDD


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Chapter 1 : Introduction to KDD

  1. 1. Chapter 1 : Introduction to KDD
  2. 2. What is Knowledge Acquisitions ? <ul><li>aka :: data mining , knowledge discovery, knowledge extraction, information discovery, information harvesting ect. </li></ul><ul><li>Process of discovering useful information,hidden pattern or rules in large quantities of data ( non-trivial, unknown data) </li></ul><ul><li>By automatic or semiautomatic means </li></ul><ul><li>It’s impossible to find pattern using manual method. </li></ul>
  3. 3. Why Knowledge Acquisitions ?
  4. 4. Why Knowledge Acquisitions ? <ul><li>Why? </li></ul><ul><ul><li>Data explosion (tremendous amount of data available) </li></ul></ul><ul><ul><li>Data is being warehoused </li></ul></ul><ul><ul><li>Computing power </li></ul></ul><ul><ul><li>Competitive pressure </li></ul></ul>Hard Disk Nowadays more than 100Ggbytes capacities
  5. 5. Is Data Mining Appropriate for My problem ? <ul><li>Four general question to consider </li></ul><ul><ul><li>Can we clearly define the problem ? </li></ul></ul><ul><ul><li>Does potentially meaningful data exist? </li></ul></ul><ul><ul><li>Does the data contain hidden knowledge or is the data factual and useful for reporting purpose only? </li></ul></ul><ul><ul><li>Will the cost of processing the data be less than the likely increase in profit seen by applying any potential knowledge gained from the data mining project. </li></ul></ul>
  6. 6. Traditional Approaches <ul><li>Traditional database queries :. Access a database using a well defined query such as SQL </li></ul><ul><li>The query output consist of data from database </li></ul><ul><li>The output usually a subset of the database </li></ul>DBMS DB SQL
  7. 7. Data Mining or Data Query <ul><li>Four general types of knowledge can be define to help us determine when data mining is appropriate. </li></ul><ul><ul><li>Shallow Knowledge </li></ul></ul><ul><ul><li>Multidimensional Knowledge </li></ul></ul><ul><ul><li>Hidden Knowledge </li></ul></ul><ul><ul><li>Deep Knowledge </li></ul></ul>
  8. 8. Shallow Knowledge <ul><li>Factual in nature </li></ul><ul><li>Can be easily stored and manipulated in a database </li></ul><ul><li>Database query language such as SQL are excellent tools for extracting shallow knowledge from data </li></ul>
  9. 9. Multidimensional Knowledge <ul><li>also Factual </li></ul><ul><li>Data are stored in a multidimensional format </li></ul><ul><li>On-line Analytical Processing (OLAP) tools are used on multidimensional data </li></ul>
  10. 10. Hidden Knowledge <ul><li>Patterns or regularities in data that cannot be easily found using database query language such as SQL </li></ul><ul><li>Data mining algorithms can find such patterns with ease. </li></ul>
  11. 11. Deep Knowledge <ul><li>Knowledge stored in database that can only be found if we are given some direction about what we are looking for. </li></ul><ul><li>Current data mining tools are not able to locate deep knowledge. </li></ul>
  12. 12. What can computers learn? <ul><li>Four level of learning can be differentiated (Merril & Tennyson, 1977) : </li></ul><ul><ul><li>Facts : simple statement of truth </li></ul></ul><ul><ul><li>Concepts : set of objects, symbols, or events grouped together because they share certain characteristics </li></ul></ul><ul><ul><li>Procedures : step by step course of action to achieve a goal. </li></ul></ul><ul><ul><li>Principles : highest level of learning. General truth or laws that are basic to other truths. </li></ul></ul>
  13. 13. What can computers learn? <ul><li>Computer are good at learning ‘ concepts ’. </li></ul><ul><li>Concepts are the output of data mining session. </li></ul><ul><li>There are three (3) common concept view: a. Classical view b. Probabilistic view c. Exemplar View </li></ul>
  14. 14. Three Concept Views <ul><li>Classical View: </li></ul><ul><li>Definite defining properties </li></ul><ul><li>These properties determine if an individual item is an example of a particular concept. </li></ul><ul><li>Crisp and leaves no room for misinterpretation. </li></ul><ul><li>Example: Good Credit Rating </li></ul>IF Annual Income >= 30,000 & Years at Current Position >= 5 & Owns Home = True THEN Good Credit Risk = True
  15. 15. Three Concept Views <ul><li>b. Probabilistic View: </li></ul><ul><li>Concepts are represented by properties that are probable of concept member. </li></ul><ul><li>Assumption is that people store and recall concept as generalization created from individual instance observation. </li></ul><ul><li>Cannot be directly applied to achieve answer – but can be used to help in decision making process. </li></ul><ul><li>Associate probability of membership with a specific classification. </li></ul>
  16. 16. Three Concept Views - The mean annual income for individuals who consistently make loan payments on time is $30,000 - Most individuals who are good credit risks have been working for the same company for at least five years. - The majority of good credit risks own their own home <ul><li>b. Probabilistic View: </li></ul><ul><li>Example: Good Credit Rating </li></ul>Home owner with an annual income of $27000, employed at the same position for 4 years might be classified as a good credit risk with a probability of 0.85
  17. 17. Three Concept Views <ul><li>c. Exemplar View: </li></ul><ul><li>A given instance is determine to be an example of a particular concept if the instance is similar enough to a set of one or more known examples of the concept . </li></ul><ul><li>Assumption is that people store and recall likely concept exemplars that are then used to classify new instances. </li></ul><ul><li>Can associate a probability of concept membership with each classification. </li></ul>
  18. 18. Three Concept Views <ul><li>c. Exemplar View: </li></ul><ul><li>Example: </li></ul>Exemplar #1: Annual Income = 32,000 Number of years at current position = 6 Homeowner Exemplar #2: Annual Income = 52,000 Number of years at current position = 16 Renter Exemplar #1: Annual Income = 28,000 Number of years at current position = 12 Homeowner
  19. 19. What can be mined?
  20. 20. Concepts that can be mined? <ul><li>a. Classes : </li></ul><ul><li>stored data is used to locate data in predetermined groups. </li></ul><ul><li>Eg: A restaurant chain could mine customer purchase data to determine when customers visit and what they typically order. </li></ul>
  21. 21. Concepts that can be mined? <ul><li>b. Clusters : </li></ul><ul><li>Data items are grouped by logical relationships. </li></ul><ul><li>Eg: Data can be mined to identify market segments or customer affinities. </li></ul>
  22. 22. Concepts that can be mined? <ul><li>c. Associations : </li></ul><ul><li>Data can be mined to identify association. </li></ul><ul><li>Eg: The beer-diaper example is typical of associative mining. </li></ul>
  23. 23. Concepts that can be mined? <ul><li>d. Sequential : </li></ul><ul><li>Patterns in which data is mined to anticipate behavior patterns and trends. </li></ul><ul><li>Eg: An outdoor equipment retailer could predict the likelihood of a backpack purchase based on sleeping bag or hiking shoes sale. </li></ul>
  24. 24. Multidisciplinary Databases Statistics Pattern Recognition KDD Machine Learning AI Neurocomputing Data Mining
  25. 25. Disciplines Of Data Mining Data Mining Information Retrival Algorithm Machine Learning Visualization Statistics Database System
  26. 26. Data Mining Model & Task Data Mining Predictive Descriptive <ul><li>Classification </li></ul><ul><li>Regression </li></ul><ul><li>Time Series Analysis </li></ul><ul><li>Prediction </li></ul><ul><li>Clustering </li></ul><ul><li>Summarization </li></ul><ul><li>Association Rules </li></ul><ul><li>Sequence Discovery </li></ul>
  27. 27. Predictive Model <ul><li>Make prediction about values of data using known results found from different data </li></ul><ul><li>Or based on the use of other historical data </li></ul><ul><li>Example:: credit card fraud, breast cancer early warning, terrorist act, tsunami and ect. </li></ul>
  28. 28. Predictive Model <ul><li>Perform inference on the current data to make predictions. </li></ul><ul><li>We know what to predict based on historical data) </li></ul><ul><li>Never accurate 100% </li></ul><ul><li>Concentrate more to input output relation ship ( x,f(x)) </li></ul><ul><li>Typical Question </li></ul><ul><ul><li>Which costumer are likely to buy this product next four month </li></ul></ul><ul><ul><li>What kind of transactions that are likely to be fraudulent </li></ul></ul><ul><ul><li>Who is likely to drop this paper? </li></ul></ul>
  29. 29. Predictive Model x x x x x x x x x x x x x x x x months Profit (RM) Current data Future data O ?
  30. 30. Descriptive Model <ul><li>Identifies pattern or relationships in data. </li></ul><ul><li>Serves as a way to explore the properties of data examined, not to predict new properties </li></ul><ul><li>Always required a domain expert </li></ul><ul><li>Example:: </li></ul><ul><ul><li>Segmenting marketing area </li></ul></ul><ul><ul><li>Profiling student performances </li></ul></ul>
  31. 31. Descriptive Model <ul><li>Discovering new patterns inside the data </li></ul><ul><li>We may don’t have any idea how the data looks like </li></ul><ul><li>Explores the properties of the data examined </li></ul><ul><li>Pattern at various granularities (eg: Student: University-> faculty->program-> major? </li></ul><ul><li>Typical Question </li></ul><ul><ul><li>What is the data </li></ul></ul><ul><ul><li>What does it look like </li></ul></ul><ul><ul><li>What does the data suggest for group of customer advertisement? </li></ul></ul>
  32. 32. Descriptive Model major Results x x x x x x x x x x o o o o o o o o o o o o o o o o y y y y y y y y y y y y y y y Group 1 Group 2 Group 3
  33. 33. View Of DM <ul><li>Data To Be Mined </li></ul><ul><ul><li>Data warehouse, WWW, time series, textual. spatial multimedia, transactional </li></ul></ul><ul><li>Knowledge To Be Mined </li></ul><ul><ul><li>Classification, prediction, summarization, trend </li></ul></ul><ul><li>Techniques Utilized </li></ul><ul><ul><li>Database, machine learning, visualization, statistics </li></ul></ul><ul><li>Applications Adapted </li></ul><ul><ul><li>Marketing, demographic segmentation, stock analysis </li></ul></ul>
  34. 34. DM In Action <ul><li>Medical Applications ::clinical diagnosis, drug analysis </li></ul><ul><li>Business (marketing segmentation & strategies, insolvency predictor, loan risk assessment </li></ul><ul><li>Education (Online learning) </li></ul><ul><li>Internet (searching engine) </li></ul><ul><li>Etc. </li></ul>
  35. 35. Data Mining Methodology <ul><li>Hypothesis Testing vs Knowledge Discovery </li></ul><ul><ul><li>Hypothesis Testing </li></ul></ul><ul><ul><ul><li>Top down approach </li></ul></ul></ul><ul><ul><ul><li>Attempts to substantiate or disprove preconceived idea </li></ul></ul></ul><ul><ul><li>Knowledge Discovery </li></ul></ul><ul><ul><ul><li>Bottom-up approach </li></ul></ul></ul><ul><ul><ul><li>Start with data and tries to get it to tell us something we didn’t already know </li></ul></ul></ul>
  36. 36. Data Mining Methodology <ul><li>Hypothesis Testing </li></ul><ul><ul><li>Generate good ideas </li></ul></ul><ul><ul><li>Determine what data allow these hypotheses to be tested </li></ul></ul><ul><ul><li>Locate the data </li></ul></ul><ul><ul><li>Prepare the data for analysis </li></ul></ul><ul><ul><li>Build computer models based on the data </li></ul></ul><ul><ul><li>Evaluate computer model to confirm or reject hypotheses </li></ul></ul>
  37. 37. Data Mining Methodology <ul><li>Knowledge Discovery </li></ul><ul><ul><li>Directed </li></ul></ul><ul><ul><ul><li>Identified sources of pre classified data </li></ul></ul></ul><ul><ul><ul><li>Prepare data analysis </li></ul></ul></ul><ul><ul><ul><li>Select appropriated KD techniques based on data characteristics and data mining goal </li></ul></ul></ul><ul><ul><ul><li>Divide data into training, testing and evaluation </li></ul></ul></ul><ul><ul><ul><li>Use the training dataset to build model </li></ul></ul></ul><ul><ul><ul><li>Tune the model by applying it to test dataset </li></ul></ul></ul><ul><ul><ul><li>Take action based on data mining results </li></ul></ul></ul><ul><ul><ul><li>Measure the effect of the action taken </li></ul></ul></ul><ul><ul><ul><li>Restart the DM process taking advantage of new data generated by the action taken </li></ul></ul></ul>
  38. 38. Data Mining Methodology <ul><li>Knowledge Discovery </li></ul><ul><ul><li>Undirected </li></ul></ul><ul><ul><ul><li>Identified available data sources </li></ul></ul></ul><ul><ul><ul><li>Prepare data analysis </li></ul></ul></ul><ul><ul><ul><li>Select appropriated undirected KD techniques based on data characteristics and data mining goal </li></ul></ul></ul><ul><ul><ul><li>Use the selected technique to uncover hidden structure in the data </li></ul></ul></ul><ul><ul><ul><li>Identify potential targets for directed KD </li></ul></ul></ul><ul><ul><ul><li>Generate new hypothesis to test </li></ul></ul></ul>
  39. 39. Question for Group Discussion
  40. 40. Revision:: Two Approaches In data Mining Data Mining Predictive Descriptive <ul><li>Classification </li></ul><ul><li>Regression </li></ul><ul><li>Time Series Analysis </li></ul><ul><li>Prediction </li></ul><ul><li>Clustering </li></ul><ul><li>Summarization </li></ul><ul><li>Association Rules </li></ul><ul><li>Sequence Discovery </li></ul>Predict the future value Define R/S among data
  41. 41. Knowledge Discovery Process
  42. 42. Knowledge Discovery Process <ul><li>1.0 Selection </li></ul><ul><ul><li>The data needs for the data mining process may be obtained from many different and heterogeneous data sources </li></ul></ul><ul><ul><li>Examples </li></ul></ul><ul><ul><ul><li>Business Transactions </li></ul></ul></ul><ul><ul><ul><li>Scientific Data </li></ul></ul></ul><ul><ul><ul><li>Video and pictures </li></ul></ul></ul>
  43. 43. Knowledge Discovery Process <ul><li>2.0 Pre Processing </li></ul><ul><li>Main idea – to ensure that data is clean (high quality of data). </li></ul><ul><ul><li>The data to be used by the process may have incorrect or missing data. </li></ul></ul><ul><ul><li>There may be anomalous data from multiple sources involving different data types and metrics </li></ul></ul><ul><ul><li>Erroneous data may be corrected or removed, whereas missing data must be supplied or predicted (Often using data mining tools) </li></ul></ul>
  44. 44. Knowledge Discovery Process <ul><li>3.0 Transformation </li></ul><ul><ul><li>Data from different sources must be converted into a common format for processing </li></ul></ul><ul><ul><li>Some data may be encoded or transformed into more usable formats </li></ul></ul><ul><ul><li>Example:: </li></ul></ul><ul><ul><ul><li>Data Reduction Data Cleaning, Data Integration, Data Transformation, Data Reduction and Data Discretization </li></ul></ul></ul>
  45. 45. Knowledge Discovery Process <ul><li>4.0 Data Mining </li></ul><ul><li>Main idea –to use intelligent method to extract patterns and knowledge from database </li></ul><ul><li>This step applies algorithms to the transformed data to generate the desired results . </li></ul><ul><li>The heart of KD process (where unknown pattern will be revealed). </li></ul><ul><li>Example of algorithms: Regression (classification, prediction), Neural Networks (prediction, classification, clustering), Apriori Algorithms (association rules), K-Means & K-Nearest Neighbor (clustering), Decision Tree (classification), Instance Learning (classification). </li></ul>
  46. 46. Knowledge Discovery Process <ul><li>5.0 Interpretation/Evaluation </li></ul><ul><ul><li>How the data mining results are presented to the users is extremely important because the usefulness of the results is dependent on it </li></ul></ul><ul><ul><li>Example:: </li></ul></ul><ul><ul><ul><li>Graphical </li></ul></ul></ul><ul><ul><ul><li>Geometric </li></ul></ul></ul><ul><ul><ul><li>Icon Based </li></ul></ul></ul><ul><ul><ul><li>Pixel Based </li></ul></ul></ul><ul><ul><ul><li>Hierarchical Based </li></ul></ul></ul><ul><ul><ul><li>Hybrid </li></ul></ul></ul>
  47. 47. Case Study: Predicting FSK Final Year’s Student Performance activities Student database {contains 30,000 records} Academics academics Selected record {matric, PMK, grades} – only 2,000 records (contains incomplete records etc. Selection academics Clean record {replace the missing value, removed the replicated} Pre-processing Using neural networks : transform into numerical. Transformation Y=w1x1+w2x2+b1 Generated Model : pattern for performance prediction Data mining Testing result: 90 % correct  accept model Knowledge (apply model) Interpretation & evaluation