• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
By Dr. Borne 2005 UMUC Data Mining Lecture 1
 

By Dr. Borne 2005 UMUC Data Mining Lecture 1

on

  • 1,058 views

 

Statistics

Views

Total Views
1,058
Views on SlideShare
1,057
Embed Views
1

Actions

Likes
1
Downloads
31
Comments
0

1 Embed 1

http://www.slideshare.net 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    By Dr. Borne 2005 UMUC Data Mining Lecture 1 By Dr. Borne 2005 UMUC Data Mining Lecture 1 Presentation Transcript

    • Data Mining UMUC CSMN 667 Lecture #1
    • So what is it? Data Mining is “an information extraction activity whose goal is to discover hidden facts contained in large databases.”
    • Class Textbooks
      • Margaret Dunham’s book: “Data Mining Introductory and Advanced Topics”
        • from Prentice Hall
        • numerous publication dates listed (2002/2003)
        • there is only one edition (just buy it)
      • APA Style Guide: Publication manual of the American Psychological Association (2001, 5 th ed.) - required by UMUC
    • Additional Assignment for first month
      • Set up database account on our class database server: dbcourse3.umuc.edu
      • Refer to WebTycho for instructions:
        • Change your passwords immediately in 2 places : your Unix server account and your Oracle database account (both passwords are initially the same, but they are completely independent).
    • Reminders
      • The word “DATA” is plural. The singular form of the word is “datum” -- one datum is okay, but many data are better.
      • Time is what prevents everything from happening at once. So, please use good time management skills to keep from falling behind in your reading and other class assignments.
    • “ Data Mining 101” An Introduction to Data Mining Data mining is defined as “ an information extraction activity whose goal is to discover hidden facts contained in (large) databases ."
    • Evolution of Data Mining < http://www.thearling.com/text/dmwhite/dmwhite.htm >
    • Data Mining is Ready for Prime Time
      • Data mining is ready for general application because it engages three technologies that are now sufficiently mature:
            • Massive data collection & delivery
            • Powerful multiprocessor computers
            • Sophisticated data mining algorithms
    • 6 Business Reasons to use Data Mining
        • Most organizations already collect and refine massive quantities of data.
        • Their most important information is in their data warehouses.
        • Data mining moves beyond the analysis of past events … to predicting future trends and behaviors that may be missed because they lie outside the experts’ expectations.
        • Data mining tools can answer complex business questions that traditionally were too time-consuming to resolve.
        • Data mining tools can explore the intricate interdependencies within databases in order to discover hidden patterns and relationships.
        • Data mining allows decision-makers to make proactive, knowledge-driven decisions.
    • Another Business Reason to use Data Mining
    • A Key Concept for Data Mining
      • Data Mining delivers actionable data :
        • data that support decision-making
        • data that lead to knowledge and understanding
        • data with a purpose
      • i.e., Data do not exist for their own sake.
      • The Data Warehouse is a corporate asset (whether in business, marketing, banking, science, telecommunications, entertainment, computer security, or Homeland Security).
    • Data Mining - the up side
      • Data mining is everywhere:
        • Huge scientific databases (NASA, Human Genome,…)
        • Corporate databases (OLAP)
        • Credit card usage histories (Capital One)
        • Loan applications (Credit Scoring)
        • Customer purchase records (CRM)
        • Web traffic analysis (Doubleclick)
        • Network security intrusion detection (Silent Runner)
        • The hunt for terrorists (DARPA TIA)
        • The NBA! … the NBA??
    • Data Mining - the down side
      • Data mining is a pejorative in the business database community (“data dredging”)
        • They prefer to call it Knowledge Discovery, or Business Intelligence, or CRM (Customer Relationship Management), or Marketing, or OLAP (On-Line Analytical Processing)
      • The Data Mining Moratorium Act of 2003
        • see first page of the bill on next slide
        • debated within the U.S.Congress
        • privacy concerns
        • directly primarily against the DARPA TIA Program (Total Information Awareness)
    • 108TH CONGRESS 1ST SESSION S. __________ IN THE SENATE OF THE UNITED STATES Mr. FEINGOLD introduced the following bill; which was read twice and referred to the Committee on _________________ A BILL To impose a moratorium on the implementation of datamining under the Total Information Awareness program of the Department of Defense and any similar program of the Department of Homeland Security, and for other purposes. 1 Be it enacted by the Senate and House of Representa- 2 tives of the United States of America in Congress assembled, 3 SECTION 1. SHORT TITLE. 4 This Act may be cited as the ‘‘Data-Mining Morato- 5 rium Act of 2003’’. 6 SEC. 2. FINDINGS. 7 Congress makes the following findings: http://www.cdt.org/legislation/108th/privacy/030122feingold.pdf
    • The Information Age is Here!
      • &quot;Data doubles about every year, but useful information seems to be decreasing.&quot;
        • Margaret Dunham, &quot;Data Mining Techniques & Algorithms&quot;, 2002
      • &quot;There is a growing gap between the generation of data and our understanding of it.&quot;
        • Witten & Frank, &quot;Data Mining: Practical Machine Learning Tools&quot;, 1999
      • &quot;The trouble with facts is that there are so many of them&quot;
        • Samuel McChord Crothers, &quot;The Gentle Reader&quot;, 1973
      • &quot;Get your facts first, and then you can distort them as much as you please.&quot;
        • Mark Twain
    • Characteristics of The Information Age:
      • Data “Avalanche”
        • the flood of Terabytes of data is already happening, whether we like it or not
        • our present techniques of handling these data do not scale well with data volume
      • Distributed Digital Archives
        • will be the main access to data
        • will need to handle hundreds to thousands of queries per day
      • Systematic Data Exploration and Data Mining
        • will have a central role
          • statistical analysis of “typical” events
          • automated search for “rare” events
    • The Data Flood is Everywhere
      • Huge quantities of data are being generated in all business, government, and research domains:
        • Banking, retail, marketing, telecommunications, other business transactions ...
        • Scientific data: genomics, astronomy, biology, etc.
        • Web, text, and e-commerce
    • 5 million terabytes created in 2002
      • UC Berkeley 2003 estimate :
      • 5 exabytes (5 million terabytes) of new data were created in 2002.
      • http://www.sims.berkeley.edu/research/projects/how-much-info-2003/
      • What is a gigabyte, terabyte, petabyte, exabyte, …?
        • Look at the definitions and examples in the following article:
        • http://www.jamesshuggins.com/h/tek1/how_big.htm
    • Data Growth Rate
      • Twice as much information was created in 2002 as in 1999 (~30% annual growth rate).
      • Other growth rate estimates are even higher.
      • Very little of these data will ever be looked at by a human.
      • Data Mining is NEEDED to make sense of and to make use of these data.
    • What is Data Mining?
      • Data mining is defined as “ an information extraction activity whose goal is to discover hidden facts contained in (large) databases .&quot;
      • Data mining is used to find patterns and relationships in data. (EDA = Exploratory Data Analysis)
      • Patterns can be analyzed via 2 types of models:
        • Descriptive : Describe patterns and create meaningful subgroups or clusters.
        • Predictive : Forecast explicit values, based upon patterns in known results.
      • How does this become useful (not just bits of data)? ...
        • … through KNOWLEDGE DISCOVERY
      • Data  Information  Knowledge  Understanding / Wisdom!
    • Historical Note: Many Names of Data Mining
      • Data Fishing, Data Dredging: 1960-
        • used by Statisticians (as a bad name)
      • Data Mining :1990-
        • used by DB & business communities
        • in 2003 – bad image because of DARPA TIA
      • Knowledge Discovery in Databases (1989-)
        • used by AI & Machine Learning communities
      • also Data Archaeology, Information Harvesting, Information Discovery, Knowledge Extraction, ...
      Currently: Data Mining and Knowledge Discovery are used interchangeably.
    • Data Mining Examples
      • Classic Textbook Example of Data Mining (Legend?) : Data mining of grocery store logs indicated that men who buy diapers also tend to buy beer at the same time.
      • Blockbuster Entertainment mines its video rental history database to recommend rentals to individual customers.
      • A financial institution discovered that credit applicants who used pencil on the form were much more likely to default on their debts than those who filled out the application using ink.
      • Credit card companies recommend products to cardholders based on analysis of their monthly expenditures.
      • Airline purchase transaction logs revealed that 9-11 hijackers bought one-way airline tickets with the same credit card.
      • Astronomers examined objects with extreme colors in a huge database to discover the most distant Quasars ever seen.
    •  
    • Data Mining Application: Marketing
      • Sales Analysis
        • associations between product sales:
          • beer and diapers
          • strawberry pop tarts and beer (and hurricanes)
      • Customer Profiling
        • data mining can tell you what types of customers buy what products
      • Identifying Customer Requirements
        • identify the best products for different customers
        • use prediction to find what factors will attract new customers
      • Auto Insurance Fraud
        • Association Rule Mining can detect a group of people who stage accidents to collect on insurance
      • Money Laundering
        • Since 1993, the US Treasury's Financial Crimes Enforcement Network agency has used a data-mining application to detect suspicious money transactions
      • Banking: Loan Fraud
        • Security Pacific/Bank of America uses data mining to help with commercial lending decisions and to prevent fraud
      Data Mining Application: Fraud Detection
    • The Necessity of Data Mining
      • Enormous interest in these data collections.
      • The environment to exploit these data does not exist!
        • 1 Terabyte at 100 Mbits/sec takes 1 day to transfer.
        • Hundreds to thousands of queries per day.
        • Data will reside at multiple locations, in many different formats.
        • Existing analysis tools do not scale to Terabyte data collections.
      • The need is acute! A solution will not just happen.
    • What is Knowledge Discovery?
      • Knowledge discovery refers to “ finding out new knowledge about an application domain using data on the domain usually stored in a database .”
        • Application domains: scientific, customer purchase records, computer network logs, web traffic logs, financial transactions, census data, basketball play-by-play histories, ...
      • Why are Data Mining & Knowledge Discovery such hot topics? --- because of the enormous interest in these huge databases and their potential for new discoveries .
      • In large databases, Data Mining and Knowledge Discovery come in two flavors :
        • Event-based mining
        • Relationship-based mining
    • Event-Based Mining
        • (Event-based mining is based upon events or trends in data.)
      • Four distinct orthogonal categorizations:
      • Known events / known models - use existing models ( descriptive models ) to locate known phenomena of interest either spatially or temporally within a large database.
      • Known events / unknown models - use clustering properties of data to discover new relationships and patterns among known phenomena.
      • Unknown events / known models - use known associations and relationships ( predictive models ) among parameters that describe a phenomenon to predict the presence of previously unseen examples of the same phenomenon within a large complex database.
      • Unknown events / unknown models - use thresholds or trends to identify transient or otherwise unique (&quot;one-of-a-kind&quot;) events and therefore to discover new phenomena.  Serendipity !
    • Relationship-Based Data Mining (Based upon associations & relationships among data items)
      • Spatial associations -- identify events or objects at the same physical spatial location, or at related locations (e.g., urban versus rural data).
      • Temporal associations -- identify events or transactions occurring during the same or related periods of time (e.g., periodically, or N days after event X).
      • Coincidence associations -- use clustering techniques to identify events that are co-located (that coincide) within a multi-dimensional parameter space.
    • Event-Based Mining (EBM) - Homeland Security Example
      • (EBM is based upon events or trends in data.)
      • Known events / known models - use existing models ( descriptive models ) to locate known phenomena of interest within a large database.
        • e.g., Terrorist activities have been financed through certain organizations. Search for similar transactions in large financial databases.
      • Known events / unknown models - use clustering properties of data to discover new relationships and patterns among known phenomena.
        • e.g., Search through credit card, travel, and phone histories of 9-11 hijackers to discover previously unknown characteristics and behavior patterns of terrorists.
      • Unknown events / known models - use known associations and relationships ( predictive models ) among parameters that describe a phenomenon to predict the presence of previously unseen examples within a large complex database.
        • e.g., Use knowledge of terrorist behavior patterns (e.g, heightened phone activity) to identify new terrorists and/or to raise new terrorist alerts.
      • Unknown events / unknown models - use thresholds or trends or outlier detection to identify transient or otherwise unique (&quot;one-of-a-kind&quot;) events, and therefore to discover new phenomena.
        • e.g., Explore all known data (including intelligence, news reports, e-mail, credit card histories, phone records, organizational memberships) to identify new threats.
    • Relationship-Based Mining (RBM) - Homeland Security Example
      • (RBM is based upon associations and relationships among data items.)
      • Spatial associations -- identify events ( e.g , airline ticket purchases ) occurring at the same location in some geospatial parameter space ( e.g , travel on the same flights ).
      • Temporal associations -- identify events occurring during the same or related periods of time ( e.g , airline ticket purchases for travel on the same flights purchased at the same time ).
      • Coincidence associations -- use clustering techniques to identify events that are co-located within a multi-dimensional parameter space ( e.g , airline tickets for the same flights purchased at the same time as one-way tickets on the same credit card, with travelers of Mid-East origin, having recent U.S. entry, were students in flight schools, having records of numerous phone calls to Afghanistan, and having visited Hamburg Germany at some time in the past few years ).
    • User Requirements for a Data Mining System (What features must a D.M. system have for your users?)
      • Cross-Identification - refers to the classical problem of associating the objects listed in one database to the objects listed in another.
      • Cross-Correlation - refers to the search for correlations, tendencies, and trends between parameters in multi-dimensional data, usually across databases.
      • Nearest-Neighbor Identification - refers to the general application of clustering algorithms in multi-dimensional parameter space, usually within a single database.
      • Systematic Data Exploration - refers to the application of the broad range of event-based and relationship-based queries to one or more databases in the hope of making a serendipitous discovery of new events/objects or a new class of events/objects.
    • Representative Data Mining Architecture < http://www.thearling.com/text/dmwhite/dmwhite.htm >
    • Data leads to Knowledge leads to Understanding
      • Remember what we said earlier :
      • EXAMPLE :
      • Data = 00100100111010100111100 (stored in database)
      • Information = ages and heights of children ( metadata )
      • Knowledge = the older children tend to be taller
      • Understanding = children’s bones grow as they get older
      Data  Information  Knowledge  Understanding / Wisdom!
    • Astronomy Example Data:
      • Information (catalogs / databases):
        • Measure brightness of galaxies from image (e.g., 14.2 or 21.7)
        • Measure redshift of galaxies from spectrum (e.g., 0.0167 or 0.346)
      • Knowledge:
        • Hubble Diagram 
        • Redshift-Brightness Correlation 
        • Redshift = Distance
      Understanding: the Universe is expanding!! (a) Imaging data (ones & zeroes) (b) Spectral data (ones & zeroes)
    • Goal of Data Mining
      • The end goal of data mining is not the data themselves, but the new knowledge and understanding that are revealed in the process = Business Intelligence (BI).
        • (Remember what we said about the business community’s opinion of D.M.)
      • This is why the research field is usually referred to as KDD = Knowledge Discovery in Databases.
    • Some words of wisdom
      • &quot;We have confused information (of which there is too much) with ideas (of which there are too few).&quot;
        • Paul Theroux
      • &quot;The great Information Age is really an explosion of non-information; it is an explosion of data ... it is imperative to distinguish between the two; information is that which leads to understanding.&quot;
        • R.S. Wurman in his book: Information Anxiety2
    • The Data Mining Process (more about this later) The most important and time-consuming step is Cleaning the Data.
    • Clustering Classification Associations Neural Nets Decision Trees Pattern Recognition Correlation/Trend Analysis Principal Component Analysis Regression Analysis Outlier/Glitch Identification Visualization Autonomous Agents Self-Organizing Maps (SOM) Link (Affinity) Analysis Data Mining Methods and Some Examples Classify new data items using the known classes & groups Find associations and patterns among different data items Organize information in the database based on relationships among key data descriptors Identify linkages between data items based on features shared in common Find all groups and classes of objects represented in the data
    • Some Data Mining Techniques Graphically Represented Self-Organizing Map (SOM) Outlier (Anomaly) Dectection Clustering Link Analysis Decision Tree Neural Network
    • Remember what it is … Data Mining is “an information extraction activity whose goal is to discover hidden facts contained in large databases.”
    • Data Mining Technique: Clustering In this case, three different groups (classes) of items were found among all of the items in the data set.
    • Data Mining Technique: Decision Tree Classification
      • Question:
      • Should I play tennis today?
      (I must really love tennis!) Similar to game “20 questions” Same technique used by bank loan officers to identify good potential customers versus poor customers.
    • Data Mining Technique: Association Rule Mining (Market Basket Analysis) transaction id customer id products bought sales records:
      • Trend (Rule): Products p5, p8 often bought together
      • Trend (Rule): Customer 12 likes product p9
    • Data Mining Algorithm: The SOM Figure : The SOM (Self-Organizing Map) is one technique for organizing information in a database based upon links between concepts. It can be used to find hidden relationships and patterns in more complex data collections, usually based on links between keywords or metadata.
    • Data Mining Application: Outlier Detection Figure : The clustering of data clouds (dc#) within a multidimensional parameter space (p#). Such a mapping can be used to search for and identify clusters, voids, outliers, one-of-kinds, relationships, and associations among arbitrary parameters in a database (or among various parameters in geographically distributed databases).
    • Link Analysis for Homeland Security: Find all connections and relationships among known terrorists.
    • Data Mining Technology: Parallel Mining Figure : Parallel Data Mining The application of parallel computing resources and parallel data access (e.g., RAID) enables concurrent drill-downs into large data collections
    • Data Mining Methods Explained
      • Clustering: Group data items according to tight relationships.
      • Classification: Assign data items to predetermined groups.
      • Associations: Associate data with similar relationships. The beer-diaper example is an example of associative mining.
      • Artificial Neural Networks (ANN): Non-linear predictive models that learn through training and resemble biological neural networks in structure.
      • Decision Trees: Hierarchical sets of decisions, based upon rules, for rapid classification of a data collection.
      • Sequential Patterns: Identify or predict behavior patterns or trends.
      • Genetic Algorithms: Rapid optimization techniques that are based on the concepts of natural evolution.
      • Nearest Neighbor Method: Classify a data item according to its nearest neighbors (records that are most similar).
      • Rule induction: The extraction of useful if-then rules from data based on statistical significance.
      • Data visualization: The illustration and visual interpretation of complex relationships in multidimensional data using graphics tools.
      • Self-Organizing Map (SOM): Graphically organizes (in a 2-dimensional map) the information stored within a database based upon similarities and links between concepts. It can be used to find hidden relationships and patterns in more complex data collections.
    • Data Mining Techniques: techniques are based on Algorithms; techniques are used in Applications
    • http://www.kdnuggets.com/polls/2002/data_mining_techniques.htm Poll of Users: Data Mining Techniques (October 2002) “ Which data mining techniques do you use regularly? (Choose several)” [825 votes total] Decision Trees/Rules (128) …. 16% Clustering (103) …...………….. 12% Statistics (101) …….………….. 12% Logistic Regression (75) ….….. 9% Neural Networks (75) …….…… 9% Association Rules (63) ………... 8% Visualization (52) ………………. 6% Nearest Neighbor (42) …………. 5% Text Mining (30) ………………... 4% Sequence Analysis (27) ….…….. 3% Genetic Algorithms (26) ……..… 3% Bayesian Nets (24) ………..…… 3% Hybrid methods (21) ………...… 3% Web mining (19) ……………..… 2% Naïve Bayes (19) ...……….…….. 2% Other (20) ……………………..… 2%
    • http://www.kdnuggets.com/polls/2003/data_mining_techniques.htm Poll of Users: Data Mining Techniques ( November 2003 ) “ Which data mining techniques do you use regularly? (Choose several)” [768 votes total] Decision Trees/Rules (120) …... 16% Clustering (93) …...…………….. 12% Statistics (92) …….…………….. 12% Neural Networks (71) …….……. 9% Logistic Regression (69) ….…... 9% Visualization (55) ………………. 7% Association Rules (42) ………... 5% Nearest Neighbor (38) …………. 5% Text Mining (30) ………………... 4% Web Mining (29) ……………..… 4% Sequence Analysis (24) ….…….. 3% Bayesian Nets (24) ………..…… 3% Support Vector Machines (24) ... 3% Hybrid methods (23) ………...… 3% Genetic Algorithms (12) ……..… 2% Other (22) ……………………..… 3%
    • http://www.kdnuggets.com/polls/data_mining_tools_2002_june2.htm Poll of Users: Data Mining Tools (June 2002) [967 votes total] SPSS Clementine (128) ……….…….. 13% Weka (101) …………………….……... 10% SAS (100) …………….………………. 10% CART/MARS (89) ….…………………. 9% SPSS/AnswerTree (76) …………….... 8% SAS Enterprise Miner (67) ……….…. 7% Other commercial tools (65) …….….. 7% Other free/open-source tools (57) ….. 6% MATLAB (52) …………………………. 5% Microsoft SQLServer/Excel (40) ……. 4% Insightful Miner (36) …………………. 4% IBM Intelligent Miner (35) …………... 4% KXEN (35) ……………………………. 4% C4.5 / C4.8 (29) …………………….... 3% Angoss (26) ……………………….….. 3% Megaputer Polyanalyst (10) ……….... 1% Neuralware (8) ………………….……. 1% Oracle Suite (Darwin) (8) ……………. 1% Quadstone (3) ………..…………….. 0.3% ThinkAnalytics (2) …..……………... 0.2% “ Which tools do you use?”
    • http://www.kdnuggets.com/polls/2003/data_mining_tools.htm Poll of Users: Data Mining Tools ( May 2003 ) [1252 votes total] SPSS Clementine (176) ……….…….. 14% SPSS/AnswerTree (110) ……………… 9% SAS (102) …………….………………… 8% Excel (92) ………………………………. 7% Your own code (87) …………………… 7% CART/MARS (76) ….………………….. 6% SAS Enterprise Miner (76) ……….…… 6% Other commercial tools (51) …….……. 4% Microsoft SQLServer (50) …………….. 4% Other free/open-source tools (49) ……. 4% Prudsys Xelopes (46) ……………......... 4% Weka (44) ……………………………….. 4% Insightful Miner (38) …………………. 3% R (37) ……………………................... 3% C4.5 / C5 (36) ………………………… 3% MATLAB (32) …………………………. 3% IBM Intelligent Miner (22) …………... 2% Oracle Suite (Darwin) (19) ………….. 2% Angoss (17) ……………………….….. 1% Megaputer Polyanalyst (12) ……….... 1% Statsoft Statistica (10) ……………….. 1% Unica(7), KXEN(4), Neuralware(4), …1% “ Which tools do you use?”
    • http://www.kdnuggets.com/polls/2002/current_application_fields.htm Poll of Users: Where do you currently apply data mining? (June 2002) “ Industries/fields where you currently apply data mining?” [608 votes total] Banking (77) ……………….………. 13% Telecommunications (56) .……..….. 9% eCommerce/Web (53) ……………... 9% Scientific data (51) ………………..... 8% Fraud Detection (51) …………..…… 8% Direct Marketing/Fundraising (42) … 7% Insurance (36)……………………….. 6% Retail (36) ..………………………….. 6% Biology/Genetics/Proteomics (32) ... 5% Pharmaceuticals (31) ………………. 5% Manufacturing (28) …………………. 5% Supply Chain Analysis (21) ……….. 3% Investment/Stocks (17) ……………. 3% Security (14) ………………………... 2% Entertainment (10) …………………. 2% Other (44) …………………………... 7% None (9) ……………………………. 1%
    • http://www.kdnuggets.com/polls/2004/data_mining_applications_industries.htm Banking (29) ………………………... 13% Scientific data (20) …………………... 9% Direct Marketing/Fundraising (19) …. 9% Fraud Detection (19) ………………… 9% Bioinformatics/Biotech (18) …………. 8% Insurance (15) ………………………... 7% Medical/Pharma (15) ………………… 7% Telecommunications (12) …………… 6% eCommerce/Web (12) ………………. 6% Investment/Stocks (9) ……………….. 4% Manufacturing (9) ……………………. 4% Retail (9) ……………………………… 4% Security (8) …………………………… 4% Travel (2) ……………………………... 1% Entertainment/News (1) ………………0.5% Other (19) ……………………………... 9% Poll of Users: Where do you currently apply data mining? ( August 2004 ) “ Industries/fields where you currently apply data mining?” [216 votes total]
    • Data Mining 101 - Summary
      • What? -- Data Mining is defined as &quot;an information extraction activity whose goal is to discover hidden facts contained in (large) databases.&quot;
      • Why? -- To explore systematically and to make discoveries in huge databases.
      • How? -- Apply one of many techniques to find patterns, relationships, groupings, classes, trends, anomalies, rare events, unusual connections, and causal connections among items in a database.
      • Example -- The standard textbook example of data mining is the legendary trend found in grocery store logs: that men who buy diapers also tend to buy beer at the same time.
      • Outcome -- “Actionable information” = make decisions based upon information discovered.
      • What is needed -- “SIFTWARE” = software that aids in isolating interesting useful information by sifting through large databases.
      • Real world application -- Data  Information  Knowledge  Understanding / Wisdom!
    • &quot;It will work in practice, yes. But will it work in theory?&quot; - Jonathan Fenby, France on the Brink