Data mining and privacy preserving in data mining


Published on

Published in: Education, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • The US Internal Revenue Service is using data mining to improve customer service. [Click] By analyzing incoming requests for help and information, the IRS hopes to schedule its workforce to provide faster, more accurate answers to questions.
  • The US DFAS needs to search through 2.5 million financial transactions that may indicate inaccurate charges. Instead of relying on tips to point out fraud, the DFAS is mining the data to identify suspicious transactions. [Click] Using Clementine, the agency examined credit card transactions and was able to identify purchases that did not match past patterns. Using this information, DFAS could focus investigations, finding fraud more costs effectively.
  • Retail banking is a highly competitive business. In addition to competition from other banks, banks also see intense competition from financial services companies of all kinds, from stockbrokers to mortgage companies. With so many organizations working the same customer base, the value of customer retention is greater than ever before. As a result, HSBC Bank USA looks to enticing existing customers to "roll over" maturing products, or on cross-selling new ones. [Click] Using SPSS products, HSBC found that it could reduce direct mail costs by 30% while still bringing in 95% of the campaign’s revenue. Because HSBC is sending out fewer mail pieces, customers are likely to be more loyal because they don’t receive junk mail from the bank.
  • Data mining and privacy preserving in data mining

    1. 1. Data mining is…… <ul><li>Data mining is the process of analyzing data from different perspectives and summarizing it into useful information - information that can be used to increase revenue, cuts costs, or both. </li></ul><ul><li>A user-centric, interactive process which leverages analysis technologies and computing power </li></ul><ul><li>A group of techniques that find relationships that have not previously been discovered </li></ul><ul><li>Not reliant on an existing database </li></ul><ul><li>A relatively easy task that requires knowledge of the business problem/subject matter expertise </li></ul>
    2. 2. Data mining is not <ul><li>“ Blind” application of algorithms </li></ul><ul><li>Going to find relationships where none exist </li></ul><ul><li>Presenting data in different ways </li></ul><ul><li>A database intensive task </li></ul><ul><li>A difficult to understand technology requiring an advanced degree in computer science </li></ul>
    3. 3. The Evolution of Data Analysis
    4. 4. Results for data mining……. <ul><li>Forecasting what may happen in the future </li></ul><ul><li>Classifying people or things into groups by recognizing patterns </li></ul><ul><li>Clustering people or things into groups based on their attributes </li></ul><ul><li>Associating what events are likely to occur together </li></ul><ul><li>Sequencing what events are likely to lead to later events </li></ul>
    5. 5. Why Should There be a Standard Process? <ul><li>Framework for recording experience </li></ul><ul><ul><li>Allows projects to be replicated </li></ul></ul><ul><li>Aid to project planning and management </li></ul><ul><li>“ Comfort factor” for new adopters </li></ul><ul><ul><li>Demonstrates maturity of Data Mining </li></ul></ul><ul><ul><li>Reduces dependency on “stars” </li></ul></ul>The data mining process must be reliable and repeatable by people with little data mining background.
    6. 6. Data mining vs OLAP <ul><li>OLAP - On-line Analytical Processing </li></ul><ul><ul><li>Provides you with a very good view of what is happening, but can not predict what will happen in the future or why it is happening </li></ul></ul>
    7. 7. Data mining vs statistical analysis <ul><li>Data Mining </li></ul><ul><ul><li>Originally developed to act as expert systems to solve problems </li></ul></ul><ul><ul><li>Less interested in the mechanics of the technique </li></ul></ul><ul><ul><li>If it makes sense then let’s use it </li></ul></ul><ul><ul><li>Does not require assumptions to be made about data </li></ul></ul><ul><ul><li>Can find patterns in very large amounts of data </li></ul></ul><ul><ul><li>Requires understanding of data and business problem </li></ul></ul><ul><li>Data Analysis </li></ul><ul><ul><li>Tests for statistical correctness of models </li></ul></ul><ul><ul><ul><li>Are statistical assumptions of models correct? </li></ul></ul></ul><ul><ul><ul><ul><li>Eg Is the R-Square good? </li></ul></ul></ul></ul><ul><ul><li>Hypothesis testing </li></ul></ul><ul><ul><ul><li>Is the relationship significant? </li></ul></ul></ul><ul><ul><ul><ul><li>Use a t-test to validate significance </li></ul></ul></ul></ul><ul><ul><li>Tends to rely on sampling </li></ul></ul><ul><ul><li>Techniques are not optimised for large amounts of data </li></ul></ul><ul><ul><li>Requires strong statistical skills </li></ul></ul>
    8. 8. Examples of what people are doing with data mining <ul><li>Fraud/Non-Compliance Anomaly detection </li></ul><ul><ul><li>Isolate the factors that lead to fraud, waste and abuse </li></ul></ul><ul><ul><li>Target auditing and investigative efforts more effectively </li></ul></ul><ul><li>Credit/Risk Scoring </li></ul><ul><li>Intrusion detection </li></ul><ul><li>Parts failure prediction </li></ul><ul><li>Recruiting/Attracting customers </li></ul><ul><li>Maximizing profitability (cross selling, identifying profitable customers) </li></ul><ul><li>Service Delivery and Customer Retention </li></ul><ul><ul><li>Build profiles of customers likely to use which services </li></ul></ul><ul><li>Web Mining </li></ul>
    9. 9. Why Should There be a Standard Process? <ul><li>Framework for recording experience </li></ul><ul><ul><li>Allows projects to be replicated </li></ul></ul><ul><li>Aid to project planning and management </li></ul><ul><li>“ Comfort factor” for new adopters </li></ul><ul><ul><li>Demonstrates maturity of Data Mining </li></ul></ul><ul><ul><li>Reduces dependency on “stars” </li></ul></ul>The data mining process must be reliable and repeatable by people with little data mining background.
    10. 10. <ul><li>Data mining is extensively used for knowledge discovery from large databases. </li></ul><ul><li>The problem with data mining is that with the availability of non-sensitive information, one is able to infer sensitive information that is not to be disclosed. </li></ul><ul><li>Thus privacy is becoming an increasingly important issue in many data mining applications. </li></ul><ul><li>This has led to the development of privacy preserving data mining. </li></ul>What is Data Mining and Privacy Preservation all about:
    11. 11. <ul><li>Perform Data-mining on union of two private databases </li></ul><ul><li>Data stays private i.e. no party learns anything but output </li></ul><ul><ul><li>meeting privacy requirements. Study of achieving some data mining goals without scarifying the privacy of the individuals </li></ul></ul><ul><li>providing valid data mining results. </li></ul>Objective
    12. 12. How Do We Do It? <ul><li>There are two approaches to preserve privacy: </li></ul><ul><li>The first approach protects the privacy of the data by using an extended role based access control approach where sensitive objects identification is used to protect an individual’s privacy. </li></ul><ul><li>The second approach uses cryptographic techniques. </li></ul>
    13. 13. Cryptographic techniques for PPDM <ul><li>To run the data mining algorithm on the union of their databases without revealing any unnecessary information. </li></ul><ul><li>consider separate medical institutions that wish to conduct a joint research while preserving the privacy of their patients. </li></ul><ul><li>Protection of privileged information, along with its use for research. </li></ul>
    14. 14. How much privacy? <ul><li>if a data mining algorithm is run against the union of the databases, and its output becomes known to one or more of the parties, it reveals something about the contents of the other databases. </li></ul><ul><li>leak of information is inevitable, however, if the parties need to learn this output. </li></ul>
    15. 15. What is Cryptography? <ul><li>The common definition of privacy in the cryptographic community limits the information that is leaked by the distributed computation to be the information that can be learned from the designated output of the computation. </li></ul>
    16. 16. Specific data mining:Applications
    17. 17. What data mining has done for….. <ul><li>The US Internal Revenue Service needed to improve customer service and... </li></ul><ul><li>Scheduled its workforce to provide faster, more accurate answers to questions </li></ul>
    18. 18. What data mining has done for………… <ul><li>The US Drug Enforcement Agency needed to be more effective in their drug “busts” and </li></ul><ul><li>analyzed suspects’ cell phone usage to focus investigations. </li></ul>
    19. 19. What data mining has done for…….. <ul><li>HSBC need to cross-sell more </li></ul><ul><li>effectively by identifying profiles </li></ul><ul><li>that would be interested in higher </li></ul><ul><li>yielding investments and... </li></ul><ul><li>Reduced direct mail costs by 30% while garnering 95% of the campaign’s revenue. </li></ul>
    20. 20. Final comments <ul><li>Data Mining can be utilized in any organization that needs to find patterns or relationships in their data. </li></ul><ul><li>By using the CRISP-DM methodology, analysts can have a reasonable level of assurance that their Data Mining efforts will render useful, repeatable, and valid results. </li></ul>
    21. 21. 5 dimensions of PPDM <ul><li>(1) the distribution of the </li></ul><ul><li>basic data, </li></ul><ul><li>(2) how basic data are modified </li></ul><ul><li>(3) which mining method is being used </li></ul><ul><li>(4) If basic data or rules are to be hidden and </li></ul><ul><li>(5) which additional methods for privacy preservation are used </li></ul>
    22. 22. Privacy guidelines <ul><li>1. Collection limitation principle – too general to be enforced in PPDM </li></ul><ul><li>2. Data quality principle – most of today’s PPDM methods or algorithms assume that </li></ul><ul><li>data are already prepared to an appropriate quality to be mined </li></ul><ul><li>3. Purpose specification principle – extremely relevant for PPDM </li></ul>
    23. 23. <ul><li>4. Use limitation principle – fundamental for PPDM </li></ul><ul><li>5. Security safeguard principle – unenforceable in the context of PPDM </li></ul><ul><li>6. Openness principle – relevant for PPDM </li></ul>
    24. 24. <ul><li>7. Individual participation principle - Oliveira and Zaïane suggest that the implications of this principle for PPDM should be carefully weighed in light of the ownership of the basic data otherwise the application could be too rigid in PPDM applications. </li></ul><ul><li>8. Accountability principle – too general for PPDM </li></ul>