Hw09 Large Scale Transaction Analysis


Published on

Published in: Economy & Finance, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Fast – On average, VisaNet fully processes transactions in approximately 1 second. Secure – VisaNet supports multiple defense layers to prevent network intrusions, protect against account fraud, and to make stolen card data unusable for criminalsincluding data encryption, sophisticated intrusion detection systems and neural network technology capable of analyzing each transaction in real-time. Over the last two years, Visa has spent more than $100 million on data security enhancements. Reliable – VisaNet runs multiple, parallel processing systems in highly redundant data centers. This ensures near 100 percent availability. In the past 15 years, VisaNet experienced less than two seconds average unavailability per year. State-of-the-art monitoring tools are in place to detect network availability and client-specific processing anomalies, such as high rates of decline. Scalable – VisaNet authorizes an average of 130 million transactions a day, totaling more than 92 billion authorization. On Dec. 21, 2008, Visa’s busiest day last year, VisaNet authorized more than 200 million transactions. Based on recent stress tests, VisaNet is capable of authorizing over a half billion transactions per day.
  • Hw09 Large Scale Transaction Analysis

    1. 1. Hadoop@Visa Joe Cunningham Technology Strategy and Innovation October 2, 2009
    2. 2. Agenda 1 Visa 2 Value-Added Information Products 3 Hadoop@Visa – research results VisaNet – an overview
    3. 3. Keeping the Brand Promise Zero Liability go Security Flexibility
    4. 4. Keeping the Brand Promise 3 Global Acceptance Reliability Zero Liability go Security Flexibility Guaranteed Payment
    5. 5. VisaNet Today <ul><li>VisaNet is the world’s largest, most advanced payments network. </li></ul>28 million acceptance locations 130 million authorizations per day 1,600 endpoints Processes transactions in 1 second 16,300 financial institutions Less than 2 seconds unavailability per year Processes in 175 currencies 1.4 million ATMs fast secure flexible reliable scalable
    6. 6. 1 VisaNet – an overview 2 3 Value-Added Information Products Hadoop@Visa – research results Agenda
    7. 7. Visa Processing Architecture Security/Access Services Message File Web VisaNet Services Integration Common Data Infrastructure Authorization Clearing & Settlement Information Disputes Risk
    8. 8. Value-Added Information Products Information Services <ul><li>Client </li></ul><ul><li>Portfolio Analysis </li></ul><ul><li>Visa Incentive Network </li></ul><ul><li>Accountholder </li></ul><ul><li>Transaction Alerts </li></ul><ul><li>Account Updater </li></ul><ul><li>Tailored Rewards </li></ul>Risk Management Services <ul><li>Authentication </li></ul><ul><li>Advanced Auth </li></ul><ul><li>Verified by Visa </li></ul><ul><li>Address Verification </li></ul><ul><li>Account Monitoring </li></ul><ul><li>Fraud Detection </li></ul><ul><li>Compromised Accounts </li></ul><ul><li>Consumer Alerts </li></ul><ul><li>Encryption </li></ul><ul><li>Dynamic Key Exchange </li></ul><ul><li>Dynamic Cryptograms </li></ul>
    9. 9. 1 VisaNet – an overview 2 Value-Added Information Products 3 Hadoop@Visa – research results Agenda
    10. 10. Research Lab Setup Hadoop Systems VM System Relational Database Custom Analytic Stacks Encryption Processing Management Stack Hadoop #2 ~300Tb / 28 nodes Hadoop #1 ~40Tb / 42 node
    11. 11. Risk Product Use Case Create critical data model elements, such as keys and transaction statistics, which feed our real-time risk-scoring systems Hadoop Cluster <ul><li>Research Sample (Synthetic Transactions) </li></ul><ul><li>500 million distinct accounts </li></ul><ul><li>100 million transactions per day </li></ul><ul><li>200 bytes per transaction </li></ul><ul><li>2 years: ~ 73 billion transactions, ~ 36 TB </li></ul><ul><li>One Month  13 minutes! </li></ul>Input: Transactions Merchant Category Country / Zip Output: Key & Statistics MCCZIP Key Statistics related to account, transaction type, approval, fraud, IP address, tx. counts
    12. 12. Financial Enterprise Fit Some key questions we are researching… <ul><li>What will the Hadoop Solution Stack(s) look like? </li></ul><ul><ul><li>File System </li></ul></ul><ul><ul><li>Transaction Sample System </li></ul></ul><ul><ul><li>Relational Back-End </li></ul></ul><ul><ul><li>Analytics Processing </li></ul></ul><ul><li>Internal vs. External cloud </li></ul><ul><ul><li>How do I get data into a cloud in a secure way? </li></ul></ul><ul><li>How does HSM and security integration work in Hadoop? </li></ul><ul><ul><li>What are the missing pieces? </li></ul></ul>
    13. 13. Why Hadoop@Visa? <ul><li>For our information products, we see promise in combining Visa’s data analytics capability with the power of Hadoop </li></ul><ul><li>Analyze volumes of data with response times that are not possible today… </li></ul><ul><ul><li>… and apply analytic models to individual client, not just client segment </li></ul></ul><ul><li>But, need to answer questions relating to stack configuration and support… </li></ul><ul><ul><li>… and, eventually must fit with our financial enterprise paradigm: </li></ul></ul>fast secure flexible reliable scalable
    14. 14. Keeping the Brand Promise Zero Liability go Security Flexibility
    15. 15. Keeping the Brand Promise 3 Global Acceptance Reliability Zero Liability go Security Flexibility Guaranteed Payment