Fast – On average, VisaNet fully processes transactions in approximately 1 second. Secure – VisaNet supports multiple defense layers to prevent network intrusions, protect against account fraud, and to make stolen card data unusable for criminalsincluding data encryption, sophisticated intrusion detection systems and neural network technology capable of analyzing each transaction in real-time. Over the last two years, Visa has spent more than $100 million on data security enhancements. Reliable – VisaNet runs multiple, parallel processing systems in highly redundant data centers. This ensures near 100 percent availability. In the past 15 years, VisaNet experienced less than two seconds average unavailability per year. State-of-the-art monitoring tools are in place to detect network availability and client-specific processing anomalies, such as high rates of decline. Scalable – VisaNet authorizes an average of 130 million transactions a day, totaling more than 92 billion authorization. On Dec. 21, 2008, Visa’s busiest day last year, VisaNet authorized more than 200 million transactions. Based on recent stress tests, VisaNet is capable of authorizing over a half billion transactions per day.
Hadoop@Visa Joe Cunningham Technology Strategy and Innovation October 2, 2009
Agenda 1 Visa 2 Value-Added Information Products 3 Hadoop@Visa – research results VisaNet – an overview
Keeping the Brand Promise Zero Liability go Security Flexibility
Keeping the Brand Promise 3 Global Acceptance Reliability Zero Liability go Security Flexibility Guaranteed Payment
VisaNet is the world’s largest, most advanced payments network.
28 million acceptance locations 130 million authorizations per day 1,600 endpoints Processes transactions in 1 second 16,300 financial institutions Less than 2 seconds unavailability per year Processes in 175 currencies 1.4 million ATMs fast secure flexible reliable scalable
1 VisaNet – an overview 2 3 Value-Added Information Products Hadoop@Visa – research results Agenda
Visa Processing Architecture Security/Access Services Message File Web VisaNet Services Integration Common Data Infrastructure Authorization Clearing & Settlement Information Disputes Risk
Value-Added Information Products Information Services
Visa Incentive Network
Risk Management Services
Verified by Visa
Dynamic Key Exchange
1 VisaNet – an overview 2 Value-Added Information Products 3 Hadoop@Visa – research results Agenda
Research Lab Setup Hadoop Systems VM System Relational Database Custom Analytic Stacks Encryption Processing Management Stack Hadoop #2 ~300Tb / 28 nodes Hadoop #1 ~40Tb / 42 node
Risk Product Use Case Create critical data model elements, such as keys and transaction statistics, which feed our real-time risk-scoring systems Hadoop Cluster
Research Sample (Synthetic Transactions)
500 million distinct accounts
100 million transactions per day
200 bytes per transaction
2 years: ~ 73 billion transactions, ~ 36 TB
One Month 13 minutes!
Input: Transactions Merchant Category Country / Zip Output: Key & Statistics MCCZIP Key Statistics related to account, transaction type, approval, fraud, IP address, tx. counts
Financial Enterprise Fit Some key questions we are researching…
What will the Hadoop Solution Stack(s) look like?
Transaction Sample System
Internal vs. External cloud
How do I get data into a cloud in a secure way?
How does HSM and security integration work in Hadoop?