Hw09 Large Scale Transaction Analysis

Hadoop@Visa Joe Cunningham Technology Strategy and Innovation October 2, 2009

Agenda 1 Visa 2 Value-Added Information Products 3 Hadoop@Visa – research results VisaNet – an overview

Keeping the Brand Promise Zero Liability go Security Flexibility

Keeping the Brand Promise 3 Global Acceptance Reliability Zero Liability go Security Flexibility Guaranteed Payment

VisaNet Today
VisaNet is the world’s largest, most advanced payments network.
28 million acceptance locations 130 million authorizations per day 1,600 endpoints Processes transactions in 1 second 16,300 financial institutions Less than 2 seconds unavailability per year Processes in 175 currencies 1.4 million ATMs fast secure flexible reliable scalable

1 VisaNet – an overview 2 3 Value-Added Information Products Hadoop@Visa – research results Agenda

Visa Processing Architecture Security/Access Services Message File Web VisaNet Services Integration Common Data Infrastructure Authorization Clearing & Settlement Information Disputes Risk

Value-Added Information Products Information Services
Client
Portfolio Analysis
Visa Incentive Network
Accountholder
Transaction Alerts
Account Updater
Tailored Rewards
Risk Management Services
Authentication
Advanced Auth
Verified by Visa
Address Verification
Account Monitoring
Fraud Detection
Compromised Accounts
Consumer Alerts
Encryption
Dynamic Key Exchange
Dynamic Cryptograms

1 VisaNet – an overview 2 Value-Added Information Products 3 Hadoop@Visa – research results Agenda

Research Lab Setup Hadoop Systems VM System Relational Database Custom Analytic Stacks Encryption Processing Management Stack Hadoop #2 ~300Tb / 28 nodes Hadoop #1 ~40Tb / 42 node

Risk Product Use Case Create critical data model elements, such as keys and transaction statistics, which feed our real-time risk-scoring systems Hadoop Cluster
Research Sample (Synthetic Transactions)
500 million distinct accounts
100 million transactions per day
200 bytes per transaction
2 years: ~ 73 billion transactions, ~ 36 TB
One Month  13 minutes!
Input: Transactions Merchant Category Country / Zip Output: Key & Statistics MCCZIP Key Statistics related to account, transaction type, approval, fraud, IP address, tx. counts

Financial Enterprise Fit Some key questions we are researching…
What will the Hadoop Solution Stack(s) look like?
File System
Transaction Sample System
Relational Back-End
Analytics Processing
Internal vs. External cloud
How do I get data into a cloud in a secure way?
How does HSM and security integration work in Hadoop?
What are the missing pieces?

Why Hadoop@Visa?
For our information products, we see promise in combining Visa’s data analytics capability with the power of Hadoop
Analyze volumes of data with response times that are not possible today…
… and apply analytic models to individual client, not just client segment
But, need to answer questions relating to stack configuration and support…
… and, eventually must fit with our financial enterprise paradigm:
fast secure flexible reliable scalable

Keeping the Brand Promise Zero Liability go Security Flexibility

Keeping the Brand Promise 3 Global Acceptance Reliability Zero Liability go Security Flexibility Guaranteed Payment

Hw09 Large Scale Transaction Analysis

by Cloudera, Inc.

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 20

Statistics

Hw09 Large Scale Transaction Analysis Presentation Transcript