Hw09   Large Scale Transaction Analysis
 

Hw09 Large Scale Transaction Analysis

on

  • 5,437 views

 

Statistics

Views

Total Views
5,437
Views on SlideShare
5,417
Embed Views
20

Actions

Likes
3
Downloads
206
Comments
0

1 Embed 20

http://www.slideshare.net 20

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Fast – On average, VisaNet fully processes transactions in approximately 1 second. Secure – VisaNet supports multiple defense layers to prevent network intrusions, protect against account fraud, and to make stolen card data unusable for criminalsincluding data encryption, sophisticated intrusion detection systems and neural network technology capable of analyzing each transaction in real-time. Over the last two years, Visa has spent more than $100 million on data security enhancements. Reliable – VisaNet runs multiple, parallel processing systems in highly redundant data centers. This ensures near 100 percent availability. In the past 15 years, VisaNet experienced less than two seconds average unavailability per year. State-of-the-art monitoring tools are in place to detect network availability and client-specific processing anomalies, such as high rates of decline. Scalable – VisaNet authorizes an average of 130 million transactions a day, totaling more than 92 billion authorization. On Dec. 21, 2008, Visa’s busiest day last year, VisaNet authorized more than 200 million transactions. Based on recent stress tests, VisaNet is capable of authorizing over a half billion transactions per day.

Hw09   Large Scale Transaction Analysis Hw09 Large Scale Transaction Analysis Presentation Transcript

  • Hadoop@Visa Joe Cunningham Technology Strategy and Innovation October 2, 2009
  • Agenda 1 Visa 2 Value-Added Information Products 3 Hadoop@Visa – research results VisaNet – an overview
  • Keeping the Brand Promise Zero Liability go Security Flexibility
  • Keeping the Brand Promise 3 Global Acceptance Reliability Zero Liability go Security Flexibility Guaranteed Payment
  • VisaNet Today
    • VisaNet is the world’s largest, most advanced payments network.
    28 million acceptance locations 130 million authorizations per day 1,600 endpoints Processes transactions in 1 second 16,300 financial institutions Less than 2 seconds unavailability per year Processes in 175 currencies 1.4 million ATMs fast secure flexible reliable scalable
  • 1 VisaNet – an overview 2 3 Value-Added Information Products Hadoop@Visa – research results Agenda
  • Visa Processing Architecture Security/Access Services Message File Web VisaNet Services Integration Common Data Infrastructure Authorization Clearing & Settlement Information Disputes Risk
  • Value-Added Information Products Information Services
    • Client
    • Portfolio Analysis
    • Visa Incentive Network
    • Accountholder
    • Transaction Alerts
    • Account Updater
    • Tailored Rewards
    Risk Management Services
    • Authentication
    • Advanced Auth
    • Verified by Visa
    • Address Verification
    • Account Monitoring
    • Fraud Detection
    • Compromised Accounts
    • Consumer Alerts
    • Encryption
    • Dynamic Key Exchange
    • Dynamic Cryptograms
  • 1 VisaNet – an overview 2 Value-Added Information Products 3 Hadoop@Visa – research results Agenda
  • Research Lab Setup Hadoop Systems VM System Relational Database Custom Analytic Stacks Encryption Processing Management Stack Hadoop #2 ~300Tb / 28 nodes Hadoop #1 ~40Tb / 42 node
  • Risk Product Use Case Create critical data model elements, such as keys and transaction statistics, which feed our real-time risk-scoring systems Hadoop Cluster
    • Research Sample (Synthetic Transactions)
    • 500 million distinct accounts
    • 100 million transactions per day
    • 200 bytes per transaction
    • 2 years: ~ 73 billion transactions, ~ 36 TB
    • One Month  13 minutes!
    Input: Transactions Merchant Category Country / Zip Output: Key & Statistics MCCZIP Key Statistics related to account, transaction type, approval, fraud, IP address, tx. counts
  • Financial Enterprise Fit Some key questions we are researching…
    • What will the Hadoop Solution Stack(s) look like?
      • File System
      • Transaction Sample System
      • Relational Back-End
      • Analytics Processing
    • Internal vs. External cloud
      • How do I get data into a cloud in a secure way?
    • How does HSM and security integration work in Hadoop?
      • What are the missing pieces?
  • Why Hadoop@Visa?
    • For our information products, we see promise in combining Visa’s data analytics capability with the power of Hadoop
    • Analyze volumes of data with response times that are not possible today…
      • … and apply analytic models to individual client, not just client segment
    • But, need to answer questions relating to stack configuration and support…
      • … and, eventually must fit with our financial enterprise paradigm:
    fast secure flexible reliable scalable
  • Keeping the Brand Promise Zero Liability go Security Flexibility
  • Keeping the Brand Promise 3 Global Acceptance Reliability Zero Liability go Security Flexibility Guaranteed Payment