Not about the Big in Big Data

It’s Not About the “Big” in Big Data

$26.5 BILLION Nick Leeson Jerome Kerviel Kweku Adoboli Bruno Iksil Barings Bank Societe Generale UBS JPMC $1.4 billion $6 billion $1.4 billion $17.5 Billion Nasa $17.6BRogue Traders $26.5B

Hadoop in 2009

Hadoop 2012

Hype Cycle We are here

Crossing the Chasm We are here

Hadoop - Enabler

Database Harddrive Unstructured ( 61.7% growths ) time to ﬁnd one record = logb N * 10ms log100(100,000,000) * 10ms = 40ms time to read record = 10ms 10,000,000 * 50ms = 5.8 days

Hadoop Harddrive throughput = 10MB/s time to transfer record = 10ms 10,000,000 * 10ms = 1.5 days random reads = (5.8 days)

Laws of Physics Random Sequential Values/Sec. 316 Disk 53,200,000 1,924 SSD 42,200,000 36,700,000Memory 358,200,000 1 10 100 1,000 10,000 100,000 1,000,000 10,000,000 100,000,000 1,000,000,000 Adam Jacobs The Pathologies of Big Data

People costHardware Cost

Time to insightTime for decision

Volume Data Size Data Complexity Sp ang es So ta Ch ee e urc Da do etyVe f ri loc Va y it

Impossible: 360 View

Game Changer Slow Static Barrier Business ETL Data Warehouse Intelligence Fast Dynamic View Raw Load Hadoop Data Pipeline

NO - SQL RDBMS Standard SQL Structured Data Response in sec. No SQL Unstructured Data Batch Hadoop

Common Applications Asset Management Analytics Security Analytics Product Cohort Analytics Advanced Web Analytics Structured + Many Unstructured Decision Makers Data Sources Data

follow us: @datameer @stefanGroschupf

Not about the Big in Big Data

by Hadoop_Summit on Jun 20, 2012

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Not about the Big in Big Data Presentation Transcript