Hw09 Real Time Business Intelligence

Real-Time BI in Hadoop
Bradford Stephens
Lead Engineer, Visible Technologies
Principal Consultant, Drawn to Scale Consulting

Topics
Scalability and BI
Costs and Abilities
Search as BI

What Is BI?

What is “Real-Time”
Understanding Latency
We aim for <5 secs.

Scalability in BI
Scalbility matters now
Social Media: Catalyst
All data is important
Data doesn’t scale with business size any more

Search as BI
Katta = Distributed Search on Haddoop
Bobo = Faceted Lucene

Doing it Cheap
100 TB, Structured and Unstructured
Oracle- $100,000,000
“ NewSQL” - $4,000,000
Hadoop + Katta - $250,000

Why We Need Hadoop
Need to process high-latency data to get the “small stuff” fast
Robust Ecosystem
Need more than SQL. RDBMS not a Swiss-Army Knife

Aggregation is Real-Time
Distributed Search w/ Katta + Facets = Aggregation-Based BI
Sum, Count, Filter, Avg, Group

Protips: Review
Understand High vs. Low Latency data
Hadoop makes it cheap
Pre-aggregate w/ Hadoop, Explore w/ Katta + Faceted Search

The Future
Search/BI as a Platform: “Google my Data Warehouse”
Real-Time MR on HBase

by Cloudera, Inc.