Real Time BI with Hadoop

by Bradford Stephens on Oct 05, 2009

12,933 views

A brief synopsis of using the Apache Hadoop stack to build a Real-Time Business Intelligence application, including data warehousing and search.

More…

A brief synopsis of using the Apache Hadoop stack to build a Real-Time Business Intelligence application, including data warehousing and search.

Less

3 Embeds 100

http://www.slideshare.net	98
http://obl8.com:8000	1
http://webcache.googleusercontent.com	1

Statistics

Favorites: 13
Downloads: 373
Comments: 2
Embed Views: 100
Views on SlideShare: 12,833
Total Views: 12,933

Real Time BI with Hadoop — Presentation Transcript

Real-Time BI in Hadoop Bradford Stephens Lead Engineer, Visible Technologies Principal Consultant, Drawn to Scale Consulting

Topics • Scalability and BI • Costs and Abilities • Search as BI

What Is BI?

What is “Real-Time” • Understanding Latency • We aim for <5 secs.

Scalability in BI • Scalbility matters now • Social Media: Catalyst • All data is important • Data doesn’t scale with business size any more

Search as BI • Katta = Distributed Search on Haddoop • Bobo = Faceted Lucene

Doing it Cheap • 100 TB, Structured and Unstructured • Oracle- $100,000,000 • “NewSQL” - $4,000,000 • Hadoop + Katta - $250,000

Why We Need Hadoop • Need to process high-latency data to get the “small stuff” fast • Robust Ecosystem • Need more than SQL. RDBMS not a Swiss- Army Knife

Aggregation is Real- Time • Distributed Search w/ Katta + Facets = Aggregation-Based BI • Sum, Count, Filter, Avg, Group

Protips: Review • Understand High vs. Low Latency data • Hadoop makes it cheap • Pre-aggregate w/ Hadoop, Explore w/ Katta + Faceted Search

The Future • Search/BI as a Platform: “Google my Data Warehouse” • Real-Time MR on HBase

David Lovering I hate to display my ignorance among such an august body, but after downloading the presentation (in .key format) absolutely nothing would recognize it - not powerpoint, not Safari, not QuickTime -- nothing. A quick search of Google revealed no plausible alternate application tied to '.key' suffix-files, and there doesn't appear to be an independent app called 'Apple Keynote'. What gives?
-- Confused in Denver 8 months ago

Are you sure you want to Yes No

Herbert Mühlburger I guess you mean NoSQL and not NowSQL, right? ;-)

Cheers,
Herbert 2 years ago

Real Time BI with Hadoop

by Bradford Stephens on Oct 05, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 100

Statistics

Real Time BI with Hadoop — Presentation Transcript