The document provides an overview of Hadoop as an open-source data management framework, its strengths in processing and storing large data sets, and its weaknesses in real-time insights and analytical tools. It discusses emerging best practices for integrating Hadoop with in-memory data management to enhance real-time data analysis and provide competitive advantages. Additionally, it outlines steps to get started with in-memory and Hadoop for businesses seeking rapid insights from data.

1. © 2013 Terracotta Inc. | Internal Use Only
In-Memory &
Hadoop:
Real-time
Big Data Intelligence

2. © 2013 Terracotta Inc. 2
Your speaker
Manish Devgan
Director of Product
Management
Terracotta

3. © 2013 Terracotta Inc. 3
What we’ll cover in this webcast
• What’s Hadoop? (quick intro)
• Hadoop’s weaknesses
• Emerging best practices for combining
Hadoop and in-memory data management
• Real-time intelligence example
• Getting started with in-memory and Hadoop
• Q & A

4. © 2013 Terracotta Inc. 4
4© 2013 Terracotta Inc. | Internal Use Only
What is Hadoop?

5. © 2013 Terracotta Inc. 5
What is ?
• Hadoop is open-source software data management framework
used to draw insights from data
Components Benefits
HDFS*: Scalable &
distributed Storage
• Data distributed across cluster
nodes
• Name node keeps track of location
MapReduce: Parallel
Processing of data
• Splits a task for processing based
on data locality and then
assembles results
• Comprises of Map() procedure for
filtering & sorting and Reduce()
procedure for summarizing
Scalable
• Efficiently store and process large
data sets
Reliable
• Get redundant storage, with failover
across cluster
Rich & Flexible
• Complimentary set of tools &
frameworks
• Store data in any format
Economical
• Deploy on commodity hardware
*Hadoop Distributed File System

6. © 2013 Terracotta Inc. 6
What is ?
• With Hadoop, you can ask interesting questions about your data
and get answers economically
Questions Hadoop can help answer
How can I target promotions to my customers for better
sales?
How risky are each of my customers?
Which advertisement should I show to optimize return?
How relevant is a result for a given search?
When will my machinery likely have a malfunction?

7. © 2013 Terracotta Inc. 7
7© 2013 Terracotta Inc. | Internal Use Only
Hadoop’s Weaknesses

8. © 2013 Terracotta Inc. 8
Hadoop’s Weaknesses
• No support for real-time insights
• No support to facilitate interactive and exploratory data analysis
• Challenging framework for computation beyond Map Reduce
• Lacks tools for business analysts

9. © 2013 Terracotta Inc. 9
9© 2013 Terracotta Inc. | Internal Use Only
Emerging best practices
for combining Hadoop and
in-memory data management

10. © 2013 Terracotta Inc. 10
Combining Hadoop and In-memory Data Management
- Businesses are looking for ways to mine real-time insights to
provide competitive advantages
- Increased adoption of transactional system data for analytics is
blurring the line between OLTP and OLAP
- New frameworks and products are bringing in-memory
technologies to the Hadoop ecosystem

11. © 2013 Terracotta Inc. 11
Real-time Data Integration with Hadoop
Web
Apps
Mobile
Apps
Dashboards
& Mashups
In-memory Data Management Platform
Real-time Data Apps
Transactional
Apps
Operational
Intelligence
Log Data POS Data Social Media Sensors
Data Sources
Events
Images/Video
s
Data Feeds
Real-time
data
Real-time
Insights

12. © 2013 Terracotta Inc. 12
12© 2013 Terracotta Inc. | Internal Use Only
Real-time intelligence example

13. © 2013 Terracotta Inc. 13
BigMemory & Hadoop in financial services
Before: Custom ETL connector pushing batch data
Hadoop Cluster
BigMemoryStore
Short Term
Transaction
Data
Long Term
Transaction
Data
Rules &
Triggers
Tagged
Accounts
Credit
Reference
Data
HDFS to BigMemory
Processing
Hadoop M/R

14. © 2013 Terracotta Inc. 14
BigMemory & Hadoop in financial services
Today: Streaming Data insights
Hadoop Cluster
Insights Hadoop M/R
BigMemory-
Hadoop
Connector
BigMemoryStore
Short Term
Transaction
Data
Long Term
Transaction
Data
Rules &
Triggers
Tagged
Accounts
Credit
Reference
Data

15. © 2013 Terracotta Inc. 15
15© 2013 Terracotta Inc. | Internal Use Only
Getting started with
in-memory and Hadoop

16. © 2013 Terracotta Inc. 16
How to get started with In-memory and Hadoop?
• If you already have a Hadoop project, look for use cases where
you want real-time access to insights
• Start with a small-to-medium sized (20-40 nodes) cluster with a
well-defined use case requiring fast access to data
• Consider exploratory use cases where you’re doing iterative
analysis on a data set to get answers faster

17. © 2013 Terracotta Inc. 17
In-Memory & Hadoop
Questions
Please type yours in the “Questions” panel or in the chat window.

18. © 2013 Terracotta Inc. 18
Connect with Terracotta
• Download “BigMemory & Hadoop” white paper
− Visit: www.terracotta.org (Resources > White Papers)
• Download “BigMemory-Hadoop Connector”
− Visit: www.terracotta.org/downloads/hadoop-connector
• Contact Manish Devgan
− Email: mdevgan@terracottatech.com
• Follow us on Twitter
− @big_memory
• Stay Tuned