This document provides an overview of the Hadoop ecosystem. It discusses what big data is, characteristics of big data like volume, velocity and variety. It then describes the problems with legacy solutions and how Hadoop provides an improved approach by processing data locally, expecting hardware failures and handling failover elegantly. The core components of Hadoop like HDFS, MapReduce, YARN and HBase are explained. Finally, it discusses what's next in terms of downloading Hadoop, using support and training resources from Hortonworks.

1. The Hadoop Ecosystem
© Hortonworks Inc. 2012

2. What is Big Data?
• Big does not have to be always
Petabytes
• Big refers to big enough for traditional
systems to handle efficiently

3. Big Data Facts
• Twitter generates 8TB of data every day
• eBay data warehouse is 10+ PB
• Facebook data warehouse is 36+ PB
• Yahoo! Has 100+ PB data
• Google scans and indexes 500+ PB data

4. Data Types
• Structured
– Pre-defined schema
– Example: relational database system
• Semi Structured
– No identifiable structure
– Cannot be stored in rows and tables in a database
– Examples : logs, tweets,
• Un Structured
– Irregular structure or it lacks structure
– Examples: free-form text, reports, customer feedback
forms
Copyright Hortonworks 2012 4

5. Characteristics of Big Data
• Volume
• Velocity
• Variety
• Value
Copyright Hortonworks 2012 5

6. Problem with Legacy Solution
• Expensive
– Scale up costs lots of $$
• Rigid
• Stale Data
Copyright Hortonworks 2012 6

7. Hadoop Approach
• Process data locally
• Expect Hardware failures
• Handle failover elegantly
• Duplicate a small percentage of the data to
small groups (versus entire database)

8. Compare with RDBMS
Copyright Hortonworks 2012 8

9. Hadoop Core Components

10. Hadoop Cluster – Basic configuration
Copyright Hortonworks 2012 10

11. MapReduce In Action
Logical
Physical
11

12. Hadoop Ecosystem
Develop Analyze Visualize
Hortonworks Data Platform
Scripting Query
Management & Monitoring
(Pig) (Hive)
(Sqoop, Talend, WebHDFS, WebHCatalog)
NoSQL Column DB
Workflow & Scheduling
Data Extraction & Load
(Ambari, Zookeeper)
(HBase)
Metadata Management
(HCatalog)
(Oozie)
Operate Integrate
Distributed Processing
(MapReduce)
Distributed Storage
(HDFS)

13. What Next?
1 Download Hortonworks Data Platform
hortonworks.com/download
2 Use the getting started guide
hortonworks.com/get-started
3 Learn more… get support
Hortonworks Support
• Expert role based training • Full lifecycle technical support
• Course for admins, developers across four service levels
and operators • Delivered by Apache Hadoop
• Certification program Experts/Committers
• Custom onsite options • Forward-compatible
hortonworks.com/training hortonworks.com/support
Page 13
© Hortonworks Inc. 2012

14. Hortonworks Support Subscriptions
Objective: help organizations to successfully develop
and deploy solutions based upon Apache Hadoop
• Full-lifecycle technical support available
– Developer support for design, development and POCs
– Production support for staging and production environments
– Up to 24x7 with 1-hour response times
• Delivered by the Apache Hadoop experts
– Backed by development team that has released every major version of
Apache Hadoop since 0.1
• Forward-compatibility
– Hortonworks’ leadership role helps ensure bug fixes and patches can be
included in future versions of Hadoop projects
Page 14
© Hortonworks Inc. 2012

15. Hortonworks Training
Objective: help organizations overcome Hadoop
knowledge gaps
• Expert role-based training for developers,
administrators & data analysts
– Heavy emphasis on hands-on labs
– Extensive schedule of public training courses available
(hortonworks.com/training)
• Comprehensive certification programs
• Customized, on-site courses available
Page 15
© Hortonworks Inc. 2012