• Save
12a architecture
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,474
On Slideshare
1,474
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Cluster Architecture 7/6/2012© 2012 MapR Technologies Architecture 1
  • 2. Architecture Agenda • Architectural Layers • Storage Architecture • Services© 2012 MapR Technologies Architecture 2
  • 3. Architecture Objectives At the end of this module you will be able to: • Describe the architectural layers and security features of MapR • Explain how the MapR lockless storage services work • List the different services running in a MapR cluster© 2012 MapR Technologies Architecture 3
  • 4. Architectural Layers© 2012 MapR Technologies Architecture 4
  • 5. Architectural Layers  MapReduce  Lockless Storage Services  Ecosystem & Applications  Management  Cluster Security© 2012 MapR Technologies Architecture 5
  • 6. MapReduce  JobTracker (with HA)  TaskTrackers  Direct Shuffle – RPC not HTTP  Fully API-compliant with stock Hadoop© 2012 MapR Technologies Architecture 6
  • 7. Lockless Storage Services  API-compatible with HDFS – Java and C++ APIs  Real-time read/write/append  Multiple concurrent sessions  NFS-mountable – Supports NFS version 3 and TCP  Lockless storage services in pure C for performance  Direct to hardware© 2012 MapR Technologies Architecture 7
  • 8. Hadoop Ecosystem and Applications  Most open Hadoop distribution  Supports – HBase – Hive – Pig – Mahout – Oozie – Flume – Informatica Hparser – Datameer Analytics Solution – Karmasphere Analyst and Karmasphere Studio – MapR ODBC Connector to Hive – and more…© 2012 MapR Technologies Architecture 8
  • 9. Management  MapR Control System  Command-Line Interface  Rest API  MapR Metrics  3rd Party integrations – Nagios – Ganglia© 2012 MapR Technologies Architecture 9
  • 10. Management - MapR Control System10 © 2012 MapR Technologies Architecture 10
  • 11. Management - MapR Metrics  Graphics on metrics about jobs, tasks, etc. – Histograms display the distribution of data about jobs and task attempts – Line charts provide historical data about jobs and task attempts© 2012 MapR Technologies Architecture 11
  • 12. Cluster Security  Security throughout the MapReduce stack – Non-Root user • All services run as MAPR_USER – Complies with SELinux policies – IP address whitelisting – Secured TaskTracker • Disallow certain user, all superusers, etc. – Integration with PAM© 2012 MapR Technologies Architecture 12
  • 13. Storage Architecture© 2012 MapR Technologies Architecture 13
  • 14. Lockless Storage Services  Nodes  Disks – operating system vs. MapR  Storage Pools  Single addressable distributed namespace  CLDB – Volumes – Containers – Blocks 12 TB 12 TB 12 TB / (Total Space = 36 TB)© 2012 MapR Technologies Architecture 14
  • 15. Services© 2012 MapR Technologies Architecture 15
  • 16. Services  Administrative services – ZooKeeper – Warden – CLDB – JobTracker – WebServer – HBase Master  Data processing services – FileServer – TaskTracker – HBase RegionServer – NFS*© 2012 MapR Technologies Architecture 16
  • 17. Questions© 2012 MapR Technologies Architecture 17