12a architecture

1,397 views
1,296 views

Published on

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,397
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

12a architecture

  1. 1. Cluster Architecture 7/6/2012© 2012 MapR Technologies Architecture 1
  2. 2. Architecture Agenda • Architectural Layers • Storage Architecture • Services© 2012 MapR Technologies Architecture 2
  3. 3. Architecture Objectives At the end of this module you will be able to: • Describe the architectural layers and security features of MapR • Explain how the MapR lockless storage services work • List the different services running in a MapR cluster© 2012 MapR Technologies Architecture 3
  4. 4. Architectural Layers© 2012 MapR Technologies Architecture 4
  5. 5. Architectural Layers  MapReduce  Lockless Storage Services  Ecosystem & Applications  Management  Cluster Security© 2012 MapR Technologies Architecture 5
  6. 6. MapReduce  JobTracker (with HA)  TaskTrackers  Direct Shuffle – RPC not HTTP  Fully API-compliant with stock Hadoop© 2012 MapR Technologies Architecture 6
  7. 7. Lockless Storage Services  API-compatible with HDFS – Java and C++ APIs  Real-time read/write/append  Multiple concurrent sessions  NFS-mountable – Supports NFS version 3 and TCP  Lockless storage services in pure C for performance  Direct to hardware© 2012 MapR Technologies Architecture 7
  8. 8. Hadoop Ecosystem and Applications  Most open Hadoop distribution  Supports – HBase – Hive – Pig – Mahout – Oozie – Flume – Informatica Hparser – Datameer Analytics Solution – Karmasphere Analyst and Karmasphere Studio – MapR ODBC Connector to Hive – and more…© 2012 MapR Technologies Architecture 8
  9. 9. Management  MapR Control System  Command-Line Interface  Rest API  MapR Metrics  3rd Party integrations – Nagios – Ganglia© 2012 MapR Technologies Architecture 9
  10. 10. Management - MapR Control System10 © 2012 MapR Technologies Architecture 10
  11. 11. Management - MapR Metrics  Graphics on metrics about jobs, tasks, etc. – Histograms display the distribution of data about jobs and task attempts – Line charts provide historical data about jobs and task attempts© 2012 MapR Technologies Architecture 11
  12. 12. Cluster Security  Security throughout the MapReduce stack – Non-Root user • All services run as MAPR_USER – Complies with SELinux policies – IP address whitelisting – Secured TaskTracker • Disallow certain user, all superusers, etc. – Integration with PAM© 2012 MapR Technologies Architecture 12
  13. 13. Storage Architecture© 2012 MapR Technologies Architecture 13
  14. 14. Lockless Storage Services  Nodes  Disks – operating system vs. MapR  Storage Pools  Single addressable distributed namespace  CLDB – Volumes – Containers – Blocks 12 TB 12 TB 12 TB / (Total Space = 36 TB)© 2012 MapR Technologies Architecture 14
  15. 15. Services© 2012 MapR Technologies Architecture 15
  16. 16. Services  Administrative services – ZooKeeper – Warden – CLDB – JobTracker – WebServer – HBase Master  Data processing services – FileServer – TaskTracker – HBase RegionServer – NFS*© 2012 MapR Technologies Architecture 16
  17. 17. Questions© 2012 MapR Technologies Architecture 17

×