Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Cluster Architecture  7/6/2012© 2012 MapR Technologies   Architecture 1
Architecture   Agenda   • Architectural Layers   • Storage Architecture   • Services© 2012 MapR Technologies      Architec...
Architecture   Objectives   At the end of this module you will be able to:   • Describe the architectural layers and secur...
Architectural Layers© 2012 MapR Technologies         Architecture 4
Architectural Layers     MapReduce     Lockless Storage Services     Ecosystem & Applications     Management     Clus...
MapReduce     JobTracker (with HA)     TaskTrackers     Direct Shuffle      –   RPC not HTTP     Fully API-compliant w...
Lockless Storage Services     API-compatible with HDFS      –   Java and C++ APIs     Real-time read/write/append     M...
Hadoop Ecosystem and Applications     Most open Hadoop distribution     Supports      –   HBase      –   Hive      –   P...
Management     MapR Control System     Command-Line Interface     Rest API     MapR Metrics     3rd Party integration...
Management - MapR Control System10   © 2012 MapR Technologies   Architecture 10
Management - MapR Metrics     Graphics on metrics about jobs, tasks, etc.      –   Histograms display the distribution of...
Cluster Security     Security throughout the MapReduce stack      –   Non-Root user          •   All services run as MAPR...
Storage Architecture© 2012 MapR Technologies     Architecture 13
Lockless Storage Services     Nodes     Disks – operating system vs. MapR     Storage Pools     Single addressable dis...
Services© 2012 MapR Technologies   Architecture 15
Services     Administrative services      –   ZooKeeper      –   Warden      –   CLDB      –   JobTracker      –   WebSer...
Questions© 2012 MapR Technologies   Architecture 17
Upcoming SlideShare
Loading in …5
×

12a architecture

1,667 views

Published on

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

12a architecture

  1. 1. Cluster Architecture 7/6/2012© 2012 MapR Technologies Architecture 1
  2. 2. Architecture Agenda • Architectural Layers • Storage Architecture • Services© 2012 MapR Technologies Architecture 2
  3. 3. Architecture Objectives At the end of this module you will be able to: • Describe the architectural layers and security features of MapR • Explain how the MapR lockless storage services work • List the different services running in a MapR cluster© 2012 MapR Technologies Architecture 3
  4. 4. Architectural Layers© 2012 MapR Technologies Architecture 4
  5. 5. Architectural Layers  MapReduce  Lockless Storage Services  Ecosystem & Applications  Management  Cluster Security© 2012 MapR Technologies Architecture 5
  6. 6. MapReduce  JobTracker (with HA)  TaskTrackers  Direct Shuffle – RPC not HTTP  Fully API-compliant with stock Hadoop© 2012 MapR Technologies Architecture 6
  7. 7. Lockless Storage Services  API-compatible with HDFS – Java and C++ APIs  Real-time read/write/append  Multiple concurrent sessions  NFS-mountable – Supports NFS version 3 and TCP  Lockless storage services in pure C for performance  Direct to hardware© 2012 MapR Technologies Architecture 7
  8. 8. Hadoop Ecosystem and Applications  Most open Hadoop distribution  Supports – HBase – Hive – Pig – Mahout – Oozie – Flume – Informatica Hparser – Datameer Analytics Solution – Karmasphere Analyst and Karmasphere Studio – MapR ODBC Connector to Hive – and more…© 2012 MapR Technologies Architecture 8
  9. 9. Management  MapR Control System  Command-Line Interface  Rest API  MapR Metrics  3rd Party integrations – Nagios – Ganglia© 2012 MapR Technologies Architecture 9
  10. 10. Management - MapR Control System10 © 2012 MapR Technologies Architecture 10
  11. 11. Management - MapR Metrics  Graphics on metrics about jobs, tasks, etc. – Histograms display the distribution of data about jobs and task attempts – Line charts provide historical data about jobs and task attempts© 2012 MapR Technologies Architecture 11
  12. 12. Cluster Security  Security throughout the MapReduce stack – Non-Root user • All services run as MAPR_USER – Complies with SELinux policies – IP address whitelisting – Secured TaskTracker • Disallow certain user, all superusers, etc. – Integration with PAM© 2012 MapR Technologies Architecture 12
  13. 13. Storage Architecture© 2012 MapR Technologies Architecture 13
  14. 14. Lockless Storage Services  Nodes  Disks – operating system vs. MapR  Storage Pools  Single addressable distributed namespace  CLDB – Volumes – Containers – Blocks 12 TB 12 TB 12 TB / (Total Space = 36 TB)© 2012 MapR Technologies Architecture 14
  15. 15. Services© 2012 MapR Technologies Architecture 15
  16. 16. Services  Administrative services – ZooKeeper – Warden – CLDB – JobTracker – WebServer – HBase Master  Data processing services – FileServer – TaskTracker – HBase RegionServer – NFS*© 2012 MapR Technologies Architecture 16
  17. 17. Questions© 2012 MapR Technologies Architecture 17

×