© Hortonworks Inc. 2013
Welcome to Ambari BoF Meetup!
Hadoop Summit
June 2013
© Hortonworks Inc. 2013
Agenda: June 25, 2013
1. Ambari Overview, Current State, Recent
Developments
Yusaku Sako @ Hortonw...
© Hortonworks Inc. 2013
“Make Hadoop Management Easy for Everyone”
Page 3
Provision, Manage,
Monitor
Hadoop Clusters
Hadoo...
© Hortonworks Inc. 2013
Apache Ambari – 100% Open Source!
• Active and growing community
• 50+ Contributors / 20+ Committe...
© Hortonworks Inc. 2013
Ambari System Architecture
5
Ambari Server
Host
Agent
gmond
Host
Agent
gmond
Ganglia
Server
Agent
...
© Hortonworks Inc. 2013
Ambari Today
• Live Demo of a cluster installed with Ambari 1.2.5
Page 6
© Hortonworks Inc. 2013
Ambari Architecture
Page 7
DB
Orchestrator SPI
REST API
Request Dispatcher
Ambari
Web
Ambari
Serve...
© Hortonworks Inc. 2013
Hadoop Security - Background
Page 8
• Non secure mode = No auth
• Kerberos tickets allow NN/JT to ...
© Hortonworks Inc. 2013
Hadoop Security with Ambari
Page 9
• Ambari 1.2.5 introduces Security Wizard
– Re-configuring serv...
© Hortonworks Inc. 2013
Password Encryption and Web Access
Page 10
• Allow trusted signed certs to be imported
into Ambari...
© Hortonworks Inc. 2013
REST API – Centralized & Consistent
Page 11
Ambari REST API
Alerts
Job
HistoryMetricsConfiguration...
© Hortonworks Inc. 2013
Ambari + Teradata Viewpoint Integration
Page 12
• Ambari = Key enabler for
integrating Hadoop moni...
© Hortonworks Inc. 2013
Stack Definitions
• Design Goals
–Ambari should be able to support choice of Hadoop stacks
–Ambari...
© Hortonworks Inc. 2013
Ambari + Redhat GlusterFS Integration
• Using Ambari to deploy / manage cluster with
distributed f...
© Hortonworks Inc. 2013
Ambari + Accumulo Integration
• Using Ambari to deploy / manage cluster with
Accumulo
–Google Summ...
© Hortonworks Inc. 2013
Splunk + Ambari Integration
• Head over to Splunk’s booth tomorrow to see Ambari
integrated into S...
© Hortonworks Inc. 2013
Thanks!
• Next Up: APIs and SPIs – How to Integrate with
Ambari
Page 17
Upcoming SlideShare
Loading in...5
×

Apache Ambari BOF - Overview - Hadoop Summit 2013

3,755

Published on

Apache Ambari BOF Meet Up @ Hadoop Summit 2013

Overview Slides

http://www.meetup.com/Apache-Ambari-User-Group/events/119184782/

Published in: Technology
2 Comments
4 Likes
Statistics
Notes
  • the hadoop tutorials
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • for beginners hadoop tutorial do follow
    http://thehadooptutorial.blogspot.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
3,755
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
112
Comments
2
Likes
4
Embeds 0
No embeds

No notes for slide
  • ProvisionEasy installation of Hadoop clusters (GUI-based or headless)ManageServices: start, stop, test, reconfigure, reconfiguring, reassign mastersHosts: add, decommission, removeStack Upgrade, HDFS Mirroring, Rolling Upgrade, moreMonitorCluster utilization (HDFS, MapReduce slots, CS queues, system metrics)Graphs and HeatmapsNotification upon critical failures, warnings, performance degradation (NameNode down, HDFS capacity low, etc)Job Analytics (MR, Pig, Hive)IntegrateREST API for doing ALL of the above!
  • Define clear wrapper around services to have consistent mgmt interface and configuration controls
  • Apache Ambari BOF - Overview - Hadoop Summit 2013

    1. 1. © Hortonworks Inc. 2013 Welcome to Ambari BoF Meetup! Hadoop Summit June 2013
    2. 2. © Hortonworks Inc. 2013 Agenda: June 25, 2013 1. Ambari Overview, Current State, Recent Developments Yusaku Sako @ Hortonworks Siddharth Wagle @ Hortonworks 2. APIs and SPIs – How to Integrate with Ambari Tom Beerbower @ Hortonworks Nate Cole @ Hortonworks 3. Ambari Blueprint Brian Swan @ Microsoft Sumit Mohanty @ Hortonworks 4. Ambari and OpenStack Himanshu Bari @ Hortonworks Page 2
    3. 3. © Hortonworks Inc. 2013 “Make Hadoop Management Easy for Everyone” Page 3 Provision, Manage, Monitor Hadoop Clusters Hadoop Operators Analyze Jobs (MR, Hive, Pig, etc) Hadoop Users Integrate Hadoop management/monitoring capabilities to software Software Vendors Ambari’s Mission
    4. 4. © Hortonworks Inc. 2013 Apache Ambari – 100% Open Source! • Active and growing community • 50+ Contributors / 20+ Committers • 140 Ambari User Group Members • Steady progress / short release cycle Page 4 Release Version Release Date JIRAs Resolved 0.9.0 Sep 2012 402 1.2.0 Feb 2013 441 1.2.1 Mar 2013 134 1.2.2 Apr 2013 106 1.2.3 Jun 2013 515 1.2.4 Jul 2013 109+ 1.2.5 Jul 2013 131+  Current Release  Today’s Demo
    5. 5. © Hortonworks Inc. 2013 Ambari System Architecture 5 Ambari Server Host Agent gmond Host Agent gmond Ganglia Server Agent Host Agent gmondgmetad gmond Ambari Web DB REST /clusters Nagios Server Agent
    6. 6. © Hortonworks Inc. 2013 Ambari Today • Live Demo of a cluster installed with Ambari 1.2.5 Page 6
    7. 7. © Hortonworks Inc. 2013 Ambari Architecture Page 7 DB Orchestrator SPI REST API Request Dispatcher Ambari Web Ambari Server Metrics AuthProvider /clusters /services /hosts /workflows/jobs /users, … User Store java RDBMS javascript RDBM S AD/ LDAP REST API for integration Auth Provider Cluster Configurations Web Client 100% REST Ambari Agents ganglia nagios Alerts Pluggable Service Providersfalcon Data Mgmt jmx python puppet
    8. 8. © Hortonworks Inc. 2013 Hadoop Security - Background Page 8 • Non secure mode = No auth • Kerberos tickets allow NN/JT to mutually authenticate (principals/keytabs) • Guidelines for Kerberos based security – Daemon principals and keytabs – Mapping of secure user to FS user – Re-configuring individual daemons – Re-starting services and testing for secure setup KDC HM JT NN Kerberosclient Client TT DN Kerberosclient
    9. 9. © Hortonworks Inc. 2013 Hadoop Security with Ambari Page 9 • Ambari 1.2.5 introduces Security Wizard – Re-configuring services with security enabled – Mapping of secure users to FS user – Re-starting services in secure mode and run smoke tests – Note: User needs to generate principals and keytabs • Future: – Automatically create principals and keytabs – Secure distributions of keytabs
    10. 10. © Hortonworks Inc. 2013 Password Encryption and Web Access Page 10 • Allow trusted signed certs to be imported into Ambari – Keystore: Open SSL with PKCS 12 keystore – User provided cert is used for server-agent communication • Database and LDAP passwords no longer in clear-text – JCEKS keystore to store credentials, locked with a user-provided master key
    11. 11. © Hortonworks Inc. 2013 REST API – Centralized & Consistent Page 11 Ambari REST API Alerts Job HistoryMetricsConfigurations Config DB Nagios Server Ganglia Server … HTTP GET, POST, PUT, DELETE :8080 HTTP Status Code / JSON core- site.xml core- site.xml Config files Config files Config files JMX Realtime Historical*-site.xml… Job History DB Hosts / ServicesCluster
    12. 12. © Hortonworks Inc. 2013 Ambari + Teradata Viewpoint Integration Page 12 • Ambari = Key enabler for integrating Hadoop monitoring capabilities to Teradata Viewpoint • Get Hadoop metrics from a non- Ambari deployed cluster – Ambari REST API + Custom Service Providers
    13. 13. © Hortonworks Inc. 2013 Stack Definitions • Design Goals –Ambari should be able to support choice of Hadoop stacks –Ambari should enable adding new components to an existing stack • Define which Services are available (services) • Define where to get the packages (repos) 13 S S S SStack B repos services S S S SStack A repos services S S S S Stack C extends Stack B repos services S S+
    14. 14. © Hortonworks Inc. 2013 Ambari + Redhat GlusterFS Integration • Using Ambari to deploy / manage cluster with distributed file system other than HDFS –HCFS: GlusterFS as first implementation –Pluggability with other HCFS’s –See AMBARI-1817 Page 14 MapReduce Hive Distributed File System HDFS GlusterFS HBasePig Other HCFS …
    15. 15. © Hortonworks Inc. 2013 Ambari + Accumulo Integration • Using Ambari to deploy / manage cluster with Accumulo –Google Summer of Code project –See AMBARI-1930 MapReduce Hive Distributed File System HBasePig
    16. 16. © Hortonworks Inc. 2013 Splunk + Ambari Integration • Head over to Splunk’s booth tomorrow to see Ambari integrated into Splunk’s Management UI for Hadoop monitoring and management Page 16 +
    17. 17. © Hortonworks Inc. 2013 Thanks! • Next Up: APIs and SPIs – How to Integrate with Ambari Page 17
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×