• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Apache Ambari BOF - Overview - Hadoop Summit 2013
 

Apache Ambari BOF - Overview - Hadoop Summit 2013

on

  • 2,665 views

Apache Ambari BOF Meet Up @ Hadoop Summit 2013

Apache Ambari BOF Meet Up @ Hadoop Summit 2013

Overview Slides

http://www.meetup.com/Apache-Ambari-User-Group/events/119184782/

Statistics

Views

Total Views
2,665
Views on SlideShare
2,651
Embed Views
14

Actions

Likes
3
Downloads
85
Comments
2

2 Embeds 14

https://twitter.com 13
http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

12 of 2 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • the hadoop tutorials
    Are you sure you want to
    Your message goes here
    Processing…
  • for beginners hadoop tutorial do follow
    http://thehadooptutorial.blogspot.com
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • ProvisionEasy installation of Hadoop clusters (GUI-based or headless)ManageServices: start, stop, test, reconfigure, reconfiguring, reassign mastersHosts: add, decommission, removeStack Upgrade, HDFS Mirroring, Rolling Upgrade, moreMonitorCluster utilization (HDFS, MapReduce slots, CS queues, system metrics)Graphs and HeatmapsNotification upon critical failures, warnings, performance degradation (NameNode down, HDFS capacity low, etc)Job Analytics (MR, Pig, Hive)IntegrateREST API for doing ALL of the above!
  • Define clear wrapper around services to have consistent mgmt interface and configuration controls

Apache Ambari BOF - Overview - Hadoop Summit 2013 Apache Ambari BOF - Overview - Hadoop Summit 2013 Presentation Transcript

  • © Hortonworks Inc. 2013 Welcome to Ambari BoF Meetup! Hadoop Summit June 2013
  • © Hortonworks Inc. 2013 Agenda: June 25, 2013 1. Ambari Overview, Current State, Recent Developments Yusaku Sako @ Hortonworks Siddharth Wagle @ Hortonworks 2. APIs and SPIs – How to Integrate with Ambari Tom Beerbower @ Hortonworks Nate Cole @ Hortonworks 3. Ambari Blueprint Brian Swan @ Microsoft Sumit Mohanty @ Hortonworks 4. Ambari and OpenStack Himanshu Bari @ Hortonworks Page 2
  • © Hortonworks Inc. 2013 “Make Hadoop Management Easy for Everyone” Page 3 Provision, Manage, Monitor Hadoop Clusters Hadoop Operators Analyze Jobs (MR, Hive, Pig, etc) Hadoop Users Integrate Hadoop management/monitoring capabilities to software Software Vendors Ambari’s Mission
  • © Hortonworks Inc. 2013 Apache Ambari – 100% Open Source! • Active and growing community • 50+ Contributors / 20+ Committers • 140 Ambari User Group Members • Steady progress / short release cycle Page 4 Release Version Release Date JIRAs Resolved 0.9.0 Sep 2012 402 1.2.0 Feb 2013 441 1.2.1 Mar 2013 134 1.2.2 Apr 2013 106 1.2.3 Jun 2013 515 1.2.4 Jul 2013 109+ 1.2.5 Jul 2013 131+  Current Release  Today’s Demo
  • © Hortonworks Inc. 2013 Ambari System Architecture 5 Ambari Server Host Agent gmond Host Agent gmond Ganglia Server Agent Host Agent gmondgmetad gmond Ambari Web DB REST /clusters Nagios Server Agent
  • © Hortonworks Inc. 2013 Ambari Today • Live Demo of a cluster installed with Ambari 1.2.5 Page 6
  • © Hortonworks Inc. 2013 Ambari Architecture Page 7 DB Orchestrator SPI REST API Request Dispatcher Ambari Web Ambari Server Metrics AuthProvider /clusters /services /hosts /workflows/jobs /users, … User Store java RDBMS javascript RDBM S AD/ LDAP REST API for integration Auth Provider Cluster Configurations Web Client 100% REST Ambari Agents ganglia nagios Alerts Pluggable Service Providersfalcon Data Mgmt jmx python puppet
  • © Hortonworks Inc. 2013 Hadoop Security - Background Page 8 • Non secure mode = No auth • Kerberos tickets allow NN/JT to mutually authenticate (principals/keytabs) • Guidelines for Kerberos based security – Daemon principals and keytabs – Mapping of secure user to FS user – Re-configuring individual daemons – Re-starting services and testing for secure setup KDC HM JT NN Kerberosclient Client TT DN Kerberosclient
  • © Hortonworks Inc. 2013 Hadoop Security with Ambari Page 9 • Ambari 1.2.5 introduces Security Wizard – Re-configuring services with security enabled – Mapping of secure users to FS user – Re-starting services in secure mode and run smoke tests – Note: User needs to generate principals and keytabs • Future: – Automatically create principals and keytabs – Secure distributions of keytabs
  • © Hortonworks Inc. 2013 Password Encryption and Web Access Page 10 • Allow trusted signed certs to be imported into Ambari – Keystore: Open SSL with PKCS 12 keystore – User provided cert is used for server-agent communication • Database and LDAP passwords no longer in clear-text – JCEKS keystore to store credentials, locked with a user-provided master key
  • © Hortonworks Inc. 2013 REST API – Centralized & Consistent Page 11 Ambari REST API Alerts Job HistoryMetricsConfigurations Config DB Nagios Server Ganglia Server … HTTP GET, POST, PUT, DELETE :8080 HTTP Status Code / JSON core- site.xml core- site.xml Config files Config files Config files JMX Realtime Historical*-site.xml… Job History DB Hosts / ServicesCluster
  • © Hortonworks Inc. 2013 Ambari + Teradata Viewpoint Integration Page 12 • Ambari = Key enabler for integrating Hadoop monitoring capabilities to Teradata Viewpoint • Get Hadoop metrics from a non- Ambari deployed cluster – Ambari REST API + Custom Service Providers
  • © Hortonworks Inc. 2013 Stack Definitions • Design Goals –Ambari should be able to support choice of Hadoop stacks –Ambari should enable adding new components to an existing stack • Define which Services are available (services) • Define where to get the packages (repos) 13 S S S SStack B repos services S S S SStack A repos services S S S S Stack C extends Stack B repos services S S+
  • © Hortonworks Inc. 2013 Ambari + Redhat GlusterFS Integration • Using Ambari to deploy / manage cluster with distributed file system other than HDFS –HCFS: GlusterFS as first implementation –Pluggability with other HCFS’s –See AMBARI-1817 Page 14 MapReduce Hive Distributed File System HDFS GlusterFS HBasePig Other HCFS …
  • © Hortonworks Inc. 2013 Ambari + Accumulo Integration • Using Ambari to deploy / manage cluster with Accumulo –Google Summer of Code project –See AMBARI-1930 MapReduce Hive Distributed File System HBasePig
  • © Hortonworks Inc. 2013 Splunk + Ambari Integration • Head over to Splunk’s booth tomorrow to see Ambari integrated into Splunk’s Management UI for Hadoop monitoring and management Page 16 +
  • © Hortonworks Inc. 2013 Thanks! • Next Up: APIs and SPIs – How to Integrate with Ambari Page 17