• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
An Overview of Ambari
 

An Overview of Ambari

on

  • 785 views

Jeff Sposetti of Ambari discusses the Apache project Ambari used to help deploy and provision Hadoop Clusters ...

Jeff Sposetti of Ambari discusses the Apache project Ambari used to help deploy and provision Hadoop Clusters
- Ambari Overview and the Community
- Ambari Architecture - Provisioning Clusters and Services -Standard Services: HDFS, YARN, MR2, Hive, new Services: Storm, Falcon
- Management and Monitoring Capabilities -Nagios and Ganglia Integration
- Key Innovation Features -Ambari Stacks providing dynamic service lifecycle -Ambari BluePrints powering Savannah OpenStack -Ambari Views enabling custom UI development

Statistics

Views

Total Views
785
Views on SlideShare
743
Embed Views
42

Actions

Likes
1
Downloads
47
Comments
0

2 Embeds 42

http://vimeo.com 22
http://www.slideee.com 20

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    An Overview of Ambari An Overview of Ambari Presentation Transcript

    • © Hortonworks Inc. 2014 Apache Ambari Provisioning, Managing and Monitoring Hadoop
    • © Hortonworks Inc. 2014 Agenda • Ambari Overview and the Community • Ambari Architecture • Provisioning Clusters and Services • Extensibility Features – Ambari Stacks – Ambari Views – Ambari Blueprints • Releases • Learn More • Q & A Page 2
    • © Hortonworks Inc. 2014 Hadoop Operations with Ambari Provision: Simplified deployment across platforms Manage: Consistent controls across the Stack Monitor: Visibility into key cluster metrics -  Single pane of glass for Hadoop & System status -  Pre-configured metrics & alerts -  Single point for cluster operations -  Advanced configurations and host controls -  Wizard-driven cluster install experience -  Cloud, virtual and physical environments Apache Ambari is a 100% open source framework for provisioning, managing and monitoring Hadoop clusters
    • © Hortonworks Inc. 2014 100% Apache Open Source • Active Community •  60+ Contributors / 30+ Committers •  200+ Ambari User Group Members Page 4 2013 Oct Apache Ambari 1.4.1 Released Adds operations for Hadoop 2 Stack Ambari Ambari 1.2.0 Released Operations for Hadoop 1 Stack 2013 Jan 2013 Dec Apache Ambari Graduates to Top Level Project
    • © Hortonworks Inc. 2014 Ambari Release Cadence Page 5 Ambari 1.4.3 Jan 2014 Ambari 1.4.4 Feb 2014 Ambari 1.5.0 Apr 2014 Ambari 1.7.0 2H2014 Ambari 1.6.0 May 2014
    • © Hortonworks Inc. 2014 Demo #1 Cluster Management Page 6
    • © Hortonworks Inc. 2014 System Architecture 7 Ambari Server Host Agent Host Agent Host Agent Ambari Web DB /clusters! Host Agent
    • © Hortonworks Inc. 2014 Ambari Server Architecture 8 DB Orchestrator Monitoring REST API Request Dispatcher Ambari Web Ambari Server Ambari Agents Metrics Providers AuthProvider /clusters! User! Repo! java! python! JS! RDBMS! LDAP! REST API Configurable Auth Provider Cluster Configurations and Topology Web Client AD!
    • © Hortonworks Inc. 2014 Installing Ambari • Install Ambari Server • Setup Options – JDK (Oracle JDK 6, Oracle JDK 7, Custom JDK) – Database (PostgreSQL, MySQL or Oracle) – Server Port and HTTPS – User Authentication (Local, LDAP or AD) – Server-Agent Two-way SSL • Start Ambari Server • Browse to Ambari Web for Cluster Install Wizard • Install Ambari Agents – Option #1: Automate via SSH – Option #2: Manual Page 9
    • © Hortonworks Inc. 2014 Installing Agents Page 10 OPTION 1! automate! provide SSH Key! OPTION 2! manual!
    • © Hortonworks Inc. 2014 Option #1: Ambari Agent via SSH Page 11 Ambari Server Ambari Agent 1) Configure Ambari Repo! 2) Copy Ambari Agent Setup script! 3) Execute Ambari Agent Setup script! 4) Install ambari-agent! 5) Configure ambari-agent.ini! 6) Start ambari-agent! 7) Begin Ambari Agent Registration! Uses the SSH
    • © Hortonworks Inc. 2014 Option #2: Ambari Agent via Manual Page 12 Ambari Server Ambari Agent 2) Install ambari-agent! 3) Configure ambari-agent.ini! 4) Start ambari-agent! 5) Begin Ambari Agent Registration! 1) Configure Ambari repo! ** User executes all ops on the Agent host(s). No requirement for SSH Private Key. No SSH
    • © Hortonworks Inc. 2014 Ambari Agent Registration Page 13 Ambari Server Ambari Agent 11) Agent Heartbeat Begins! 1) Connect on Handshake port 8441! 4) Sign Agent Cert! 2) Download Server Cert! 6) Connect on Registration port 8440! 7) Perform 2WAY auth using Agent Cert! 10) Complete Host Registration! Agent Host 8) Get FQDN! 9) Register host! 8a) Host Script! 3) Request to Sign Agent Cert! 5) Download Agent Cert + Disconnect!
    • © Hortonworks Inc. 2014 Demo #2 Cluster Install Page 14
    • © Hortonworks Inc. 2014 Extensibility Features Ambari Stacks Ambari Blueprints Ambari Views Page 15
    • © Hortonworks Inc. 2014 Ambari Stacks Page 16
    • © Hortonworks Inc. 2014 Ambari Stacks • Separate “the Stack” from “Ambari” • Make it easier to bring more than a traditional “Hadoop Stack” under Ambari management • Defining a consistent Service lifecycle management interface that can be extended • Dynamically add Stacks + Services definitions Page 17
    • © Hortonworks Inc. 2014 Stack Details • Stacks define Services + Repos – What is in the Stack, and where to get the bits • Each Service has a definition – What Components are part of the Service • Each Service has defined lifecycle commands – start, stop, status, install, configure • Lifecycle is controlled via command scripts • Ability to define “custom” commands • Ability to “extend” Stacks Page 18 AMBARI   SERVER   Stack Command   Scripts   Service   Defini:ons   AMBARI   AGENT/S   AMBARI   AGENT/S   AMBARI   AGENT/S   python!xml! Repos
    • © Hortonworks Inc. 2014 Stack Mechanics • Ambari Server reads Stack definitions on start • Ambari Server sends a command to Agents • Agents download Stack definition + command scripts • Agent executes command • If the Stack definition changes, Agent will request latest Stack definition + command scripts Page 19
    • © Hortonworks Inc. 2014 Ambari Blueprints Page 20
    • © Hortonworks Inc. 2014 Ambari Blueprints Page 21 STACK   HOSTS   BLUEPRINT  
    • © Hortonworks Inc. 2014 Automating Installs •  Blueprint defines a cluster layout and component configuration for a Stack •  Provide Blueprint to Ambari to perform cluster installation (no wizard required) •  Export Blueprint from existing cluster BLUEPRINT! Ambari   Submit Blueprint to Ambari Hadoop   Cluster Ambari provisions cluster BLUEPRINT! <stack>! <host>! <service>! <component>! <config>! HOST! MANIFEST! <host>! <meta>! SERVICE! CONFIGS! <props>! Page 22
    • © Hortonworks Inc. 2014 Ambari + OpenStack Page 23 OpenStack  Infrastructure   Horizon Savanna AMBARI   SERVER   Hadoop  Cluster
    • © Hortonworks Inc. 2014 Ambari Views Page 24
    • © Hortonworks Inc. 2014 Ambari Views •  Goal: Customize the Ambari Web experience •  Views compliment Stack Extensibility – Stack Extensibility makes custom Stack Services available to Ambari – Views expose custom UI features for Services •  Ambari Admins can entitle “views” to Ambari Web users –  Entitlements framework for finer-grained permissions control for Ambari users Page 25
    • © Hortonworks Inc. 2014 View Usage • Developer Creates a View • Deploys View to Ambari Server • Ambari Admin creates instances of Views • Ambari Admin entitles Views to users Page 26
    • © Hortonworks Inc. 2014 View Configuration + Entitlement Page 27
    • © Hortonworks Inc. 2014 Customized Views Page 28 “admin” gets operator controls “joe.user” gets specific views based on entitlements
    • © Hortonworks Inc. 2014 Releases What’s Up Next Page 29
    • © Hortonworks Inc. 2014 What’s New in Ambari 1.5.0 • Maintenance Mode • Rolling Restarts • Bulk Host Operations • Decommission TTs, NMs, RSs • Add Service • Refresh Client Configs • Default JDK 7 • For a complete list of changes: https://issues.apache.org/jira/browse/AMBARI/fixforversion/12325019 Page 30
    • © Hortonworks Inc. 2014 Ambari Futures • Operations – Ambari Blueprints – Component log access + search – Ubuntu and Windows support • Extensibility – Finer-grain user entitlements – Ambari Views • Insight – Troubleshooting + Performance Tuning – Capacity Planning Page 31
    • © Hortonworks Inc. 2014 Learn More Page 32 Resource Location Apache Ambari Project Page http://ambari.apache.org Ambari Project Wiki https://cwiki.apache.org/confluence/display/AMBARI Ambari Project JIRA https://issues.apache.org/jira/browse/AMBARI
    • © Hortonworks Inc. 2014 Q & A Page 33