Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Apache Ambari BOF - APIs - Hadoop Summit 2013

1,791 views

Published on

Apache Ambari BOF Meet Up @ Hadoop Summit 2013

APIs and SPIs – How to Integrate with Ambari

http://www.meetup.com/Apache-Ambari-User-Group/events/119184782/

Published in: Technology, Health & Medicine
  • Be the first to comment

Apache Ambari BOF - APIs - Hadoop Summit 2013

  1. 1. © Hortonworks Inc. 2013 Ambari API June 25, 2013 Page 1
  2. 2. © Hortonworks Inc. 2013 Agenda • API Overview • Monitoring • API Constructs • Management Page 2
  3. 3. © Hortonworks Inc. 2013 API Overview - Features • REST • Monitoring and Management of Hadoop Cluster • Partial Response • Query Predicates Page 3
  4. 4. © Hortonworks Inc. 2013 Monitoring • Read state of Hadoop resources. Example: api/v1/clusters/cluster1/services/HDFS/components/NAMENODE { "href" : "http://ec2…1/services/HDFS/components/NAMENODE", "metrics" : { "boottime" : 1.364912931E9, "process" : { "proc_total" : 752.333333333, "proc_run" : 1.64444444444 }, "rpc" : { "rpcAuthorizationSuccesses" : 141, "SentBytes" : 251758, "rpcAuthorizationFailures" : 0,… Page 4
  5. 5. © Hortonworks Inc. 2013 API Overview – Resource Types • Collection Resource: This resource type doesn’t refer to any specific resource; rather it refers to a collection of resources. For example: api/v1/clusters/cluster1/services Returns a collection of services • Instance Resource: This resource type refers to a single specific resource. For example: api/v1/clusters/cluster1/services/HDFS Refers to the service resource identified by the id “HDFS”. Page 5
  6. 6. © Hortonworks Inc. 2013 API Overview - Response • JSON • Collection resource • Always includes request href. • Inlcludes “items” of the collection. • Each item includes href and primary id fields. • Instance resource • Always includes request href. • Always includes primary id fields. • May include metrics and properties for instance. • May include href and primary id fields for sub-resources. Page 6
  7. 7. © Hortonworks Inc. 2013 API Overview - Response Collection Resource Response Example: api/v1/clusters/cluster1/services/ { "href" : "http://ec2…api/v1/clusters/cluster1/services/", "items" : [ { "href" : "http://ec2…/clusters/cluster1/services/HDFS", "ServiceInfo" : { "cluster_name" : "cluster1", "service_name" : "HDFS" } }, { "href" : "http://ec2…/cluster1/services/MAPREDUCE",… Page 7
  8. 8. © Hortonworks Inc. 2013 API Overview - Response Instance Resource Response Example: api/v1/clusters/cluster1 { "href" : "http://ec2...com:8080/api/v1/clusters/cluster1", "Clusters" : { "cluster_name" : "cluster1", "cluster_id" : 1, "version" : "HDP-1.2.1" }, "services" : [ { "href" : "http://ec2...com:8080/api/v1/clusters/cluster1/services/MAPR EDUCE", "ServiceInfo" : { "cluster_name" : "cluster1", "service_name" : "MAPREDUCE" } }, … Page 8
  9. 9. © Hortonworks Inc. 2013 API Constructs – Partial Response • Used to control which fields are returned by a query. –restrict which fields are returned. –reach down and return data from sub-resources. • Properties, categories and sub-resources can be specified. • The wildcard ‘*’ can be used to show all… –categories, fields and sub-resources. –provides „expand‟ functionality for sub-components. • Primary id fields of a resource are always shown regardless of the specifies partial response. Page 9
  10. 10. © Hortonworks Inc. 2013 API Constructs – Partial Response Query Specific Field Example: api/v1/clusters/cluster1/services/MAPREDUCE/components/JOBTRACKE R?fields=metrics/rpc/SentBytes { "href" : "http://…/api/v1/clusters/cluster1/services/MAPREDUCE/components/ JOBTRACKER?fields=metrics/rpc/SentBytes", "metrics" : { "rpc" : { "SentBytes" : 2763921 } }, "ServiceComponentInfo" : { "cluster_name" : "cluster1", "component_name" : "JOBTRACKER", "service_name" : "MAPREDUCE" } } Page 10
  11. 11. © Hortonworks Inc. 2013 API Constructs – Partial Response Expand Sub-Resource Example: api/v1/clusters/cluster1/services/MAPREDUCE?fields=components/met rics/jvm/gcCount { "href" : "http://…/services/MAPREDUCE?fields=components/metrics/jvm/gc Count", … }, "components" : [ { "href" : "http://…/MAPREDUCE/components/JOBTRACKER", "metrics" : { "jvm" : { "gcCount" : 47 } },… Page 11
  12. 12. © Hortonworks Inc. 2013 API Constructs – Query Predicates • Limits the set of resources returned by a query. • Consists of at least one relational expression. • Can only be applied to collection resources. • Relational operators ( =, !=, <, >, <=, >= ) • Logical operators ( |, &, !) • Functions (in(), isEmpty()) • Brackets can be used to provide explicit grouping of expressions. Expressions within brackets have the highest precedence. Page 12
  13. 13. © Hortonworks Inc. 2013 API Constructs – Query Predicates Query For Started Services Example: api/v1/clusters/cluster1/services?ServiceInfo/state=STARTED { "href" : "http://…/clusters/cluster1/services?ServiceInfo/state=STARTE D", "items" : [ { "href" : "http://…/clusters/cluster1/services/NAGIOS", "ServiceInfo" : { "cluster_name" : "cluster1", "state" : "STARTED", "service_name" : "NAGIOS" } },… Page 13
  14. 14. © Hortonworks Inc. 2013 Management - Create • Create a resource. Example: Create a cluster named „c1‟ with the property „Clusters/version‟ = „HDP-1.2.0‟ POST http://…:8080/api/v1/clusters/c1 { "Clusters": { "version" : "HDP-1.2.0” } } Page 14
  15. 15. © Hortonworks Inc. 2013 Management - Update • Update a resource. Example: Update the state of all „INSTALLED‟ services to be „STARTED‟ PUT http://…:8080/api/v1/clusters/c1/services?ServiceInf o/state=INSTALLED { "ServiceInfo": { "state" : "STARTED” } } Page 15
  16. 16. © Hortonworks Inc. 2013 Management - Delete • Delete a resource. Example: Delete the cluster named „c1‟ DELETE http://…:8080/api/v1/clusters/c1 Page 16
  17. 17. © Hortonworks Inc. 2013: Q & A Page 17

×