Ambari Meetup: APIs and SPIs of Ambari

2,920 views

Published on

Ambari Meetup: APIs and SPIs of Ambari

  1. 1. AmbariAPIs and SPIs of AmbariTom Beerbower @ HortonworksApril 02, 2013© Hortonworks Inc. 2013 Page 1
  2. 2. Agenda• API Overview• Monitoring• API Constructs• Management• Error Handling• SPI Overview• SPI Usage © Hortonworks Inc. 2013 Page 2
  3. 3. API Overview - Features• REST• Monitoring and Management of Hadoop Cluster• Partial Response• Query Predicates © Hortonworks Inc. 2013 Page 3
  4. 4. Monitoring• Read state of Hadoop resources.Example:api/v1/clusters/cluster1/services/HDFS/components/NAMENODE{ "href" : "http://ec2…1/services/HDFS/components/NAMENODE", "metrics" : { "boottime" : 1.364912931E9, "process" : { "proc_total" : 752.333333333, "proc_run" : 1.64444444444 }, "rpc" : { "rpcAuthorizationSuccesses" : 141, "SentBytes" : 251758, "rpcAuthorizationFailures" : 0,… © Hortonworks Inc. 2013 Page 4
  5. 5. API Overview – Resource Types• Collection Resource: This resource type doesn’t refer to any specific resource; rather it refers to a collection of resources. For example: api/v1/clusters/cluster1/services Returns a collection of services• Instance Resource: This resource type refers to a single specific resource. For example: api/v1/clusters/cluster1/services/HDFS Refers to the service resource identified by the id “HDFS”. © Hortonworks Inc. 2013 Page 5
  6. 6. API Overview - Response• JSON• Collection resource • Always includes request href. • Inlcludes “items” of the collection. • Each item includes href and primary id fields.• Instance resource • Always includes request href. • Always includes primary id fields. • May include metrics and properties for instance. • May include href and primary id fields for sub-resources. © Hortonworks Inc. 2013 Page 6
  7. 7. API Overview - ResponseCollection Resource Response Example:api/v1/clusters/cluster1/services/{ "href" : "http://ec2…api/v1/clusters/cluster1/services/", "items" : [ { "href" : "http://ec2…/clusters/cluster1/services/HDFS", "ServiceInfo" : { "cluster_name" : "cluster1", "service_name" : "HDFS" } }, { "href" : "http://ec2…/cluster1/services/MAPREDUCE",… © Hortonworks Inc. 2013 Page 7
  8. 8. API Overview - ResponseInstance Resource Response Example:api/v1/clusters/cluster1{ "href" : "http://ec2...com:8080/api/v1/clusters/cluster1", "Clusters" : { "cluster_name" : "cluster1", "cluster_id" : 1, "version" : "HDP-1.2.1" }, "services" : [ { "href" :"http://ec2...com:8080/api/v1/clusters/cluster1/services/MAPREDUCE", "ServiceInfo" : { "cluster_name" : "cluster1", "service_name" : "MAPREDUCE" } }, … © Hortonworks Inc. 2013 Page 8
  9. 9. API Constructs – Partial Response• Used to control which fields are returned by a query. – restrict which fields are returned. – reach down and return data from sub-resources.• Properties, categories and sub-resources can be specified.• The wildcard ‘*’ can be used to show all… – categories, fields and sub-resources. – provides „expand‟ functionality for sub-components.• Primary id fields of a resource are always shown regardless of the specifies partial response. © Hortonworks Inc. 2013 Page 9
  10. 10. API Constructs – Partial ResponseQuery Specific Field Example:api/v1/clusters/cluster1/services/MAPREDUCE/components/JOBTRACKER?fields=metrics/rpc/SentBytes{ "href" :"http://…/api/v1/clusters/cluster1/services/MAPREDUCE/components/JOBTRACKER?fields=metrics/rpc/SentBytes", "metrics" : { "rpc" : { "SentBytes" : 2763921 } }, "ServiceComponentInfo" : { "cluster_name" : "cluster1", "component_name" : "JOBTRACKER", "service_name" : "MAPREDUCE" }} © Hortonworks Inc. 2013 Page 10
  11. 11. API Constructs – Partial ResponseExpand Sub-Resource Example:api/v1/clusters/cluster1/services/MAPREDUCE?fields=components/metrics/jvm/gcCount{ "href" :"http://…/services/MAPREDUCE?fields=components/metrics/jvm/gcCount", … }, "components" : [ { "href" : "http://…/MAPREDUCE/components/JOBTRACKER", "metrics" : { "jvm" : { "gcCount" : 47 } },… © Hortonworks Inc. 2013 Page 11
  12. 12. API Constructs – Query Predicates• Limits the set of resources returned by a query.• Consists of at least one relational expression.• Can only be applied to collection resources.• Relational operators ( =, !=, <, >, <=, >= )• Logical operators ( |, &, !)• Functions (in(), isEmpty())• Brackets can be used to provide explicit grouping of expressions. Expressions within brackets have the highest precedence. © Hortonworks Inc. 2013 Page 12
  13. 13. API Constructs – Query PredicatesQuery For Started Services Example:api/v1/clusters/cluster1/services?ServiceInfo/state=STARTED{ "href" :"http://…/clusters/cluster1/services?ServiceInfo/state=STARTED", "items" : [ { "href" : "http://…/clusters/cluster1/services/NAGIOS", "ServiceInfo" : { "cluster_name" : "cluster1", "state" : "STARTED", "service_name" : "NAGIOS" } },… © Hortonworks Inc. 2013 Page 13
  14. 14. Management - Create• Create a resource.Example:Create a cluster named „c1‟ with the property„Clusters/version‟ = „HDP-1.2.0‟POST http://…:8080/api/v1/clusters/c1{ "Clusters": { "version" : "HDP-1.2.0” }} © Hortonworks Inc. 2013 Page 14
  15. 15. Management - Update• Update a resource.Example:Update the state of all „INSTALLED‟ services to be„STARTED‟PUThttp://…:8080/api/v1/clusters/c1/services?ServiceInfo/state=INSTALLED{ "ServiceInfo": { "state" : "STARTED” }} © Hortonworks Inc. 2013 Page 15
  16. 16. Management - Delete• Delete a resource.Example:Delete the cluster named „c1‟DELETE http://…:8080/api/v1/clusters/c1 © Hortonworks Inc. 2013 Page 16
  17. 17. Error Handling HTTP Code Description 200 OK 400 Bad Request 401 Unauthorized 403 Forbidden 404 Not Found 500 Internal Server Error © Hortonworks Inc. 2013 Page 17
  18. 18. Error Handling - Examplesapi/v1/clusters/BAD_CLUSTER_NAME404 Not Found{ "status" : 404, "message" : "The requested resource doesnt exist: Cluster not found,clusterName=BAD_CLUSTER_NAME"}api/v1/clusters/cluster1?fields=BAD_FIELD400 Bad Request{ "status" : 400, "message" : "The properties [BAD_FIELD] specified in the request or predicate are not supported for the resource type Cluster."} © Hortonworks Inc. 2013 Page 18
  19. 19. SPI Overview• Service framework for handling requests for the various resources that it manages and monitors. /api/v1/clusters/c1/hosts/host1/host_components Handler getResources(Resource.Type.HostComponent, request, predicate) Cluster ResourceProvider Service ResourceProvider Host ClusterController ResourceProvider Component ResourceProvider getResources(request, predicate) HostComponent ResourceProvider © Hortonworks Inc. 2013 Page 19
  20. 20. SPI Use Case• Pluggable resource providers allow API usage with a GSInstaller installed cluster. API SPI GSInstaller Default ResourceProviders ResourceProviders Ambari DB GSInstaller Artifacts © Hortonworks Inc. 2013 Page 20
  21. 21. Q&A© Hortonworks Inc. 2013 Page 21

×