Building a Web API Platform with Open Source oAuth 2.0, REST, and NoSQL (JavaOne 2012)

32,308 views

Published on

More and more companies provide Web APIs for their core services as an effective way to foster an ecosystem, but you need to have an API platform to host and manage the Web APIs. Building one from scratch can be challenging. This session teaches you how to create your API platform based on oAuth 2.0, REST, and NoSQL technologies by using open source stacks, including Apache projects such as Tomcat, Tuscany, Wink, Amber, and HTTP clients together with NoSQL solutions such as MongoDB and Redis. From real-world experience, you’ll learn the key components and techniques for creating a robust and scalable Web API server.

Published in: Technology

Building a Web API Platform with Open Source oAuth 2.0, REST, and NoSQL (JavaOne 2012)

  1. 1. BUILDING A WEB API PLATFORM WITH OPENSOURCE OAUTH 2.0, REST, AND NOSQL SESSION 6946 JAVAONE 2012
  2. 2. About the speakers Raymond Feng Software Engineer – Shutterfly, Inc.  Staff  Member – Apache Software Foundation  Committer – Apache Tuscany, Wink, Amber  Co-author – Tuscany SCA In Action
  3. 3. Agenda Why a Web API platform The key components of Web API platform stack  OAuth 2.0  Http Reverse Proxy  Metrics  API discovering, exploring and playing Q&A
  4. 4. Overview of a Web APIPlatform
  5. 5. Why a Web APIs platform? Why Web API?  A great way to build the ecosystem  For some companies, APIs = products  Proliferation of mobile clients  Universal access for internal systems/web or mobile fronts/third party apps This talk is about the platform/infrastructure behind the curtain to support Web API calls. It’s NOT about Web API design/development/security.
  6. 6. What’s behind the scene? What’s behind an oAuth 2.0 protected REST API call?  GET https://api.<mycompany>.com/me/albums?acces s_token=<oAuth 2.0 access token>  POST https://api.<mycompany>.com/me/albums?acces s_token=<oAuth 2.0 access token>  Content-Type: application/json  Accept: application/json  Request body: {“name”: “Summer 2012”}
  7. 7. A Web API Platform QoS plugins HTTP Reverse Proxy Auth Protected Protected Resource Resource Metrics Client ClientApplication Throttling Application Mediation User Sign-in Page … Infrastructure Distributed data grid (service registry, metrics, quota/usage) Persistenc Security e (Identity management, client application registration, user authentication, token management, resource ACLs)
  8. 8. Key components Authentication/Authorization (oAuth 2.0) Reverse proxy (API dispatching) Mediation Monitoring Analytics Backend infrastructure  Client/Token management  Service registry  Metrics aggregation  Metrics visualization Developer portal  Application registration  Dashboard  API documentation  API playground
  9. 9. API invocation flow An API request (REST/JSON) comes in using HTTPS Threat detection Authentication (oAuth 2.0 access_token) Authorization (token – scopes – resources) Rate limiting Caching Mediation Routing Analytics API Logic
  10. 10. oAuth 2.0
  11. 11. oAuth 2.0 Based on Apache Amber which implements the latest oAuth 2.0 spec  http://tools.ietf.org/html/draft-ietf-oauth-v2-31  http://incubator.apache.org/amber/ Additional SPIs are identified for oAuth 2.0 related metadata management and integration with the security infrastructure
  12. 12. oAuth 2.0 concepts OAuth defines four roles:  resource owner: An entity capable of granting access to a protected resource (e.g. end-user).  resource server: The server hosting the protected resources, capable of accepting and responding to protected resource requests using access tokens.  client: An application making protected resource requests on behalf of the resource owner and with its authorization.  authorization server: The server issuing access tokens to the client after successfully authenticating the resource owner and obtaining authorization.
  13. 13. oAuth 2.0 flows/grant types An authorization grant is a credential representing the resource owners authorization (to access its protected resources) used by the client to obtain an access token. oAuth 2.0 specification defines four grant types:  authorization code  implicit  resource owner password credentials  client credentials
  14. 14. User cases: Trusted clientapplications Trusted clients include:  Internal applications (running at the server side or client side)  Mobile clients Mapping to oAuth 2.0  Clientcredentials for access token (super user)  Resource owner password credentials for access token (run as the “resource owner”)
  15. 15. Use cases: Third partyapplications We plan to roll out web APIs to 3rd party applications oAuth 2.0 will be used as the authentication/authorization mechanism Mapping to oAuth 2.0  Authorization code flow  Implicit grant flow
  16. 16. oAuth 2.0 SPI – Clientregistration Client applications will be registered  Contact e-mail  Application name/description  Callback URIs  Authentication scheme
  17. 17. oAuth 2.0 SPI – resource ACL  Determine if a resource is protected  Public  Client-id specific  Resource-owner specific  Who are the resource owners?  Scopes of the access{"scope": "media", {"id": "media_resource","description": "Media", "operations": [ "ALL”],"expiresIn": 604800, "path": "/media","requiredAccessLevel": 1, "scopes": [ "media”]"resourceOwnerAuthorizationRequired": }false}
  18. 18. oAuth 2.0 SPI – Authentication &Authorization Make sure a client is registered Make sure a token/code is valid Authenticate a client using client_id/client_secret Authenticate a resource owner using user name/password Check the token against the protected resources (ACL) Establish the principal/subject
  19. 19. oAuth 2.0 SPI – TokenManagement Generate access/refresh tokens for a given client, resource owner, and scopes Generate authorization codes for a given client, resource owner, and scopes Look up the token metadata based on the token string Look up the authorization code metadata based on the code string Expiration Refresh tokens MongoDB as the backend store for tokens
  20. 20. oAuth 2.0 infrastructure Client Client Authorization Registration Client Registration Server Manager Registrations Endpoint Resourc e Owner Authorization Users Authenticator Endpoint (Resource Protected Owners) Resources User Internet (such as Agent user media(browser) or address Token Endpoint Token Tokens and book) Manager Authorization Codes Resource Client Access Manager Resource oAuth 2.0 Permissions Resource and Scope Filter Definitions HTTP Resource proxy Server
  21. 21. Mapping oAuth 2.0 scopes Scope Scope Scope Protected Protected Resource oAuth 2.0 Protected Resource token GET /services/addressbook/… Resource POST/services/addressbook/… Client GET /services/media/… … … Resource Owner See an example at: http://developers.facebook.com/docs/reference/api/permissions/
  22. 22. oAuth 2.0 metadata model
  23. 23. Client/Resource management(administrative) We need to have UI to  manage the resource endpoints (URI patterns and HTTP operations)  define oAuth 2.0 scopes to map to a list of resource access permissions  Manage client applications (enable/disable, setting quota, …)  Manage access tokens/authorization codes
  24. 24. Dashboard Monitor the API usages (administrative)  By client id  By user id  By resource URIs  By timeline Let developers see their client applications (developer)  Registrations  Usages  Granted permissions
  25. 25. REST APIs for APImanagement Get the list of defined scopes Display a scope Get the list of defined resources Display a token Display an authorization code Display a client registration Get the list of registered clients for a given user List tokens by client id/user id List authorization codes by client id/user id List granted permissions
  26. 26. Performance
  27. 27. oAuth 2.0
  28. 28. HTTP reverse proxy DO NOT deploy the services on the api server directly API server dispatches API calls to the back- end services Open source tools:  Apache http components: http://hc.apache.org/
  29. 29. HTTP reverse proxy – connectionmanagement HTTP connection pool  Persistent connections (keep-alive)  Check the keep alive settings at the backend web servers  Chunked transfer encoding support  Some Nginx servers don’t support chunked mode  Make sure the http entity is “consumed” so that the http connection will be released back to the pool http://hc.apache.org/httpcomponents-client- ga/tutorial/html/connmgmt.html
  30. 30. Routing/Proxying URL mapping  service registry URL templating <uriMapping> <source>/addressbook/{uid}</source> <target>http://backend.xyz.com/services/addressbook/{uid}</target> </uriMapping>  api.xyz.com/addressbook/me/contatcs  The “me” or “self” will be replaced with the resource owner from the oAuth 2.0 access token
  31. 31. Sync vs. Async Sync:  Servlet 2.5 and Apache HTTP client 4.x Async:  Servlet 3.0 async filter (Tomcat 7.0.x or Jetty 8.x)  Apache HttpAsyncClient 4.0 beta 2 final AsyncContext asyncContext = request.startAsync(); asyncContext.start(new Runnable() { public void run() { asyncDispatch(asyncContext, target); } });
  32. 32. Java vs. Node.js Which one is better?  Overhead  Scalability Servlet 2.5 sync mode + Apache http client Servlet 3.0 async mode + Apache http async client Node.js event-driven mode + http
  33. 33. Mediations Some examples:  Protocol/data translation  XML  JSON  API tracking  https://github.com/codahale/metrics  http://graphite.wikidot.com/  Analytics  CORS enablement
  34. 34. API discovery anddocumentation
  35. 35. API discovery anddocumentation Some apis/tools that help developers to discover and explore the apis  A spec to describe the apis (urls, methods, input/output/exception data model)  A UI to discover and explore the apis  Some integration with the api implementation stacks such as JAX-RS to introspect the api signatures  Client code generation tool Open source tools  http://swagger.wordnik.com/  https://developers.google.com/discovery/
  36. 36. Swagger UI
  37. 37. Q&A Thank you!

×