Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Getting started with DSpace 7 REST API


Published on

Workshop held at Open Repository 2018, Bozeman, Montana

As of late 2016, a DSpace 7 UI Working Group has begun developing an Angular User Interface which will replace the existing UIs in DSpace 7. This effort also includes the development of a new REST API for DSpace, designed to follow the principles of a RESTful webservice and adopt emerging standards and formats. The goals of the REST API are twofold: (1) to fully support the new Angular UI, and (2) to provide a rich, RESTful integration point for third-party services and tools.

This workshop will allow developers to become more familiar with the new REST API framework before DSpace 7 is released.

This hands-on developers workshop will provide attendees with an overview of the DSpace 7 REST framework:

- standards / best practices that the API is based on (HAL, JSON+PATCH, JWT)

- DSpace 7 REST Contract (documentation of all endpoints)

- interacting with the REST API (via HAL browser, curl and/or postman)

- how to build new endpoints into the REST API

- where to look when issues arise

- how to document and test existing/new endpoints

Attendees will be expected to setup a virtual machine (or install the DSpace 7 codebase locally) to get more familiar with the codebase/development tools.

Published in: Technology
  • ♣♣ 10 Easy Ways to Improve Your Performance in Bed... ♥♥♥
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Getting started with DSpace 7 REST API

  1. 1. Getting Started with DSpace 7 REST API Andrea Bollini, 4Science Terry Brady, Georgetown Tim Donohue, DuraSpace
  2. 2. Workshop Schedule ❖ Why a new REST API? ❖ Interacting with REST API ❖ (Morning Tea 10:30-11:00) ❖ Design Principles of REST API ❖ Developing the REST API
  3. 3. Version 7.0 goals ★ Single, Angular UI (new) ★ Fully-featured REST API (new) ★ Configurable Entities (new) ★ Align w/ NGR recommendations ★ Existing, core Java backend (DB, Solr, Assetstore)
  4. 4. Current Status ❏ Enhanced backend (object store, caching, speed improvements) ❏ Browse (Homepage, Community, Collection, Item pages) ❏ Search ❏ Authentication and Authorization ❏ Submission ❏ Approval Workflows ❏ Content Management (Create/Edit/Delete tasks) ❏ Administration ❏ Statistics ❏ Configurable Entities support Development Planning Spreadsheet
  5. 5. Why a new REST API?
  6. 6. Why a new REST API? Covers only a subset of DSpace functionality Not based on current REST best practices or standards Handcrafted in Jersey, while most DSpace code uses Spring technologies 4.x - 6.x No search No submit / workflows Limited admin operations Limited write / delete (4.x was read only)
  7. 7. Why a new REST API? All features MUST be in REST API (for Angular UI) Defined REST Contract. HATEOAS, ALPS, HAL format Built using Spring technologies (Spring Boot, MVC, HATEOAS) 7.x Bonus: better third-party app integration!
  8. 8. HATEOAS, HAL, & ALPS, oh my! ALPS = Application Level Profile Semantics* Describes the operations (actions) available for all REST endpoints. (Machine-readable) metadata about how to interact with the API. HAL Format = Hypertext Application Language (JSON or XML) A standard format for making REST APIs browseable (think HTML for machines). Open source HAL Browser available. RESULT: A standards-based, browseable, self-describing REST API HATEOAS = Hypertext As The Engine Of Application State In each response, include “links” to available next requests. Results in better decoupling, as API is self-describing.
  9. 9. Stateless (no session) Specifications • JSON web tokens (JWT) • Cross-Origin Resource Sharing (CORS) Headers HTTP Resources ● URIs reference individual resources or collections (of resources) ● /items/{uuid} and /items Formats: JSON* or XML HTTP methods GET (read), HEAD (read headers only) POST (create) PUT (replace), PATCH (partial update) DELETE (remove) OPTIONS (verify access, e.g. via CORS) HTTP return codes 2xx (Success) 3xx (Redirect) 4xx (Client error) 5xx (Server error) REST Terminology
  10. 10. Interacting with the REST API (HAL browser, Postman, command line?)
  11. 11. Interacting with a REST API You can use the command line… curl "" But, there are better ways… using a REST client
  12. 12. Goals for this section • Understand the endpoints that are available • Learn how to call these endpoints – As an anonymous user – As an authenticated user • Appreciate the self-describing nature of the DSpace REST design
  13. 13. We will look at 2 tools • HAL Browser • Postman The REST API currently supports very limited modification actions (create/update/delete). More support will be added in the future.
  14. 14. HAL Browser • Development public instance (provided by 4science) – / • Workshop Link – • Navigate to the link above to explore
  15. 15. HAL Browsing - Example
  16. 16. Response Object Components • Response Properties – Often pagination details • Links – What you can do next • Embedded Resources – List of objects which may contain • Properties • Links
  17. 17. Open the API Landing Page • Open the Entry Point – • Request (in HAL Browser) – /api
  18. 18. Request + Headers
  19. 19. Response Properties
  20. 20. Links: All endpoints are available from the Entry Point
  21. 21. Response Headers
  22. 22. Response Body
  23. 23. Example: Items Endpoint
  24. 24. Items Response
  25. 25. Links - A generic approach to paginated results
  26. 26. Embedded Item Properties
  27. 27. Embedded Item Links
  28. 28. Exercise 1: Explore HAL Browser Exercise 1 - Exploring Endpoints in HAL Browser Short Link To Exercises
  29. 29. The REST contract • – Explore the list of endpoints • Implemented • Unimplemented • Under discussion – Use GitHub pull requests to suggest changes
  30. 30. The future of REST API Documentation PR#1915
  31. 31. REST Authentication • The existing DSpace authentication methods will be supported • The REST API returns an authentication token • This token is passed to subsequent REST requests as a header
  32. 32. Why Authenticate? • View restricted items/bitstreams • Create/Update/Delete Objects – Submit items • Perform admin operations • View restricted metadata (provenance) • Note: most of these actions are not yet supported in DSpace REST
  33. 33. A subset of these operations are available in HAL
  34. 34. Exercise 2: Authenticating in HAL Exercise 2 - Authentication in HAL Browser • For this workshop, we will use password authentication
  35. 35. Limitations of HAL Authn • Credentials only passed for GET operations • File uploads not supported
  36. 36. Postman - A development client for REST API’s
  37. 37. Postman - A tool for sending REST requests • Postman is a tool for interacting with various REST services – Even ones without a HAL Browser • Using a web browser, it is difficult to construct complex requests
  38. 38. Postman
  39. 39. Tabs and Collections help you organize and re-use requests
  40. 40. Request Area
  41. 41. Parameters and Headers
  42. 42. Response Panel
  43. 43. Response Headers
  44. 44. Browsing with Postman • View all items – /api/core/items • Change page size – /api/core/items?size=5 • Change starting page – /api/core/items?size=5&page=2
  45. 45. Exercise 3: Browsing with Postman Exercise 3 - Browsing with Postman
  46. 46. Postman - Authenticating as a User
  47. 47. AuthN Status in Postman (no AuthN token)
  48. 48. Authenticating in Postman
  49. 49. AuthN Status in Postman (passing Bearer token)
  50. 50. Let’s Attempt to change data • The current REST API allows a user to start a submission • POST – /api/submission/workspaceitems – Body: {} • The response will return an object with an id
  51. 51. Retrieving the Item that was created • GET – /api/submission/workspaceitems/[id] • DELETE – /api/submission/workspaceitems/[id] • GET – /api/submission/workspaceitems/[id] The second GET request will fail
  52. 52. Exercise 4: Modifying data with Postman Exercise 4 - Modifying data with Postman
  53. 53. Managing Postman • Postman allows you to use variables and scripts to manage credentials Tips on using postman
  54. 54. Design Principles
  55. 55. Stateless (no session) Specifications • JSON web tokens (JWT) • Cross-Origin Resource Sharing (CORS) Headers HTTP Resources ● URIs reference individual resources or collections (of resources) ● /items/{uuid} and /items Formats: JSON* or XML HTTP methods GET (read), HEAD (read headers only) POST (create) PUT (replace), PATCH (partial update) DELETE (remove) OPTIONS (verify access, e.g. via CORS) HTTP return codes 2xx (Success) 3xx (Redirect) 4xx (Client error) 5xx (Server error) REST Terminology
  56. 56. Hypermedia as the Engine of Application State - HATEAOS HAL Format Links are expressed in a standard/well-known way in the json response (it is also suitable for xml but DSpace7 will focus only on JSON) → enable the interactive navigation → abstract routing (URL can change without break the client) → support documentation ALPS Available methods, semantics of request and response objects discoverable in a standard way (/profile) and expressed in multiple standards format (Alps JSON, JSON Schema, XML, etc) → enable to expose only available methods (i.e. GET on resourcepolicies is disallowed) → document in a machine-readable way the request and response semantics
  57. 57. HAL Document
  58. 58. ALPS will enable...
  59. 59. REST Maturity level Image from:
  60. 60. Spring technologies Spring Bootstrap: pre-configured managed Spring platform Spring MVC: Front Controller, data binding, error handling Spring REST: MVC extension to easily support Content-Negotiation, Response in JSON format
  61. 61. Spring technologies Spring HATEOAS: library to deal with HAL Document (Resources, Resource, Links, Curie) Spring Data REST: only for inspiration (consistent design choices, HTTP error handling, support classes, etc.)
  62. 62. Spring technologies Spring Data REST Hal Browser: WebJar of the HAL browser customized to work at best with Spring Data REST Spring REST Docs: Self contained documentation with snippets from Integration Tests
  63. 63. Pagination Resource Collections are always paginated → pagination information are returned in a consistent way across all the endpoints → pagination parameters are expected in the same form in all the endpoints → common JAVA API to deal with page jump and offset
  64. 64. Search Methods Collection endpoints that allowing filtering, search or precise lookup of resources MUST provide a search link listing all the available methods
  65. 65. Right use of the HTTP Verbs: collection - POST Adds a new element to the collection. - GET Returns the first page of the resources in the collection
  66. 66. Right use of the HTTP Verbs: element GET Returns a single entity. HEAD Returns whether the item resource is available. PUT Replaces the state PATCH Similar to PUT but partially updating the resources state DELETE Deletes the resource exposed.
  67. 67. How to deal with PATCH JSON Patch specification RFC6902 Express change to a JSON Object in JSON Array of Operations executed in an atomic transaction Each successful Patch operation will return a HTTP 200 CODE with the new state of the patched resource as body similar to what is returned for a GET request.
  68. 68. How to deal with PATCH ADD / REMOVE / REPLACE / MOVE [ { "op": "test", "path": "/a/b/c", "value": "foo" }, { "op": "remove", "path": "/a/b/c" }, { "op": "add", "path": "/a/b/c", "value": [ "foo", "bar" ] }, { "op": "replace", "path": "/a/b/c", "value": 42 }, { "op": "move", "from": "/a/b/c", "path": "/a/b/d" }, { "op": "copy", "from": "/a/b/d", "path": "/a/b/e" } ] TEST & COPY (no plan to support them)
  69. 69. PATCH (More examples: [{ "op": "add", "path": "/sections/traditionalpageone/", "value": {"value": "Another, Author"} }]
  70. 70. HTTP status codes - 2xx, 3xx 200 Ok - Normal success state 201 Created - Returned when a resource is created 204 No content - Returned when the operation succeed but no content is available (i.e. hit the logo endpoint of a community without logo 206 Partial Content - DSpace 7 provides range support for bitstream download allowing streaming 302 Found - the PID endpoint redirect to the target resource
  71. 71. HTTP status codes - 4xx 400 Bad Request - if the request is invalid (not a json, etc.) 401 Unauthorized - if the request require a logged-in user 403 Forbidden - if the requester doesn't have enough privilege to execute the request 404 Not found - if the requested resource doesn't exists 405 Method Not Allowed - if the requested verb is not supported for the resource 422 Unprocessable Entity - if the request is semantically invalid (i.e. attempt to add undefined metadata, deposit an invalid workspace)
  72. 72. Developing the REST API
  73. 73. Code used in the workshop Branch: or2018-workshop (final result) | tags will help to move step by step or2018-workshop-start
  74. 74. How URL are translated to JAVA code Where is my web.xml? It is a spring boot application with web support (MVC) Controller → are the “new” Servlet @RestController
  75. 75. What about the lovable spring xml files? sorry now we use annotations :) Component scanning enabled on repository, converter, utils packages @Component @Service
  76. 76. Goal: Enhance the EPerson endpoint • List EPersons (already exists) • Get EPerson (already exists) Walkthrough the implementation and the Integration Tests [tag: or2018-workshop-start]
  77. 77. Rest Model Class • POJO used as DTO between the REST layer and the services. • Clean and stable version of our data model • The converter class transcode the DSpace-API objects in REST objects
  78. 78. Repository Class • It is defined in the Repository design pattern • It implements the Data Access Layer in DDD • Close to the domain model, abstracting access to a collection
  79. 79. Resource Class • Wrap the Domain Class inside an HAL document • Provide supports to add links • Manage embedded objects
  80. 80. Integration Test Builder and Matcher helper classes to keep code compact & cleanup the test database between runs One or more test for each repository methods - Separate test for different scenarios (exception, limit case, normal case) - Multiple calls in the same test to cross check the intent changes
  81. 81. Add security check Learn how to authenticate an user in the Integration Test [or18-ws1-it] Secure the main eperson endpoint (only administrators can access) [or18-ws1-fix]
  82. 82. Enhancing EPerson endpoint Implement Search by email, by name (walkthrough) 1. Write Integration Tests in advance [or18-ws2-it] 2. Implement the endpoint, run the test [or18-ws2-search / or18-ws2-search2] 3. Interact using the HAL browser [or18-ws2-fix]
  83. 83. Enhancing EPerson endpoint Implement DELETE (walkthrough) 1. ITs successful DELETE, failures due to Authorization issues, DB constraints [or18-ws3-it] 2. Check the implementation [or18-ws3-fix]
  84. 84. Enhancing EPerson endpoint Implement POST (walkthrough) • How to test a create operation • How to implement [or18-ws4]
  85. 85. Enhancing EPerson endpoint Implement Patch (walkthrough) • Change the CanLogin flag • Change the Password [or18-ws5]
  86. 86. Enhancing EPerson endpoint BONUS (discussion only) • List/Create/Delete RegistrationToken • How to transform a RegistrationToken in an EPerson (text/uri-list)
  87. 87. DSpace 7 will arrive more quickly with more help! ❏ 1st DSpace 7 Community Sprint: May 7 - 18 ❏ 14 pull requests, 8 contributors (2 first time) ❏ 2nd DSpace 7 Community Sprint: July 9 - 20 No availability requirements! Choose your own tickets/tasks! No DSpace 7 experience necessary! (Coaches available) Join a Development Sprint! Sign-up now!
  88. 88. Questions? Slides available at Sl