Published on

Connecting to remote repositories via the docproxy interface in Sakai OAE

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • The 0.9 release I used is from March of this year. It is pre-sparse content. Some of the changes I made were submitted and incorporated into the code base (a small API change and corresponding search servlet change to allow properties for results sets to be passed back). Changes I have made since are a bit more fundamental, have been submitted.
  • Sakai3 UX talks to the Nakamura back end by http. These can be resources or service endpoints
  • Servlet resolution is magic; things like extensions do not filter, but rather contribute to a score. Servlet bindings are magically registered using annotation incantations. In this case, a servlet in the files bundle checks to see if the URL had the extension, and if so, it sends JSON. If not, the default servlet is used and just returns the bits.
  • You can access these resources directly by entering the sakai server URL and then /p/<jcr:name> For example, this is what you get by accessing Without the extensions, you get the bits.
  • This is similar to the last example, same idea, same extensions. Without the extensions, you get the contents of the file, which is just the external URL.
  • Not the same as just pointing out to a web resource. There may be access permissions involved, for example. Also, we will want to allow other methods like creating, updating and deleting. This requires more infrastructure. the docproxy bundle addresses this.
  • So there will be a service endpoint for searching against each repository type.
  • Search endpoint probably could also use the repository-type property instead of repository-ref (it would be more elegant) – this would require a small change in the search servlet The first two of these are placed in the source tree and are created when the instance is first fired up.
  • Have changed resourceType for metadata servlet
  • Trace through an example search ... hits the service endpoint (in our case, /var/search/docproxy/dcs.json) This request is resolved to the ExternalDocumentSearchServlet, which calls the search method in the processor. This returns an ExternalSerachResultSet of ExternalDocumentResults to the servlet, which in turn serializes this in JSON format. This is presented to the UX.
  • Our data model is derived from OAIS and PLANETS. Have Collections, DeliverableUnits, Manifestations, Files Most useful metadata for searching is in the Dus, so we search for them, give a list ... user selects an interesting one, then gets to see all the Files in there, can then import the files to sakai.
  • Create could be interesting, because we need to submit in a Submission Information Package (SIP) Will probably need a tool to do this.
  • Sakai11docproxy

    1. 1. Connecting to Remote Repositories: Implementing the Docproxy Interface in Sakai OAE Jim Martino, Software Engineer, Johns Hopkins University
    2. 2. Outline of talk <ul><li>Disclaimers </li></ul><ul><li>How docproxy works </li></ul><ul><li>Our Implementation details </li></ul><ul><li>Demo </li></ul><ul><li>Future directions </li></ul>12th Sakai Conference – Los Angeles, California – June 14 - 16
    3. 3. Disclaimers <ul><li>Sakai3 / Nakamura is currently a moving target. This work was done against Nakamura 0.9 </li></ul><ul><li>I have had to tweak a few things to get stuff to work according to my understanding of how they should work. My understanding may be incomplete. </li></ul>12th Sakai Conference – Los Angeles, California – June 14 - 16
    4. 4. Simple diagram (not to scale)
    5. 5. What happens when the UX requests something from Nakamura? <ul><li>UX hits a URL, which Sling handles </li></ul><ul><li>Sling looks at the resourceType and path of the node at the URL, and resolves which servlet to hand it off to </li></ul><ul><li>In the case for content, the URL is of resourceType sakai/pooled-content. </li></ul><ul><li>Giving Sling an undecorated pooled content URL returns the bits if there is no extension. </li></ul><ul><li>If the extension metadata.tidy.json is appended, the metadata for the file is returned in JSON </li></ul>12th Sakai Conference – Los Angeles, California – June 14 - 16
    6. 6. Locally stored file example { &quot;jcr:path&quot;: &quot;/_p/m/da/qi/i2/jqnmZuUQc63Z7mAoIA9iisGc&quot;, &quot;jcr:name&quot;: &quot;jqnmZuUQc63Z7mAoIA9iisGc&quot;, &quot;sakai:fileextension&quot;: &quot;usage&quot;, &quot;sakai:permissions&quot;: &quot;public&quot;, &quot;sakai:directory&quot;: &quot;default&quot;, &quot;sling:resourceType&quot;: &quot;sakai/pooled-content&quot;, &quot;jcr:uuid&quot;: &quot;17c29862-6431-41de-8bcf-6efb7e087527&quot;, &quot;jcr:mixinTypes&quot;: [ &quot;rep:AccessControllable&quot; ], &quot;jcr:createdBy&quot;: &quot;admin&quot;, &quot;sakai:pooled-content-file-name&quot;: &quot;usage&quot;, &quot;sakai:copyright&quot;: &quot;creativecommons&quot;, &quot;jcr:created&quot;: &quot;2011-05-19T16:28:24-04:00&quot;, &quot;sakai:pool-content-created-for&quot;: &quot;jhopkins&quot;, &quot;jcr:primaryType&quot;: &quot;sakai:pooled-content&quot; }
    7. 7. Sakai link node example
    8. 8. docproxy <ul><li>What do we need to handle external files? The docproxy bundle addresses this. There are: </li></ul><ul><ul><ul><li>several new Sling resourceTypes: </li></ul></ul></ul><ul><ul><ul><li>servlets to handle these resourceTypes </li></ul></ul></ul><ul><ul><ul><li>interfaces to ensure the action of the servlets will be generic, and to ensure compatibility with everything else </li></ul></ul></ul>12th Sakai Conference – Los Angeles, California – June 14 - 16
    9. 9. ExternalDocumentResultMetadata interface
    10. 10. ExternalRepositoryProcessor interface
    11. 11. resourceTypes <ul><ul><ul><li>We have several new Sling resourceTypes: </li></ul></ul></ul><ul><ul><ul><li>One type for external resources themselves, of course </li></ul></ul></ul><ul><ul><ul><li>One type for the repository itself for operations which do not address a specific external resource </li></ul></ul></ul><ul><ul><ul><li>One type for searching against the external repository </li></ul></ul></ul>12th Sakai Conference – Los Angeles, California – June 14 - 16
    12. 12. Docproxy nodes needed for an implementation
    13. 13. Servlets We also have servlets to handle the new resource types. Servlet resolution will depend primarily on the resourceType and node path, with further refining from any extensions and selectors appended to the URL. These servlets then call the appropriate methods in the repository processors. 12th Sakai Conference – Los Angeles, California – June 14 - 16
    14. 14. Some binding information for docproxy servlets, and which processor methods they call CreateExternalDocumentProxyServlet: resourceTypes = { &quot;sakai/external-repository&quot; }, selectors = { &quot;create&quot; } Calls: updateDocument DeleteExternalDocumentProxyServlet: resourceTypes ={ &quot;sakai/external-repository&quot; }, selectors = { &quot;delete&quot; } Calls: removeDocument ExternalDocumentSearchServlet: resourceTypes = { &quot;sakai/external-repository-search&quot; }, extensions = { &quot;json&quot; } Calls: search ExternalDocumentMetadataServlet: resourceTypes = {&quot;sling/nonexisting&quot;, &quot;sakai/external-repository -document &quot; } selectors = &quot;metadata”, extensions = &quot;json&quot; Calls: getDocumentMetadata ExternalDocumentProxyServlet: resourceTypes = { &quot;sling/nonexisting&quot;, &quot;sakai/external-repository-document&quot; } Calls: getDocument
    15. 15. Example transaction
    16. 16. Sample ExtgernalSearchResultSet from dcs search service
    17. 17. Implementation Details <ul><li>Our implementation is based on the url processor in the docproxy bundle, since our repository exposes RESTful services. </li></ul><ul><li>Data packaging from the remote repository affects repository processor implementation. </li></ul><ul><li>UX integration may also be affected by data packaging structure. </li></ul>12th Sakai Conference – Los Angeles, California – June 14 - 16
    18. 18. Initial Design <ul><li>The first implementation was for a read-only repository, with as tight an integration as possible with native content in the UX. </li></ul><ul><li>Requirements: </li></ul><ul><li>Remote content should be an option in the content search menu </li></ul><ul><li>Should be able to merge remote search results with other remote searches and with local searches </li></ul><ul><li>Imported content should behave as much like native content as possible </li></ul>12th Sakai Conference – Los Angeles, California – June 14 - 16
    19. 19. Design decisions <ul><ul><ul><li>Use sakai links to point into docproxy document nodes </li></ul></ul></ul><ul><ul><ul><li>Insert additional methods, configurations, etc. into existing UX files </li></ul></ul></ul>12th Sakai Conference – Los Angeles, California – June 14 - 16
    20. 20. Data Conservancy link node
    21. 21. Sakai OAE login screen
    22. 22. Skip Class logs in ...
    23. 23. Skip's home page
    24. 24. The content tab, with Data Conservancy added to the list of facets
    25. 25. We enter the search term, with “Content I Manage” as the active facet
    26. 26. Sakai does not find any matching content
    27. 27. Change the facet to Data Conservancy, and we see remote hits (DUs)
    28. 28. We select the first one, and get a list of files in the Rock Sample A-399 DU
    29. 29. We click on A-399_2_r.jpg to see if it's something we want
    30. 30. We go back, then import it, and now see it in our content
    31. 31. Clicking on this, we pull up the link. Let's add some tags.
    32. 32. Ok ... now let's search for it under Content I Manage
    33. 33. Ta-da!
    34. 34. Further Directions <ul><li>Clean up current implementation </li></ul><ul><li>Implement Create/Update method </li></ul><ul><li>May need to develop a widget to handle submissions </li></ul><ul><li>Continue to review code to see if the new use cases are supported </li></ul>12th Sakai Conference – Los Angeles, California – June 14 - 16