Your SlideShare is downloading. ×
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply



Published on

Connecting to remote repositories via the docproxy interface in Sakai OAE

Connecting to remote repositories via the docproxy interface in Sakai OAE

Published in: Technology, Business

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • The 0.9 release I used is from March of this year. It is pre-sparse content. Some of the changes I made were submitted and incorporated into the code base (a small API change and corresponding search servlet change to allow properties for results sets to be passed back). Changes I have made since are a bit more fundamental, have been submitted.
  • Sakai3 UX talks to the Nakamura back end by http. These can be resources or service endpoints
  • Servlet resolution is magic; things like extensions do not filter, but rather contribute to a score. Servlet bindings are magically registered using annotation incantations. In this case, a servlet in the files bundle checks to see if the URL had the extension, and if so, it sends JSON. If not, the default servlet is used and just returns the bits.
  • You can access these resources directly by entering the sakai server URL and then /p/<jcr:name> For example, this is what you get by accessing Without the extensions, you get the bits.
  • This is similar to the last example, same idea, same extensions. Without the extensions, you get the contents of the file, which is just the external URL.
  • Not the same as just pointing out to a web resource. There may be access permissions involved, for example. Also, we will want to allow other methods like creating, updating and deleting. This requires more infrastructure. the docproxy bundle addresses this.
  • So there will be a service endpoint for searching against each repository type.
  • Search endpoint probably could also use the repository-type property instead of repository-ref (it would be more elegant) – this would require a small change in the search servlet The first two of these are placed in the source tree and are created when the instance is first fired up.
  • Have changed resourceType for metadata servlet
  • Trace through an example search ... hits the service endpoint (in our case, /var/search/docproxy/dcs.json) This request is resolved to the ExternalDocumentSearchServlet, which calls the search method in the processor. This returns an ExternalSerachResultSet of ExternalDocumentResults to the servlet, which in turn serializes this in JSON format. This is presented to the UX.
  • Our data model is derived from OAIS and PLANETS. Have Collections, DeliverableUnits, Manifestations, Files Most useful metadata for searching is in the Dus, so we search for them, give a list ... user selects an interesting one, then gets to see all the Files in there, can then import the files to sakai.
  • Create could be interesting, because we need to submit in a Submission Information Package (SIP) Will probably need a tool to do this.
  • Transcript

    • 1. Connecting to Remote Repositories: Implementing the Docproxy Interface in Sakai OAE Jim Martino, Software Engineer, Johns Hopkins University
    • 2. Outline of talk
      • Disclaimers
      • How docproxy works
      • Our Implementation details
      • Demo
      • Future directions
      12th Sakai Conference – Los Angeles, California – June 14 - 16
    • 3. Disclaimers
      • Sakai3 / Nakamura is currently a moving target. This work was done against Nakamura 0.9
      • I have had to tweak a few things to get stuff to work according to my understanding of how they should work. My understanding may be incomplete.
      12th Sakai Conference – Los Angeles, California – June 14 - 16
    • 4. Simple diagram (not to scale)
    • 5. What happens when the UX requests something from Nakamura?
      • UX hits a URL, which Sling handles
      • Sling looks at the resourceType and path of the node at the URL, and resolves which servlet to hand it off to
      • In the case for content, the URL is of resourceType sakai/pooled-content.
      • Giving Sling an undecorated pooled content URL returns the bits if there is no extension.
      • If the extension metadata.tidy.json is appended, the metadata for the file is returned in JSON
      12th Sakai Conference – Los Angeles, California – June 14 - 16
    • 6. Locally stored file example { "jcr:path": "/_p/m/da/qi/i2/jqnmZuUQc63Z7mAoIA9iisGc", "jcr:name": "jqnmZuUQc63Z7mAoIA9iisGc", "sakai:fileextension": "usage", "sakai:permissions": "public", "sakai:directory": "default", "sling:resourceType": "sakai/pooled-content", "jcr:uuid": "17c29862-6431-41de-8bcf-6efb7e087527", "jcr:mixinTypes": [ "rep:AccessControllable" ], "jcr:createdBy": "admin", "sakai:pooled-content-file-name": "usage", "sakai:copyright": "creativecommons", "jcr:created": "2011-05-19T16:28:24-04:00", "sakai:pool-content-created-for": "jhopkins", "jcr:primaryType": "sakai:pooled-content" }
    • 7. Sakai link node example
    • 8. docproxy
      • What do we need to handle external files? The docproxy bundle addresses this. There are:
          • several new Sling resourceTypes:
          • servlets to handle these resourceTypes
          • interfaces to ensure the action of the servlets will be generic, and to ensure compatibility with everything else
      12th Sakai Conference – Los Angeles, California – June 14 - 16
    • 9. ExternalDocumentResultMetadata interface
    • 10. ExternalRepositoryProcessor interface
    • 11. resourceTypes
          • We have several new Sling resourceTypes:
          • One type for external resources themselves, of course
          • One type for the repository itself for operations which do not address a specific external resource
          • One type for searching against the external repository
      12th Sakai Conference – Los Angeles, California – June 14 - 16
    • 12. Docproxy nodes needed for an implementation
    • 13. Servlets We also have servlets to handle the new resource types. Servlet resolution will depend primarily on the resourceType and node path, with further refining from any extensions and selectors appended to the URL. These servlets then call the appropriate methods in the repository processors. 12th Sakai Conference – Los Angeles, California – June 14 - 16
    • 14. Some binding information for docproxy servlets, and which processor methods they call CreateExternalDocumentProxyServlet: resourceTypes = { "sakai/external-repository" }, selectors = { "create" } Calls: updateDocument DeleteExternalDocumentProxyServlet: resourceTypes ={ "sakai/external-repository" }, selectors = { "delete" } Calls: removeDocument ExternalDocumentSearchServlet: resourceTypes = { "sakai/external-repository-search" }, extensions = { "json" } Calls: search ExternalDocumentMetadataServlet: resourceTypes = {"sling/nonexisting", "sakai/external-repository -document " } selectors = "metadata”, extensions = "json" Calls: getDocumentMetadata ExternalDocumentProxyServlet: resourceTypes = { "sling/nonexisting", "sakai/external-repository-document" } Calls: getDocument
    • 15. Example transaction
    • 16. Sample ExtgernalSearchResultSet from dcs search service
    • 17. Implementation Details
      • Our implementation is based on the url processor in the docproxy bundle, since our repository exposes RESTful services.
      • Data packaging from the remote repository affects repository processor implementation.
      • UX integration may also be affected by data packaging structure.
      12th Sakai Conference – Los Angeles, California – June 14 - 16
    • 18. Initial Design
      • The first implementation was for a read-only repository, with as tight an integration as possible with native content in the UX.
      • Requirements:
      • Remote content should be an option in the content search menu
      • Should be able to merge remote search results with other remote searches and with local searches
      • Imported content should behave as much like native content as possible
      12th Sakai Conference – Los Angeles, California – June 14 - 16
    • 19. Design decisions
          • Use sakai links to point into docproxy document nodes
          • Insert additional methods, configurations, etc. into existing UX files
      12th Sakai Conference – Los Angeles, California – June 14 - 16
    • 20. Data Conservancy link node
    • 21. Sakai OAE login screen
    • 22. Skip Class logs in ...
    • 23. Skip's home page
    • 24. The content tab, with Data Conservancy added to the list of facets
    • 25. We enter the search term, with “Content I Manage” as the active facet
    • 26. Sakai does not find any matching content
    • 27. Change the facet to Data Conservancy, and we see remote hits (DUs)
    • 28. We select the first one, and get a list of files in the Rock Sample A-399 DU
    • 29. We click on A-399_2_r.jpg to see if it's something we want
    • 30. We go back, then import it, and now see it in our content
    • 31. Clicking on this, we pull up the link. Let's add some tags.
    • 32. Ok ... now let's search for it under Content I Manage
    • 33. Ta-da!
    • 34. Further Directions
      • Clean up current implementation
      • Implement Create/Update method
      • May need to develop a widget to handle submissions
      • Continue to review code to see if the new use cases are supported
      12th Sakai Conference – Los Angeles, California – June 14 - 16