www.ariadne-eu.org       www.cs.kuleuven.be/~hmdb
                     1
OAI workshop:
                OAI-PMH - Practical
                     session

                            Bram Vandeputt...
Overview




www.ariadne-eu.org              www.cs.kuleuven.be/~hmdb
                        2
Overview

         •                    How OAI works
         •         Harvesting metadata : the simple way
         •  ...
Overview


         •         Harvesting metadata : the simple way
         •             SettingHow OAI works target
    ...
How OAI works

    •   OAI “VERBS”               Service Provider      Metadata Provider


        •   Identify           ...
GetRecord

         • Purpose
           • Returns the metadata for a single item in the form
              of an OAI reco...
ListRecords
         • Purpose
           • Retrieves metadata records for multiple items
         • Parameters
          ...
ListRecords – from until
          http://localhost:8080/oaicat/OAIHandler?verb=ListRecords&
        from=1999-01-15&until...
ListIdentifiers
         • Purpose
           •   List headers for all items corresponding to the specified
               p...
ListSets

         • Purpose
           • Provide a listing of sets in which records may be
              organized (may b...
Overview




www.ariadne-eu.org              www.cs.kuleuven.be/~hmdb
                        9
Overview

         ✓                How OAI works
         •     Harvesting metadata : the simple way
         •         S...
Overview

         ✓                  How OAI works

         •         Harvesting up an OAI-PMH targetway
               ...
Harvesting metadata
 • the simple way : harvesting through browser
   • Identify
      - http://ariadne.cs.kuleuven.be/oai...
Overview




www.ariadne-eu.org              www.cs.kuleuven.be/~hmdb
                        11
Overview

         ✓                How OAI works
         ✓     Harvesting metadata : the simple way
         •         S...
Overview

         ✓                How OAI works
         ✓     Harvesting metadata : the simple way
         •         S...
Setting up a target




www.ariadne-eu.org               www.cs.kuleuven.be/~hmdb
                          12
Setting up a target


      Wiki page with detailed instructions :
      http://ariadne.cs.kuleuven.be/lomi/index.php/Sett...
OAI-PMH Target Software
                                             2. Mapping Process

                         Reposito...
Setting up a target




www.ariadne-eu.org               www.cs.kuleuven.be/~hmdb
                          14
Setting up a target
       • Install Java, Ant & apache-tomcat
       • Set environment vars
           set JAVA_HOME=C:j2...
Setting up a target




www.ariadne-eu.org               www.cs.kuleuven.be/~hmdb
                          15
Setting up a target


• Testing the target
        - http://localhost:8080/oaitarget/OAIHandler?verb=Identify
        - ht...
Setting up a target




www.ariadne-eu.org               www.cs.kuleuven.be/~hmdb
                          16
Setting up a target
       • Binding to example lucene index
            • Download the index :
                - http://a...
Setting up a target




www.ariadne-eu.org               www.cs.kuleuven.be/~hmdb
                          17
Setting up a target

• Testing the target
        - http://localhost:8080/oaitarget/OAIHandler?verb=ListRecords&metadataPr...
Setting up a target




www.ariadne-eu.org               www.cs.kuleuven.be/~hmdb
                          18
Setting up a target


         • Validate the result metadata in the
            validationService online :
           • h...
Overview




www.ariadne-eu.org              www.cs.kuleuven.be/~hmdb
                        19
Overview

         ✓                How OAI works
         ✓     Harvesting metadata : the simple way
         ✓         S...
Overview

         ✓                How OAI works
         ✓     Harvesting metadata : the simple way
         ✓         S...
Binding it to your DB




www.ariadne-eu.org          www.cs.kuleuven.be/~hmdb
                      20
Binding it to your DB
     • Implement /adapt 3 classes :
          •   org.ariadne.oai.server.X.catalog.XCatalog.java
   ...
Binding it to your DB




www.ariadne-eu.org          www.cs.kuleuven.be/~hmdb
                      21
Binding it to your DB

     • XCatalog.java
          • For each “verb” :
              - get parameters
              - q...
Binding it to your DB




www.ariadne-eu.org          www.cs.kuleuven.be/~hmdb
                      22
Binding it to your DB

         • XRecordFactory.java
              • Two important methods :
                     -   get...
Binding it to your DB




www.ariadne-eu.org          www.cs.kuleuven.be/~hmdb
                      23
Binding it to your DB


         • XCrosswalk.java
              • Map fields of your object to LOM XML object :
          ...
Overview




www.ariadne-eu.org              www.cs.kuleuven.be/~hmdb
                        24
Overview

         ✓                How OAI works
         ✓     Harvesting metadata : the simple way
         ✓         S...
Overview

         ✓                How OAI works
         ✓     Harvesting metadata : the simple way
         ✓         S...
Harvesting metadata - take 2




www.ariadne-eu.org        www.cs.kuleuven.be/~hmdb
                     25
Harvesting metadata - take 2

      Download ARIADNE harvester :
      http://www.cs.kuleuven.be/~bramv/oaiharvester.war

...
References
         • http://ariadne.cs.kuleuven.be/lomi/index.php/
            Setting_Up_OAI-PMH

         • http://aria...
Upcoming SlideShare
Loading in …5
×

Oai Workshop Extended

1,292 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,292
On SlideShare
0
From Embeds
0
Number of Embeds
48
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Oai Workshop Extended

  1. 1. www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 1
  2. 2. OAI workshop: OAI-PMH - Practical session Bram Vandeputte K.U.Leuven ARIADNE www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 1
  3. 3. Overview www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 2
  4. 4. Overview • How OAI works • Harvesting metadata : the simple way • Setting up an OAI-PMH target • Binding your database to the OAI-PMH target • Harvesting metadata - take 2 www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 2
  5. 5. Overview • Harvesting metadata : the simple way • SettingHow OAI works target up an OAI-PMH • Binding your database to the OAI-PMH target • Harvesting metadata - take 2 www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 2
  6. 6. How OAI works • OAI “VERBS” Service Provider Metadata Provider • Identify H HTTP Request R E • ListMetadataFormats A R (OAI Verb) P • GetRecord V O • ListIdentifiers E OAI OAI S S I • ListRecords T T O • ListSets E HTTP Response R R (Valid XML) Y www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 3
  7. 7. GetRecord • Purpose • Returns the metadata for a single item in the form of an OAI record • Parameters • identifier – unique id for item (R) • metadataPrefix – metadata format for the record (R) www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 4
  8. 8. ListRecords • Purpose • Retrieves metadata records for multiple items • Parameters • from – start date (O) - greater than or equal to • until – end date (O) - less than or equal to • set – set to harvest from (O) • resumptionToken – flow control mechanism (X) • metadataPrefix – metadata format (R) www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 5
  9. 9. ListRecords – from until http://localhost:8080/oaicat/OAIHandler?verb=ListRecords& from=1999-01-15&until=2005-12-31&metadataPrefix=oai_lom UTCdatetime Dates and times are uniformly encoded using ISO8601 and are expressed in UTC throughout the protocol. When time is included, the special UTC designator ("Z") must be used. UTC is implied for dates although no timezone designator is specified. For example, 1957-03-20T20:30:00Z is UTC 8:30:00 PM on March 20th 1957. UTCdatetime is used in both protocol requests and protocol replies, in the way described in the following sections. www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 6
  10. 10. ListIdentifiers • Purpose • List headers for all items corresponding to the specified parameters • Parameters • from – start date (O) • until – end date (O) • set – set to harvest from (O) • metadataPrefix – metadata format to list identifiers for (R) • resumptionToken – flow control mechanism (X) www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 7
  11. 11. ListSets • Purpose • Provide a listing of sets in which records may be organized (may be hierarchical, overlapping, or flat) • Parameters • None www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 8
  12. 12. Overview www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 9
  13. 13. Overview ✓ How OAI works • Harvesting metadata : the simple way • Setting up an OAI-PMH target • Binding your database to the OAI-PMH target • Harvesting metadata - take 2 www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 9
  14. 14. Overview ✓ How OAI works • Harvesting up an OAI-PMH targetway Setting metadata : the simple • Binding your database to the OAI-PMH target • Harvesting metadata - take 2 www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 9
  15. 15. Harvesting metadata • the simple way : harvesting through browser • Identify - http://ariadne.cs.kuleuven.be/oaitarget/OAIHandler?verb=Identify • ListMetadataFormats - http://ariadne.cs.kuleuven.be/oaitarget/OAIHandler?verb=ListMetadataFormats • ListRecords - http://ariadne.cs.kuleuven.be/oaitarget/OAIHandler?verb=ListRecords&metadataPrefix=oai_lom - http://ariadne.cs.kuleuven.be/oaitarget/OAIHandler?verb=ListRecords&resumptionToken= • GetRecord - http://ariadne.cs.kuleuven.be/oaitarget/OAIHandler?verb=GetRecord &identifier=oai:ariadne.cs.kuleuven.be:BLKLKP1382&metadataPrefix=oai_lom • ... www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 10
  16. 16. Overview www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 11
  17. 17. Overview ✓ How OAI works ✓ Harvesting metadata : the simple way • Setting up an OAI-PMH target • Binding your database to the OAI-PMH target • Harvesting metadata - take 2 www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 11
  18. 18. Overview ✓ How OAI works ✓ Harvesting metadata : the simple way • Setting up an OAI-PMH target • Binding your database to the OAI-PMH target • Harvesting metadata - take 2 www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 11
  19. 19. Setting up a target www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 12
  20. 20. Setting up a target Wiki page with detailed instructions : http://ariadne.cs.kuleuven.be/lomi/index.php/Setting_Up_OAI-PMH www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 12
  21. 21. OAI-PMH Target Software 2. Mapping Process Repository metadata Repository LOM metadata 1. Get metadata - identifier - lom.general.identifier out of database - title - lom.general.title - url - lom.general.description - project description - lom.technical.location - ... - ... 3. Copy results in OAI-PMH service Repository OAI-PMH result DYNAMO LOM 4. Serve results ARIADNE DYNAMO LOM metadata LOM Harvester DYNAMO metadata Repository LOM metadata metadata www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 13
  22. 22. Setting up a target www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 14
  23. 23. Setting up a target • Install Java, Ant & apache-tomcat • Set environment vars set JAVA_HOME=C:j2sdk1.5.0 set ANT_HOME=C:apache-ant-1.6.5 set PATH=%JAVA_HOME%bin;%ANT_HOME%bin;%PATH% • Download oaicat source and build cd <your build-dir> type “ant” • Deploy code put “oaicat_5.0/dist/oaicat.war” under “apache-tomcat/webapps/” www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 14
  24. 24. Setting up a target www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 15
  25. 25. Setting up a target • Testing the target - http://localhost:8080/oaitarget/OAIHandler?verb=Identify - http://localhost:8080/oaitarget/OAIHandler?verb=ListMetadataFormats www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 15
  26. 26. Setting up a target www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 16
  27. 27. Setting up a target • Binding to example lucene index • Download the index : - http://ariadne.cs.kuleuven.be/MeltSqiOai/OAI/lucene.zip • Extract • open properties file with text editor : - oaicat_5.0/WEB-INF/oaicat.properties • Enter the full path of index under the "LuceneLomCatalog.lucenePath" property • reload the target www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 16
  28. 28. Setting up a target www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 17
  29. 29. Setting up a target • Testing the target - http://localhost:8080/oaitarget/OAIHandler?verb=ListRecords&metadataPrefix=oai_lom - http://localhost:8080/oaitarget/OAIHandler?verb=ListRecords&resumptionToken= - http://localhost:8080/oaitarget/OAIHandler?verb=GetRecord&identifier= oai:oaicat.ariadne.org:CS_LKP_v_3.0_nr_1162 &metadataPrefix=oai_lom www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 17
  30. 30. Setting up a target www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 18
  31. 31. Setting up a target • Validate the result metadata in the validationService online : • http://ariadne.cs.kuleuven.be/validationService www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 18
  32. 32. Overview www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 19
  33. 33. Overview ✓ How OAI works ✓ Harvesting metadata : the simple way ✓ Setting up an OAI-PMH target • Binding your database to the OAI-PMH target • Harvesting metadata - take 2 www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 19
  34. 34. Overview ✓ How OAI works ✓ Harvesting metadata : the simple way ✓ Setting up an OAI-PMH target • Binding your database to the OAI-PMH target • Harvesting metadata - take 2 www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 19
  35. 35. Binding it to your DB www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 20
  36. 36. Binding it to your DB • Implement /adapt 3 classes : • org.ariadne.oai.server.X.catalog.XCatalog.java ➡ Implements the OAI-PMH verbs • org.ariadne.oai.server.X.catalog.XRecordFactory.java ➡ Creates OAI-PMH headers • org.ariadne.oai.server.X.crosswalk.XCrosswalk.java ➡ Maps arbitrary object to LOM XML object www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 20
  37. 37. Binding it to your DB www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 21
  38. 38. Binding it to your DB • XCatalog.java • For each “verb” : - get parameters - query database to get matching metadata - parse metadata into java objects (Vector, HashMap, ...) - call RecordFactory and Crosswalk to create records www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 21
  39. 39. Binding it to your DB www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 22
  40. 40. Binding it to your DB • XRecordFactory.java • Two important methods : - getLocalIdentifier(Object nativeItem); - getDatestamp(Object nativeItem); www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 22
  41. 41. Binding it to your DB www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 23
  42. 42. Binding it to your DB • XCrosswalk.java • Map fields of your object to LOM XML object : - createMetadata(Object nativeItem); www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 23
  43. 43. Overview www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 24
  44. 44. Overview ✓ How OAI works ✓ Harvesting metadata : the simple way ✓ Setting up an OAI-PMH target ✓ Binding your database to the OAI-PMH target • Harvesting metadata - take 2 www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 24
  45. 45. Overview ✓ How OAI works ✓ Harvesting metadata : the simple way ✓ Setting up an OAI-PMH target ✓ Binding your database to the OAI-PMH target • Harvesting metadata - take 2 www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 24
  46. 46. Harvesting metadata - take 2 www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 25
  47. 47. Harvesting metadata - take 2 Download ARIADNE harvester : http://www.cs.kuleuven.be/~bramv/oaiharvester.war Wiki page with detailed instructions : http://ariadne.cs.kuleuven.be/lomi/index.php/Harvesting_Metadata www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 25
  48. 48. References • http://ariadne.cs.kuleuven.be/lomi/index.php/ Setting_Up_OAI-PMH • http://ariadne.cs.kuleuven.be/lomi/index.php/ Harvesting_Metadata • http://oai.grainger.uiuc.edu/FinalReport/ JCDL_2003_OAI_Intro.ppt • http://www.cs.kuleuven.ac.be/~hmdb/SqiOaiAspect/ • http://www.oclc.org/research/software/oai/cat.htm • http://ariadne.cs.kuleuven.be/validationService www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 26

×