0
ARIADNE                                   K.U.Leuven
www.ariadne-eu.org                     www.cs.kuleuven.be/~hmdb




H...
ARIADNE                                              K.U.Leuven
www.ariadne-eu.org                                www.cs.k...
ARIADNE                              K.U.Leuven
www.ariadne-eu.org                www.cs.kuleuven.be/~hmdb



            ...
ARIADNE                                K.U.Leuven
www.ariadne-eu.org                  www.cs.kuleuven.be/~hmdb



        ...
ARIADNE                                          K.U.Leuven
www.ariadne-eu.org                            www.cs.kuleuven....
ARIADNE                                             K.U.Leuven
www.ariadne-eu.org                              www.cs.kule...
ARIADNE                                                                         K.U.Leuven
www.ariadne-eu.org             ...
ARIADNE                     K.U.Leuven
www.ariadne-eu.org       www.cs.kuleuven.be/~hmdb




                     8
      ...
ARIADNE                     K.U.Leuven
www.ariadne-eu.org       www.cs.kuleuven.be/~hmdb




     The ARIADNE Harvester
  ...
ARIADNE                                     K.U.Leuven
www.ariadne-eu.org                       www.cs.kuleuven.be/~hmdb

...
ARIADNE                                               K.U.Leuven
www.ariadne-eu.org                                www.cs....
ARIADNE                                                  K.U.Leuven
www.ariadne-eu.org                                   w...
ARIADNE                                                      K.U.Leuven
www.ariadne-eu.org                                ...
ARIADNE                                                      K.U.Leuven
www.ariadne-eu.org                                ...
ARIADNE                                                      K.U.Leuven
www.ariadne-eu.org                                ...
ARIADNE                                                      K.U.Leuven
www.ariadne-eu.org                                ...
ARIADNE                                                              K.U.Leuven
www.ariadne-eu.org                        ...
ARIADNE                                                          K.U.Leuven
www.ariadne-eu.org                            ...
ARIADNE                                                         K.U.Leuven
www.ariadne-eu.org                             ...
ARIADNE                                                                          K.U.Leuven
www.ariadne-eu.org            ...
ARIADNE                        K.U.Leuven
www.ariadne-eu.org          www.cs.kuleuven.be/~hmdb



          Harvesting in ...
ARIADNE                                          K.U.Leuven
www.ariadne-eu.org                            www.cs.kuleuven....
ARIADNE                                                                               K.U.Leuven
www.ariadne-eu.org       ...
ARIADNE                        K.U.Leuven
www.ariadne-eu.org          www.cs.kuleuven.be/~hmdb



                     MAC...
ARIADNE                                            K.U.Leuven
www.ariadne-eu.org                              www.cs.kuleu...
ARIADNE                                                                           K.U.Leuven
www.ariadne-eu.org           ...
ARIADNE                      K.U.Leuven
www.ariadne-eu.org        www.cs.kuleuven.be/~hmdb




                     16
   ...
ARIADNE                                        K.U.Leuven
www.ariadne-eu.org                          www.cs.kuleuven.be/~...
ARIADNE                                                                                                                   ...
ARIADNE                      K.U.Leuven
www.ariadne-eu.org        www.cs.kuleuven.be/~hmdb




                     19
   ...
ARIADNE                               K.U.Leuven
www.ariadne-eu.org                www.cs.kuleuven.be/~hmdb




          ...
ARIADNE                                                                                       K.U.Leuven
www.ariadne-eu.or...
ARIADNE                      K.U.Leuven
www.ariadne-eu.org        www.cs.kuleuven.be/~hmdb




 • Questions ?




        ...
ARIADNE                                                             K.U.Leuven
www.ariadne-eu.org                         ...
Upcoming SlideShare
Loading in...5
×

Ariadne Harvesting

1,351

Published on

Harvesting metadata in repository networks, including ARIADNE, GLOBE, MACE and MELT

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,351
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
20
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Ariadne Harvesting"

  1. 1. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb HARVESTING METADATA IN REPOSITORY NETWORKS Bram Vandeputte K.U.Leuven ARIADNE 1 1
  2. 2. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb WHO AM I • Working since 2006 with Erik Duval • ARIADNE harvester, OAI-PMH target support, validation service, etc. • International projects : MACE, MELT, Globe, Ariadne, Aspect, iCoper,... • Other interests : Information visualisation, Interoperability, TEL 2 2
  3. 3. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb Overview • OAI-PMH • Validation Service • Harvester Infrastructure • Projects • MACE • MELT • GLOBE • OAI-PMH Target 3 3
  4. 4. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb OAI-PMH • Server - Client model • Pull mechanism • options : • selective harvesting • incremental harvesting 4 4
  5. 5. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb OAI-PMH • Verbs : Identify, ListRecords, GetRecord • Parameters : • baseUrl • from & until date • metadataPrefix • sets 5 5
  6. 6. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb Validation Service • Interoperability : Application Profile (AP) • Manual check : very time consuming • Need a tool for enforcing an AP => validation scheme • A set of validation rules • Reusable 6 6
  7. 7. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb Validation Service LOM loose MELTv1.0/ MACEv1.0 minimal melt.xsd lomloose.xsd mace.xsd classification vocab service MELTv1.0/full vcard validator Legend empty attribute uses fields extends MACEv1.0 validationScheme conditional LRE taxon field fields check numerical vcard validator validation component 7 7
  8. 8. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 8 8
  9. 9. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb The ARIADNE Harvester Infrastructure 8 8
  10. 10. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb ARIADNE Harvester harvester log 8 8
  11. 11. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb ARIADNE Harvester ARIADNE Repository SPI SQI harvester log validation service 8 8
  12. 12. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb ARIADNE Harvester ARIADNE Repository SPI SQI External Repository OAI OAI-PMH LOM LOM LOM harvester log validation service 8 8
  13. 13. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb ARIADNE Harvester ARIADNE Repository OAI-PMH SPI SQI External Repository LOM LOM LOM OAI OAI-PMH LOM LOM LOM harvester log validation service 8 8
  14. 14. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb ARIADNE Harvester ARIADNE Repository OAI-PMH SPI SQI External Repository LOM LOM OAI OAI-PMH LOM LOM LOM LOM harvester log validation service 8 8
  15. 15. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb ARIADNE Harvester ARIADNE Repository OAI-PMH SPI SQI External Repository LOM LOM OAI OAI-PMH LOM LOM LOM LOM LOM harvester log validation service 8 8
  16. 16. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb ARIADNE Harvester ARIADNE Repository OAI-PMH SPI SQI External Repository LOM LOM OAI OAI-PMH LOM LOM LOM LOM Validation Msg harvester log validation service 8 8
  17. 17. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb ARIADNE Harvester ARIADNE Repository OAI-PMH SPI SQI External Repository LOM LOM OAI OAI-PMH Validation LOM Msg LOM LOM LOM harvester log validation service 8 8
  18. 18. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb ARIADNE Harvester ARIADNE Repository OAI-PMH SPI SQI External Repository LOM LOM OAI OAI-PMH LOM LOM LOM LOM Validation Msg harvester log validation service 8 8
  19. 19. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb ARIADNE Harvester ARIADNE Repository OAI-PMH SPI SQI External Repository LOM LOM OAI LOM OAI-PMH LOM LOM LOM Validation Msg harvester log validation service 8 8
  20. 20. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb ARIADNE Harvester ARIADNE Repository OAI-PMH SPI SQI External Repository LOM LOM LOM OAI 2 6 LOM OAI-PMH 1 Validation LOM Msg LOM LOM LOM 4 5 3 Validation Validation LOM Msg Msg harvester log validation service 9 9
  21. 21. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb Harvesting in networks • MACE • MELT • GLOBE 10 10
  22. 22. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb MACE • Metadata for Architectural Contents in Europe • www.mace-project.eu • Harvesting, Searching, Enriching, (geo-) Tagging, attention metadata 11 11
  23. 23. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb MACE Uses Lucene for index search WINDS Validation OAI-PMH service SPI Harvested metadata OAI-PMH SPI IRB ARIADNE Harvester OAI-PMH MACE metadata SPI OAI-PMH DYNAMO Enriched metadata CuminCAD 12 12
  24. 24. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb MACE 13 13
  25. 25. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb MELT • MELT : a Metadata Ecology for Learning and Teaching • Ministries of education across europe • enriching, translating, better quality metadata • practical test for scalability of harvester framework 14 14
  26. 26. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb MELT Multilingual Vocabulary Thesaurus Registry Validation BMBWK EUN MEC RVP Enrichment service Toolkit EUN AMG OAI-PMH SQI ARIADNE C-H FWU MESC TLF Harvester EUN MDS SPI CFL INDIRE MSU UL-FRI ARIADNE OAI Harvester SQI (EUN) ARIADNE METADATASTORE EDUCATIO KUL NCTE XTEC 15 15
  27. 27. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 16 16
  28. 28. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb GLOBE • Global Learning Objects Brokered Exchange • Worldwide network of networks of repositories • => Both SQI and OAI-PMH are used on various levels in the network 17 17
  29. 29. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb GLOBE BMBWK EUN MEC RVP SQI UDDI OAI OAI-PMH Registry SQI ARIADNE C-H FWU MESC TLF SQI Client Harvester OAI CFL INDIRE MSU UL-FRI MELT SQI SQI Federated EDUCATIO KUL NCTE XTEC GLOBE Search SPI OAI OAI-PMH ARIADNE SQI Harvester WINDS DYNAMO ARIADNE Harvester MACE IRB CuminCAD OAI OAI OAI SQI SQI SQI SQI SQI SQI SQI LACLO NIME EUN EdNA CGIAR Pubelo Merlot OAI OAI SQI SQI SQI SQI SQI SQI ISKME KERIS III COSL LORNET Ariadne 18 18
  30. 30. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb 19 19
  31. 31. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb OAI-PMH target 20 20
  32. 32. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb OAI-PMH Target Software 2. Mapping Process Repository metadata Repository LOM metadata 1. Get metadata - identifier - lom.general.identifier out of database - title - lom.general.title - url - lom.general.description - project description - lom.technical.location - ... - ... 3. Copy results in OAI-PMH service Repository OAI-PMH result DYNAMO LOM 4. Serve results ARIADNE DYNAMO LOM metadata LOM Harvester DYNAMO metadata Repository LOM metadata metadata 21 21
  33. 33. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb • Questions ? 22 22
  34. 34. ARIADNE K.U.Leuven www.ariadne-eu.org www.cs.kuleuven.be/~hmdb References • SPI : http://ariadne.cs.kuleuven.be/lomi/index.php/SimplePublishingInterface • SQI : http://ariadne.cs.kuleuven.be/lomi/index.php/LorInteroperability • IEEE LOM : http://ltsc.ieee.org/wg12/ • OAI-PMH : http://www.openarchives.org/ • OAI-PMH target wiki : http://ariadne.cs.kuleuven.be/lomi/index.php/Setting_Up_OAI-PMH 23 23
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×