LibreCat::Catmandu

1,002 views

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,002
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • LibreCat::Catmandu

    1. 1. LibreCat::Catmandu
    2. 2. Philosophy
    3. 3. Kahn-Wilensky Web Handle I search a paper about... Repository Service Repository Provider
    4. 4. Hypothesis 1: One Network & Common SchemaHypothesis 2: Object Oriented DesignHypothesis 3: The Resource is the Message
    5. 5. Hypothesis 1: One Network & Common Schema OpenAIRE Student Papers Google Europeana CRIS PhD Inst. Repository Digitized Data Material Sets
    6. 6. Hypothesis 2: Object Oriented Design •(Complex) Digital Objects • Unique identifiers • Key-Metadata + 1 or more data streams • Metadata schemes invariant over repositories • Specialized data types + service bindings on these types • Fragmented input • Incomplete • Metadata, Files, Metadata + Files • Many schemas • Simple data types (String, Array, Map) many native functions on these types
    7. 7. ResearcherHypothesis 3: The Resource is the Message DNS Dr Müller Repository GoogleISI Researcher PubMed Homepage
    8. 8. Extract, Transform & Load Fulltext SearchRelational Database Report Excel
    9. 9. Schemaless databases $store->add({ ...} ); $store->search({ ...} );
    10. 10. Copy & Paste Functional Style of Programming on native hashes and arrays“It is better to have 100 functions operate on one data structure than to have 10 functions operate on 10 data structures.” Alan J. Perlis
    11. 11. Project Catmandu
    12. 12. Download, Install & Play• Perl• http://librecat.org• cpan Catmandu• https://github.com/LibreCat/Catmandu
    13. 13. Anatomy of Searchdatabase export index def fix def convert filter/map fix index store field def display
    14. 14. Store Schemaless storage Elastic Solr Search JSON store Mongo DBI DB $store->add({}) $store->search() title:“krank” and subject.local:“cycle”title any “krank” and subject.local any “cycle”
    15. 15. Import Import from many sourcesAtom LDAP DBI JSON importerMARC OAI SRU $store->each({}) $store->first() $store->rest() $store->select({}) $store->any({}) $store->many({})
    16. 16. ETL upcase(job);JSON capitalize(first); capitalize(last); capitalize(my.deep.nested.0); upcase(my.deep.nested.0);fix downcase(my.deep.nested.0); substring(my.deep.nested.0,0,2);fix add_field(test); add_field(income,0); add_field(a.0.0.0,1); marc_map(100,my.authors.$append);JSON marc_map(710,my.authors.$append); marc_map(600x,my.subjects.$append); marc_map(008_/35-37,my.language); join_field(colors.0,/);
    17. 17. MVC Dancerapp/ |--bin/ |--public/ hello.tt | |-- images/ <html> | |-- css/ <body> | -- javascript/ <h1>[% txt %]</h1> |--views/ [% FOREACH obj IN res %] | -- hello.tt <p>[% obj.title %]</p> --environments/ [% END %] </body> </html> get ‘/hello’ => sub { my $res = store->bag ->search(query => ...) ->reduce( ... ); template ‘hello’ , { res => $res, txt => “Hello, World!” }; };
    18. 18. Project LibreCat
    19. 19. http://lup.lub.lu.se ISI PubMedLibreCat Catalog Fedora Mongo Store Store
    20. 20. http://pub.uni-bielefeld.de/ AnySRU/REST/Lucene/ Fix CSL JSON Store
    21. 21. http://biblio.ugent.be Fix AnySRU/REST/Lucene/ JSON Elastic Mongo Store Search LDAP JCR Project CRIS
    22. 22. http://adore.ugent.be MySQL FixAleph Store SOLR IIPImageFedora Store
    23. 23. Project Plan• Catmandu: Open Source Data Toolkit• LibreCat: Example Programs: • LibreCat-Search, LibreCat-Citation, LibreCat-Grim, LibreCat-Archive ...• Suite of repository add-ons: • Project Database, Research Groups, Authority Files
    24. 24. • Nicolas Steenlant - Ghent• Nicolas Franck - Ghent• Patrick Hochstenbach - Ghent• Snorri Briem - Lund• Dave Sherohman - Lund• Jörgen Eriksson - Lund Thanks!• Maria Hedberg - Lund• Friedrich Summann - Bielefeld• Najko Jahn - Bielefeld• Vitali Peil - Bielefeld• Petra Kohorst - Bielefeld http://librecat.org

    ×