• Share
  • Email
  • Embed
  • Like
  • Private Content

Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

System Update 2010 CrossRef Workshops Chuck Koscher

on

  • 1,185 views

 

Statistics

Views

Total Views
1,185
Views on SlideShare
1,181
Embed Views
4

Actions

Likes
0
Downloads
19
Comments
0

2 Embeds 4

http://www.crossref.org 2
http://crossref.org 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    System Update 2010 CrossRef Workshops Chuck Koscher System Update 2010 CrossRef Workshops Chuck Koscher Presentation Transcript

    • CrossRef Annual Meeting – London Workshops 15 November 2010 CrossRef 2010 Annual Member Meeting - London Page
    • CrossRef 2010 Annual Member Meeting - London Page Workshops Agenda 9:30-10:00 Coffee & Tea 10:00-11:30 System Update ….. Andrew Gilmartin, Senior Software Developer Chuck Koscher , Director of Technology 11:30-12:00 CrossMark …………Geoff Bilder, Director of Strategic Initiatives 12:00-12:30 CrossCheck ………. Kirsty Meddings, Product Manager 12:30-1:15 Lunch 1:15-2:15 Metadata Quality …. Patricia Feeney, Product Support Manager 2:15-2:45 Cited-by Linking ……Carol Anne Meyer, Business Development and Marketing Manager Chuck Koscher 2:45-3:00 Break 3:00-4:00 DOI Workflow Issues, Working with Vendors ……. Carol Anne Meyer 4:00-4:45 Boot Camp …………Carol Anne Meyer Tim Pickard, System Support Analyst/Administrator 4:45-5:15 Books ………………. Carol Anne Meyer,
    • System Update CrossRef 2010 Annual Member Meeting - London Page
        • System status
        • Rewrite review
        • Rewrite implementation
        • Discussion
    • CrossRef 2010 Annual Member Meeting - London Page System status
    • CrossRef 2010 Annual Member Meeting - London Page System status
    • CrossRef 2010 Annual Member Meeting - London Page
    • CrossRef 2010 Annual Member Meeting - London Page
    • CrossRef 2010 Annual Member Meeting - London Page Old system New Q system The switch
    • CrossRef 2010 Annual Member Meeting - London Page System status
      • Deposit processing
        • Suspended for 2+ weekends for Oracle DB upgrade (to 11g)
        • Processing times remain the same. (50% under 5 min, 30% more under 1 hour)
        • Large re-deposits (Elsevier plans for 2011)
        • Schema relatively unchanged in 2+ years (keep adding MIME types)
      • Deposit focus areas for 2011 (other than the re-write)
        • Investigating a PDF upload option
        • (for depositing a DOI and the article’s references)
        • Modify WebDeposit to allow users to edit an existing DOI’s metadata
        • Maintenance on NLM DTD deposit tool
    • CrossRef 2010 Annual Member Meeting - London Page
    • CrossRef 2010 Annual Member Meeting - London Page
    • CrossRef 2010 Annual Member Meeting - London Page System rewrite
      • The Query System (QS), where are we?
        • Its taking longer than we thought.
        • QS is 99% ready, periodically in service since starting mid Sept.
        • Last vexing problem solved (database connection dead-lock)?
        • Performance improvement is very encouraging.
        • Metrics and measurement capability greatly improved.
      • The Deposit System (DS), where are we?
        • Initial design discussions have been held, documentation is under way.
        • Implementation to start in January
        • Development will take until mid year, then lots of testing
        • Data clean up will be part of the migration process (mainly titles)
    • CrossRef 2010 Annual Member Meeting - London Page ⋅ Modularity of design ⋅ Utility of APIs where possible ⋅ Data stores that enable XML capabilities ⋅ Minimize dependency on proprietary systems
      • That CrossRef should ultimately own the intellectual property in the software at the heart of its operations
      • • That CrossRef should not risk or jeopardize the reliability and throughput offered by the existing system
      • • That CrossRef should remain free to develop further applications for other purposes which need to interface to the reference-linking systems and/or its data
      System rewrite
      • Rewrite 2 Working Group – Final report November 2008
    • CrossRef 2010 Annual Member Meeting - London Page O Unit testing (regression testing) O Scriptable data ingestion work flow F Richer metadata querying capability F Integrated data harvesting capabilities F Dealing with references using other character sets F Crawling of content to ingest it Vs. making deposits F Depositing of non journal content F Matching unstructured references using full text of equiv F Querying of non journal content F Real time, cited-by queries - with data-driven APIs F More content types, including language variants F More granular typing of journal articles F Improved reporting facilities F More useful user interface for members System rewrite
      • Rewrite 2 Working Group – Final report November 2008
      A Solve NFS issue A Federate architecture A Database redesign A Redesign event notification model (replace email) O Improved title management and control O Better publisher/member management model O Daily testing/monitoring (data integrity) O Built in health and status monitoring O Performance improvements and queue management Now Soon Later
    • CrossRef 2010 Annual Member Meeting - London Page System rewrite
      • Technical Objectives
        • Rework a 9 year old system
        • Address a declining performance situation
        • Improve administrative aspects (better control and reporting)
        • Facilitate extensibility
        • Staff’s better able to respond due to operational insight
      • Business Objectives
        • Develop internal capabilities ($ for every change Atypon makes)
        • Secure an independent path (continuity)
        • Benefit of being on a ‘shared’ platform nearing zero
        • Maintain access to technical expertise
    • CrossRef 2010 Annual Member Meeting - London Page Late 2010 thru mid 2011 HAProxy HTTP Traffic MySQL Lucene BerkelyDB FrontEnd QS (Spring) (Tomcat) Deposit System (old Atypon EDS) BackEnd Services Active MQ (messaging) Oracle (prime) Oracle (active-stndby) Constant Replication Oracle Group New System External messaging (email, etc) System rewrite
    • CrossRef 2010 Annual Member Meeting - London Page Q3 2011 HAProxy HTTP Traffic MySQL Lucene BerkelyDB FrontEnd QS (Spring) (Tomcat) BackEnd Services Active MQ (messaging) Oracle (prime) Oracle (active-stndby) Constant Replication Oracle Group New System External messaging (email, etc) Deposit Processing
      • FrontEnd DS
      • (Spring)
      • (Tomcat)
      • File Upload
      • Deposit reports
      System rewrite
    • CrossRef 2010 Annual Member Meeting - London Page Deposit DB (prime) Oracle Group System rewrite Deposit DB (standby) Oracle Replication Query DB (prime) Query DB (secondary) Oracle Replication New Deposit System Database Updater Primary Datacenter Deposit DB (prime) Query DB (prime) Recovery Datacenter
    • CrossRef 2010 Annual Member Meeting - London Page
      • Query system feature changes
        • Tweaks to the matching logic (discoveries made porting the code)
        • Fixed some nagging characteristics
        • Aggregate email notices for alerts
        • Implement HTTP free-text matching (still needs work, ‘alpha’)
        • Process free-text references for cited-by (done, stable, uses refXpress)
        • Establish better user model:
          • Username & passwords for members (Query and deposit)
          • Registered email address of non members (Query only)
      System rewrite Use Registration Form Receive Email Use Validation Form
    • CrossRef 2010 Annual Member Meeting - London Page
    • CrossRef 2010 Annual Member Meeting - London Page System rewrite Simple Text Query
    • CrossRef 2010 Annual Member Meeting - London Page
      • Uses refXpress to break free-text into XML suitable for running a metadata query
    • CrossRef 2010 Annual Member Meeting - London Page
      • Uses QS Formatted Citation Parse to break free-text into XML suitable for running a metadata query, if that fails uses QS Formatted Citation Search (with high threshold) to search Lucene index for a DOI.
    • CrossRef 2010 Annual Member Meeting - London Page But be careful ! <citation key=&quot;b53_366&quot;> <unstructured_citation> 53. O.S. Gudmundsson, S.D.S. Jois, D.G. Vander Velde, T.J. Siahaan, B. Wang, and R.T. Borchardt (1999 ) The effect of conformation on the membrane permeability of coumarinic acid- and phenylpropionic acid-based cyclic prodrugs of opioid peptides.J. Pept. Res.53 , 383 -392 . </unstructured_citation> </citation> <doi type=&quot;journal_article&quot;> 10.1034/j.1399-3011.1999.00076.x</doi> <issn type=&quot;print&quot;>1397-002X</issn> <issn type=&quot;electronic&quot;>1399-3011</issn> <journal_title>Journal of Peptide Research</journal_title> <contributors> <contributor sequence=&quot;first&quot; contributor_role=&quot;author&quot;> <given_name>O.S.</given_name> <surname>Gudmundsson</surname> </contributor> </contributors> <volume>53</volume> <issue>4</issue> <first_page>383</first_page> <last_page>392</last_page> <year media_type=&quot;print&quot;>1999</year> <publication_type>full_text</publication_type> <article_title> The effect of conformation on the membrane permeation of coumarinic acid- and phenylpropionic acid-based cyclic prodrugs of opioid peptides </article_title> <doi type=&quot;journal_article&quot;> 10.1034/j.1399-3011.1999.00077.x</doi> <issn type=&quot;print&quot;>1397-002X</issn> <issn type=&quot;electronic&quot;>1399-3011</issn> <journal_title>Journal of Peptide Research</journal_title> <contributors> <contributor sequence=&quot;first&quot; contributor_role=&quot;author&quot;> <given_name>O.S.</given_name> <surname>Gudmundsson</surname> </contributor> </contributors> <volume>53</volume> <issue>4</issue> <first_page>403</first_page> <last_page>413</last_page> <year media_type=&quot;print&quot;>1999</year> <publication_type>full_text</publication_type> <article_title> The effect of conformation of the acyloxyalkoxy-based cyclic prodrugs of opioid peptides on their membrane permeability </article_title> Still yields this But the correct answer is this
    • CrossRef 2010 Annual Member Meeting - London Page
      • Deposit system feature changes
        • Parse the XML prior to accepting the upload
        • Process XML, register DOIs regardless of metadata ingestion problems
        • Provide aggregated deposit reports (daily?)
        • Integrate Schematron checks into deposit process
        • Robust title ownership model, not based on prefix, with shared ownership options
        • Separate deposit metadata organization from query metadata organization (ex. Allow title substitution
      System rewrite
    • CrossRef 2010 Annual Member Meeting - London Page Andrew