Making your data work harder than you do

1,369 views

Published on

From Shapeshifting: A Practical Look at Metadata Interoperability. The use of planning documents, data dictionaries and xslt in transforming data, particularly for use in image delivery systems. Also addresses the uses of VRA Core 4 display and index fields.

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,369
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
14
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Making your data work harder than you do

  1. 1. Images of the Babbage Engine taken from the Computer History Museum website http://www.computerhistory.org
  2. 2. Learning to Make Your Data Work Harder Than You Do
  3. 3. Do it once, do it right Its about transforming and repurposing the same data for different tools and users in the most automated way possible, not recreating it
  4. 4. Three Words: USE PLANNING DOCUMENTS
  5. 5. In the age of Google, why have fielded data? •More efficient for both data entry and for systems to search, retrieve and ingest •Parsed, discretely fielded data can be recombined mechanically for a variety of outputs and uses, including XML
  6. 6. Data flow and use Cataloging Institutional utility Digital XML (relational Repository database) xslt xslt Web 2.0 Delivery Systems XMP (in images) Users ARTstor RSS feeds XMP (in MDID Websites images) CONTENTdm tools Flat Excel LUNA Insight etc. PDF
  7. 7. Data flow and use Cataloging Institutional utility Digital XML (relational Repository database) xslt Web 2.0 Delivery Systems XMP (in images) Users ARTstor RSS feeds XMP (in MDID Websites images) CONTENTdm tools Flat Excel LUNA Insight etc. PDF
  8. 8. A pithy answer to ―why relational?‖ (for cataloging) • Message from Jan Eklund to VRA-L, Feb 20, 2008, subject: Re: CONTENTdm and metadata (now posted on VRA web under Resources) – Complexity: ―complexity cannot be captured efficiently in a flat data model because basically you have to leave space in every record to accommodate the most complex object you will ever encounter. This adds up to a lot of wasted space, and wasted space means more money…‖ – Consistency: ―all the descriptive data about the work is entered once, and every image that shows this work inherits the same information‖
  9. 9. Excel sample (―flat file‖ output) Notice that each row represents an image file and conflates the work and image records (repeats the information about the work for each image). Each repeating value (like Artist) must have a column reserved for possible use.
  10. 10. XML sample—more like a flexible accordion—expands as needed
  11. 11. ER Diagrams show related tables
  12. 12. Authority record All the information about the agent is supplied from this file on the basis of the numeric key Numeric key
  13. 13. Repeating values are supported for each element (using portals or subforms) ―indexed‖ value (in this case the sort name) Numeric key A note field is possible ―display‖ value done to CCO for every recommended formatting. Note that Core 4 the Agent Nationality is supplied element automatically here by the Link (numeric key) to the Agent Authority
  14. 14. Creation of an XSLT • XSLT stands for Extensible Stylesheet Language Transformation. XSLT is XML-based. You can use a stylesheet to take an XML document and turn it into plain text, PDF documents, web pages, or to import fielded data into other applications. • In this case sample it creates a tab delimited file and specifies the field names in the headers when it is converted into Excel (extra step is to preserve diacritics with Unicode)
  15. 15. VCat • Begun 2004 with goal of being fully relational, VRA Core 4 and CCO compliant and capable of Core 4 XML output—that goal met • Reality in 2010—flattened Excel is still the lingua franca. The XML export stylesheet was used as basis to create a flattened Excel export • So, there are now 2 exports (XSLTs) that can provide XML and Excel
  16. 16. You can create as many stylesheets as you like for specific purposes VCat folder .xsd=schema; .xsl=stylesheet; .xml=document
  17. 17. Data flow and use Cataloging Institutional utility Digital XML (relational Repository database) xslt Web 2.0 Delivery Systems XMP (in images) Users ARTstor RSS feeds XMP (in MDID Websites images) CONTENTdm tools Flat Excel LUNA Insight etc. PDF
  18. 18. Sample XML output (small clip)
  19. 19. Data flow and use Cataloging Institutional utility Digital XML (relational Repository database) xslt Web 2.0 Delivery Systems XMP (in images) Users ARTstor RSS feeds XMP (in MDID Websites images) CONTENTdm tools Flat Excel LUNA Insight etc. PDF
  20. 20. Creation of a mapping document to a standard • Flattened Core 4
  21. 21. • Flattening repeating fields
  22. 22. Sample Excel output (a small clip)
  23. 23. Data flow and use Cataloging Institutional utility Digital XML (relational Repository database) xslt Web 2.0 Data XMP (in images) Dictionary Systems Delivery Users ARTstor RSS feeds XMP (in MDID Websites images) CONTENTdm tools Flat Excel LUNA Insight etc. PDF
  24. 24. Creation of a Data Dictionary for each tool • Data dictionaries help set the display look of the data that the patron sees—this can be customized and where the use of ―index‖ and ―display‖ values of Core 4 are crucial • They also set the things the patron does not see—under the surface search parameters, like using early and late date (index fields) to do ―fuzzy‖ searching
  25. 25. Display data is like publishing: arranges data attractively for user
  26. 26. Difference in user display and cataloger mode We are used to seeing this in OPACs
  27. 27. Sample MDID Data Dictionary Set import, field labels, thumb captions, sorting, searching, keyword searching, DC mapping for cross collection searching, advance search pop-down lists, etc.
  28. 28. VCat ARTstor
  29. 29. VCat-ARTstor Data Dictionary Concatenate fields; Set display order ―prepend‖ global of grouped fields information or labels
  30. 30. Data Dictionary settings in action Clustered (grouped) fields; ability to concatenate information or ―preprend‖ information
  31. 31. Setting thumb captions In this case, ARTstor has a floating information window; in other tools this would be a place to use the INDEX value of the name (which is the sort value instead of the display) Also allows user to change thumb sort
  32. 32. Users use keywords
  33. 33. Same data, different tools and users • The following 3 slides show the same data prepared for our stock and royalty-free publishing site hosted on SmugMug—the educational data is reduced and compressed into an IPTC-like caption and keywords only and written into the image header. This means it can be seen using Cooliris as well (which is fun).
  34. 34. The Cooliris ―wall‖ of images with captions
  35. 35. In general, you seek to adapt the xslt stylesheet and the data dictionary as needed rather than changing the data that you produce centrally—that should remain consistent to a standard and you should seek the ability to express that in a standard xml schema, as well as any other stylesheets. Hopefully future tools will ingest from the standard schema.
  36. 36. Thinking about how to present grouped or complex objects • Think about this upfront so that your cataloging can help facilitate groupings— use of data values • Also think about what needs to be consistently fielded data (including local field structure) to help order and sequence manuscripts and time-based works • These will require local fields and data dictionary mapping and settings
  37. 37. Pragmatic, phased approaches • Being able to find and update older records easily and consistently into full Core 4 when it is better supported in tools • Supplying the data now in some useful form
  38. 38. ―Collection‖ record

×