Which Came First, the Data Structure or the Website?: Lessons Learned in Building a New Collections Website with Existing Collections Data
•Download as PPTX, PDF•
3 likes•928 views
Presentation for Museum Computer Network 2015 conference on 11/5/15, Minneapolis, MN.
Recommended
Recommended
More Related Content
Similar to Which Came First, the Data Structure or the Website?: Lessons Learned in Building a New Collections Website with Existing Collections Data
Similar to Which Came First, the Data Structure or the Website?: Lessons Learned in Building a New Collections Website with Existing Collections Data (20)
Recently uploaded
Recently uploaded (20)
Which Came First, the Data Structure or the Website?: Lessons Learned in Building a New Collections Website with Existing Collections Data
- 1. Which Came First, the Data Structure or the Website?: Lessons Learned in Building a New Collections Website with Existing Collections Data Ellice Engdahl, Digital Collections & Content Manager Museum Computer Network November 5, 2015
- 2. The Edison Institute, Founded 1929 Francis Jehl, Thomas Edison, President Herbert Hoover, and Henry Ford at Menlo Park Laboratory in Greenfield Village, 1929 (P.O.4556)
- 3. Five Venues Henry Ford Museum Greenfield Village Benson Ford Research Center The Henry Ford IMAX Theater Ford Rouge Factory Tour Henry Ford Museum, 1953 (P.B.3002) Old Car Festival, 1959 (2005.0.9.18)
- 4. • ~250,000 objects • ~25 million archival documents and photographs • Historic audio and video • 200+ objects on loan to dozens of institutions • ~21,700 artifacts on display in Museum and Village (~5- 10% of non-archives collection) • More than 30 distinct collections storage areas totaling nearly 200,000 sq. ft. Eye Portrait, circa 1800 (61.151.40) Our Collections: “The Bottomless Pit of Wonderfulness”
- 5. pre-2010: “Random Acts of Digitization” 2010: THF announces intent to digitize all collections holdings 2011: THF begins using EMu as its collections management system and repository of record for digital collections data and image metadata 2010-2015: Artifacts online grow from 300 to 43,000+ Factory Workers Testing Newly Manufactured Toasters, circa 2000 (2008.122.188) Collections Digitization at THF
- 7. Collections Access User Types/User Goals User types • Students • Educators • Researchers • Campus visitors • Enthusiasts High-level User goals • Understand what the archive/THF is • Find artifacts • Use & share artifact records • Order (images, etc.) School's Out Target Game, 1902-1914 (00.4.5224)
- 8. New Requirement: Archival Collection Records • Display archival collection- level records within THF’s digital collections • Associate individual items with their collection • Cross-search archival collections records with individual artifact records Photo Archivist Win Sears and Co-Worker in the Ford Archives at Fair Lane, Dearborn, Michigan, 1952-1957 (P.O.6193)
- 9. Why? • Make our collections cross- searchable • Provide context for individual archival items already digitized • Allow users to understand what items from a collection have and have not yet been digitized Detroit Public Library, Detroit, Michigan, circa 1921 (37.102.136)
- 10. Challenge #1: Define search & display for collection records • Determine what fields will be used within our collections management system, incl. those that overlap with individual item records • Identify useful fields to include in basic and advanced searches • Decide metadata fields to identify collection level records and finding aid PDFs Jigsaw Puzzle of Manhattan Island, New York, circa 1932 (41.214.1593)
- 11. Challenge #2: Create plan to populate collections records • Determine which collections have finding aids, and which of these we have or have not digitized material from • Determine which collections without finding aids we’ve digitized material from • Finalize number of collections records to be created for launch • Integrate creation and publishing of collection records into archival workflows Providence and Worcester Canal Boat Company Schedule from Newspaper, Rhode Island, 1829 (82.129.190)
- 12. Challenge #3: Draw the line on scope • Defer desired requirement to display digitized collections within a functional finding aid hierarchy • Defer desired requirement to create and search transcriptions for documents Draftsmen Working in the Lofting Division for B-24 Assembly, Willow Run Bomber Plant, 1942 (P.189.17502)
- 13. New Requirement: Online Image Delivery & Image Zoom • Allow users to zoom into high-res artifact imagery details • Automate previously manual delivery of high- resolution collections images • Revise service fee structure and licensing policies Advertising Poster, "Mother Needs a Ford, Buy It Now," circa 1925 (69.114.2.2)
- 14. Why? Actors inside Logan County Courthouse, Greenfield Village, 1983 (EI.1929.880) • Institutional revenue funds the deployment of our digital repository and content • We want to make our collections as accessible as possible • Rights are complex and we are not lawyers • Free up additional digitization bandwidth through automation and less rights-checking
- 15. Challenge #1: Using High-Res Images Online for the First Time • Standardize & resolve issues with storage for master image files • Populate collections management system records for images with filepaths to network storage location of master image files • Create mechanisms for access, selection, FTP of files Handmade Tape Recording Featuring Live Performance of Jim Croce, 1976-1977 (96.12.10)
- 16. Challenge #2: Guidelines for Online Delivery of Images • Move away from misapplied Creative Commons statements on individual artifacts to general policies encouraging legal use • Develop brief, simple guidelines for staff to determine when images can be downloaded • Populate download status of images within collections management system Star Wars Action Figure, Stormtrooper, 1978-1985 (2000.84.2)
- 17. Challenge #3: Automate Manual Image Delivery • Develop new workflows to minimize staff time delivering image files & processing payments • Create basic metadata standards in order to get new records online quickly for online order fulfillmentStarwood Hotel Card Key, 2010 (2011.88.1)
- 18. New Requirement: Enhance Online Artifact Data • Create & display links to related content & artifacts • Display and/or search “new” fields from collections mgmt. system • Make collections mgmt. system changes in conjunction with website changes First Ford Model T Production Card, September 27, 1908 (64.167.641.2)
- 19. Why? • Share as much as we can of what we know about our collection • Help users find out more from us (or others) • Make our data as granular as possible to enable internal and external future use World War II Poster, "Keep Your Machinery in the Fight, Share with Neighbors," 1943 (94.5.2)
- 20. Challenge #1: Previously Internal Data Becomes Public • Clean up data (measurement notes, technique, color, inscriptions) • Review our cataloging practices and vocabularies with an eye toward users • Ensure internal consistency in the fields and terms used Computer Perfection Game, 1979-1985 (99.224.1)
- 21. Challenge #2: Populate New Fields • Move titles of works from general subject field to title-specific subject field • Move names of events from general subject field to event-specific subject field • Time switchover to avoid duplication Racing Poster, Race Cars at Le Mans, "24 Heures du Mans," 1966 (89.1.1734.120)
- 22. Challenge #3: Develop Rules & Delivery Strategies • Related content delivered dually—via algorithms in website CMS or through collections mgmt. system • Related objects populated within collections mgmt. system Henry Ford Posing with Painting by C. Bennett Linder, circa 1930 (P.O.6684)
- 23. Oh, and… • Define fields to be included in simple search • Define fields to be included in advanced search • Implement ability to shift default image from first sequential image for archival material • Migrate 800+ oral history video clips to digital collections • Determine how to handle objects with large numbers of multimedia • Add “Other Names” as searchable (but not display) field) • Release constraint requiring creation of duplicate multimedia records when attached to more than one catalog record • Discuss (and reject) idea of “short title” field • Develop plan, formats, metadata, data structure to support 360-degree artifact multimedia credit field • Support geotagging data (for future app development • Rethink existing “set” concept & consider delivery mechanism • Capture collections-related metadata & narratives from existing microsites before they sunset, & digitize as many of these objects as possible • Develop or modify systems & processes to support delivery of data via daily XML export from our collections management system & FTP for automated loading into indexing tool • Modify existing or create new workflows for approving & tracking harvested records, deleting records, error handling • Develop most useful possible requirements for supporting date searching, given limitations of our existing data (e.g. formats, ranges, “circa” dates, unknown dates, subject vs. creation date, etc.) • Clean up miscellaneous video formats (WMV vs. MP4) • Locate and reload videos that cannot be reprocessed/optimized through 3rd-party video platform
- 24. Lessons Learned • Until you use any piece of data, you cannot be sure how consistent you’ve been with it • Your data is never “done” • It’s a challenge to find a balance between planning for future needs and the number of hours in the day • Building a system is a related but distinct task from populating that system Students in a Classroom at Henry Ford Institute of Agricultural Engineering, England, 1939 (P.188.26304)
- 25. Sneak Peek
- 26. Thank you! Ellice Engdahl Digital Collections & Content Manager, The Henry Ford @ErisuEEE
Editor's Notes
- Introduction Will be discussing state of our digital collections data & related challenges we’ve discovered in building a new collections website
- Bit about us… Currently called The Henry Ford, but founded 1929 by Henry Ford as Edison Institute, on 50th anniversary of Thomas Edison’s invention of electric light Henry interested in significant achievements of industrial history (like Edison’s) but also everyday life for Americans before changes from automobile and industrialization.
- Two large venues: Henry Ford Museum Museum = 600,000 sq. ft. (plus 45,000 sq. ft. for IMAX) Greenfield Village, open-air historical village Village (developed) = 81 acres ~80 historic structures Also have Benson Ford Research Center = portal to our collex, incl. reading room and behind-the-scenes tours/access IMAX theater – largest screen in Michigan Run Ford Rouge Plant factory tour for FMC Henry Ford Academy, a public charter high school with 485 students Numbers People ~300 full-time staff 1100+ part-time staff (summer peak) Over 600 volunteers contributing nearly 100,000 hours annually Campus ~236 acre campus 100+ buildings
- Quote from Curator of Public Life Donna Braden– gets at both good & bad things about vast collection About ¼ million objects--estimates vary depending on how you count, but that’s estimate for accreditation Educated guess for our archives #—but lots, plus historic audio/video in prob any format ever existed Vast storage facilities chock full of material
- Brief history of our digitization efforts…. Pre-2010: Lots of collections metadata and images created as “random acts of digitization” (another phrase I stole, this time from the Smithsonian digitization director). Created as one-offs, no standardization between various projects Can give us a leg-up on updating these records to current standards, but very little of it completely ready to go 2010: Really started to standardize, using consistent cataloging standards for metadata and consistent multimedia formats 2011: EMu became our collex mgmt system & repository of record for collex info + consistently sized/formatted imgs for web Result of standardization: Have increased number of digitized artifacts 140-fold over last 5 years— A drop in the bucket compared to the size of the whole collection, but we keep finding ways to increase our efficiency without sacrificing standards
- THF currently in the middle of “digital transformation”—a rethinking of how digital world & physical campus intersect & how we can increasingly integrate the two. One initial output of this process: Brand-new website coming in early 2016 for our entire institution, which will include sorely-needed update to our digital collections. Digital collections will be integrated into our main website (unlike now), indexed (unlike now), will support granular advanced searching (unlike now), and will have a number of cool new features, a few of which I’ll be talking about.
- High-level look at user types & user goals we initially established for our new digital collections We are a public history institution, so general public is primary audience, but also want to support other groups. Finding, using, & sharing information about our artifacts is key. Will focus today on 3 of the requirements that have been most challenging & have had the most interdependence with & impact on our digital collections metadata.
- First requirement: to integrate records for archival collections within currently all item-level digital collections. Has been on our radar for a while, and in fact we had a project to tackle this on the data side a few years ago, but we’re finally making it happen online.
- Why do we want to do this? Currently, finding aids are published within DALNET, our library cataloging system--but individual items from archival collections can be found within digital collections. Want to bring the two together to provide the context for the individual items Also, to help users understand how much of any given collection may NOT have been digitized (as it’s relatively rare for us to digitize an entire archival collection).
- First challenge we faced with this requirement was figuring out how to define search and display for these new records These records represent a grouping of individual items, while rest of our existing digital collections data represents those individual items themselves. Were able to use a lot of work we’d done as part of that data project I mentioned, which was great—at least (for the most part) we weren’t having to figure out new fields in our collections management system in which to populate values. However, we did have to look at fields to figure out which it made sense to publish online, what order they should appear in, & which should be searchable, both through basic & advanced search. Also had to figure out how we wanted to handle the finding aids themselves—we finally settled on attaching the finding aids in PDF format--but we had to add a new value to the metadata so they’re identifiable as finding aids (versus other types of PDFs).
- This is a challenge that is sometimes seen as secondary to creating a system: Actually coming up with a plan to populate it. Since our method of publishing digital finding aids has always been adding them as PDFs to our library catalog, we don’t yet have systems in place where staff populate these values in our collections management system as part of their daily work, & we don’t have a backlog of collection-level data as we do with our digital collections. We had to settle on a number of collection-level records to complete for launch (which ended up being 80-100), and then develop a plan to add more over time. This is complicated by the fact that relatively few of our archival collections have a full, complete finding aid—so for some collections, this work entails processing and describing the collection. Will be a slow road, but we’ll start with what we have, & add as we go.
- Third challenge encountered with integration of archival collection records & item-level records: more potential work here to fill desired requirements than we can tackle in the middle of this larger digital transformation project—had to move couple of desired features out of scope. One is creation of a functional finding aid hierarchy—which would mean that for any archival item, you could see where it falls within the entire collection & navigate up or down hierarchical structure to see immediately related items. Have a way to handle this within our collections management system, but work to implement this online, and figure out how we would populate it going forward, was too much for us to take on right now. Second is creating searchable transcriptions of textual documents. Again, definitely a direction we want to go, but we don’t have systems like OCR capture or staff in place for this internally, so had to call it out of scope for launch. Hoping to revisit both of these in future phases of our digital transformation.
- Second new requirement = actually two requirements that entail much the same work— Delivering high resolution images (for a service fee) through our digital collections site, Allowing users to zoom deeply into high-res images on the web in order to view details, read small text, etc. Current site features a medium-res image as the largest available, with no zoom, so many details are essentially inaccessible.
- Think everyone will get the “why” behind being able to zoom into images on the site—basically, the higher resolution we can provide, the more access users have to examine/understand that artifact. Delivering image files online for a fee is likely more controversial. Some museums are moving to a model to provide completely free access for users to download high-resolution collections I personally have no issue with charging a reasonable fee for access to our high-resolution images, as I’m acutely aware how much money it takes for us to a) digitize this content, & b) make it accessible through modern digital portals. 5 years into our institutional commitment to digitizing our collections, we still have 4 digitization staff members who remain on soft money, which will at some point be used up. (I happen to be one of them.) If we can provide good access to all users through the site, continue to allow free and clear download of medium-resolution images, & charge anyone who needs more than that, seems like a fair trade-off to me. In addition, we already deliver high-res for a fee, but we do it manually & with one-by-one “rights-checking” by internal staff. Since we have a limited number of digitization staff, & none of us are lawyers, by finding ways to simplify this work & the delivery of requests, we free up more bandwidth to provide access to more artifacts.
- First challenge: since we don’t currently deliver high-res images to the web, have no system in place to do this. Currently use the digital asset management functionality of our collections management system to deliver derivative JPG images to the web, but preservation TIF files live on our network. Sadly, first two bullet points on this slide have probably already encompassed hundreds of hours of work—making sure we have copies of all collections images within a certain universe of networked drives, & that we know the correct path to each image. Finding all the places where we hadn’t been 100% consistent in what we were doing, because up till now it hasn’t mattered. Beyond that, are having to develop systems to access, select, FTP, & process image files appropriately so they’ll be available to the new site—but without endangering original assets.
- Challenge #2: Creating guidelines for delivering images online Around 2010/2011, started applying non-commercial Creative Commons statements to collections images. Went a little gung-ho with it & began applying them to nearly all of these images—regardless of whether rights were ours to grant or not. Turns out, Creative Commons only applies if rights are yours to grant. As part of new online image delivery feature, we’re trying to move away from (previously misapplied) letter of the law to more spirit of the law: Want people to use our images in all legal ways & we want to minimize the degree to which we interfere in that. Replacing individual Creative Commons statements with more general policies encouraging use. Also worked with copyright attorney to develop a handful of simple guidelines for staff to determine what will/will not be available for high-res online download Now working through backlog of 80K+ digitized images to apply the guidelines, & applying them on all new images we’re digitizing as well
- Challenge #3: another example of systems vs. workflow. Will have a system to deliver high-res images online in an automated fashion, but the way we currently deliver high-res images lives completely outside that system Involves e-mail or CD delivery of image files as well as manual charging of credit cards. Trying to develop new workflows where any image ordered can quickly & efficiently be put online with minimal but sufficient metadata within as little as 24 hours--from there the interested party (or anyone else) can order it. If zooming into high-res image online takes care of their needs, won’t even have to order it. In any case, more time we can save from manual fulfillment of image requests, more digitization bandwidth we can free up.
- Last requirement to cover is general enhancements to artifact metadata we display online. For any given artifact, want to be able to show users other concretely related artifacts within our collection, as well as other assets that might provide more information. Will also take advantage of some of the cataloging we already do in collections management system by including additional metadata that doesn’t currently display within digital collections.
- “Why” on this one pretty straightforward—want to share as much information as we can with the world. Some information we can’t share—specific locations for objects in our storage areas, donor information, or valuations for objects. Everything else we want to get out there for interested parties to use. Also making our collections metadata a bit more granular.
- Always fun to take data no one ever imagined would be seen by the outside world & make it public. You find out exactly how inconsistently or non-usefully you’ve been populating those values. Actually have pretty good data consistency at The Henry Ford—in nearly two decades of working with large datasets, ours is among the cleanest I’ve ever seen. Still, are having to go back through these additional fields and make sure the data in them is a) good, b) consistent, and c) useful for the general public (classic example for the last one is “do people search for automobile or for car?” LOC would say “automobile”; the people say “car.”). Have refined the number and types of terms we use.
- Are taking advantage of the switchover between metadata delivery systems to move data around within our cataloging system as well. Right now, we include titles of works (like plays, novels, artworks, etc.) & names of events (Indianapolis 500, Panama-Pacific International Exposition) within general subjects field. Are breaking each of these out: titles into title field, and events into event field. However, since current system doesn’t pick up these new fields, we’re also having to plan a schedule to remove them from the existing field not too early to compromise our existing website, but not too late to have duplicate data display on the new website.
- Based on requirements, we knew we wanted to tie together assets that relate to any given artifact, but What we’re still working on figuring out is what those rules are. One of scenario we’re thinking about is if you’re looking at a 1960s era auto race photo within our collection, you might also be interested to know that the car shown in that photo is in our collection. Another example is to link any historic image of one of Greenfield Village’s buildings on its original site to the record for that building. We also want to link to other assets, such as blog posts or videos, that relate to a given artifact—but we don’t want to overwhelm the user—for example, tying together related artifacts for topics as general as Henry Ford or Thomas Edison could result in hundreds or thousands of matches Are trying to find balance between providing information and being judicious in our choices so everything we relate to any given artifact is of high value to end-users. Targeting as highest-value either connections that cannot be easily made through metadata, or those where the exact connection would be overwhelmed in a metadata-only search (for ex “Henry Ford,” “Thomas Edison”).
- Those are the three largest, most complex requirements we’ve been tackling as a part of this project, but there are a host of other things we’re also working on in order to get ready for the new site. Not going to read out this list, and it’s not a complete inventory, but it gives an idea of the amount of work we’re putting into revamping our collections website--even when we’ve already had an existing site, with air-quote “standardized data” for five years.
- Which brings me to lessons learned through our efforts. Even if you’re populating a piece of metadata, if you’re not actually using it anywhere & nothing is depending on it, you can’t be sure how consistent or appropriate your work has been. You find out the hard way when you take a closer look. Data is never done! A bit of a converse to the first point—no matter how consistently & appropriately you populate data, changing interfaces & changing requirements always require going back and modifying or adding to what you are doing. Can mitigate some of this by planning smartly for the future—but last thing you want to do is to have people spend hours populating data that never ends up getting used. There’s a difference between creating a system to use and/or display data & the resources and workflow needed to create that data—latter is a totally different issue. Again, there’s a balance between building a killer system that you can’t really populate, & creating something before you’re fully ready to ramp up and use it (with knowledge that you’ll grow into it over time).
- And last…. We’re a ways out from release of our new collections site, & there’s not too much I can share yet, but here’s a quick sneak peek at how an artifact record will look on our new site in that image on the left. It’s a design comp, so take data, images, pretty much everything with a grain of salt. Smaller image on the right is an alternate, condensed view of an artifact record that will allow users to (and us) to embed artifacts within other websites & social media channels. Going to be great, so keep an eye out for the release of our new site.
- That’s it! Couple of ways to contact me--feel free to contact me with questions today or any time. Thanks so much!