6-6-12 Presentation Slides, “Creating Access to Audio & Video Digital Media: The Variations on Video Project & the Rock and Roll Hall of Fame” Presented by: Karen Cariani, Adam Wead, & Jon Dunn

1. Hot Topics: The DuraSpace
Community Webinar Series
Series Two:
Managing and preserving audio and
video in your digital repository
Curated by Karen Cariani
June 6, 2012 Hot Topics: DuraSpace Community Webinar Series

2. Webinar 2: Creating Access to
Audio & Digital Media:
The Variations on Video Project &
the Rock and Roll Hall of Fame
Presented by:
Karen Cariani,
Jon Dunn & Adam Wead
June 6, 2012 Hot Topics: DuraSpace Community Webinar Series

3. About our presenters
• Karen Cariani, Director, Media Library
& Archives, WGBH Boston
• Jon Dunn, Director, Library
Technologies & Digital Libraries, Indiana
University
• Adam Wead, Systems & Digital
Collections Librarian, Rock and Roll Hall
of Fame
June 6, 2012 Hot Topics: DuraSpace Community Webinar Series

4. Rock and Roll Hall of Fame
Adam Weed,
Systems & Digital Collection Librarian
June 6, 2012 Hot Topics: DuraSpace Community Webinar Series

5. Rock and Roll Hall Of Fame Library
and Archives
• Hall of Fame was founded in 1985
• Original library was in the Museum, but was
overrun due to space limitations
• Archival materials were moved to off-site
storage
• In 2000, the Rockhall Foundation (NYC) began
a capital campaign
• Raised $10 million
• New facility built at Cuyahoga Community
College Metro campus in 2010
Hot Topics: DuraSpace Community Webinar Series

6. Library and Archives (cont.)
• Began archival processing and
cataloging in 2010
• Grand opening in April 2012
• 256 processed archival collections
• Over 8,000 cataloged library items
• Free and open to the public 9–5, M-F
• Serves researchers, Museum staff and
anyone who wants to come in
Hot Topics: DuraSpace Community Webinar Series

7. Library and Archives Staff
• 8 full time staff (permanent)
• Librarians: Director, Public Services
Librarian, Metadata Librarian, Library
Assistant, Systems Librarian
• Archivists: Head Archivist, A/V
Archivist, Assistant Archivist
• 3 project archivists and 3 project
catalogers (2010-2012)
• Interns and volunteers
Hot Topics: DuraSpace Community Webinar Series

8. Digital Collections: Museum
• No pre-existing asset management system
• No digitization
• No archival processing
• Needed to have something in place by April
2012
• Decided to focus on institutional content first
• Other archival collections from storage were
unprocessed and their contents unknown
• Institutional content had the greatest need for
preservation and access requirements
Hot Topics: DuraSpace Community Webinar Series

9. Digital Collections: Museum (cont.)
• Approx. 1,500 hours of video in various
tape formats: Betacam, MiniDV,
DigiBeta
• Educational department lectures and
presentations
• Songwriters to Soundmen TV series
• Some induction footage
• No born-digital content (yet)
• Outsourced to vendor for digitization
Hot Topics: DuraSpace Community Webinar Series

10. Digital Collections: Foundation
• Induction ceremony footage
• Over 800 additional tapes
• Selected 150 for priority digitization
• Remaining tapes can be processed “as
needed” – mostly ISOs and
miscellaneous footage
• Formats include BetacamSP, DigiBeta
and HDCam
• 90 mins to 2 hours each
Hot Topics: DuraSpace Community Webinar Series

11. Video Content Challenges
• Archival-quality: uncompressed, 10-bit,
4:2:2 chroma video yields very large
files
• 125 GB/hr for SD, the majority of our
content, and 500 GB/hr for HD
• Currently, only 60 hours of priority HD
content from HDCam tapes but 400
total tapes exist
• Future induction footage will be HD
Hot Topics: DuraSpace Community Webinar Series

12. Access Issues
• Access limited to onsite only
• Surrogate files are primary means of
access: compressed H.264 video
• Uncompressed files only needed for
future transcoding or for third-party
distribution if H.264 video is unsuitable
• Have not had any requests for
preservation video yet
Hot Topics: DuraSpace Community Webinar Series

13. Storage Solutions
• Tailored our storage system to fit our access
needs: high data volume, low access rate
• Small, surrogate video files are kept on disk,
in a SAN for immediate access
• Uncompressed video files are kept on LTO-5
tape using hierarchical storage so that they
can be moved back to disk when needed
• For archiving, all files are backed up to two
LTO-5 tapes: one stored at the Museum (3 mi.
away), and a second at Iron Mountain in
Boyers, PA
Hot Topics: DuraSpace Community Webinar Series

14. Storage Pros and Cons
• Start small, grow as needed
• Archiving data to tape is time-consuming
• Ex. 8 TB ingest takes about a week
• Retrieving uncompressed data from tape is
time-consuming
• Requests for large files must be scheduled:
limited hardware resources for multiple
retrievals
• Modular design of our storage system offers
easy expansion should requirements change
Hot Topics: DuraSpace Community Webinar Series

15. Tools
• Hydra software stack
• Fedora repository
• Large video files are stored as external
datastreams accessible via HTTP
• PBCore for descriptive metadata
• Records are exported to Blacklight for
public access
• Video streamed using open source
software: Wowza with Flowplayer
Hot Topics: DuraSpace Community Webinar Series

16. System Overview
Hot Topics: DuraSpace Community Webinar Series

17. Fedora Model
Hot Topics: DuraSpace Community Webinar Series

18. Costs and Limitations
• Storage systems represent the bulk of
the monetary costs
• Digitization vendor expenses come
second: George Blood Audio/Video
digitizes our video
• Staff hours spent developing software
(me)
• Staff hours spent cataloging items: one
librarian, one archivist, plus volunteers
Hot Topics: DuraSpace Community Webinar Series

19. Pros and Cons
• Costs saved through in-house development vs.
ability to produce working software
• Atomistic fedora model can work with any kind
of A/V content
• Staffing determines cataloging rate
• Lack of organization make processing
collections time-consuming
• We have a digital “backlog” for content that we
can’t yet ingest
Hot Topics: DuraSpace Community Webinar Series

20. Future Work/Challenges
• Images and documents: institutional
and from archival collections
• Audio from archival collections
• Models in Hydra to deal with new
content types
• Mapping archival description to Fedora
objects to create coherent finding aids
• Merging physical archival content with
digital
Hot Topics: DuraSpace Community Webinar Series

21. For More Info
• About Us: http://library.rockhall.com
• Our Catalog:
http://catalog.rockhall.com
• About Hydra: http://hydraproject.org/
• Our Hydra “head”:
https://github.com/awead/Hydra-Rock
Hot Topics: DuraSpace Community Webinar Series

22. Variations on Video:
Building the Next
Generation Library Media
Management System
DuraSpace Hot Topics Webinar
Jon Dunn
Indiana University
June 6, 2012

23. Presentation Outline
 Background and motivations
 Technical architecture
 Development roadmap and process

24. Background: Variations
 Open source digital music library system
developed at Indiana University
 In use at IU since 1996, in various forms
 Used at about 20 institutions, mainly for
streaming audio course reserves
 variations.sourceforge.net

25. Variations on Video
 Educational institutions collaborating with open
source software communities
 Led by Indiana University and Northwestern
University Libraries
 Goal: Create an open source system to enable
libraries and archives to provide online access to
video and audio collections

26. Variations on Video
 Planning grant from IMLS
 August 2010 – June 2011
 Implementation grant from IMLS
 October 2011 – September 2014

27. Variations on Video:
Project Objectives
 Develop a digital audio/video management and
delivery system, focused on needs of libraries and
archives
 Follow an agile, open source development model
 Leverage existing technologies, where feasible
 Communicate and market the project broadly to
increase awareness and grow the community of
users and developers

28. Partners and Collaborators

29. Variations on Video:
Motivators
 Demand from Variations implementers and other
institutions
 Increased video digitization and creation at IU
 IU Media Preservation Initiative
 IU IT strategic plan: Empowering People
 No existing system serves needs of libraries
 History of involvement in open and community
source software
 Desire to create a sustainable foundation for
Variations development and maintenance

30. Indiana University Bloomington
Media Preservation Plan
www.indiana.edu/~medpres/

31. Serving the needs of libraries
and archives
 Wide variety of audio/video collections, uses, and
access needs
 Licensed educational video collections
 Video/audio e-reserves
 Archival collections
 Open access collections
 Research and teaching & learning use
 Long-term management and access
 Tie-in with preservation

32. Variations on Video: Content
Video Live University
digitized Performances produced
from Feature Films video
library Research-
collections related
Video Archival
TV shows collections
Files with Lecture
purchased Series
Faculty-
or licensed produced
streaming video
rights Field
Documentaries
Recordings

33. Existing Products
 Institutional and digital library repositories
 DSpace, Fedora, Digital Commons, ContentDM
 Web video services
 YouTube, Vimeo
 Streaming servers
 Flash Media Server, Real Helix, Wowza, Red5
 Classroom lecture capture
 Echo360, Mediasite, Opencast Matterhorn
 Digital asset management systems
 OpenText, NetXposure
 Online video environments
 Kaltura, Brightcove

34. Variations on Video
 Designed to meet needs of libraries and archives
 Storage requirements
 Streaming
 Transcoding
 Access control
 Media players
 Structural metadata / navigation
 Ease of use and administration

35. Variations on Video:
Basic Functionality
 Video file upload/ingestion
 Transcoding
 Descriptive metadata entry / bulk upload
 Basic structural metadata
 Discovery
 Delivery and navigation
 Reuse

36. Key Architectural Components
 Hydra framework
 Fedora repository
 Opencast Matterhorn
 Streaming Server

37. Hydra Framework
http://projecthydra.org

38. Opencast Matterhorn
http://opencast.org/matterhorn/

39. Architecture Revisited
 Hydra Framework
 Search for free
 Rich toolset for quick development
 “One body, many heads”
 Opencast Matterhorn
 Flexible processing pipeline; highly modular
 Streaming Server
 Flash/RTMP, HTML5/HTTP, and Apple Live HTTP
Streaming all required
 Red5 – open source and support in Matterhorn
 Flash Media Server – popular and iOS support

40. Variations on Video: Roadmap
 Release 0 (July 2012)
 Manual video and basic metadata ingest
 Transcoding
 Search and basic playback in desktop browsers
 Release 1 (December 2012)
 Desktop and mobile audio/video playback
 Clip and playlist creation
 Authentication and group-based authorization
 Manual and batch ingest
 More complete discovery interface
 Release 2+ (2013-) …

41. Development Process
 Scrum agile approach, based at IU and NU
 Single virtual distributed team
 Other partners will install, test, and provide
feedback
 Transparent planning and development process
 Open wiki, Jira, mailing lists
 Regular public demos
 Engage additional community involvement over
time

42. For more information
 http://variationsonvideo.org/
 Variations on Video e-mail list
 E-mail vov-l-subscribe@indiana.edu to subscribe
twitter.com/varvideo
facebook.com/varvideo

43. Questions?
Jon Dunn
Indiana University
jwd@iu.edu
Adam Wead
Rock and Roll Hall of Fame
awead@rockhall.org
June 6, 2012 Hot Topics: DuraSpace Community Webinar Series

44. Solutions for
Digital Media Preservation
• DuraCloud is simple, flexible and cost
effective
• DuraCloud offers expandable storage
space as needed for your media files
• Streaming media services available
June 6, 2012 Hot Topics: DuraSpace Community Webinar Series

45. Solutions for
Digital Media Preservation
• Sign up for a free 2 month trial account
today www.duracloud.org
• Priced for every budget
http://duracloud.org/pricing
June 6, 2012 Hot Topics: DuraSpace Community Webinar Series