I was going to call this 5 easy steps to a robust program of assessment and analysis – but I can’t honestly say any of this is easy. However, I can say that it’s worth it!
Since we were starting from scratch, we wanted to make sure the data and reports we would develop would answer the questions asked by the stakeholders. Here was our initial list of questions:
Creating the Service of Your Patrons' Dreams: A Short History of Scan & Deliver
Creating the Service of Your Patrons’ Dreams (a short history of Scan & Deliver) Tom Bruno and Sarah Tudesco Harvard College Library 6/8/12
The Simpsons On Patron ExpectationsAgnes: And you, start over. Iwant everything in one bag. Homer’s ILLiadPimple Faced Kid: Yes, maam!Agnes: But I dont want thebag to be heavy.Pimple Faced Kid: I dont thinkthats possible!Agnes: What are you, the (Beware of Greeks bearingpossible police? Just do it! PDFs)Simpson Safari, Season 12 Episode 17 (Airdate: April 1, 2001)
Guess What?• WE ARE THE POSSIBLE POLICE!• Your job: figure out how to Make It Happen without breaking the law or your budget• User engagement + data assessment + continuous improvement = making yourself indispensable, therefore JP Porcaro, Patron Saint of Making It Happen Awesome (LJ 2012 Mover & shaker)
Overview• History of the service – Project Planning, Launch, Growing Pains, Assessment, Future• How is Scan & Deliver like Angry Birds?
Overview Part 2- Meet The Data • Sarah Tudesco- (Re)building the reporting workflows • Meeting stakeholder data needs • Prioritizing & Building reports • Presenting the dataDisclaimer: Sarah is not an android, butshe is a wizard
Scan & Deliver Basic Facts • Launched April 22nd, 2009 • 9 “Hub” libraries + 6 additional participating collections • Dumbarton Oaks added in 2011, new collections TBA • Open to current faculty, students, and staff • No charge- 2 request/patron/business day • 4 business-day turnaround
Timeline65 million 1636 2008 2009 2010 2011 2015years agoDinosaurs Harvard ULC authorizes Launch! 1st DDO Hoverboardsroam Earth Founded S&D service Assessment added NOTE: Time axis should be read from left to right only, unless you own a TARDIS or a DeLorean
Origins• Before, ILL units used OCLC as mechanism for requesting/fulfilling article requests• Cumbersome, negatively impacted ILL operations, patron confusion about what items/collections were eligible, what were not• Short-lived scanning pilot at HD proved too costly to implement permanently
A New Mandate• University Library Council authorized the formation of an Electronic Document Delivery project in 2008• Each school would be represented, ensuring 100% coverage of eligible Harvard Library patrons• Harvard Library’s first “shared service,” requiring unprecedented coordination and cooperation across previously independent entities (a.k.a. “tubs”)• Before S&D, think of Harvard Libraries as a consortium of 70+ libraries
Project Planning Phase • All participating units wouldSAVE A TREE adopt ILLiad for resource sharing operations • Existing ILLiad units (i.e., HMS, HVL, HLS) would be merged into shared hosted server • New units would be brought online at staggered intervals of 1-2 weeksSCAN A BOOK • Scan & Deliver links would go live in OPAC on April 22, 2009 – Earth Day
Flies in the Ointment- #1• In Fall 2008 many universities lost significant amounts on their endowments, including Harvard• Questions arose about wisdom of adding an ambitious new library service at this time “Brother, can you spare a billion?”• Mandate was sent back to ULC and was reaffirmed unanimously
Flies in the Ointment- #2 • In February 2009 (less than two months before launch!), we realized our workflow would not work as proposed • A staff-mediated workaround was developed, documented, and communicated • Lesson Learned: Make sure your project has the right combination of stakeholders!
LAUNCH!• First request received was from the Biblioteca Berenson in Florence, Italy• Enabling remote access to Harvard Library patrons overseas has been a huge selling point for the service• One graduate student in China was able to complete her dissertation without having to return to the United States to access the library collection
Growing Pains • Every year has seen double- digit growth in request volume over previous year • At HCL, request volume exceeded capacity of our resource sharing unit • New cross-divisional workflow established to meet demand • Service Level Agreement defined each division’s responsibilities and expectations
User Assessment• In Fall 2010 Scan & Deliver conducted extensive user assessment of service• Short survey sent to all recent users via an email link• Focus Groups asked patrons from four select groups (Faculty, Grad Students, Undergrads, Staff) to share impressions of service
Assessment Findings • Scan & Deliver had quickly been incorporated into curriculum support role • Service worked well with “just in time” research methods • Patrons wanted more microform eligibility, OCR- ready PDFs • Wide misunderstanding of how copyright operated
Listening to the Patron #1• Initially 2 request/patron/day was a “hard” limit – 3rd+ requests were queued for processing next day• Managing this quickly became a nightmare• Our solution: treat request limits as “Service Minimums” – process them if there was capacity• Hypothetical abuse was impacting actual service
Listening to the Patron #2• Patrons were routinely asking for Tables of Contents, Index, Bibliography, Accompanying Images/Plates, and Title & Verso pages – no standard form of entry• Modified request forms so that these “menu options” could be selected via checkboxes, appear in unused CitedIn fields• Less cut & paste for staff, more transparent and reliable for patrons!
Listening to the Patron #3• Resends often unwittingly reproduced original error, leading to patron frustration• New workflow where resend requests went to different scanning unit, staff mediation if needed – 24-hour turnaround on resends• The Takeaway: If you’re going to adopt a high- performance workflow, you need to adopt a “high touch” troubleshooting workflow to keep your patrons happy
The Dreaded “C”-Word • When service was launched, our OGC said to use Section 108 guidelines • Poor fit when scanning our material for our own patrons • Closest analogy= Reserves, but that’s for entire classes, not individuals • Using S&D scans for curriculum support further complicates this
Fair Use, Anyone?• Bottom Line: We could probably fill a lot more than we currently do• Huge potential benefit to distance students, faculty abroad, preservation of originals• Anyone want to get sued so we know exactly how much?• Our solution: central oversight removes burden from local units
Assessment• Per unit cost studies suggested economies of scale made service feasible• Ongoing problems of cost at smaller units and HD, where labor primarily performed by staff, not students• Development of “success Library Science Dog metrics” to evaluate services (Don’t Try This At Home!)
Scan & Deliver vs. Angry Birds• Launched in 2009 Today the world…• Wildly successful beyond expectation• Fulfilled previously unacknowledged need• Iterative development- new features, new levels (i.e., collections) being added …tomorrow the universe!• Totally addictive
Forward the Future • HD ILL article scanning pilot, Borrow Direct • Campus book delivery • Adding more collections, automation • Better integration with e-reserves, Ares? • Collaboration with preservation to save public domain scans
Questions? Comments?Tom BrunoHead of Resource SharingHarvard College Library617firstname.lastname@example.orgIM: email@example.com firstname.lastname@example.org
• Step 1 – AskingScan and Deliver Statistics Questions5 steps to creating a robust program ofassessment and analysis. • Step 2 – Setting Priorities • Step 3 – Analyzing & Building • Step 4 – Presenting the Data • Step 5 – Feedback & Continuous Improvement
Step 1 – Asking Questions• How many requests?• Who is making requests?• How much gets filled?• Whats the turnaround from request to delivery?• Are we meeting the service agreements?• How many requests are unfilled?• How many requests get routed to ILL?• Who is filling the requests?• How are people using the service?• Are people satisfied with the service?
Step 2 – Setting Priorities• Patrons: – How many patrons use the service? – Which faculty or school are they associated with (Law, Medicine, Faculty of Arts & Sciences)? – What is their role at the University (student – undergraduate and graduate, faculty, staff, other)?• Requests: – How many Scan & Deliver requests placed daily, weekly, monthly? – How many requests were filled? By which school?
Step 3 – Analyzing and BuildingCore Data Set: Data Table with all Scan&DeliverRequests Placed by Patrons • Document Type = “Scan&Deliver” (Transactions) • ChangedTo = “Submitted by Customer” (Tracking)
Step 3 – Analyzing and BuildingCore Data Set: Data Table with all Scan&DeliverRequests Filled by Participating Libraries • Document Type = “Scan&Deliver” (Transactions) • ChangedTo = “Delivered to Web”(Tracking)
Step 3 – Analyzing and Building Access Form: Input Dates – when ‘Run Reports’ is clicked, an Access Macro runs in the background and builds reporting tables. Access Form: Download Report Data, form displays the dates in the current available reporting table and user can download data for requests, users, and cancellations.
Step 4 – Presenting the Data Who? How much do they want? What did they get?FY2012 Report: Requests Submitted 7/1/11 – 2/29/12 [Screen Shots from Excel Spreadsheet]
Scan & Deliver User Stats• 15,860 unique patrons since service began in 2009• Average Requests Per User: 9• Most Requests: 1,807 by a Graduate Student• 36% of our patrons make a single request• Graduate Students are our biggest customer (42% of all requests placed).
Step 5 – Feedback & Process Improvement• New workflows – Do they impact the reporting data?• New questions – Can the existing reporting data answer the question? – What about other resource sharing services (Borrow Direct? Interlibrary Loan?)• Improving existing data – Make the data clear to a wider audience
Questions? Comments?Sarah TudescoCollection Management Analyst and Reporting Librarian for Harvard College Library617email@example.comIM: studesco (Yahoo, Google Chat)