This document describes the development and testing of CrowdAsk, a crowdsourced library help system created by researchers at Purdue University Libraries. It received funding from an IMLS grant and allows users to ask questions that can be answered by other students, faculty experts, or librarians. The system was beta tested by undergraduate students in classes and specialists in the special collections department. Analysis found that students asked a variety of questions and engagement was high based on website usage statistics. While badges and points were not strong motivators, expert users were motivated by reciprocity. The system is now launched on the Purdue Libraries website.

1. A Crowdsourced Library Help System
Internet Librarian 2014
Funded by 2013 IMLS Sparks! Ignition Grant
https://github.com/crowdask/crowdask
Tao Zhang, PhD
Digital User Experience
Specialist
Purdue University Libraries
zhan1022@purdue.edu
Ilana Stonebraker
Business Information Specialist
Purdue University Libraries
stonebraker@purdue.edu

2. Introduction
• Crowdsourced library help system (with
gamification)
• crowdask.lib.purdue.edu
• Year long IMLS funded project
• Github:
https://github.com/crowdask0/crowdask
2

3. Agenda
• Challenges/Solutions
• Project timeline
• Project results
– CrowdAsk System
– Classroom, Usability Testing and Archives beta-test
– Reference service rollout
• Discussion of Implications
3

4. How we got to this
• Killing a FAQ
• How do we know which users want what in
help?
4

5. Results
Research Project-
Characteristics/ Design
5
Library Familiarity
Perceived
Competence
Work Avoidance
Task Orientation
(Learning-oriented)
Task Orientation
(Performance-oriented)
0.758
Conceptual
0.358
Unobtrusive
0.341
Relevant to the user’s
immediate situation
-0.358
0.304 -0.351
-0.414
-0.43
-0.448 Easy to access
0.36
Help should be…
How we got to this

6. Results
Research Project-
Characteristics/ Format
How we got to this
6
Library Familiarity
Perceived
Competence
Work Avoidance
Task Orientation
(Learning-oriented)
Task Orientation
(Performance-oriented)
0.758
Index
Screenshots
Table of Contents
Pictures/Images
0.616
0.36
0.523
Video Tutorials
0.332
0.311
-0.301
-0.377
-0.495
-0.664
0.36

7. 7

8. 8
Librarian
If patron is
satisfied
• They learn
• They tell
their friends
• They teach
others
If patron is not
satisfied
• Have to ask
again
• They ask
again later
If a patron is
confused
• They ask
again later
• They ask
someone
else

9. Challenges
9
Questions are all treated alike
• Majority of reference questions are lower level (Bishop &
Bartlett, 2013)
• Questions are context-based
• The process of reference decontextualizing then has to add
context in the reference interview process- inefficient
• Lack of utilization of other information sources such as
graduate students, instructors

10. Challenges
• Reference service model flawed
10
Librarians
Other
Students
Friends
Professor

11. Challenges
11

12. Solutions
• “Crowd” sourced
12
– Content experts (such as graduate students) better
utilized
– Novices learning together
• Better reflect participatory culture, metaliteracy

13. Solutions
• Single channel
• Focus on librarians as community builders
13
versus information sources
Librarians
Students
Experts

14. The Project Team
14

15. Project Timeline
Received
Grant,
Hired GA
• August 2013
Prototype
Completed
• December
2013
Beta-testing
• January-
April 2014
Usability
testing
• March- April
2014
Reporting,
Sharing of
Code
• April-
October
2014

16. How it works
• Demo
16

25. Three Types of Beta Tests
Classroom Beta Test
26
Spring 2014
Special Collections
Summer 2014
Website Rollout
Fall 2014

26. Metrics
27
Qualitative
(Usability and
And Interviews)
Quantitative
Qs and As
Google
Analytics Return on
Investment

27. Beta Test- Students
• 370 undergraduate students at Purdue
University
• MGMT 175 (Information Strategies for MGMT
Students)
• English 106 (First-Year Composition)
• GS 175 (Information Strategies for Hospitality
and Tourism Management Students)
28

28. Quantitative Assessment- Students
• Systems statistics
– 211 registered users
– 99 users posted questions, 106 voted
– 122 questions, 232 answers
– Most views on a question: 92
– Most answers to a question: 8
– Most votes on a question: 35
29

29. Quantitative Assessment- Students
• Google Analytics (January 5, 2014 to April 2,
2014)
– 1,150 visits from 474 unique visitors
– 14,715 page views
• average 12.8 pages per visit.
– 6 minutes and 7 seconds average visit duration.
30

30. Types of Questions
• Course-related.
– “Do you know how to retake the quiz on blackboard (for Management
175)?” and “How much will my writing improve throughout the semester?”.
• CrowdAsk-related.
– “How do you earn points on CrowdAsk?”; “May I know the full list of badges
and how to achieve them?”; and “Are we only allowed to ask academic
related questions here?”.
• Library services or resources.
– “Is there a way to search the libraries catalog just for movies?”; “How to
reserve a study room at library”; and “How do you get the actual article to
come up on Business Source Premier instead of just the abstract?”.
• How-to.
– “What is a good website to use to do a voiceover on Prezi?” and “How to
analyze the financial tables of a company?”.
• Conceptual.
– “What is the best citation management software?” and “Could someone tell
me what is the meaning of APA citation?”.
31

31. Qualitative Assessment- Students
• Usability Test of 4 students (2 novice and 2
expert)
• Average duration of task: 2 minutes
• Unclear items
– Value of assigning tags
– Badges- Meaning
– Awareness of Extra privileges
• Motivation for expert users: reciprocity, not
points
32

32. Second Beta Test- Special
Collections
• Usability Testing and Interviews- Usability test of
three Special Collection experts outside the
libraries
• Selection of Special Collection FAQ- integrated
into system.
33

33. Second Beta Test- Special
Collections
34

34. 35

35. User Testing- Special Collections
• Echoed many of students’ usability testing
• Concern over expertise
• Badges, points not much of a motivator
36

36. Launch on Purdue Libraries Website
• August 2014
• Secondary Choice for users to get help
• PUL reference staff
– Enthusiastic meeting at beginning of semester
– Will ensure that CrowdAsk questions answered in one
day timeframe.
• PUL Web Team
– Work with CrowdAsk team to support system
37

37. 38

38. 39

39. Career Information- Purdue Career
Wiki
40

40. Launch on Purdue Libraries Website
Goal: develop sustainable
user engagement and
community involvement as
part of the Purdue University
Libraries website.
41

41. Next Steps
• We want you! Looking for partners and test
cases
• GNU General Public License Version 2 on
GitHub
43

42. How could you use CrowdAsk?
Where are the
Opportunities?
44
Where are the possible
Threats or Weakness?
Where are the possible
Benefits?

43. Thank You
CrowdAsk code
https://github.com/crowdask0/crowdask
Short video on CrowdAsk
http://youtu.be/-kaNIPJ82yA
Ilana Stonebraker
Business Information Specialist
Purdue University Libraries
stonebraker@purdue.edu
Tao Zhang, PhD
Digital User Experience Specialist
Purdue University Libraries
zhan1022@purdue.edu
45

44. Works cited and Image Credit
• Bishop, B. W., & Bartlett, J. A. (2013). Where Do We Go from Here?
Informing Academic Library Staffing through Reference Transaction
Analysis. College & Research Libraries, 74(5), 489-500.
• http://static.guim.co.uk/sys-images/
Money/Consumer/financialservicesbrochures/2014/2/28/139
3599196845/Angry-man-about-to-throw--011.jpg
46