Downloaded 43 times
![Avoid jargon and colloquialisms
Think of your audience.
Wording should be simple.
Avoid technical words.
X "How many times last week did you use the library's Internet-
enabled public access computers?"
O "How many times last week did you use the library's
computers [to access the Internet]?"](https://image.slidesharecdn.com/surveyworkshop-ala-part2-170208211606/85/Using-Surveys-to-Improve-Your-Library-Part-2-45-320.jpg)
![Abbreviations and acronyms
Assume your audience does not know any
of these.
X “CCPL is interested in replacing its OPAC."
O “Cleveland County Public Library is interested in replacing its
online catalog.”
X “How frequently do you use our ILL services?”
O “How frequently do you use our Interlibrary Loan services?
[This service allows us to request material from another library
for you if we do not have it here].”](https://image.slidesharecdn.com/surveyworkshop-ala-part2-170208211606/85/Using-Surveys-to-Improve-Your-Library-Part-2-47-320.jpg)
![Appropriate response choices
– Surveys can be frustrating when the questions are
fixed response without appropriate answer choices.
– Provide answer choices such as “Don’t know,” “N/A,”
“Unsure,” and “Other” where appropriate.
X Does the laptop lending program meet your needs?
Yes__ No__
O Does the laptop lending program meet your needs?
Yes__ No__ N/A__ [or “I’ve never used this program__”]](https://image.slidesharecdn.com/surveyworkshop-ala-part2-170208211606/85/Using-Surveys-to-Improve-Your-Library-Part-2-66-320.jpg)
Uploaded byALATechSource
Using Surveys to Improve Your Library - Part 2
Using surveys to improve your library was a presentation about developing and distributing effective surveys to gather feedback from library patrons. The presentation covered topics like survey validation and piloting, basic survey structure, writing unbiased questions, and acting on survey results. The presenters emphasized designing surveys based on the information needed, validating questions with experts, and using clear unambiguous language to get accurate feedback.
More Related Content
Similar to Using Surveys to Improve Your Library - Part 2
Using Surveys to Improve Your Library - Part 2
- 1. Using surveys toimprove your library Emily Daly & Joyce Chapman Duke University Libraries ALA eLearning Workshop, February 2017
- 2. Introductions Emily Daly, Headof Assessment & User Experience Joyce Chapman, Assessment Analyst & Consultant
- 3. Agenda • Survey validationand piloting • Basic survey structure • Writing actionable survey questions • Survey tools (briefly!) • Acting on survey data • Key takeaways
- 4. Time to hearfrom you! Which methods do you use to distribute surveys in your library?
- 5. SURVEY VALIDATION ANDPILOTING
- 6. What is surveyvalidation? • The process of assessing the survey questions for their dependability • Have two parties review the survey if possible: 1. People familiar with the topic 2. “Expert” in survey question design
- 7. Who are these“experts”? Public libraries might get help from: – A staff person at the State Library – City/county government staff working in assessment/planning College/university library might get help from: – Assessment/UX staff in the library – Assessment office on campus
- 8. Validating the survey •Start with a simple text document • Have validators go through the survey and make notes – Do all questions and answer choices make sense, are they unbiased, etc.? – Will the resulting data help you answer your questions? Is all topical content accurate? • Make changes based on validation!
- 9. What is surveypiloting? • Select a small subset of your target population to take your survey • Even one pilot tester is better than none! • Try to get a range of different people who represent your target group
- 10. Pilot testing yoursurvey • Pilot one by one • Pilot using your final tool (digital or paper) • Observe, ask them to think out loud, or to write notes • Time respondents • Revise your survey and re-test if desired
- 11.
- 12. STRUCTURE OF THESURVEY
- 13. Form follows function •What are you most interested in learning? • Who will analyze the data? How? • Who will see the results? • Who is your target response group? • How much time do they have? • How invested are your respondents?
- 14.
- 15. Some examples “The DuplinCounty Public Library is interested in learning more about what community members think of services provided by the library.”
- 16. Some examples “The DuplinCounty Public Library is interested in learning more about what community members think of services provided by the library.” “The purpose of this study is to identify areas undergraduate students feel should be addressed in order to maintain an effective academic library.”
- 17. More examples “The libraryis particularly interested in the opinions of patrons who live in the western part of the county. You have been selected at random from community residents in western Greene County.”
- 18. More examples “The libraryis particularly interested in the opinions of patrons who live in the western part of the county. You have been selected at random from community residents in western Greene County.” “Your opinions are very important to us. There are no right or wrong answers. Your responses will be treated confidentially. Survey results will in no way be traceable to individual respondents.”
- 19.
- 20. Strategies for goodsequence • Start with well formatted, engaging questions • Don’t lead with questions on sensitive topics • Place questions you care about most in the first half • Avoid repetitive, consecutive questions that lead to reflexive responses • Consider ending with demographics questions
- 21.
- 22.
- 23.
- 24. Closed responses Advantages • Uniformresponse set facilitates comparison, choices clarify meaning of question, reminder of alternatives, pre-establishment for sensitive questions, increased response rate and speed
- 25. Closed responses Advantages • Uniformresponse set facilitates comparison, choices clarify meaning of question, reminder of alternatives, pre-establishment for sensitive questions, increased response rate and speed Disadvantages • Random selection, “closest representation” issue, loss of distinction
- 26. Open responses Advantages • Allowfor deep explanations, only way to get responses you wouldn’t otherwise know
- 27. Open responses Advantages • Allowfor deep explanations, only way to get responses you wouldn’t otherwise know Disadvantages • Requires communication skills from respondents, more time consuming for respondent and analysis
- 28. Compromise Would you recommendthis search results screen to a friend or colleague? Yes Maybe No
- 29. Compromise Would you recommendthis search results screen to a friend or colleague? Yes Maybe No Why, or why not?
- 30. Last call forfeedback!
- 31. Last call forfeedback! Any additional comments about the Natrona County Public Library?
- 32. Last call forfeedback! Any additional comments about the Natrona County Public Library? What else would you like to tell us about your experience using this page?
- 33. Last call forfeedback! Any additional comments about the Natrona County Public Library? What else would you like to tell us about your experience using this page? Is there anything else you’d like to share?
- 34. Closing your survey •Demographics questions for data you can’t get elsewhere • Call for volunteers • Clear SUBMIT button • Thank respondents for participating! • Web-based: Redirect to a related page or the library homepage
- 35. Invitations to participate •“Would you be willing to participate in future discussions or focus groups about the library? If so, please provide your contact information below.”
- 36. Invitations to participate •“Would you be willing to participate in future discussions or focus groups about the library? If so, please provide your contact information below.” • “Would you be interested in joining a community of makers to present their work in the Maker Space?” – Yes, I'm interested in being contacted about this opportunity – No, I'm not interested in being contacted about this opportunity
- 42.
- 43.
- 44. Avoid jargon andcolloquialisms Think of your audience. Wording should be simple. Avoid technical words. X "How many times last week did you use the library's Internet- enabled public access computers?"
- 45. Avoid jargon andcolloquialisms Think of your audience. Wording should be simple. Avoid technical words. X "How many times last week did you use the library's Internet- enabled public access computers?" O "How many times last week did you use the library's computers [to access the Internet]?"
- 46. Abbreviations and acronyms Assumeyour audience does not know any of these. X “CCPL is interested in replacing its OPAC." X “How frequently do you use our ILL services?”
- 47. Abbreviations and acronyms Assumeyour audience does not know any of these. X “CCPL is interested in replacing its OPAC." O “Cleveland County Public Library is interested in replacing its online catalog.” X “How frequently do you use our ILL services?” O “How frequently do you use our Interlibrary Loan services? [This service allows us to request material from another library for you if we do not have it here].”
- 48. Avoid ambiguity Look atthe survey from every angle: are there ways that someone could interpret a question to have two meanings? X "What is your income?" X "How many people are there in your household?"
- 49. Avoid ambiguity Look atthe survey from every angle: are there ways that someone could interpret a question to have two meanings? X "What is your income?" O "What is your income before taxes? Include salary as well as other sources of income." X "How many people are there in your household?" O "Including yourself, how many people are there in your household?"
- 50. Confusing phrasing The respondentshould not have to spend time re-reading/interpreting the question. X "Does it seem likely or does it seem unlikely to you that you would use a Maker Space if the library had one?" It seems likely __ It seems unlikely __ I’m not sure __
- 51. Confusing phrasing The respondentshould not have to spend time re-reading/interpreting the question. X "Does it seem likely or does it seem unlikely to you that you would use a Maker Space if the library had one?" It seems likely __ It seems unlikely __ I’m not sure __ O "If the library had a Maker Space, would you use it?" Yes__ No__ Unsure__
- 52. Avoid double barreledquestions A question that introduces two or more issues with the expectation of a single response X "Is our staff friendly and professional?"
- 53. Avoid double barreledquestions A question that introduces two or more issues with the expectation of a single response X "Is our staff friendly and professional?" O Question1: "Is our staff friendly?" O Question 2: "Is our staff professional?"
- 54. Avoid non-specific questions Donot leave questions open to a wide range of interpretations. X “How do you feel about public transportation?”
- 55. Avoid non-specific questions Donot leave questions open to a wide range of interpretations. X “How do you feel about public transportation?” O “How do you feel about the DATA bus system in Durham County, North Carolina?”
- 56. Manipulative information Certain questionsrequire some background. Be careful that explanatory statements do not unduly influence responses. X The county government spends approximately $10 per resident on landscaping public areas. Do you believe that the county government is adequately allocating funds for our library by designating only $1.15 per resident?"
- 57. Manipulative information Certain questionsrequire some background. Be careful that explanatory statements do not unduly influence responses. X The county government spends approximately $10 per resident on landscaping public areas. Do you believe that the county government is adequately allocating funds for our library by designating only $1.15 per resident?" O "Do you believe that the county government is adequately allocating funds for our library by designating $1.15 per resident?"
- 58. Manipulative information We oftenare interested to know how knowledge of the difference in spending might affect responses, first ask straightforward, then with additional info. O "Do you believe that the county government is adequately allocating funds for our library by designating $1.15 per resident?" O "If you were to learn that the county government spends approximately $10 per resident on landscaping public areas, would that change your opinion about the adequacy of allocating $1.15 per resident to the library?"
- 59. Order of responseoptions Often there is a logical, inherent order. If order is irrelevant, list choices alphabetically so respondents don't assume answers at the top are more important to the interviewer, or have software randomize them. X Group study rooms Digital media lab Laptop lending E-books Printing/copying
- 60. Order of responseoptions Often there is a logical, inherent order. If order is irrelevant, list choices alphabetically so respondents don't assume answers at the top are more important to the interviewer, or have software randomize them. X Group study rooms O Digital media lab Digital media lab E-books Laptop lending Group study rooms E-books Laptop lending Printing/copying Printing/copying
- 61. Interval categories Do notallow to overlap. Provide an unbounded final category if appropriate. X Age 0-10 Age 10-15 Age 15-20 Age 20-50 Age 50-75
- 62. Interval categories Do notallow to overlap. Provide an unbounded final category if appropriate. X Age 0-10 O Age 0-9 Age 10-15 Age 10-19 Age 15-20 Age 20-29 Age 20-50 Age 30-39 Age 50-75 Age 40-49 Age 50+
- 63. Multiple response clarification Sometimeswe allow respondents to choose only one option and sometimes we let them choose multiple. Be very clear that you are allowing multiple! Otherwise results are unclear. X For which of the following reasons do you use the library? X__ Y__ Z__
- 64. Multiple response clarification Sometimeswe allow respondents to choose only one option and sometimes we let them choose multiple. Be very clear that you are allowing multiple! Otherwise results are unclear. X For which of the following reasons do you use the library? X__ Y__ Z__ O For which of the following reasons do you use the library? Choose all that apply. X__ Y__ Z__
- 65. Appropriate response choices –Surveys can be frustrating when the questions are fixed response without appropriate answer choices. – Provide answer choices such as “Don’t know,” “N/A,” “Unsure,” and “Other” where appropriate. X Does the laptop lending program meet your needs? Yes__ No__
- 66. Appropriate response choices –Surveys can be frustrating when the questions are fixed response without appropriate answer choices. – Provide answer choices such as “Don’t know,” “N/A,” “Unsure,” and “Other” where appropriate. X Does the laptop lending program meet your needs? Yes__ No__ O Does the laptop lending program meet your needs? Yes__ No__ N/A__ [or “I’ve never used this program__”]
- 67.
- 68.
- 69. Poll What survey toolshave you used? (check all that apply) – Google Forms – Paper-based – Poll Everywhere – Qualtrics – Survey Monkey – Other (let us know in the chat box!)
- 70. Survey tools There aremany tools! I’ll mention a few, feel free to share your experiences in the chat • Google Forms • Qualtrics • Survey Monkey • Poll Everywhere • Paper!
- 71.
- 72.
- 73.
- 74.
- 75.
- 76.
- 77.
- 78.
- 79.
- 80.
- 81. ACTING ON SURVEYDATA
- 82. Involve your colleagues Image:http://www.flickr.com/photos/dukeunivlibraries/
- 83. Share your results •Short email with key points about the results • Presentation for key staff or departments or even the entire library • Brief report, sent only to the staff most interested in the results • Links to the data for staff to explore on their own
- 84. Code free textresponses
- 85.
- 86. Outreach opportunities “Funding forstudents to purchase articles that Duke Libraries doesn't have access to would be nice. Often I find current articles that I can use for research but they are over $60.” “It would be nice to be able to return Lilly Library DVDs to Perkins & Bostock.”
- 87. Triangulating data • Methodsto consider when triangulating data: – thoughtfully planned focus groups – semi-structured interviews – observational studies – targeted, more focused surveys – usage statistics or other numerical data (e.g., gate counts, circulation stats, web metrics)
- 88. Follow-up focus groups Image:http://www.flickr.com/photos/dukeunivlibraries/
- 89. Reports, presentations, dashboards Tableaudashboards: bit.ly/SurveyDashboards-2016
- 90. Staff workshop Chart: emba.mit.edu/images/uploads/Impact_Matrix_MIT_EMBA.jpg Image:http://www.flickr.com/photos/dukeunivlibraries/
- 91. Staff workshop Chart: emba.mit.edu/images/uploads/Impact_Matrix_MIT_EMBA.jpg Image:http://www.flickr.com/photos/dukeunivlibraries/
- 92. Organizing all ofthe great ideas
- 93. Organizing all ofthe great ideas • Potential expenditures
- 94. Organizing all ofthe great ideas • Potential expenditures • Service improvements
- 95. Organizing all ofthe great ideas • Potential expenditures • Service improvements • Marketing opportunities
- 96. Organizing all ofthe great ideas • Potential expenditures • Service improvements • Marketing opportunities • Assessment opportunities
- 97.
- 98. Sample projects • Librarye-newsletter • Dry erase marker kits • Low-cost earbuds • Posting hours to the coffee shop • Improved signage • FitDesks
- 99. Process from startto finish 1. Conduct survey 2. Analyze data 3. Share findings 4. Develop recommendations 5. Prioritize according to ease of implementation and potential impact 6. Assign ownership 7. Follow through
- 101.
- 102.
- 103. Involve your colleagues… Image:www.flickr.com/photos/dukeunivlibraries/
- 104.
- 105.
- 106.
- 107. Test early andoften
- 108.
- 109. Act on whatyou learn Image: http://www.flickr.com/photos/dukeunivlibraries/
- 110. Thank you! ALA eLearningwill send participants a link to the recorded workshop and slide deck. With questions about our content, contact – Emily Daly: emily.daly@duke.edu – Joyce Chapman: joyce.chapman@duke.edu With questions for ALA eLearning, contact – editionscoursehelp@ala.org
Editor's Notes
- #23 Photo by Gary Simmons
- #24 Photo by Mark Seton
- #88 For instance, you might lead a group discussion or individual semi-structured interview. You might observe users in your library to get a better sense of some of the behavior they may have hinted at in the survey. There may be existing data like circulation stats, usage stats, or website hits and clicks that will help further inform your survey results. You might even decide to conduct another survey so you can get more information on a few targeted aspects of your initial survey. Joyce and I rarely, if ever, base decisions or track trends entirely on a survey, or any single assessment measure, for that matter – again, we consider surveys to be one tool in a larger assessment effort.
- #89 As was the case last time around, we know that a survey only scratches the surface in understanding student feedback. We use the survey as a way to figure out more about what we want to know from our users. After preliminary analysis, we contacted the numerous students who offered to participate in follow-up interviews or focus groups and invited them to spend time with us delving more deeply into the issues and topics that survey respondents appeared to care most about. We conducted 6 highly participative semi-structured group interviews. From there, Images of Duke University available at https://www.flickr.com/photos/dukeunivlibraries/.
- #90 Mentioned Comments dashboard previously Also developed dashboards to view and analyze the two main sections of our survey data (can go into more depth about this later if you like) – point here, though, is that we made the data available and highly accessible to library staff so they could dig in on their own. We also developed a hands-on workshop so staff could set aside time to consider survey data most relevant to their work and begin to respond to findings. SLIDE CHANGE http://library.duke.edu/about/depts/assessment-user-experience/student-survey
- #91 We were pleased to have 47 staff from across the Libraries and representing tech services, public services, IT, building services, and administration, register to attend a workshop to explore Tableau dashboards and talk about ways we might follow up on survey findings and then consider the potential impact and ease of actually implementing the recommendations they brainstormed using an impact matrix like the one here Images from https://emba.mit.edu/images/uploads/Impact_Matrix_MIT_EMBA.jpg and https://www.flickr.com/photos/dukeunivlibraries/.
- #92 We were pleased to have 47 staff from across the Libraries and representing tech services, public services, IT, building services, and administration, register to attend a workshop to explore Tableau dashboards and talk about ways we might follow up on survey findings and then consider the potential impact and ease of actually implementing the recommendations they brainstormed using an impact matrix like the one here Images from https://emba.mit.edu/images/uploads/Impact_Matrix_MIT_EMBA.jpg and https://www.flickr.com/photos/dukeunivlibraries/.
- #105 Images of Duke University available at https://www.flickr.com/photos/dukeunivlibraries/.
- #109 Image: https://commons.wikimedia.org/wiki/File:Japanese_Map_symbol_(Triangulation_point).svg Surveys are an extremely useful starting point – again, they’re a great way to reach a lot of people quickly and fairly easily, and results are typically very easy to analyze. They are not, however, the best way to collect information about users’ actual behavior, since surveys rely on users to self report their habits and experiences. Additionally, it can be difficult to be sure that survey respondents understand exactly what you mean by a particular question or answer choice, and there is, of course, no opportunity in a survey to follow up on a particular question or response with individual respondents. So, you might start by distributing a survey to get a sense of broader trends. If you decide after your survey that you need more information about a particular topic, or if you are results are inconclusive or incomplete (which is not uncommon for surveys!), it can be very useful to follow up with additional assessment and triangulate results from multiple studies. For instance, you might lead a group discussion or individual semi-structured interview. You might observe users in your library to get a better sense of some of the behavior they may have hinted at in the survey. There may be existing data like circulation stats, usage stats, or website hits and clicks that will help further inform your survey results. You might even decide to conduct another survey so you can get more information on a few targeted aspects of your initial survey. Joyce and I rarely, if ever, base decisions or track trends entirely on a survey, or any single assessment measure, for that matter – again, we consider surveys to be one tool in a larger assessment effort.