Marie L. RadfordChair, Department of Library &Information ScienceRutgers Universitymradford@rutgers.eduLynn Silipigni Conn...
Seeking Synchronicity& Cyber Synergy The world’s libraries. Connected.
Transcript Analysis                                          Seeking Synchronicity 2004-2006                              ...
Research Questions – Cyber Synergy    • How have the mix & frequency of types of VRS      questions changed?    • How freq...
Seeking Synchronicity & Cyber Synergy            Transcript Analysis Results Types of Questions   The world’s libraries. C...
Types of VR Questions • Subject Search • Ready Reference • Procedural • No Question • Holdings • Research • Inappropriate ...
Subject Search or Specific Search “Almost always takes the form of giving the user a document, for         example, a list...
Ready Reference  “These are the typical ready-reference or data queries that require only a single, usually uncomplicated,...
Procedural    “Questions pertaining to the policies or procedures within           the library system” (Arnold & Kaske, 20...
Holdings     “Questions about specific holdings of a library in print            or digital form” (Arnold & Kaske, 2005). ...
Research     “Research questions… involve trial-and-error searching     or browsing… [and] are usually identified as comin...
Inappropriate  “Questions which are not appropriate for a reference service        including personal questions” (Radford,...
Directional    “The general information or directional question is of the      information booth variety…” (Arnold & Kaske...
Reader‟s Advisory     Reader‟s Advisory questions are “focused on helping   readers find materials they want to read, list...
Query Type: 2004-2006 35%        32%           n=293 30%                                27%                               ...
Query Type: 2010 35%            31.1%                   31.1%            n=179                   n=181 30% 25%            ...
Query Type: 2004-2006 vs. 2010 35%       32%                                            31%                               ...
Ready ReferenceAccuracy The world’s libraries. Connected.
Accuracy in VR Ready Reference• How has the rate of accuracy of answers to VRS ready  reference questions changed?• Is the...
Accuracy Transcript Analysis                                          • Subset of ready reference                         ...
Checking for Accuracy       You were   interested in the                         • Teams of coders checked  kids‟ (up to a...
Categories                                1. Correct with Citation                             2. Correct without Citation...
Ready Reference Questions (2004-2006 vs. 2010)Received by Type of Chat Service  45%                   40%                 ...
Ready ReferenceAccuracyResults The world’s libraries. Connected.
Improving Accuracy: 2004-2006 vs. 2010 100%                            90%  90%                      n=151                ...
Ready Reference Accuracy Correct Answers and Citationsfor 2004-2006 vs. 2010   100%                  89%                  ...
Ready Reference is Alive!• % of Ready Reference Queries   • Relatively stable   • Slight increase over time (4%)• Accuracy...
Explanations for Increase                                          • VRS service providers more                           ...
Where to Go fromHere The world’s libraries. Connected.
Recommendations to Improve Accuracy • Clarify question • Provide citations • Check for broken links • Make sure all parts ...
Future Research Directions                                          Investigate:                                          ...
ReferencesArnold, J., & Kaske, N. (2002). Chat with a Librarian. Computers in Libraries Annual Conference and Exhibition  ...
References Pew Internet & American Life Project. (2012b). Teens, smartphones & texting. Retrieved on April   13, 2012 from...
Funding & Acknowledgements     Cyber Synergy: Seeking Sustainability through        Collaboration between Virtual Referenc...
Questions &               Discussion                                             Marie Radford                            ...
Upcoming SlideShare
Loading in …5
×

Not Dead Yet: A Longitudinal Study of Query Type and Ready Reference Accurancy in Live Chat and IM Reference

1,483 views

Published on

Presented at the 2013 ALISE Annual Conference, 22-25 January 2013, Seattle WA (USA).

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,483
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Seeking SynchronicityPartners: OCLC & Rutgers$1,090,072 project, 2005-2008, funded by:Institute of Museum & Library Services (IMLS) $684,996 grant$405,076 in kind contributionsCo-PIs Marie Radford (RU), Lynn Silipigni Connaway (OCLC)http://www.oclc.org/research/activities/synchronicity/default.htmCyber Synergy: Seeking Sustainability through Collaboration between Virtual Reference and Social Q&A Sites$250,000 for 2011-2013Funded by IMLS, OCLC, & Rutgers UniversityCo-PIs Marie Radford (RU), Lynn SilipigniConnaway (OCLC), & Chirag Shah (RU)http://www.oclc.org/research/activities/synergy.html
  • Image: http://www.flickr.com/photos/badjonni/7131927517/Connaway, L. S., Radford, M. L., & OCLC Research. (2011). Seeking synchronicity: Revelations and recommendations for virtual reference. Dublin, Ohio: OCLC Research. http://www.oclc.org/research/activities/synchronicity/default.htm
  • Cyber Synergy: Seeking Sustainability through Collaboration between Virtual Reference and Social Q&A Sites$250,000 for 2011-2013Funded by IMLS, OCLC, & Rutgers UniversityCo-PIs Marie Radford (RU), Lynn SilipigniConnaway (OCLC), & Chirag Shah (RU)http://www.oclc.org/research/activities/synergy.html
  • All transcripts were stripped of identifying information at OCLC. Live Chat Transcripts 2004-2006 (QP1)8/2004 to 10/2006Live Chat & Qwidget Transcripts 2010 (QP2 & QW)6/2010 - 2/2010
  • Image: Microsoft Clip Art
  • Image: http://www.flickr.com/photos/marcobellucci/3534516458/Our study draws on categories for questions defined by Katz (1997) and enhanced by Arnold & Kaske (2002, 2005).Directional, Ready Reference, Subject Search, Research, & Holdings were taken from coding schemes from Katz (1997), Kaske & Arnold (2002), and Arnold & Kaske (2005). Arnold, J., & Kaske, N. (2002). Chat with a Librarian. Computers in Libraries Annual Conference and Exhibition Usa-, 1-6.Arnold , J., & Kaske, N. K. (2005). Evaluating the quality of a chat service. portal: Libraries and the Academy, 5(2), 177-193.Katz, W. A. (1997). Introduction to reference work. (7th ed.). New York: McGraw-Hill. Reader’s Advisory was taken from Ross, Nilsen, & Radford (2009). Ross, C. S., Nilsen, K., & Radford, M. L. (2009). Conducting the reference interview (2nd ed.). New York: Neal-Schuman.Inappropriate, No Question were added by Radford & Connaway (2005-2008).Radford, M. L., & Connaway, L. S. 2005–2008. Seeking Synchronicity: Evaluating virtual reference services from user, non-user, and librarian perspectives. Funded by the Institute of Museum and Library Services (IMLS), Rutgers University, and OCLC http://www.oclc.org/research/activities/synchronicity/default.htm
  • A search on a topic. Arnold and Kaske called this “Specific Search,” we are calling it “Subject Search”.Note that all quotes are verbatim and have not been corrected if there are errors.
  • QP2-153U: WAT DOES it mean when a guy said he like me more as a friend doesnt it mean like a gf?L: what do you think it means?U: well like a relationship but wuts u think?L:It is a nice way of saying he wants to be friends and nothing more.U: are you for reallllll?L: Yes I am.U: oh wtffffgrrrrrr now am fkin MAD CRYING*U:cause he said he likes me and how am cute etc and he even kisses me etc so idont think thatsbein a friend lolL: Sorry.L: If you have to ask, you already know, right?U: cause i thought when a guy says more then a friend i thought it meant as a bf / gfL: You wrote "more as a friend" not "more than a friend".L: Very different meaning.U: he said more as a friendL: Be honest with yourself.U: he told his friend about me yoo how hot i am etc and he likes me alot and loves to hang out with meU: i am being honest he said he wants us to be more then friends like a relationship
  • Includes virtual directions like 2nd example asking for URL
  • 575 is actual number of questions from 560 transcripts (includes both QP and QW)Subject Search (n=97, 17%)Ready Reference (n=179, 31%)Procedural (n=181, 31%)No Question (n=25, 4%)Holdings (n=49, 9%)Research (n=19, 3%)Inappropriate (n=4, <1%)Directional (n=15, 3%)Reader’s Advisory (n=6, 1%)
  • 2004-2006QP1:n = 915 questions vs. 575 questionsSubject Search (n=293, 32%) vs. (n=97, 17%)Ready Reference (n=243, 27%) vs. (n=179, 31%)Procedural (n=162, 18%) vs. (n=181, 31%)No Question (n=104, 11%) vs. (n=25, 4%)Holdings (n=77, 8%) vs. (n=49, 9%)Research (n=23, 3%) vs. (n=19, 3%)Inappropriate (n=10, 1%) vs. (n=4, 1%)Directional (n=2, <1%) vs. (n=15, 3%)Reader’s Advisory (n=1, <1%) vs. (n=6, 1%)2010QP2:N=366Directional (n=7, 2%) Ready Reference (n=127, 35%)Subject Search (n=74, 20%)Research (n=16, 3%)Procedural (n=99, 27%)Holdings (n=25, 7%)No Question (n=11, 3%)Inappropriate (n=4, 1%)Reader’s Advisory (n=3, 1%)QW2:N=209Directional (n=8, 4%) Ready Reference (n=52, 25%)Subject Search (n=23, 11%)Research (n=3, 1%)Procedural (n=82, 39%)Holdings (n=24, 11%)No Question (n=14, 7%)Inappropriate (n=0, 0%)Reader’s Advisory (n=3, 1%)
  • Total n= 575 questionsQP2:N=366 questionsDirectional (n=7, 1%) Ready Reference (n=127, 22%)Subject Search (n=74, 13%)Research (n=16, 3%)Procedural (n=99, 17%)Holdings (n=25, 4%)No Question (n=11, 2%)Inappropriate (n=4, 1%)Reader’s Advisory (n=3, 1%)QW2N=209Directional (n=8, %) Ready Reference (n=52, 9%)Subject Search (n=23, 4%)Research (n=3, 1%)Procedural (n=82, 14%)Holdings (n=24, 4%)No Question (n=14, 2%)Inappropriate (n=0, 0%)Reader’s Advisory (n=3, 1%)TotalN=575Directional (n=15, 3%)Ready Reference (n=179, 31%)Subject Search (n=97, 17%)Research (n=19, 3%)Procedural (n=181, 31%)Holdings (n=49, 9%)No Question (n=25, 4%)Inappropriate (n=4, 1%)Reader’s Advisory (n=6, 1%)
  • Image: http://www.webdesign.org/photoshop/drawing-techniques/magnify-glass.13819.html
  • Image: Microsoft Clip ArtHernon, P., & McClure, C. R. (1987). Unobtrusive testing and library reference services. Norwood, New Jersey: Ablex Publishing.
  • All transcripts were stripped of identifying information at OCLC. Following Arnold and Kaske method, ready reference queries were dropped out of the accuracy analysis if they were referred for follow-up iva email, phone, fax, or to another library/librarian or if there were technical difficulties, such as abrupt disconnect prior to an answer being offered.
  • Categories 1-5 are consistent with Arnold & Kaske (2005), Category 6 was created by Seeking Synchronicity coders (Radford & Connaway, 2005-2008; Radford et al., 2011) for answers that were incorrect because they provided general information and/or a citation(s) that did not answer the specific question. Category 7 was created to identify complete ready reference interactions that did not fit into other categories (Radford & Connaway, 2005-2008; Radford et al., 2011). Arnold , J., & Kaske, N. K. (2005). Evaluating the quality of a chat service. portal: Libraries and the Academy, 5(2), 177-193.Radford, M. L., & Connaway, L. S. 2005–2008. Seeking Synchronicity: Evaluating virtual reference services from user, non-user, and librarian perspectives. Funded by the Institute of Museum and Library Services (IMLS), Rutgers University, and OCLC http://www.oclc.org/research/activities/synchronicity/default.htmRadford, M. L., Connaway, L. S., Confer, P., Sabolsci-Boros, S., & Kwon, H. (2011). “Are we getting warmer?” Query clarification in live chat virtual reference. Reference & User Services Quarterly,50(3), 259-279.
  • Image: Microsoft Clip Art
  • This slide shows percentages of correct answers with citations, no citations, and instances where no citation was necessary and demonstrates increasing accuracy from 2004-2006 to the 2010 sample. Correct with Citation decreased from 89 percent (125 of 141 correct answers) in 2004-2006 to 75 percent (114 of 151 correct answers) in 2010. Correct without Citation improved from 6 percent in 2004-2006 (9 of 141 correct answers) to 14 percent in 2010 (21 of 151 correct answers), and Correct No Citation Needed improved from 5 percent (7 of 141 correct answers) in 2004-2006 to 11 percent (16 of 151 correct answers) in 2010.Ready Reference queries were dropped out of the accuracy analysis if they were referred for follow-up to another library/librarian or had technical difficulties; therefore, 63 of 243 Ready Reference questions in 2004-2006, and 11 of 179 Ready Reference questions in 2010 were eliminated. Accuracy of the subset of ready reference questions for 2004-2006 (180) and 2010 (168) was verified. There were 141 correct answers (78 percent) in 2004-2006. In 2010, there were 151 correct answers (90 percent) as Figure 5 illustrates. In 2004-2006 there were 36 incorrect answers (20 percent) and in 2010 there were 7 incorrect answers (4 percent). Questions coded as “Other,” experienced a small increase from three answers (2 percent) in 2004-2006, compared to 10 answers (6 percent) in 2010.Ready reference accuracy has been found to increase over time in this longitudinal study of live chat reference from 78 percent in 2004-2006 to 90 percent in 2010. As shown in Figure 5, the percentage of correct responses to ready reference questions in both 2004-2006 (141, 78 percent) and 2010 datasets (151, 90 percent) are considerably higher than the benchmark 55 percent for FtF reference transactions reported by Hernon and McClure (1987), noting that their method and analytic categories were quite different from this transcript analysis. *Other is a new categorycoveringavariety of circumstances, including librarian unable to find answer and users finding information themselves. Does not include responses that were terminated without an answer due to technical problems or inquiries that were referred elsewhere.Example: U: “What is the number to financial aid” L: “Okay. Let me see if I can find the number for you. Please hold on.”U: “thanks sorry found it” (QP 24/7-100)-------- U: who is selena? (Then signs off.) (QP2- 199) --------This chart compares the Accuracy for 2004-2006 to 2010. 2004 – 2006 (n=180 ready reference questions only)Correct: n=141, 78%Incorrect: n=36, 20%Other: n=3, 2%2010 (n=168 ready reference questions for both QP2 and QW) Correct: n=151, 90%Incorrect: n=7, 4%Other: n=10, 6%Also: note that referred questions are not counted in either 04-06 or 10 dataset.
  • Connaway, L. S. & Radford, M. L. (2011). Seeking Synchronicity: Revelations and recommendations for virtual reference. Dublin, OH: OCLC Research. Retrieved on February 26, 2012 from http://www.oclc.org/reports/synchronicity/full.pdf Radford, M. L., Connaway, L. S., Confer, P., Sabolsci-Boros, S., & Kwon, H. (2011). “Are we getting warmer?” Query clarification in live chat virtual reference. Reference & User Services Quarterly,50(3), 259-279.
  • Image: Microsoft Clip Art
  • Image from: http://www.flickr.com/photos/34316967@N04/3838807132/Radford, M. L., Connaway, L. S., Confer, P., Sabolsci-Boros, S., & Kwon, H. (2011). “Are we getting warmer?” Query clarification in live chat virtual reference. Reference & User Services Quarterly,50(3), 259-279.
  • Image from: http://www.flickr.com/photos/41118175@N05/6679478873/Pearce, A., Collard, S., & Whatley, K. (2010). SMS reference: Myths, markers, and modalities. Reference Services Review, 38(2), 250-263.Pew Internet & American Life Project. (2012a). Nearly half of American adults are smartphone owners. Retrieved on April 13, 2012 from http://www.pewinternet.org/Reports/2012/Smartphone-Update-2012/Findings.aspx Pew Internet & American Life Project. (2012b). Teens, smartphones & texting. Retrieved on April 13, 2012 from http://www.pewinternet.org/~/media//Files/Reports/2012/PIP_Teens_Smartphones_and_Texting.pdfRadford, M. L., & Vine, S. (2011). An exploration of the hybrid service model: Keeping what works. In: Diane Zabel (Ed.), Reference Reborn: Breathing New Life into Public Services Librarianship, pp. 79-89. CO: Libraries Unlimited.READ Scale:Gerlich, B., Berard, G. (2010). Testing the Viability of the READ Scale (Reference Effort Assessment Data): Qualitative Statistics for Academic Reference Services. Coll Res Libr, 71, (2) 116-37.Hudock, S., Sullivan, D. (2011). Hit the Reference Transaction Mark: Incorporating READ into DART. ColoLibr ,36 (1) 1-6.University of Illinois at Urbana-Champaign University Library READ scale. Retrieved from: http://www.library.illinois.edu/assessment/readscale.html
  • Not Dead Yet: A Longitudinal Study of Query Type and Ready Reference Accurancy in Live Chat and IM Reference

    1. 1. Marie L. RadfordChair, Department of Library &Information ScienceRutgers Universitymradford@rutgers.eduLynn Silipigni ConnawaySenior Research Scientist ALISE/Bohdan S. Wynar Research Paper AwardOCLC ALISE, January 22-25, 2013connawal@oclc.org http://www.oclc.org/research/activities/synergy.html The world’s libraries. Connected.
    2. 2. Seeking Synchronicity& Cyber Synergy The world’s libraries. Connected.
    3. 3. Transcript Analysis Seeking Synchronicity 2004-2006 Live Chat Transcripts 2004-2006 (QP1) (QP1) • Total QuestionPoint & 24/7 = 651,687 • 850 randomly selected • 550 QP & 300 24/7 2010 Cyber Synergy (QP2 & QW) Live Chat & Qwidget Transcripts 2010 (QP2 & QW) • Total QP2 & QW = 296,797 • QP2 221,158 & QW 75,639 • 560 randomly selected • 350 QP2 & 210 QW The world’s libraries. Connected.
    4. 4. Research Questions – Cyber Synergy • How have the mix & frequency of types of VRS questions changed? • How frequently are ready reference questions asked in VRS? • How has the rate of accuracy of answers to VRS ready reference questions changed? • What differences are there in mix, frequency, & accuracy for live chat compared to IM/QW VRS modes? The world’s libraries. Connected.
    5. 5. Seeking Synchronicity & Cyber Synergy Transcript Analysis Results Types of Questions The world’s libraries. Connected.
    6. 6. Types of VR Questions • Subject Search • Ready Reference • Procedural • No Question • Holdings • Research • Inappropriate • Directional • Reader‟s Advisory The world’s libraries. Connected.
    7. 7. Subject Search or Specific Search “Almost always takes the form of giving the user a document, for example, a list of citations, a book, or a report” (Arnold & Kaske, 2005). How was the labeling theory important in deinstitutionalization? (QW–023) can you help me Where can i find find poems by information about Maya Angelou and ADHD? criticism on them? (QP2–005) (QP1–230) The world’s libraries. Connected.
    8. 8. Ready Reference “These are the typical ready-reference or data queries that require only a single, usually uncomplicated, straightforward answer...Who? What? When? Why? Where?” (Arnold & Kaske, 2005). who won the world cup game between south africa and france? (QW–024) How do i cite a political Who was Bentonville, talk show in MLA NC named after? format? (QP1–147) (QP2–013) The world’s libraries. Connected.
    9. 9. Procedural “Questions pertaining to the policies or procedures within the library system” (Arnold & Kaske, 2005). Can I get summer long term renewals online? How? (QP1–004) what is the max for need access logon info checkout on blu ray to lexis from scool dvids? (QW–033) library website (QP2– 027) The world’s libraries. Connected.
    10. 10. Holdings “Questions about specific holdings of a library in print or digital form” (Arnold & Kaske, 2005). I was wondering if you have textbooks to rent for an hour or 2? (QP2–172) Do you have any hi there; i was books on Paris Hilton? wondering if you guys (doing a project on have Moby Dick in her) stock? (QP 1–195) (QW–014) The world’s libraries. Connected.
    11. 11. Research “Research questions… involve trial-and-error searching or browsing… [and] are usually identified as coming from an adult specialist who is seeking detailed information to assist in specific work” (Arnold & Kaske, 2005). Looking for information on national traffic survey and incident reports. (QP2–169) Hi, I am trying to find out information on how the Hi I‟m looking for Learn Direct initiative was studies similar to one started, the criteria that was I‟ve already found. How set and if local government do I go about this? had to be involved in the (QW–013) first stage. (QP 1–155) The world’s libraries. Connected.
    12. 12. Inappropriate “Questions which are not appropriate for a reference service including personal questions” (Radford, 2005). How do I have sex? (QP1–159) Are WAT DOES it mean You when a guy said he Typing like me more as a War friend doesnt it mean and Peace like a gf? (QP2–153) (QW–020) The world’s libraries. Connected.
    13. 13. Directional “The general information or directional question is of the information booth variety…” (Arnold & Kaske, 2005). what is the URL for the summer reading program for teens? (QP2–032) I heard that you have adapted book kits for Where is the 67th street ESL speakers. Where library? Is it on the west can I find these or east side? online? (QP1–387) (QW–008) The world’s libraries. Connected.
    14. 14. Reader‟s Advisory Reader‟s Advisory questions are “focused on helping readers find materials they want to read, listen to, or view for pleasure” (Ross, Nilsen, & Radford, 2009). Hello I am looking for recommendations for the author jonathan kellerman. (QP1–218) Hi, my 7 year old son is looking for a copy of the Hobbit suitable I was wondering if u for children, but we can‟t seem to would happen to know find anything in the library any good fantasy catalogue. Can you please offer books… any advice? Thanks. (QP2 – 050) (QW– 120) (from Australia) The world’s libraries. Connected.
    15. 15. Query Type: 2004-2006 35% 32% n=293 30% 27% n=243 25% 04 - 06 (n = 915) 18% 20% n=162 15% 11% n=104 8% 10% n=77 5% 0% Subject Search Ready Procedural No Question Holdings Reference The world’s libraries. Connected.
    16. 16. Query Type: 2010 35% 31.1% 31.1% n=179 n=181 30% 25% 10 (n=575) 20% 16.9% n=97 15% 8.5% 10% n=49 4.3% 5% n=25 0% Ready Procedural Subject Search Holdings No Question Reference The world’s libraries. Connected.
    17. 17. Query Type: 2004-2006 vs. 2010 35% 32% 31% 31% n=293 n=179 n=181 30% 27% n=243 25% 04 - 06 (n = 915) 18% 20% 17% n=162 n=97 10 (n = 575) 15% 11% n=104 8% 9% 10% n=77 n=49 4% 5% n=25 0% Subject Search Ready Procedural No Question Holdings Reference The world’s libraries. Connected.
    18. 18. Ready ReferenceAccuracy The world’s libraries. Connected.
    19. 19. Accuracy in VR Ready Reference• How has the rate of accuracy of answers to VRS ready reference questions changed?• Is the 55% rule in effect? (Hernon & McClure, 1987) The world’s libraries. Connected.
    20. 20. Accuracy Transcript Analysis • Subset of ready reference questions • 8/2004 - 10/2006 • n=162 ready reference queries • 6/2010 - 12/2010 • n=168 ready reference queries The world’s libraries. Connected.
    21. 21. Checking for Accuracy You were interested in the • Teams of coders checked kids‟ (up to age 12) responses summer reading, right? • Authoritative websites • Subscription-based databases Yes Yes • Any citations/links provided by the librarian/staff memberHere is a link to info on it.http://host8.evanced.info/yorklibraries/sr/homepage.aspHopefully, that page willanswer your questions. (QP2-024) The world’s libraries. Connected.
    22. 22. Categories 1. Correct with Citation 2. Correct without Citation 3. Correct without Citation, No Citation Needed 4. Incorrect with Citation 5. Incorrect without Citation 6. Incorrect (without specific information requested) 7. Other The world’s libraries. Connected.
    23. 23. Ready Reference Questions (2004-2006 vs. 2010)Received by Type of Chat Service 45% 40% 39% n=68 40% n=65 36% n=59 35% 04-06 (n=162) 28% 30% n=45 10 (n=168) 25% 21% 20% n=34 n=33 20% 15% 12% n=20 10% 5% <1% 2% <1% n=4 n=1 n=1 0% Consortium Academic & Law Public National Other The world’s libraries. Connected.
    24. 24. Ready ReferenceAccuracyResults The world’s libraries. Connected.
    25. 25. Improving Accuracy: 2004-2006 vs. 2010 100% 90% 90% n=151 78% n=141 80% 04 - 06 (n=180) 70% 10 (n = 168) 60% 50% 40% 30% 20% n=36 20% 6% n=10 4% 10% n=7 2% n=3 0% Correct Incorrect Other The world’s libraries. Connected.
    26. 26. Ready Reference Accuracy Correct Answers and Citationsfor 2004-2006 vs. 2010 100% 89% n=125 90% 75% 80% n=114 04 - 06 (n=141) 70% 10 (n = 151) 60% 50% 40% 30% 14% 20% 11% n=21 6% n=16 5% 10% n=9 n=7 0% Correct w/ Citation Correct w/o Citation Correct No Citation Needed The world’s libraries. Connected.
    27. 27. Ready Reference is Alive!• % of Ready Reference Queries • Relatively stable • Slight increase over time (4%)• Accuracy much higher than 55% benchmark • 78% („04-‟06) to („10) 90% • Both ready reference accuracy & percentages for correct w/ citation improved The world’s libraries. Connected.
    28. 28. Explanations for Increase • VRS service providers more facile in chat & IM environment • Providers feel less rushed • Have followed authors‟ previous recommendations? The world’s libraries. Connected.
    29. 29. Where to Go fromHere The world’s libraries. Connected.
    30. 30. Recommendations to Improve Accuracy • Clarify question • Provide citations • Check for broken links • Make sure all parts of a query are answered • Make sure specific information requested in presented before pushing a general website The world’s libraries. Connected.
    31. 31. Future Research Directions Investigate: • Accuracy in other question types & SQA • Data-derived taxonomy of accuracy • SQA accuracy ask a • Level of difficulty (using READ librarian model) • Staffing models • Both FtF & VR desks • Managing multiple chat windows • Lean staffing in tight budgets • More collaboration/crowd-sourcing • Mobile access The world’s libraries. Connected.
    32. 32. ReferencesArnold, J., & Kaske, N. (2002). Chat with a Librarian. Computers in Libraries Annual Conference and Exhibition Usa-, 1-6.Arnold , J., & Kaske, N. K. (2005). Evaluating the quality of a chat service. portal: Libraries and the Academy, 5(2), 177-193.Connaway, L. S., Dickey, T. J., & Radford, M. L. (2011) “If it is too inconvenient Im not going after it:” Convenience as a critical factor in information-seeking behaviors. Library & Information Science Research, 33(3), 179-190.Connaway, L. S., Radford, M. L., & OCLC Research. (2011). Seeking synchronicity: Revelations and recommendations for virtual reference. Dublin, Ohio: OCLC Research. http://www.oclc.org/research/activities/synchronicity/default.htmHernon, P., & McClure, C. R. (1987). Unobtrusive testing and library reference services. Norwood, New Jersey: Ablex Publishing.Janes, J. (2003). What is reference for? Reference Services Review, 31(1), 22-25.Nummien, P., & Vakkari, P. (2009). Question types in public libraries digital reference service in Finland: Comparing 1999 and 2006. Journal of the American Society for Information Science and Technology, 60(6), 1249-1257.Pearce, A., Collard, S., & Whatley, K. (2010). SMS reference: Myths, markers, and modalities. Reference ServicesReview, 38(2), 250-263.Pew Internet & American Life Project. (2012a). Nearly half of American adults are smartphone owners. Retrieved on April 13, 2012 from http://www.pewinternet.org/Reports/2012/Smartphone-Update-2012/Findings.aspx The world’s libraries. Connected.
    33. 33. References Pew Internet & American Life Project. (2012b). Teens, smartphones & texting. Retrieved on April 13, 2012 from http://www.pewinternet.org/~/media//Files/Reports/2012/PIP_Teens_Smartphones_and_Texting.pdf Radford, M. L., & Connaway, L. S. 2005–2008. Seeking Synchronicity: Evaluating virtual reference services from user, non-user, and librarian perspectives. Funded by the Institute of Museum and Library Services (IMLS), Rutgers University, and OCLC http://www.oclc.org/research/activities/synchronicity/default.htm Radford, M. L., & Connaway, L. S. (2013). Not dead yet! A longitudinal study of query type and ready reference accuracy in live chat and IM reference. Library & Information Science Research, 35(1), 2-13. [Available: http://www.oclc.org/resources/research/publications/library/2012/radford-connaway-lisr.pdf] Radford, M. L., Connaway, L. S., Confer, P., Sabolsci-Boros, S., & Kwon, H. (2011). “Are we getting warmer?” Query clarification in live chat virtual reference. Reference & User Services Quarterly, 50(3), 259-279. Radford, M.L., Connaway,L.S. & Shah, C. (2011-2013). Cyber synergy: Seeking sustainability through collaboration between virtual reference and social Q&A sites. Funded by the Institute of Museum and Library Services (IMLS). http://www.oclc.org/research/activities/synergy/default.htm Radford, M. L., & Vine, S. (2011). An exploration of the hybrid service model: Keeping what works. In: Diane Zabel (Ed.), Reference Reborn: Breathing New Life into Public Services Librarianship, pp. 79-89. CO: Libraries Unlimited. Ross, C. S., Nilsen, K., & Radford, M. L. (2009). Conducting the reference interview (2nd ed.). New York: Neal- Schuman. Stacy-Bates, K. (2003). E-mail reference responses from academic ARL libraries. Reference & User Services Quarterly, 43(1), 59-70. The world’s libraries. Connected.
    34. 34. Funding & Acknowledgements Cyber Synergy: Seeking Sustainability through Collaboration between Virtual Reference and Social Q&A Sites • $250,000 for 2011-2013 • Funded by IMLS, OCLC, & Rutgers University • Co-PIs Marie Radford (RU), Lynn Silipigni Connaway (OCLC), & Chirag Shah (RU) http://www.oclc.org/research/activities/synergy.html • We thank Alyssa Darden, Lisa DeLuca, Kathy Juliano, Stephanie Mikitish, Janet Torsney, & Jacqueline Woolcott for assistance in coding, analysis, & data presentation. The world’s libraries. Connected.
    35. 35. Questions & Discussion Marie Radford mradford@rutgers.edu Lynn Silipigni Connaway connawal@oclc.orgThe world’s libraries. Connected.

    ×