Qualitative Inquiry in Social and Cultural Contexts

  • 48 views
Uploaded on

Panel presented at CoLIS 2013: 8: International Conference on Conceptions of Library and Information Science, August 22, 2013, Copenhagen, Denmark.

Panel presented at CoLIS 2013: 8: International Conference on Conceptions of Library and Information Science, August 22, 2013, Copenhagen, Denmark.

More in: Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
48
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
1
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Image from http://www.flickr.com/photos/matt_hintsa/2963782492/Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358.Used in business and marketing
  • Image from http://www.flickr.com/photos/manintheorangeshirt/3947383369/Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358.The roots of the present procedures can be traced back directly to the studies of Sir Francis Galton nearly 70 years ago, and to later developments such as time sampling studies of recreational activities, controlled observation tests, and anecdotal records (Flanagan, 1954, p. 2)
  • Image from http://www.flickr.com/photos/soldiersmediacenter/2561212534/Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358.“The critical incident technique…can best be regarded as an outgrowth of studies in the Aviation Psychology Program of the United States Army Air Forces in World War II. The Aviation Psychology Program was established in the summer of 1941 to develop procedures for the selection and classification of aircrews. One of the first studies (40) carried out in this program was the analysis of the specific reasons for failure in learning to fly that were reported for 1,000 pilot candidates eliminated from flight training schools in the summer and early fall of 1941. The basic source used in this analysis was the proceedings of the elimination boards. In these proceedings the pilot instructors and check pilots reported their reasons for eliminating the particular pilot. It was found that many of the reasons given were clichés and stereotypes such as "lack of inherent flying ability" and ''inadequate sense of sustentation," or generalizations such as ''unsuitable temperament," "poor judgment," or "insufficient progress." However, along with these a number of specific observations of particular behaviors were reported. This study provided the basis for the research program on selecting pilots. Although it was found very useful, it also indicated very clearly the need for better procedures for obtaining a representative sample of factual incidents regarding pilot performance. A second study (13), which emphasized the importance of factual reports on performance made by competent observers, was carried out in the winter of 1943-1944 in the 8th, 9th, 12th, and 15th Air Forces. This study collected the reasons for the failures of bombing missions as reported in the Group Mission Reports. In the summer of 1944 a series of studies (74) was planned on the problem of combat leadership in the United States Army Air Forces. These represent the first large-scale, systematic effort to gather specific incidents of effective or ineffective behavior with respect to a designated activity. The instructions asked the combat veterans to report incidents observed by them that involved behavior which was especially helpful or inadequate in accomplishing the assigned mission. The statement finished with the request, "Describe the officer's action. What did he do?" Several thousand incidents were collected in this way and analyzed to provide a relatively objective and factual definition of effective combat leadership. The resulting set of descriptive categories was called the "critical requirements" of combat leadership” (Flanagan, 1954, pp. 2-3).
  • Image from Microsoft Clip ArtFisher, S., & Oulton, T. (1999). The critical incident technique in library and information management research. Education for Information, 17(2), 113–126.Example from Fisher: Hamer, J. S. (2003). Coming-out: Gay males’ information seeking. School Libraries Worldwide, 9(2), 73–79.Radford, M. L. (2006). The critical incident technique and the qualitative evaluation of the Connecting Libraries and Schools Project. Library Trends 54(1), 46-64. Tice, M. (2001). Queens Borough Public Library and the Connecting Libraries and Schools Project. Journal of Youth Services in Libraries, 14(3), 11–13Wilkins, J. L. H., & Leckie, G. J. (1997). University Professional and Managerial Staff: Information Needs and Seeking. College & Research Libraries, 58(6), 561-74.Tice CIT Example: “A sample of fifth and seventh graders was asked to describe a good public library experience and what made it good, describe a bad public library experience and what made it bad, and relate whether or not CLASP has made a difference. Overall, CLASP had a higher positive impact on fifth graders (39 percent) as opposed to seventh graders (25 percent), and 75 percent of all students surveyed were able to remember a CLASP visit to their class” (Tice, 2001, p.13) Hamer: “The interview schedule was developed using Flanagan's (1954) critical incident technique, a method complementary to this study's social constructivist outlook.” The following questions were asked to solicit responses about specific incidents: • What questions did you initially have about coming-out? • How did your questions change as you continued to think about coming-out? • How did you go about trying to find answers to these questions? • What was most challenging about answering these questions? •What, if any, barriers did you encounter when seeking answers to questions? • What are some of the thoughts and feelings you experienced during these experiences? • Please describe the resources or experiences that helped you most (Hamer, 2003, p. 79).
  • Connaway, L. S. & Radford, M. L. (2011). Seeking Synchronicity: Revelations and recommendations for virtual reference. Dublin, OH: OCLC Research. Retrieved from http://www.oclc.org/reports/synchronicity/full.pdf
  • Image from http://www.flickr.com/photos/stuartpilbrow/2967327441/Connaway, L. S. & Radford, M. L. (2011). Seeking Synchronicity: Revelations and recommendations for virtual reference. Dublin, OH: OCLC Research. Retrieved from http://www.oclc.org/reports/synchronicity/full.pdf
  • Image from Microsoft Clip ArtConnaway, L. S. & Radford, M. L. (2011). Seeking Synchronicity: Revelations and recommendations for virtual reference. Dublin, OH: OCLC Research. Retrieved from http://www.oclc.org/reports/synchronicity/full.pdf
  • Image from Microsoft Clip ArtConnaway, L. S. & Radford, M. L. (2011). Seeking Synchronicity: Revelations and recommendations for virtual reference. Dublin, OH: OCLC Research. Retrieved from http://www.oclc.org/reports/synchronicity/full.pdf
  • White, D., & Connaway, L. S. (2011). Visitors and residents: What motivates engagement with the digital information environment. Funded by JISC, OCLC, and Oxford University. Retrieved from http://www.oclc.org/research/activities/vandr/
  • Image from Microsoft Clip Art“Several methods of data collection are being utilized in this study: semi-structured interviews, diaries, and an online survey. The multi-method design enables triangulation, which provides a cross examination of the data analysis and results. The quantitative and qualitative methods, including ethnographic methods that devote individual attention to the subjects, yield a very rich data set enabling multiple methods of analysis.” Connaway, L. S., Lanclos, D., White, D. S., Le Cornu, A., & Hood, E. M. (2012). User-centered decision making: A new model for developing academic library services and systems. IFLA 2012 Conference Proceedings, August 11-17, Helsinki, Finland. White, D., & Connaway, L.S. (2011). Visitors and Residents: What Motivates Engagement with the Digital Information Environment. Funded by JISC, OCLC, and Oxford University. http://www.oclc.org/research/activities/vandr/.
  • Image from Microsoft Clip ArtThe diaries are an ethnographic data collection technique and a form of event sampling, which can focus participant attention on those areas which most interest researchers. Connaway and Powell (2010) point out that instruments (like diaries) that are intended to get people to describe what has just happened to them may be affected by distortions of memory and retrospection. They recommend that the question under review “center on discrete, defined events or moments so that such recording effort becomes reasonable and recall efforts are relatively straightforward” (Connaway and Powell 2010, 222).Ethnography(From Basic Research Methods 5th Edition, p. 175-176)Ethnography involves establishing rapport, selecting research participants, transcribing observations and conversations, and keeping diaries, although Geertz believes that none of these techniques or procedures adequately defines the venture. He believes ethnography is defined by the kind of intellectual effort it is, “an elaborate venture in ‘thick description.’ ” Reality, as perceived by the observer, is described in such detail that the reader can experience the total event as if he or she had actually been involved in it. The goal of ethnography is to establish rapport with target communities, via a flexible toolkit of methods including participant observation, structured and unstructured interviews, reliance on selected research participants as “key informants,” and keeping diaries. The analytical intellectual work of ethnography involves being able to engage in a particular way of seeing (Wolcott 2008) that is informed by the ethnographer’s immersion in the reality of other people’s existence. Such qualitative data must be approached and interpreted in a way that recognizes and retains this richness (Connaway and Powell 2010).Geertz, Clifford. 1973. The interpretation of cultures: selected essays. New York: Basic Books, 6. Connaway, L. S., & Powell, R. R. (2010). Basic research methods for librarians. Santa Barbara, CA: Libraries Unlimited.
  • Image from Microsoft Clip ArtScreenshot: USU7 Emg Dry1 Ph1 (Digital Visitors and Residents, USU7, Female, Age 19)“March was a little more serious. We’re nearing election time and all that jazz, and this is the first election I’ll be able to vote in, so I’ve been doing a bit of reading on a few of the candidates (not much, mind you). As a side note, I’m kind of debating whether or not I should actually vote in this election since I have such little knowledge on the candidates.” (Digital Visitors and Residents, USU12, Male, Age 19) USU12 Est Dry2 Ph2“Um they do have a search box but I didn’t—I didn’t use that. Um I pretty much just, I pretty much just clicked links.” (Digital Visitors and Residents, USU12, Male, Age 19, 30:37) USU12 EstFollup Int1 Ph3
  • Image from http://www.flickr.com/photos/susyna/3643831785/White, D., & Connaway, L.S. (2011). Visitors and Residents: What Motivates Engagement with the Digital Information Environment. Funded by JISC, OCLC, and Oxford University. http://www.oclc.org/research/activities/vandr/.Example: Diarist Follow Up InterviewsInterviewer: Lets see, explain a time in the past month when you were successful in getting what you needed in a personal situation. What steps did you take? UKU7: Um, get what I wanted... Ah, I bought an iPad. [laughter] But I had to convince my husband. [laughter] So, I almost did like a business plan. [laughter] Just because I... The, um, other girls at university have... Had one. And it was getting to be a standard joke, that every week somebody else would turn up with one. Um, but I could see all the benefits, I was very antsy of them beforehand. I just thought it's just a toy and a gimmick, but once I've seen them in use, I could really see the benefits and how it would help me. So, I went to the business plan with my husband, um, and then I took him into the Apple store, had my list of questions that I wanted to ask the salesman in front of my husband so that he could see that I wasn't making it up. Because it, it, you know, it is a lot of money for one.-From Digital Visitors and Residents Diarist Follow-up Interviews (UKU7, Female, Age 34, Social Sciences)
  • Image from Microsoft Clip ArtWhite, D., & Connaway, L.S. (2011). Visitors and Residents: What Motivates Engagement with the Digital Information Environment. Funded by JISC, OCLC, and Oxford University. http://www.oclc.org/research/activities/vandr/.
  • Radford, M. L., Connaway, L. S., & Shah, C. (2011-2013). Cyber Synergy: Seeking Sustainability through Collaboration between Virtual Reference and Social Q&A Sites. Funded by the Institute of Museum and Library Services (IMLS), Rutgers University, and OCLC. Retrieved from http://www.oclc.org/research/activities/synergy/default.htmProposes new model for VRS to remain viable in today's economic environmentInvestigate possibility of collaboration between knowledge institutions and Social Q&A (SQA) communityProvide evidence for modeling new library collaborative servicesThree phasesAnalysis of transcripts500 QuestionPoint1000 Yahoo Answers Q & A pairs200 QuestionPoint live chat 200 QuestionPointQwidget session transcriptsTelephone interviews & analysisConstruct design specifications
  • Radford, M. L., Connaway, L. S., & Shah, C. (2011-2013). Cyber Synergy: Seeking Sustainability through Collaboration between Virtual Reference and Social Q&A Sites. Funded by the Institute of Museum and Library Services (IMLS), Rutgers University, and OCLC. Retrieved from http://www.oclc.org/research/activities/synergy/default.htm
  • Image from http://www.flickr.com/photos/myxi/4327438430/Radford, M. L., Connaway, L. S., & Shah, C. (2011-2013). Cyber Synergy: Seeking Sustainability through Collaboration between Virtual Reference and Social Q&A Sites. Funded by the Institute of Museum and Library Services (IMLS), Rutgers University, and OCLC. Retrieved from http://www.oclc.org/research/activities/synergy/default.htm
  • Image from http://www.flickr.com/photos/sylvain_masson/4195880838/

Transcript

  • 1. CoLIS, Copenhagen, Denmark August 22, 2013 Qualitative Inquiry in Social and Cultural Contexts The Critical Incident Technique Lynn Silipigni Connaway, Ph.D. Senior Research Scientist OCLC @LynnConnaway connawal@oclc.org The world’s libraries. Connected.
  • 2. Critical Incident Technique (CIT) • Flanagan (1954) • Qualitative technique • Focuses on most memorable event/experience • Allows categories or themes to emerge rather than be imposed (Flanagan, 1954) The world’s libraries. Connected.
  • 3. Origins of CIT • Sir Francis Galton • Aviation Psychology Program of US Army Air Forces in WWII • Analysis of failure in learning to fly • Used in proceedings of elimination boards • Research for selecting pilots • Reasons for failures of bombing missions (Flanagan, 1954, p. 2) The world’s libraries. Connected.
  • 4. Origins of CIT • Combat leadership studies • Gather specific incidents of behavior • Helpful or inadequate in accomplishing mission • “Describe the officer’s action. What did he do?” • Resulted in “critical requirements” of combat leadership (Flanagan, 1954, 2) The world’s libraries. Connected.
  • 5. CIT in LIS • Radford (2006) Tice (2001) • Evaluate CLASP (Connecting Libraries and Schools Project) • YA attitudes toward public libraries • Fisher & Oulton (1999) • Staff development needs • Library decision-making • Tool for librarian entering management positions • Hamer (2003) • Information seeking of LGBT youth coming out • Wilkins & Leckie (1997) • Investigate information needs & informationseeking behavior of university staff • Used in addition to questionnaire The world’s libraries. Connected.
  • 6. Critical Incident Technique Examples: Our Research The world’s libraries. Connected.
  • 7. Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User & Librarian Perspectives • Telephone interviews • Studied habits & needs of virtual reference services (VRS) users and non-users to identify characteristics for informing library system and service development • Four phases: • Focus group interviews • Analysis of 850 QuestionPoint live chat transcripts • 76 VRS Users • 100 VRS Librarians • 107 VRS Non-users • Generalizable through large sample sizes, multiple methods of data collection, and triangulation of results. • Online surveys of VRS • 137 VRS Users • 173 VRS Librarians • 134 VRS Non-users http://www.oclc.org/research/activities/synchronicity/default.htm The world’s libraries. Connected.
  • 8. Example: Seeking Synchronicity CIT VRS User Online Survey Questions Think about one experience using VRS in which you felt achieved (or did not achieve) a positive result • Please describe the circumstances and nature of your question. • Describe why you felt the encounter was successful (or unsuccessful). • Did the chat format help your experience to be successful (or unsuccessful)? If yes, how? (Connaway & Radford, 2011) The world’s libraries. Connected.
  • 9. Example: Seeking Synchronicity CIT VRS Potential User Online Survey Questions Think about one experience in which you felt you achieved (or did not achieve) a positive result after seeking library reference services in any format. • Think about one experience in which you felt you did (or did not) achieve a positive result after seeking library reference services in any format. • Describe each interaction. • Identify the factors that made these interactions positive or negative. (Connaway & Radford, 2011) The world’s libraries. Connected.
  • 10. Example: Seeking Synchronicity CIT VRS User Result “The librarian threw in a cordial sign-off and encouraged me to pursue the reading. It was like talking to a friendly librarian in person.” –VRS User (online survey) The world’s libraries. Connected.
  • 11. Digital Visitors and Residents: What Motivates Engagement with the Digital Information Environment? • Investigate theory of digital residents & visitors with students in the various stages of education • Increase understanding of learners’ online engagement • How can educational services & systems attract & sustain new group of lifelong learners? • Trans-Atlantic partnership supports comparison of students' digital learning strategies in different cultural contexts. The world’s libraries. Connected. (Connaway and White for OCLC Research, 2012.) http://www.oclc.org/research/activities/vandr/default.htm
  • 12. Example: Digital Visitors and Residents Triangulation of Data • Several methods: • Semi-structured interviews (qualitative) • Diaries (qualitative) • Online survey (quantitative) • Enables triangulation of data (Connaway et al., 2012) The world’s libraries. Connected.
  • 13. Diaries • Ethnographic data collection technique • Get people to describe what has happened • Center on defined events or moments (Connaway & Powell, 2010) The world’s libraries. Connected.
  • 14. Example: Digital Visitors and Residents Diaries The world’s libraries. Connected.
  • 15. Example: Digital Visitors and Residents CIT Diarist Follow Up Interviews Think of a time when you had a situation where you needed answers or solutions and you did a quick search and made do with it. You knew there were other sources but you decided not to use them. Please include sources such as friends, family, teachers, coaches, etc. The world’s libraries. Connected.
  • 16. Example: Digital Visitors and Residents CIT Diarist Follow Up Interviews • Explain a time in the past month when you were SUCCESSFUL in completing an ACADEMIC assignment. What steps did you take? • Think of a time fairly recently when you struggled to find appropriate resources to help you complete an ACADEMIC assignment. What happened? • Explain a time in the past month when you were successful in getting what you needed in a PERSONAL situation. What steps did you take? • Explain a time in the past month when you were NOT successful in getting what you needed in a PERSONAL situation. What steps did you take? The world’s libraries. Connected.
  • 17. Cyber Synergy: Seeking Sustainability through Collaboration between Virtual Reference & Social Q&A Sites • New model for VRS • Collaboration with Social Q&A (SQA) community • Provide evidence for modeling new library collaborative services • Three phases • Analysis of transcripts • 500 QuestionPoint • 1000 Yahoo Answers Q & A pairs • 200 QuestionPoint live chat • 200 QuestionPoint Qwidget session transcripts • Telephone interviews & analysis • 50 librarian interviews, 50 user interviews • Construct design specifications The world’s libraries. Connected. http://www.oclc.org/research/activities/synergy/default.htm
  • 18. Example: Cyber Synergy CIT User Phone Interviews Please recall one specific VR Please recall one specific VR interaction...that you would interaction...that you would consider successful and consider unsuccessful and describe. describe. (Radford, Connaway, & Shah, 2011-2013) The world’s libraries. Connected.
  • 19. “I was looking for books on bilingual communication in the setting of sports teams and I needed at least three book references and I wasn't able to get them on time, so I think it was unsuccessful because I ended up not getting the information I needed for research I was working on.” Cyber Synergy (VS43, Male, Age 19-25) The world’s libraries. Connected.
  • 20. Using CIT in YOUR Research • Can be applied to a range of studies • Allows categories or themes to emerge rather than be imposed • Provides clear examples that prove the value of your services • Users speak for you The world’s libraries. Connected.
  • 21. References Connaway, L. S. & Radford, M. L. (2011). Seeking Synchronicity: Revelations and recommendations for virtual reference. Dublin, OH: OCLC Research. Retrieved from http://www.oclc.org/reports/synchronicity/full.pdf Connaway, L. S., Lanclos, D., White, D. S., Le Cornu, A., & Hood, E. M. (2012). User-centered decision making: A new model for developing academic library services and systems. IFLA 2012 Conference Proceedings, August 11-17, Helsinki, Finland. Connaway, L. S., & Powell, R. R. (2010). Basic research methods for librarians. Santa Barbara, CA: Libraries Unlimited. Fisher, S., & Oulton, T. (1999). The critical incident technique in library and information management research. Education for Information, 17(2), 113–126. Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358. Geertz, Clifford. (1973). The interpretation of cultures: selected essays. New York: Basic Books, 6. Hamer, J. S. (2003). Coming-out: Gay males’ information seeking. School Libraries Worldwide, 9(2), 73–79. Radford, M. L. (summer, 2006). The critical incident technique and the qualitative evaluation of the Connecting Libraries and Schools Project. Library Trends 54(1), 46-64. Radford, M. L. (1999). The Reference Encounter: Interpersonal Communication in the Academic Library. Chicago: Association of College & Research Libraries, A Division of the American Library Association. (Publications in Librarianship #52). Radford, M. L., Connaway, L. S., & Shah, C. (2011-2013). Cyber Synergy: Seeking Sustainability through Collaboration between Virtual Reference and Social Q&A Sites. Funded by the Institute of Museum and Library Services (IMLS), Rutgers University, and OCLC. Retrieved from http://www.oclc.org/research/activities/synergy/default.htm Tice, M. (2001). Queens Borough Public Library and the Connecting Libraries and Schools Project. Journal of Youth Services in Libraries, 14(3), 11–13 White, D., & Connaway, L. S. (2011). Visitors and residents: What motivates engagement with the digital information environment. Funded by JISC, OCLC, and Oxford University. Retrieved from http://www.oclc.org/research/activities/vandr/ Wilkins, J. L. H., & Leckie, G. J. (1997). University Professional and Managerial Staff: Information Needs and Seeking. College & Research Libraries, 58(6), 561-74. The world’s libraries. Connected.
  • 22. Lynn Silipigni Connaway, Ph.D. connawal@oclc.org Questions? The world’s libraries. Connected.