Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Developing a Mixed Qualitative andQuantitative Research Design toInform Library Policy Decision-makingDavid P. KennedyRAND...
Outline• Background• Overview of Cultural Domain Analysis• Results of use of two methods that combinequalitative and quant...
Systematic Data Collection
Cultural Domain Analysis• 2 Key Methods–Free Listing–Pile Sorting
What is a Cultural Domain?• Cognitive Domain:– How people think (cognition) about categories(domains)– Typically named• ex...
Cultural Domain Analysis atWilliam H. Hannon Library
Cultural Domain Analysis atWilliam H. Hannon Library• The WHH Library is the library atLoyola Marymount University (LMU)• ...
Goal: Efficiently and effectively determineitems in a domain and understand howthey are related to each other.• Step One: ...
Free Listing• Goal: Native categories and terminology• Basic idea:– “Tell me all the <category name> you can think of”– Ty...
Free Listing:Cultural Domain of Library Usage• Systematic: Everyone Given the SameQuestion:• “Think about all of the thing...
Free List Analysis• Different ways to analyze free list data– Sort in descending order of frequency mentioned– Tally avera...
FREE LIST RESULTS
Free List Results• 4 Interviewers– Library Staff– 3 Female, 1 male• 21 Respondents– Student Workers– 16 Freshman/Sophomore...
Free List Results: > 50%Item Freq. % Rank Avg. SalienceUse printers 19 86.4 6.6 0.5Use computers 17 77.3 6.8 0.5Use group ...
Free lists: Scree Plot, % mentioned0.010.020.030.040.050.060.070.080.090.0100.0• No sharp scree fall• No strong core set o...
Goal: Efficiently and effectively determineitems in a domain and understand howthey are related to each other.• Step Two: ...
Pile Sorting Example• Fruits and Vegetables
Free List Text to CardsApplePear13CornBanana24
Cards to PilesAppleBananaBreadfruitKumquatCurrantPile 1: “Fruit I eatat lunch”Pile 2: “Fruit Idon’t like”Pile 3: “Fruit I’...
Pile Sort Analysis• Different ways to analyze pile sort data– Produce Aggregate Proximity Matrix• How often did each item ...
Pile Sort Analysis• Software: Anthropac (Visual Anthropac1.0)• Goal: Identify Patterns in How PeopleGroup Items• 35 items ...
PILE SORT RESULTS
Pile Sort Results• 4 Interviewers– Library Staff– 3 Female, 1 male• 21 Respondents– Student Workers– 12 Freshman/Sophomore...
Use computersUse printersBorrow booksUse group study roomsAttend classesStudyGet research helpUse copierDo researchSpend t...
Reactions from Data Collectors• “I was also surprised by students who did notimmediately identify functions in their own a...
Conclusions and Recommendations• Three primary sub-domains– Research– Getting help / borrow things– Socialize• Electronic ...
Thank You!davidk@rand.orgmarie.kennedy@lmu.edu
Developing a Mixed Qualitative and Quantitative Research Design to Inform Library Policy Decision-making
Upcoming SlideShare
Loading in …5
×

Developing a Mixed Qualitative and Quantitative Research Design to Inform Library Policy Decision-making

1,358 views

Published on

Presented at QQML 2013: Qualitative and Quantitative Methods in Libraries International Conference. Rome, Italy.

Published in: Technology, Business
  • Be the first to comment

Developing a Mixed Qualitative and Quantitative Research Design to Inform Library Policy Decision-making

  1. 1. Developing a Mixed Qualitative andQuantitative Research Design toInform Library Policy Decision-makingDavid P. KennedyRAND CorporationMarie R. KennedyLoyola Marymount UniversityQQML 2013: Qualitative and QuantitativeMethods in Libraries International Conference
  2. 2. Outline• Background• Overview of Cultural Domain Analysis• Results of use of two methods that combinequalitative and quantitative approaches– Free Listing– Pile Sorting
  3. 3. Systematic Data Collection
  4. 4. Cultural Domain Analysis• 2 Key Methods–Free Listing–Pile Sorting
  5. 5. What is a Cultural Domain?• Cognitive Domain:– How people think (cognition) about categories(domains)– Typically named• examples: illnesses, vegetables, countries– Categories contain items– Items have semantic relationships with each other• Examples: X is part of Y, X is similar to Y, X causes Y, etc.• Some may be categories themselves– For example– Books -> reference books -> dictionaries -> types of dictionaries ->and so on• When it is Cultural?– Shared
  6. 6. Cultural Domain Analysis atWilliam H. Hannon Library
  7. 7. Cultural Domain Analysis atWilliam H. Hannon Library• The WHH Library is the library atLoyola Marymount University (LMU)• Private, Catholic• In Los Angeles, California (USA)• Primary attention given to undergraduates• Strong student employee program in the library
  8. 8. Goal: Efficiently and effectively determineitems in a domain and understand howthey are related to each other.• Step One: Free ListsEvaluating a Cultural Domain
  9. 9. Free Listing• Goal: Native categories and terminology• Basic idea:– “Tell me all the <category name> you can think of”– Typically loosely timed, no questions allowed– “Grand tour” question• Contrasts with survey open-ended question– Open-ended is typically about the respondent:• What do you like about this product? What ice-cream flavors doyou like? What illnesses have you had?– Free list is about the domain:• What ice-cream flavors are there? What illnesses exist? Whatare all of the fruits and vegetables?
  10. 10. Free Listing:Cultural Domain of Library Usage• Systematic: Everyone Given the SameQuestion:• “Think about all of the things that people onLMU campus do when they use libraryservices. Off the top of your head, list all ofthe ways that people on LMU campus use thelibrary.” NOTE: Not just what they do!• Exploratory: How they answer is upto them
  11. 11. Free List Analysis• Different ways to analyze free list data– Sort in descending order of frequency mentioned– Tally average position in lists– Combine frequency and position to create“salience” measure• Software: Anthropac (Visual Anthropac 1.0)• Goal: Identify core set of items– Explore patterns in what people say
  12. 12. FREE LIST RESULTS
  13. 13. Free List Results• 4 Interviewers– Library Staff– 3 Female, 1 male• 21 Respondents– Student Workers– 16 Freshman/Sophomores– 13 Had worked in library less than 1 year– 18 Female• 324 responses– Collapsed into 62 Unique responses
  14. 14. Free List Results: > 50%Item Freq. % Rank Avg. SalienceUse printers 19 86.4 6.6 0.5Use computers 17 77.3 6.8 0.5Use group study rooms 15 68.2 5.9 0.5Borrow books 15 68.2 5.3 0.5Study 14 63.6 4.1 0.5Attend classes 13 59.1 12.1 0.2Use copier 11 50.0 11.4 0.2Get research help 11 50.0 8.0 0.3Spend time in a quiet place 11 50.0 7.1 0.3Do research 11 50.0 3.3 0.4
  15. 15. Free lists: Scree Plot, % mentioned0.010.020.030.040.050.060.070.080.090.0100.0• No sharp scree fall• No strong core set of Items• Short Tail• Relatively fewidiosyncratic answers• Top electronic resource13.6%
  16. 16. Goal: Efficiently and effectively determineitems in a domain and understand howthey are related to each other.• Step Two: Pile Sorting items identified infree list interviewsEvaluating a Cultural Domain
  17. 17. Pile Sorting Example• Fruits and Vegetables
  18. 18. Free List Text to CardsApplePear13CornBanana24
  19. 19. Cards to PilesAppleBananaBreadfruitKumquatCurrantPile 1: “Fruit I eatat lunch”Pile 2: “Fruit Idon’t like”Pile 3: “Fruit I’m not surewhat they are”OrangeCoconut
  20. 20. Pile Sort Analysis• Different ways to analyze pile sort data– Produce Aggregate Proximity Matrix• How often did each item end up in the same pileacross all pile sorters– Multivariate Analysis Techniques• Multidimensional Scaling (MDS)• Cluster Analysis• Consensus Analysis– Compare results of quantitative analysis toqualitative descriptions of piles
  21. 21. Pile Sort Analysis• Software: Anthropac (Visual Anthropac1.0)• Goal: Identify Patterns in How PeopleGroup Items• 35 items from free list – top frequency– 5 additional items not named: electronicresources• Better understand how electronic resources fit intocultural domain• Provide insight into how to better market electronicresources• Increase their salience
  22. 22. PILE SORT RESULTS
  23. 23. Pile Sort Results• 4 Interviewers– Library Staff– 3 Female, 1 male• 21 Respondents– Student Workers– 12 Freshman/Sophomores– 13 Had worked in library less than 1 year– 18 Female• 102 piles total across respondents– Nearly 5 piles on average– Range 2 to 8 piles
  24. 24. Use computersUse printersBorrow booksUse group study roomsAttend classesStudyGet research helpUse copierDo researchSpend time in a quiet placeGo to the CaféUse scannersReadWork on group projectsConduct archival researchAttend eventsAccess the InternetDo homeworkEmploymentBorrow DVDs / moviesWatch moviesBorrow technical equipmentMeeting placeAccess course reservesUse presentationequipment/roomUse media roomsUse fax machineRead newspapersGet technical helpAsk questionsLook at art/exhibitsSee the viewUse databasesRead periodicals/magazinesHang outProQuestEBSCOGoogle ScholarWikipedia LibGuidesResearchBorrow things and get helpSocial and enjoymentrelated activitiesStress: .133High Consensus : Strongevidence of one cultureMDS and Cluster Analysis
  25. 25. Reactions from Data Collectors• “I was also surprised by students who did notimmediately identify functions in their own areasof work as things that people do when they cometo the library! (For example, ref desk studentswho listed research help only at the very end oftheir list, if at all.)”• “Not a single student separated the ideas of usingthe library to explore and gratify personal orintellectual curiosities or independentlybroadening their knowledge of their own majorareas of study as separate from the overall ideaof research.”
  26. 26. Conclusions and Recommendations• Three primary sub-domains– Research– Getting help / borrow things– Socialize• Electronic resources are not highly salientelements of the cultural domain of library activities– Loyola Marymount Campus– Student workers• Market resources as aspects of other maindomains– Peer educators
  27. 27. Thank You!davidk@rand.orgmarie.kennedy@lmu.edu

×