2016 qqml kyrillidou_worldviews, methods, tools, and protocols
•Download as PPTX, PDF•
1 like•357 views
A look at our assumptions and healthy ways of questioning them as we deploy different worldviews, methods and tools in library assessment! Don't be afraid to question assumptions -- most of the time the data are in front of us and we are just blind because of our own fundamental assumptions. There is a better way than resting on erroneous assumptions!
Recommended
More Related Content
What's hot
What's hot (20)
Similar to 2016 qqml kyrillidou_worldviews, methods, tools, and protocols
Similar to 2016 qqml kyrillidou_worldviews, methods, tools, and protocols (20)
More from Martha Kyrillidou, PhD
More from Martha Kyrillidou, PhD (6)
Recently uploaded
Recently uploaded (20)
2016 qqml kyrillidou_worldviews, methods, tools, and protocols
- 1. Martha Kyrillidou Library Worldviews, Design and Methods Mixed Methods Simple and Complex QualityMetrics, LLC Research Associate, UIUC Part-time instructor, Kent State Qualitative and Quantitative Methods in Libraries, London, UK, May 24, 2016
- 2. It ain’t what you don’t know that gets you into trouble It’s what you know for sure that just ain’t so Mark Twain
- 4. 1. Library worldviews 2. Research Design and Methods 3. Mixed Methods 4. Simple and Complex
- 6. 1. Your problem/issue is not as unique as you think Adapted from Douglas Hubbard, “How to Measure Anything” (2014)
- 7. 1. Library worldviews 2. Research Design and Methods
- 8. 1. Your problem/issue is not as unique as you think 2. You have more data/information than you think Adapted from Douglas Hubbard, “How to Measure Anything” (2014)
- 9. 1. Library worldviews 2. Research Design and Methods 3. Mixed Methods
- 10. 1. Your problem/issue is not as unique as you think 2. You have more data/information than you think 3. You need less data/information than you think Adapted from Douglas Hubbard, “How to Measure Anything” (2014)
- 11. 1. Library worldviews 2. Research Design and Methods 3. Mixed Methods 4. Simple and Complex
- 12. Four Useful Assessment Assumptions 1. Your problem/issue is not as unique as you think 2. You have more data/information than you think 3. You need less data/information than you think 4. There are useful methods that are much simpler than you think Adapted from Douglas Hubbard, “How to Measure Anything” (2014)
- 14. Scarcity We made decisions on advice from experts using them as north stars for insights.
- 16. Five laws of library science – S.R. Ranganathan (1931) 1. Books are for use. 2. Every reader his / her book. 3. Every book its reader. 4. Save the time of the reader. 5. The library is a growing organism.
- 20. Five laws of library science – S.R. Ranganathan (1931) 1. Books are for use. 2. Every reader his / her book. 3. Every book its reader. 4. Save the time of the reader. 5. The library is a growing organism. • Connaway, Lynn Silipigni, and Ixchel M. Faniel. 2014. Reordering Ranganathan: Shifting User Behaviors, Shifting Priorities. Dublin, OH: OCLC Research.http://www.oclc.org/content/dam/research/publications/library/ 2014/oclcresearch-reordering-ranganathan-2014.pdf.
- 22. Four Useful Assessment Assumptions 1. Your problem/issue is not as unique as you think Adapted from Douglas Hubbard, “How to Measure Anything” (2014)
- 24. • Connaway and Powell • Wildemuth • Matthews • Markless and Steadfield • Ben Showers • Creswell • Better evaluation: www.betterevaluation.org • Anthropology, Sociology, Economics, Education
- 30. Explanatory Tell a story Why am I collecting these data? What’s the story?
- 31. Persuasion and the Power of Story: Jennifer Aaker (Future of StoryTelling 2013) DATA – intellectual + STORIES- emotional
- 34. 1. Library worldviews 2. Research Design and Methods
- 35. Four Useful Assessment Assumptions 1. Your problem/issue is not as unique as you think 2. You have more data/information than you think Adapted from Douglas Hubbard, “How to Measure Anything” (2014)
- 36. Mixed Methods
- 37. Criteria for Selecting a Research Approach • Research Problem and Questions: Quantitative approach is best when: • Testing causal relationships (factor/s that influence a particular outcome). • Evaluating the usefulness or successes of an intervention. • Establishing which factors best predict an outcome. • Testing theories or explanations. Qualitative approach is best when: • The Researcher is uncertain about which are the most important variables to be examined. • If the topic is new, sample population is unexplored by the topic or the dominant explanations may not apply to a given sample population.
- 38. Data Analysis Qualitative • Get to know your data. • Narrow your focus; determine what you want to find out. • Identify themes/categories. • Preset or evolving • Code themes/categories. • Look for patterns/connections. • Interpret the findings. Quantitative • Counts/frequencies • Percentages • Central tendency • Mean, median, mode • Variability • Range, standard deviation • Ranks • Cross-tabulations & subsorting
- 42. Assessment Methods Commonly Used in Libraries • Surveys • Local, multi-library, and institution • Qualitative methods • Focus groups, interviews • Observation, ethnographic • Usability • Rubrics/evaluations • Tests • Advisory groups • Critical incident, logs • Comments • Social media • Usage/Data mining
- 43. 1. Library worldviews 2. Research Design and Methods 3. Mixed Methods
- 44. Four Useful Assessment Assumptions 1. Your problem/issue is not as unique as you think 2. You have more data/information than you think 3. You need less data/information than you think Adapted from Douglas Hubbard, “How to Measure Anything” (2014)
- 46. Methods Dependent on Information Needed • What do you want to know? • When do you need it? • How will you get it? • Who else might you collaborate with • How will you analyze data and communicate it • How will you use the information?
- 47. Assessment Methods and Tools/Protocols • Customer Surveys • Interviews/Focus Groups • Observational/Ethnographic Studies • User-Centered Design/Usability… • Usage Data/Data Mining For each method… Simple and complex
- 48. More than Big Data Analysis • Big data in the form of behaviors and small data in the form of surveys complement each other and produce insights rather than simple metrics • Small data requires more qualitative approaches – use of social psychologists, anthropologists, and sociologists to find what simple measures miss • Small data can find holes in big data • "How Not to Drown in Numbers", 2015
- 49. So Many Methods . . . . So Little Time, Resources Etc. It’s All About Reducing Uncertainty and Risk • How much do you already know? • If you have a lot of uncertainty you don’t need much new data to tell you something useful • If you have a lot of certainty already, then you need a lot of data to reduce uncertainty significantly. . . • But how important is it to do this? It isn’t brain surgery. Is 90% confidence sufficient for most library decisions? Is it “good enough”?
- 50. There is a better way
- 51. Four Useful Assessment Assumptions 1. Your problem/issue is not as unique as you think – Library worldviews 2. You have more data/information than you think – research design/methods 3. You need less data/information than you think – mixed methods 4. There are useful methods that are much simpler than you think Adapted from Douglas Hubbard, “How to Measure Anything” (2014)
Editor's Notes
- It’s a real honor to be here at the opening of the 8th QQML Conference in London and have the opportunity to explore with you how far we have come in our application of sound research approaches in libraries as well as how we can move ahead in agile, flexible and useful ways. Anthi has consistently put together year after year a unique gathering of colleagues across the globe – probably the most diverse conference among those emphasizing evidence, assessment and research in libraries. I am also truly honored to have the opening slot at this particular event during a time that I am diversifying myself, my work and my future with libraries – as many of you know I have worked at ARL for 22 years and many things have we accomplished over the last two decades which allowed us through ARL to bring useful protocols to libraries across the globe. In my new incarnation presented to you here as Principal of QualityMetrics, a Research Associate at UIUC and a part-time instructor at Kent State, I have the pleasure of not only continuing to work with ARL as a consultant but expand to include different types of libraries, public libraries, school libraries and other special agencies, as well as delve into the broader spectrum of cultural heritage institutions that includes museums and archives. So, I am hoping, given the new flexibility I have as a consultant, that I will get to work with some of you much more closely in the future. The talk of today’s presentation is inspired by the wider recognition that research skills are fundamental to the information professions these days – we have not only developed a great appreciation of qualitative and quantitative methods over the years as we can see from the growing literature but we are witnessing new specializations in this area manifested through ‘user experience librarian’ roles, ‘assessment librarian’ roles, and ‘planning and engagement’ roles to just mention a few. Today it’s an opportunity to step back and reflect a bit for the next hour or so – why is it that this is happening and hopefully this reflection will help us gain a fresh perspective on future exciting possibilities. So, why is it that we care about research – quantitative and qualitative methods?
- Mark Twain gave an answer in his own unique diction many years ago -- … This quote is on the opening scene of a recent movie – anyone cares to guess the title? ‘The Big and Short
- When four outsiders saw what the big banks, media and government refused to see – the global collapse of the economy, they had an idea: The Big Short In our library world we have the Big Short opportunities as well Next slide