What Can We Learn from Library
    Instruction Research?
A Content Analysis of Evidence Summaries from
Evidence Based Libr...
•   Open access, non-profit
•   Peer-reviewed
•   Published quarterly since 2006
•   Open Journal Systems (University of A...
Evidence Based Practice

                  Assess                           Ask




              Apply                   ...
Evidence Summaries

Structured abstract
  objective – design – setting – subjects –
  method – main results – conclusion

...
WILU 2009   Kloda, Koufoginannakis & Mallan   5
Objectives
              •         Determine common
                        strengths and weaknesses
                     ...
Design
                  Content analysis.




WILU 2009   Kloda, Koufoginannakis & Mallan   7
Setting
                  LIS literature, as
                  represented in the
                  journal, Evidence Base...
Subjects
                  Commentaries of 21
                  evidence summaries on
                  library instructio...
Method
             •    Data extraction form
             •    Pre-testing
             •    Emerging categories
        ...
Validity

1.    Focused issue/question
2.    Conflict of interest
3.    Appropriate and replicable method
4.    Population...
Reliability

1.    Results clearly explained
2.    Response rate
3.    Useful analysis
4.    Appropriate analysis
5.    Re...
Applicability


1. Implications reported in original study
2. Applicability to other populations
3. More information requi...
Main Results
                • General attributes (domain,
                  setting, source, length)
                • Va...
Domain Overlap
            Education                                       21

            Collections                    ...
Setting

            Academic                                    14

            Health                                   ...
Publications
• Canadian Journal of                    • Libraries & the Academy
  Learning and Technology                •...
Length of Commentaries

             Word Count           Frequencies

             200-299                               ...
Topics Covered in Original Studies

    Effect of instruction program or service on
    learning                          ...
Validity

      Focused issue/question (n=10)
      Conflict of interest (n=2)
      Appropriate and replicable method (n=...
[appropriateness of method]

    This study employs a quantitative
    methodology to answer a question about
    understa...
[population and representative sample]


    Participants were randomly contacted but it
    is unclear how randomization ...
[focused question]




            The aims of the study were clear: ...




WILU 2009                Kloda, Koufoginannak...
Reliability

      Results clearly explained (n=7)
      Response rate (n=8)
      Useful analysis (n=7)
      Appropriate...
[appropriate analysis]

    The study reports several statistically significant
    results in relation to the research qu...
Applicability


      Implications reported in original study (n=13)
      Applicability to other populations (n=9)
      ...
[applicability]


    … suggests that Web-based tutorials are at
    least as effective as face to face teaching
    sessi...
Other Findings of Note

•   Commentary length / categories coded
•   Situated research in wider setting
•   Significance o...
[literature review]


    What is perhaps more valuable in this paper
    is the extensive use of the research literature
...
Limitations

• Small set of commentaries
• Writers have varying styles of writing,
  appraisal experience
• Bias of resear...
Conclusions

• Aspects of validity and reliability in studies
  that were critically appraised in EBLIP were
  more often ...
Implications for Practice

• Consider aspects of validity, reliability and
  applicability when you are developing a
  res...
Upcoming SlideShare
Loading in …5
×

What Can We Learn from Library Instruction Research

1,983 views

Published on

A content analysis of Evidence Summaries from Evidence Based Library and Information Practice. Presented at WILU 2009, Montreal, Concordia University

Published in: Education
1 Comment
1 Like
Statistics
Notes
  • Very interesting ppt. The research indicated PPT only contains 30% of information; therefore the 70% valuable information comes from the presenter himself/herself. soEZLecturing.com provides you a chance to record your voice with your PowerPoint presentation and upload to the website. It can share with more readers and also promote your presentation more effectively on soEZLecturing.com.
    www.soezlecturing.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
1,983
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
23
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

What Can We Learn from Library Instruction Research

  1. 1. What Can We Learn from Library Instruction Research? A Content Analysis of Evidence Summaries from Evidence Based Library and Information Practice Lorie Kloda Denise Koufogiannakis Katrine Mallan WILU 2009 | Concordia University | Montreal, QC
  2. 2. • Open access, non-profit • Peer-reviewed • Published quarterly since 2006 • Open Journal Systems (University of Alberta) • International audience • 2000+ registered readers WILU 2009 Kloda, Koufoginannakis & Mallan 2
  3. 3. Evidence Based Practice Assess Ask Apply Acquire Appraise WILU 2009 Kloda, Koufoginannakis & Mallan 3
  4. 4. Evidence Summaries Structured abstract objective – design – setting – subjects – method – main results – conclusion Commentary • 300-400 words • appraisal of validity, reliability, applicability • significance, implications for practice WILU 2009 Kloda, Koufoginannakis & Mallan 4
  5. 5. WILU 2009 Kloda, Koufoginannakis & Mallan 5
  6. 6. Objectives • Determine common strengths and weaknesses of research relevant to Library and Information Studies (LIS), as reported in the commentary section of published evidence summaries; • Identify commonalities in existing evidence summary commentaries. WILU 2009 Kloda, Koufoginannakis & Mallan 6
  7. 7. Design Content analysis. WILU 2009 Kloda, Koufoginannakis & Mallan 7
  8. 8. Setting LIS literature, as represented in the journal, Evidence Based Library and Information Practice (EBLIP). WILU 2009 Kloda, Koufoginannakis & Mallan 8
  9. 9. Subjects Commentaries of 21 evidence summaries on library instruction research published in EBLIP between 2006 and 2008. WILU 2009 Kloda, Koufoginannakis & Mallan 9
  10. 10. Method • Data extraction form • Pre-testing • Emerging categories • Each commentary analyzed by 2 researchers independently; discrepancies resolved by 3rd WILU 2009 Kloda, Koufoginannakis & Mallan 10
  11. 11. Validity 1. Focused issue/question 2. Conflict of interest 3. Appropriate and replicable method 4. Population and representative sample 5. Validated instrument WILU 2009 Kloda, Koufoginannakis & Mallan 11
  12. 12. Reliability 1. Results clearly explained 2. Response rate 3. Useful analysis 4. Appropriate analysis 5. Results address research questions 6. Limitations 7. Conclusions based on actual results WILU 2009 Kloda, Koufoginannakis & Mallan 12
  13. 13. Applicability 1. Implications reported in original study 2. Applicability to other populations 3. More information required WILU 2009 Kloda, Koufoginannakis & Mallan 13
  14. 14. Main Results • General attributes (domain, setting, source, length) • Validity • Reliability • Applicability • Other findings of note WILU 2009 Kloda, Koufoginannakis & Mallan 14
  15. 15. Domain Overlap Education 21 Collections 0 Information access & 2 retrieval Reference 1 Management 1 Professional issues 2 WILU 2009 Kloda, Koufoginannakis & Mallan 15
  16. 16. Setting Academic 14 Health 5 Public 1 Special 0 School 3 WILU 2009 Kloda, Koufoginannakis & Mallan 16
  17. 17. Publications • Canadian Journal of • Libraries & the Academy Learning and Technology • Library & Information • College & Research Libraries Science Research • First Monday • Library Review • Health Information & • Library Trends Libraries Journal • New Library World • Information Technology & • Ohio Educational Library Libraries Media Association • Internet Reference Services • School Libraries Worldwide Quarterly • School Library Media • Journal of Academic Research Librarianship WILU 2009 Kloda, Koufoginannakis & Mallan 17
  18. 18. Length of Commentaries Word Count Frequencies 200-299 1 300-399* 5 Mean = 400-499 7 476 500-599 4 words 600-699 3 700-799 1 WILU 2009 Kloda, Koufoginannakis & Mallan 18
  19. 19. Topics Covered in Original Studies Effect of instruction program or service on learning 8 Exploration of attitudes/perceptions 6 The literature of the field 2 Library or librarian effect on learner outcomes 2 Librarian roles/professional issues 2 Factors that impact student outcomes 1 WILU 2009 Kloda, Koufoginannakis & Mallan 19
  20. 20. Validity Focused issue/question (n=10) Conflict of interest (n=2) Appropriate and replicable method (n=17) Population and representative sample (n=18) Validated instrument (n = 11) WILU 2009 Kloda, Koufoginannakis & Mallan 20
  21. 21. [appropriateness of method] This study employs a quantitative methodology to answer a question about understanding or meaning of terminology used in library instruction. A qualitative research design, such as one using focus groups, would have also been appropriate for discovering more information about the students’ perceptions of library jargon…. WILU 2009 Kloda, Koufoginannakis & Mallan 21
  22. 22. [population and representative sample] Participants were randomly contacted but it is unclear how randomization was done or whether there was a self-selection bias in the type of respondent who agreed to participate (response rates were not provided). WILU 2009 Kloda, Koufoginannakis & Mallan 22
  23. 23. [focused question] The aims of the study were clear: ... WILU 2009 Kloda, Koufoginannakis & Mallan 23
  24. 24. Reliability Results clearly explained (n=7) Response rate (n=8) Useful analysis (n=7) Appropriate analysis (n=10) Results address research questions (n=5) Limitations (n=13) Conclusions based on actual results (n=7) WILU 2009 Kloda, Koufoginannakis & Mallan 24
  25. 25. [appropriate analysis] The study reports several statistically significant results in relation to the research questions, yet the analyses seem misinterpreted. For example, self-efficacy and use of electronic information jointly contributed 9% of the variance of academic performance. A large amount of variance and thus other contributing factors (91%) remain unaccounted. Both the R2 and the adjusted R2 (0.05531) indicate that these data do not represent a good statistical model … WILU 2009 Kloda, Koufoginannakis & Mallan 25
  26. 26. Applicability Implications reported in original study (n=13) Applicability to other populations (n=9) More information required (n=8) WILU 2009 Kloda, Koufoginannakis & Mallan 26
  27. 27. [applicability] … suggests that Web-based tutorials are at least as effective as face to face teaching sessions and that these may be successfully delivered either in the classroom or via the Web. WILU 2009 Kloda, Koufoginannakis & Mallan 27
  28. 28. Other Findings of Note • Commentary length / categories coded • Situated research in wider setting • Significance of research • Literature review WILU 2009 Kloda, Koufoginannakis & Mallan 28
  29. 29. [literature review] What is perhaps more valuable in this paper is the extensive use of the research literature to inform the various ideas throughout. The literature review is robust, and the author includes results from previous studies all though the paper to strengthen his statements and conclusions. WILU 2009 Kloda, Koufoginannakis & Mallan 29
  30. 30. Limitations • Small set of commentaries • Writers have varying styles of writing, appraisal experience • Bias of researchers WILU 2009 Kloda, Koufoginannakis & Mallan 30
  31. 31. Conclusions • Aspects of validity and reliability in studies that were critically appraised in EBLIP were more often noted as weaknesses of the study. Whether this was due to general poor study design or the focus of the writer in trying to point out faults rather than positives, is unknown. • Despite the criticisms of validity and reliability, applicability was still viewed positively – why? WILU 2009 Kloda, Koufoginannakis & Mallan 31
  32. 32. Implications for Practice • Consider aspects of validity, reliability and applicability when you are developing a research study. • Think critically when reading a research article – regardless of where it was published, was it well done and can you apply its findings to your own environment? • Improvements to EBLIP evidence summaries: content/structure of the commentaries. WILU 2009 Kloda, Koufoginannakis & Mallan 32

×