Learning & Teaching Conference, Leicester 2013


    Helping students develop their
         information literacy


                                Dr Chris Willmott
                            Dept of Biochemistry,
                           University of Leicester
                                   cjrw2@le.ac.uk
                                        University of
                                        Leicester
Background
• There was a time, not SO long ago…
• The internet provides an apparently bottomless pool
  of information
• There are concerns that students may not possess
  the skills to appraise the merits of online sources 1,2

                                  • Information literacy
                                  • Digital literacy




http://tinyurl.com/indexmedicus
Activity
• Students are asked to evaluate eight online sources
• One week before group tutorial, sent instructions,
plus:
     • File listing top eight hits on a “Google search”
     • Link to proforma for recording evaluation of the
       sources
• Student complete activity in their own time prior to
  the tutorial
     • Ensures they can work at own pace
     • Enables me to collate cohort response
Activity
Activity
• For each source students are asked to consider:
• The “academic quality” of the source
  • Veracity?
  • Credibility?
  • Would person marking your work think that it is
    an appropriate source document for the purpose?
• 1 (low) to 10 (high)
Activity
• For each source students are asked to consider:
• The “suitability” and “relevance” for two assignments
  • A first year essay on the role of mitochondria in
    energy production
  • A final year dissertation on the role of
    mitochondria in ageing
• For each category 5-point Likert scale offered
• Open-text box to give short explanation of their
  reasoning
Source 1
• Mean student rating*: 7.7
• Tutor rating: 8
• What is it?
  Chapter from Molecular
 Biology of the Cell
• Why is this example included?
  Alerts students to fact some major texts are online
  Raises point that textbooks = good, but not pinnacle




                                    * 2010-12 (n=299)
Source 1
• Suitability for 1st year essay on energy production?
  Definitely Suitable
• Relevance for 1st year essay on energy production?
  Definitely Relevant
• Suitability for Final year dissertation on ageing?
  Not Sure – textbooks unlikely to play major role in
  final year
• Relevance for Final year dissertation on ageing?
  Probably Irrelevant – cursory mention of topic
Source 2
• Mean student rating*: 6.1
• Tutor rating: 4
• What is it?
  Wikipedia
• Why is this example included?
  First port of call for casual research
  Students need to be aware of differences of opinion
  Raising notion of Wikipedia as a portal not endpoint




                                     * 2010-12 (n=299)
Source 2
• Suitability for 1st year essay on energy production?
  Not Sure – Definitely Unsuitable? who authored?
• Relevance for 1st year essay on energy production?
  Probably Relevant – some relevant level content, but…
• Suitability for Final year dissertation on ageing?
  Probably Unsuitable – raise notion of “portal”, “good”
  Wikipedia pages are well referenced
• Relevance for Final year dissertation on ageing?
  Not Sure
Source 3
• Mean student rating*: 5.3
• Tutor rating: 2
• What is it?
  Essay from “papermill”
• Why is this example included?
  Important to raise educational concerns (and quality
  concerns) regarding sub-contraction services
  [Worry – alerting students to their existence?]




                                     * 2010-12 (n=299)
Source 3
• Suitability for 1st year essay on energy production?
  Definitely Unsuitable – academic dishonesty issues
• Relevance for 1st year essay on energy production?
  Not Sure – title almost identical, but can it be trusted?
• Suitability for Final year dissertation on ageing?
  Definitely Unsuitable
• Relevance for Final year dissertation on ageing?
  Definitely Irrelevant
Source 4
• Mean student rating*: 8.3
• Tutor rating: 9
• What is it?
  A short review article in an
  important journal
• Why is this example included?
  An example of a top peer-reviewed journal
  An opportunity to discuss peer review and validation?




                                     * 2010-12 (n=299)
Source 4
• Suitability for 1st year essay on energy production?
  Probably Unsuitable or Definitely Suitable? – tougher
  than expected for students at Yr1. Setting reasonable
  expectations?
• Relevance for 1st year essay on energy production?
  Probably Irrelevant – energy production mentioned,
  but focus is on ageing
• Suitability for Final year dissertation on ageing?
  Definitely Suitable – perfect for job
• Relevance for Final year dissertation on ageing?
  Definitely Relevant
Source 5
• Mean student rating*: 8.7
• Tutor rating: 9
• What is it?
  A research article from a
  top journal
• Why is this example included?
  Raise differences between “primary literature” and
  review articles
  Good point to discuss citation scores?
Source 5
• Suitability for 1st year essay on energy production?
  Definitely Unsuitable
• Relevance for 1st year essay on energy production?
  Definitely Irrelevant
• Suitability for Final year dissertation on ageing?
  Definitely Suitable – perfect for job
• Relevance for Final year dissertation on ageing?
  Definitely Relevant
Source 6
• Mean student rating*: 6.8
• Tutor rating: 6
• What is it?
  Educational website with
  peer-reviewed articles

• Why is this example included?
  A “good” website – run by American Institute of
  Biological Sciences



                                    * 2010-12 (n=299)
Source 6
• Suitability for 1st year essay on energy production?
  Not Sure – claims of “peer review” but likely to be
  sceptically received by markers
• Relevance for 1st year essay on energy production?
  Probably Irrelevant – not main thrust of article
• Suitability for Final year dissertation on ageing?
  Definitely Unsuitable
• Relevance for Final year dissertation on ageing?
  Probably Relevant – topic aligns, but…
Source 7
• Mean student rating*: 4.7
• Tutor rating: 3
• What is it?
  CBS News website
• Why is this example included?
  The title is alluringly close to
  one of the set assignments
  Written by staff writer “Jamie Holguin”, no evidence of
  scientific training



                                     * 2010-12 (n=299)
Source 7
• Suitability for 1st year essay on energy production?
  Probably Unsuitable – an illustration for introduction?
  Not authoritative on matters of scientific fact
• Relevance for 1st year essay on energy production?
  Definitely Irrelevant
• Suitability for Final year dissertation on ageing?
  Probably Unsuitable – as above
• Relevance for Final year dissertation on ageing?
  Not Sure – material is on same topic, but…
Source 8
• Mean student rating*: 5.2
• Tutor rating: 5
• What is it?
  TED talk

• Why is this example included?
  Raising notion that not all sources are written
  (though a transcript is available)




                                      * 2010-12 (n=299)
Source 8
• Suitability for 1st year essay on energy production?
  Probably Unsuitable – videos and podcasts can be
  valuable, probably not this one!
• Relevance for 1st year essay on energy production?
  Probably Irrelevant
• Suitability for Final year dissertation on ageing?
  Probably Unsuitable – as above
• Relevance for Final year dissertation on ageing?
  Probably Relevant – concepts under discussion are
  relevant to dissertation, but concern over authority
Comments at close of tutorial
• Reminder that although internet useful tool for
  accessing fantastic resources, other resources
• Library offers access to books and expertise of staff
Summary of outcomes
Evaluation
• 2010-2012, 169 students completed anonymous
  evaluation of activity
• 91% of students gave positive feedback:
 This was a useful exercise
     Agreed (56%)
     Strongly Agreed (35%)
Evaluation
• “In some instances my opinion and the actual
  answers differed by quite a lot. This activity gave me
  greater insight into how sources are viewed in terms
  of authority at a university level”
• “It really opened my eyes to some aspects I had
    never given a lot of thought to”
• “It made me aware of how important the quality of
  sources is and how to judge this”
•    “It was really useful, as I was a bit apprehensive
    about using the wrong sources in essays.”
• “I didn’t learn much new, but the practice and the
  feedback session helped me to understand better.”
My reflections
• A valuable, if imperfect, exercise
• Terminology: explaining key issues hard to get across
  succinctly a priori
• Works best as blended (online then F2F) activity
• Could be quicker without requesting explanation
 “After about five sources the task got tedious,
 however it was still useful”
Adapting for your course?
• Researching possible sources and shortlisting to about
  eight takes time
• Check that sources are available off campus (and
  expect some tinkering to be required as URLs change)
• Search pages prepared as PowerPoint slides and
  uploaded to connect.le.ac.uk using Adobe Presenter
• Data collection form prepared in Plone
References
1. Beetham H. et al. (2009) Thriving in the 21st
   Century: Learning literacies for the digital age (JISC)
2. Pan B. et al. (2010) Assessing the dynamics of
   search results in Google. Information and
   Communication Technologies in Tourism 11:405-416
Acknowledgements
• Stuart Johnson
• Paul Smith
• Alan Cann & Jo Badge




• Ethical approval cjrw2-7991b
Any Questions?
E-mail:    cjrw2@le.ac.uk
Twitter:   cjrw
Slideshare: cjrw2
Delicious: chriswillmott
Blogs:     www.bioethicsbytes.wordpress.com
           www.biosciencecareers.wordpress.com
           www.lefthandedbiochemist.wordpress.com

                                       University of
                                       Leicester

Helping students develop their information literacy

  • 1.
    Learning & TeachingConference, Leicester 2013 Helping students develop their information literacy Dr Chris Willmott Dept of Biochemistry, University of Leicester cjrw2@le.ac.uk University of Leicester
  • 2.
    Background • There wasa time, not SO long ago… • The internet provides an apparently bottomless pool of information • There are concerns that students may not possess the skills to appraise the merits of online sources 1,2 • Information literacy • Digital literacy http://tinyurl.com/indexmedicus
  • 3.
    Activity • Students areasked to evaluate eight online sources • One week before group tutorial, sent instructions, plus: • File listing top eight hits on a “Google search” • Link to proforma for recording evaluation of the sources • Student complete activity in their own time prior to the tutorial • Ensures they can work at own pace • Enables me to collate cohort response
  • 4.
  • 5.
    Activity • For eachsource students are asked to consider: • The “academic quality” of the source • Veracity? • Credibility? • Would person marking your work think that it is an appropriate source document for the purpose? • 1 (low) to 10 (high)
  • 6.
    Activity • For eachsource students are asked to consider: • The “suitability” and “relevance” for two assignments • A first year essay on the role of mitochondria in energy production • A final year dissertation on the role of mitochondria in ageing • For each category 5-point Likert scale offered • Open-text box to give short explanation of their reasoning
  • 7.
    Source 1 • Meanstudent rating*: 7.7 • Tutor rating: 8 • What is it? Chapter from Molecular Biology of the Cell • Why is this example included? Alerts students to fact some major texts are online Raises point that textbooks = good, but not pinnacle * 2010-12 (n=299)
  • 8.
    Source 1 • Suitabilityfor 1st year essay on energy production? Definitely Suitable • Relevance for 1st year essay on energy production? Definitely Relevant • Suitability for Final year dissertation on ageing? Not Sure – textbooks unlikely to play major role in final year • Relevance for Final year dissertation on ageing? Probably Irrelevant – cursory mention of topic
  • 9.
    Source 2 • Meanstudent rating*: 6.1 • Tutor rating: 4 • What is it? Wikipedia • Why is this example included? First port of call for casual research Students need to be aware of differences of opinion Raising notion of Wikipedia as a portal not endpoint * 2010-12 (n=299)
  • 10.
    Source 2 • Suitabilityfor 1st year essay on energy production? Not Sure – Definitely Unsuitable? who authored? • Relevance for 1st year essay on energy production? Probably Relevant – some relevant level content, but… • Suitability for Final year dissertation on ageing? Probably Unsuitable – raise notion of “portal”, “good” Wikipedia pages are well referenced • Relevance for Final year dissertation on ageing? Not Sure
  • 11.
    Source 3 • Meanstudent rating*: 5.3 • Tutor rating: 2 • What is it? Essay from “papermill” • Why is this example included? Important to raise educational concerns (and quality concerns) regarding sub-contraction services [Worry – alerting students to their existence?] * 2010-12 (n=299)
  • 12.
    Source 3 • Suitabilityfor 1st year essay on energy production? Definitely Unsuitable – academic dishonesty issues • Relevance for 1st year essay on energy production? Not Sure – title almost identical, but can it be trusted? • Suitability for Final year dissertation on ageing? Definitely Unsuitable • Relevance for Final year dissertation on ageing? Definitely Irrelevant
  • 13.
    Source 4 • Meanstudent rating*: 8.3 • Tutor rating: 9 • What is it? A short review article in an important journal • Why is this example included? An example of a top peer-reviewed journal An opportunity to discuss peer review and validation? * 2010-12 (n=299)
  • 14.
    Source 4 • Suitabilityfor 1st year essay on energy production? Probably Unsuitable or Definitely Suitable? – tougher than expected for students at Yr1. Setting reasonable expectations? • Relevance for 1st year essay on energy production? Probably Irrelevant – energy production mentioned, but focus is on ageing • Suitability for Final year dissertation on ageing? Definitely Suitable – perfect for job • Relevance for Final year dissertation on ageing? Definitely Relevant
  • 15.
    Source 5 • Meanstudent rating*: 8.7 • Tutor rating: 9 • What is it? A research article from a top journal • Why is this example included? Raise differences between “primary literature” and review articles Good point to discuss citation scores?
  • 16.
    Source 5 • Suitabilityfor 1st year essay on energy production? Definitely Unsuitable • Relevance for 1st year essay on energy production? Definitely Irrelevant • Suitability for Final year dissertation on ageing? Definitely Suitable – perfect for job • Relevance for Final year dissertation on ageing? Definitely Relevant
  • 17.
    Source 6 • Meanstudent rating*: 6.8 • Tutor rating: 6 • What is it? Educational website with peer-reviewed articles • Why is this example included? A “good” website – run by American Institute of Biological Sciences * 2010-12 (n=299)
  • 18.
    Source 6 • Suitabilityfor 1st year essay on energy production? Not Sure – claims of “peer review” but likely to be sceptically received by markers • Relevance for 1st year essay on energy production? Probably Irrelevant – not main thrust of article • Suitability for Final year dissertation on ageing? Definitely Unsuitable • Relevance for Final year dissertation on ageing? Probably Relevant – topic aligns, but…
  • 19.
    Source 7 • Meanstudent rating*: 4.7 • Tutor rating: 3 • What is it? CBS News website • Why is this example included? The title is alluringly close to one of the set assignments Written by staff writer “Jamie Holguin”, no evidence of scientific training * 2010-12 (n=299)
  • 20.
    Source 7 • Suitabilityfor 1st year essay on energy production? Probably Unsuitable – an illustration for introduction? Not authoritative on matters of scientific fact • Relevance for 1st year essay on energy production? Definitely Irrelevant • Suitability for Final year dissertation on ageing? Probably Unsuitable – as above • Relevance for Final year dissertation on ageing? Not Sure – material is on same topic, but…
  • 21.
    Source 8 • Meanstudent rating*: 5.2 • Tutor rating: 5 • What is it? TED talk • Why is this example included? Raising notion that not all sources are written (though a transcript is available) * 2010-12 (n=299)
  • 22.
    Source 8 • Suitabilityfor 1st year essay on energy production? Probably Unsuitable – videos and podcasts can be valuable, probably not this one! • Relevance for 1st year essay on energy production? Probably Irrelevant • Suitability for Final year dissertation on ageing? Probably Unsuitable – as above • Relevance for Final year dissertation on ageing? Probably Relevant – concepts under discussion are relevant to dissertation, but concern over authority
  • 23.
    Comments at closeof tutorial • Reminder that although internet useful tool for accessing fantastic resources, other resources • Library offers access to books and expertise of staff
  • 24.
  • 25.
    Evaluation • 2010-2012, 169students completed anonymous evaluation of activity • 91% of students gave positive feedback: This was a useful exercise Agreed (56%) Strongly Agreed (35%)
  • 26.
    Evaluation • “In someinstances my opinion and the actual answers differed by quite a lot. This activity gave me greater insight into how sources are viewed in terms of authority at a university level” • “It really opened my eyes to some aspects I had never given a lot of thought to” • “It made me aware of how important the quality of sources is and how to judge this” • “It was really useful, as I was a bit apprehensive about using the wrong sources in essays.” • “I didn’t learn much new, but the practice and the feedback session helped me to understand better.”
  • 27.
    My reflections • Avaluable, if imperfect, exercise • Terminology: explaining key issues hard to get across succinctly a priori • Works best as blended (online then F2F) activity • Could be quicker without requesting explanation “After about five sources the task got tedious, however it was still useful”
  • 28.
    Adapting for yourcourse? • Researching possible sources and shortlisting to about eight takes time • Check that sources are available off campus (and expect some tinkering to be required as URLs change) • Search pages prepared as PowerPoint slides and uploaded to connect.le.ac.uk using Adobe Presenter • Data collection form prepared in Plone
  • 29.
    References 1. Beetham H.et al. (2009) Thriving in the 21st Century: Learning literacies for the digital age (JISC) 2. Pan B. et al. (2010) Assessing the dynamics of search results in Google. Information and Communication Technologies in Tourism 11:405-416
  • 30.
    Acknowledgements • Stuart Johnson •Paul Smith • Alan Cann & Jo Badge • Ethical approval cjrw2-7991b
  • 31.
    Any Questions? E-mail: cjrw2@le.ac.uk Twitter: cjrw Slideshare: cjrw2 Delicious: chriswillmott Blogs: www.bioethicsbytes.wordpress.com www.biosciencecareers.wordpress.com www.lefthandedbiochemist.wordpress.com University of Leicester