Recommended
Recommended
More Related Content
Similar to How students judge the relevance and reliability of information in the digital environment
Similar to How students judge the relevance and reliability of information in the digital environment (20)
Recently uploaded
Recently uploaded (20)
How students judge the relevance and reliability of information in the digital environment
- 1. How students judge the relevance and reliability of information in the digital environment Curtis Watson ALIA National Library & Information Technicians Symposium, 1 November 2013
- 5. Participants Secondary school students undertaking information search tasks • • • • 37 students Between 14 and 17 years of age Years 9 to 11 From a school in south-eastern Australia From: http://us.123rf.com/400wm/400/400/stockbroker/stockbroker0806/stockbroker080604128/3204223-secondary-school-students-ina-school-hallway.jpg
- 6. Data • • • • • Students’ journals Structured and semi-structured interviews Think-aloud reports and video screen captures Video-stimulated recall interviews Questionnaires From: http://www.bgsu.edu/departments/greal/llc/german From: http://www.freewareguy.co m/wpcontent/uploads/2009/11/s From: http://languageartsgames.4you4fr
- 10. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance First result
- 12. ‘the top one [is] easier to click’ ‘with Google, the first one normally is the one you’re looking for’ (Sophie)
- 13. ‘it was the first one on the top of the list’ (Victoria)
- 14. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance First result Snippets
- 15. Sometimes the words are […] like they just have like a ‘Rome’ over here and a ‘dependence’ over there and […] and I just, I don’t go to them, I go to the ones where the words are like what I want and they’re all close together kind of thing. (Karen)
- 16. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance First result Snippets Titles
- 17. ‘if the title, like the blue thing, matches what I’ve typed in, then I just go to it’ (Karen)
- 18. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance First result Snippets Titles URLs
- 19. ‘Quite certain that a composer like him would have his own page ‘ (Paul)
- 20. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance First result Snippets Titles URLs Subjective clues to relevance
- 21. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance First result Snippets Titles URLs Subjective clues to relevance Comprehensibility
- 22. ‘if it was very cluttered and a bit over my head, then I sort of went away from that to a more, to information that I could understand ‘ (Richard)
- 23. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance Subjective clues to relevance First result Comprehensibility Snippets Completeness Titles URLs
- 24. ‘it’s either going to mean I’ve got to buy it or I’m not really going to be able to view much of it’ (Sharon)
- 25. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance Subjective clues to relevance First result Comprehensibility Snippets Completeness Titles Price URLs
- 26. ‘Ah, Amazon, I don’t want to have to, I can’t buy a book’ (Chloe)
- 27. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance Subjective clues to relevance First result Comprehensibility Snippets Completeness Titles Price URLs Utility of video sources
- 28. ‘Generally I just tend to skip videos’ ‘it’s just harder to analyse [videos] and, if I’m doing an assignment, I just want what’s basic, given to me in plain English’ (Stefano)
- 29. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance Subjective clues to relevance First result Comprehensibility Snippets Completeness Titles Price URLs Utility of video sources Fact or opinion
- 30. ‘people’s opinions on analysis stuff generally aren’t very good’ (Gerard)
- 31. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance Post-access criteria of relevance Subjective clues to relevance First result Comprehensibility Snippets Completeness Titles Price URLs Utility of video sources Fact or opinion
- 32. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance Post-access criteria of relevance Subjective clues to relevance First result Comprehensibility Snippets Completeness Titles Price URLs Utility of video sources Fact or opinion Priming
- 33. ‘usually I just go for the basics first, so I’ll get a basic outline of what I’m trying to find out and then I’ll just build on it after a few researching sessions’ (Mary-Ann)
- 34. Chloe ‘I generally start with Wiki, just if, I don’t know, just to give me a general idea’ … ‘Wikipedia, good place to start’ David ‘Wikipedia gave me good background information about both the topics’ Edward ‘It’s [Wikipedia’s] just a starting reference point for me’ Emily ‘I always use it [Wikipedia] to get a summary of the things’ Jenny ‘Wikipedia […] I just go to for a general overview’ Lionel ‘I would tend to use a Wikipedia article, not as the only source but as something you can refer to and you can get a grounding knowledge in’ Mark ‘I just get a base knowledge from Wikipedia […] it just starts me off, ’cause it always has like the start of everything’
- 35. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance Post-access criteria of relevance Subjective clues to relevance Priming Chaining First result Comprehensibility Snippets Completeness Titles Price URLs Utility of video sources Fact or opinion
- 36. Once you have a good site, sometimes you’ll take a piece of information from it, and search it from that, so like the physicist, so maybe I’ll search him from that first site that gave me the information on him. […] What I’d next do is just have a read through, see what else I can find and then search from that new information. (Mary-Ann)
- 37. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance Post-access criteria of relevance Subjective clues to relevance Priming Chaining Depth of topic treatment First result Comprehensibility Snippets Completeness Titles Price URLs Utility of video sources Fact or opinion
- 38. ‘they just went through it more thoroughly’ (Elizabeth) ‘they had the most information in them’ (Sophie)
- 39. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance Post-access criteria of relevance Subjective clues to relevance Priming Chaining Depth of topic treatment First result Comprehensibility Snippets Completeness Titles Price URLs Utility of video sources Fact or opinion Structural clues
- 40. Relevance criteria Pre-access criteria of relevance System-provided metadata clues to relevance Post-access criteria of relevance Subjective clues to relevance Priming Chaining Depth of topic treatment First result Comprehensibility Snippets Completeness Titles Price URLs Utility of video sources Fact or opinion Structural clues Topic sentences First paragraph Headings
- 41. sort of scan-reading like the first sentence of each sort of paragraph and if it was, appeared that it would be helpful, I continued reading the paragraph and stuff’ (Victoria)
- 43. Reliability criteria Pre-access criteria Post-access criteria
- 45. ‘There was just so much and it was a ‘.gov’ site […] so that usually does mean that it is reliable’ (Kristian) ‘weird-looking website but we’ll go to it, it’s got a dot org, I like dot orgs’ (Chloe)
- 47. ‘BBC, very, very reliable’ (Stefano) [Referring to Geoscience Australia’s website] ‘really, really reliable’ (Gerard)
- 48. ‘It’s by the […] Geological Society, so they have a lot of, like, people contributing, so it’s not very biased but it’s got a lot of facts in it, which is good for this, um, I trust it, I do, and they have a very long reference list, which is also nice when you’re looking for things; looking through their references is also very good, so I think that’s a very trustable source.’ (Gerard)
- 50. Reliability criteria Pre-access criteria URL Source reputation Post-access criteria Corroboration
- 51. ‘I’ll normally just go to the, take the information from that, double-check it with, um, newspapers and independent articles that have been written’ (Edward) ‘I compared this information to the basic information I had got from the Wikipedia site’ (David)
- 52. Reliability criteria Pre-access criteria Post-access criteria URL Corroboration Source reputation Design
- 53. ‘they don’t look like there’s much design put into them … so I look at them and I think, well, that’s probably not the best one to choose’ (Richard) ‘I think if it’s too showy and has like too many advertisements, like, I’d kind of stay away, wouldn’t stay away, […] but I’d be, like, “This probably isn’t as, like, dependable as, like, a boring one” ’ (Karen) ‘[a] bit gimmicky but not too bad’ (Edward)
- 55. Reliability criteria Pre-access criteria Post-access criteria URL Corroboration Source reputation Design Writing style
- 56. ‘if it was poorly written, I probably wouldn’t bother with them, even, no matter how good the, um, author is’ (Edward) ‘it sounded like a really good argument … he actually had evidence ’n’ discussed it in depth and stuff’ (Karen)
- 57. Reliability criteria Pre-access criteria Post-access criteria URL Corroboration Source reputation Design Writing style Authorship
- 58. ‘I don’t usually check for the authors’ (Karen) ‘where the information comes from, it’ll usually say on the web page, if it’s come from a specific person, um, it might be a person who’s well-known or a person that’s, um, you know, properly trained, like a professor or something particular’ (Charlotte) ‘if it says like a doctor wrote it or something, […] it needs to actually have proof that it’s a really good, you know, piece of work’ (Michael)
- 59. Convenient or pragmatic approach First result Dependence on search engine's relevance clues Decreasing relevance beyond first page of results Easy access to or abundance of information sources Unconscious or intuitive response Pre-access judgements of relevance and reliability
- 62. Relevance chaining Information gathering Prior knowledge Source-tosource chaining Origin of source
- 63. Building knowledge Filtering Metadata clues to relevance Depth of topic treatment Structural clues Design Matching Reputation of source URL Writing style Topic Adding Matching material to topic Comparing information Corroboration Inclusion as relevant Continued engagement Further information processing
- 64. Convenient or pragmatic approach First result Dependence on search engine's relevance clues Decreasing relevance beyond first page of results Easy access to or abundance of information sources Relevance priming Unconscious or intuitive response Pre-access judgements Topic overview of relevance and reliability Socio-academic context Topic choice/ motivation Author credentials Reputation of resource's reliability Reliability of books Relevance chaining Building knowledge Filtering Metadata clues to relevance Depth of topic treatment Structural clues Design Matching Reputation of source URL Writing style Topic Information gathering Adding Matching material to topic Comparing information Corroboration Inclusion as relevant Continued engagement Further information processing Prior knowledge Source-tosource chaining Origin of source
- 65. Theoretical statement 1 Convenience and pragmatism • convenient and pragmatic approach to information search tasks
- 66. Theoretical statement 1 Convenience and pragmatism • convenient and pragmatic approach to information search tasks • research process unchallenging
- 67. Theoretical statement 1 Convenience and pragmatism • convenient and pragmatic approach to information search tasks • research process unchallenging • least challenging path to task completion
- 68. Theoretical statement 1 Convenience and pragmatism • • • • convenient and pragmatic approach to information search tasks research process unchallenging least challenging path to task completion depend on relevance rankings of search engines, favouring results that occur early in a set, and do not demonstrate any understanding of the methods used by the search engine
- 69. Theoretical statement 1 Convenience and pragmatism • • • • convenient and pragmatic approach to information search tasks research process unchallenging least challenging path to task completion depend on relevance rankings of search engines, favouring results that occur early in a set, and do not demonstrate any understanding of the methods used by the search engine • heuristics save the effort of accessing the full resource
- 70. Theoretical statement 1 Convenience and pragmatism • • • • convenient and pragmatic approach to information search tasks research process unchallenging least challenging path to task completion depend on relevance rankings of search engines, favouring results that occur early in a set, and do not demonstrate any understanding of the methods used by the search engine • heuristics save the effort of accessing the full resource • intuitive judgements
- 71. Theoretical statement 1 Convenience and pragmatism • • • • convenient and pragmatic approach to information search tasks research process unchallenging least challenging path to task completion depend on relevance rankings of search engines, favouring results that occur early in a set, and do not demonstrate any understanding of the methods used by the search engine • heuristics save the effort of accessing the full resource • intuitive judgements • prevalence of information for relevance judgements and deciding topic’s main points
- 72. Theoretical statement 2 Search for an overview • strongly motivated to find overview of topic of interest
- 73. Theoretical statement 2 Search for an overview • strongly motivated to find overview of topic of interest • provides framework for learning from information sources found later
- 74. Theoretical statement 2 Search for an overview • strongly motivated to find overview of topic of interest • provides framework for learning from information sources found later • early adoption of information at hand may prevent deeper/broader exploration of topic
- 75. Theoretical statement 3 Incidental nature of establishing reliability • information accords with that found in earlier sources
- 76. Theoretical statement 3 Incidental nature of establishing reliability • information accords with that found in earlier sources • students’ perceptions that teachers consider Wikipedia unreliable
- 77. Theoretical statement 3 Incidental nature of establishing reliability • information accords with that found in earlier sources • students’ perceptions that teachers consider Wikipedia unreliable • perceptions do not prevent use because provides overview of topics
- 78. Theoretical statement 4 Prior knowledge • two forms: information remembered from an earlier experience preconceptions of ability of genre, or information provider, to offer relevant and reliable information
- 79. Theoretical statement 4 Prior knowledge • two forms: information remembered from an earlier experience preconceptions of ability of genre, or information provider, to offer relevant and reliable information • reputation of a source’s reliability based on a generalised personal belief (outcome of socialisation or personal experience?)
- 80. Theoretical statement 5 Process of building knowledge • filtering of information occurs in process of relevance judgements
- 81. Theoretical statement 5 Process of building knowledge • filtering of information occurs in process of relevance judgements • rare explicit judgements of reliability of information; depend on impressions of trustworthiness
- 82. Theoretical statement 5 Process of building knowledge • filtering of information occurs in process of relevance judgements • rare explicit judgements of reliability of information; depend on impressions of trustworthiness • matching employs judgements of relevance and reliability simultaneously
- 83. Theoretical statement 5 Process of building knowledge • filtering of information occurs in process of relevance judgements • rare explicit judgements of reliability of information; depend on impressions of trustworthiness • matching employs judgements of relevance and reliability simultaneously • adding to finished product mostly occurs with little apparent discrimination
- 84. Implications for practice • checklist of features to gauge reliability of information not consonant with naturalistic decision making • information adopted because of its early appearance ought to be questioned in later stages of the ISP • prior knowledge should be harnessed more explicitly in the learning process • background knowledge should be developed from more than one source. From: http://mhpbooks.com/mobylives/wp-content/uploads/2010/07/ebooks1.jpg
- 86. Follow-up study?
- 87. Follow-up study? I use Wikipedia to give me an introduction to my topic. Wikipedia is a reliable source. When I find information in one source, I check that information with information in another source. I trust websites that have .org in their URLs.