Critical Reading and Thinking Skills
•Download as PPTX, PDF•
16 likes•7,664 views
The document provides an overview of critical reading, critical thinking, and evaluating research information. It discusses what critical reading and thinking entail, and how the ecology of resources and cognitive biases can influence researchers. It emphasizes the need for researchers to critically evaluate information by considering relevance, authority, objectivity, methodology, and other factors. The document uses examples to illustrate cognitive biases and strategies for critically analyzing sources, noting researchers must consider what may be filtering their evaluations.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Critical Reading and Thinking Skills
Similar to Critical Reading and Thinking Skills (20)
More from Jamie Bisset
More from Jamie Bisset (20)
Recently uploaded
Recently uploaded (20)
Critical Reading and Thinking Skills
- 1. Critical Evaluation Critical Reading Critical Thinking James Bisset (james.bisset@uwe.ac.uk ) Academic Liaison Librarian (Research Support)
- 2. Session outline - What is Critical Reading / Critical Thinking - Ecology of Resources - Cognitive bias and you, the researcher - Evaluating Research Information
- 5. The non-critical reader - Reads a text as a source for... - memorising facts & statements - repeating facts & statements - building a narrative around facts & statements without analysing validity, reliability or applicability
- 6. The critical reader - Reads a text as a... - One interpretation of facts - Recognises the importance of... - what a text says - how the text evidences and portrays the subject matter
- 7. Critical Reading “ Critical Reading involves understanding the content of a text as well as how the subject matter is developed. Critical reading takes in the facts, but goes further. “ http://www.rimt.edu.au/studyandlearningcentre/
- 9. Critical Thinking “Critical thinking involves reflecting on the validity of what you have read in light of our prior knowledge and understanding of the world.“ http://www.criticalreading.com/critical_reading_thinking.htm
- 10. • Much training is about directing you to the right information = searching and retrieval • As postgraduate researchers you have to be critical and reflect on what you find. • Be aware of your impact on your own research, and the research of others. • What defines your evaluative criteria? The need to evaluate information
- 11. • Resources are interconnected and they evolve • Information resources are transformed into knowledge • Knowledge becomes a resource • Therefore prior knowledge shapes what we go on to create Ecology
- 12. • Resources are interconnected and they evolve • Information resources are transformed into knowledge • Knowledge becomes a resource • Therefore prior knowledge shapes what we go on to create Ecology
- 13. • In theory we can select almost any information to complete a task • In practice we filter it by selecting resources we think most appropriate • Motivation - affected by the learning we have already done Role of the
- 14. Role of the • In theory we can select almost any information to complete a task • In practice we filter it by selecting resources we think most appropriate • Motivation - affected by the learning we have already done
- 15. Other factors • But, filtering is done for us BEFORE we get the chance to make a judgement • People • Technologies • Cost • Skills • Copyright, IP
- 16. Other factors • But, filtering is done for us BEFORE we get the chance to make a judgement • People • Technologies • Cost • Skills • Copyright, IP
- 17. Other factors • But, filtering is done for us BEFORE we get the chance to make a judgement • People • Technologies • Cost • Skills • Copyright, IP
- 18. Other factors • But, filtering is done for us BEFORE we get the chance to make a judgement • People • Technologies • Cost • Skills • Copyright, IP
- 19. Other factors • But, filtering is done for us BEFORE we get the chance to make a judgement • People • Technologies • Cost • Skills • Copyright, IP
- 20. • You need to be critical and reflect on all of the sources you find and use. • You have a professional responsibility, as your research will impact on others. • You are creating knowledge… … which evolves, and will shape what others create… … similarly, the information you discover will shape the knowledge you create. • You need to be aware of the filters already impacting upon the information you use. Part 1 Summary
- 23. Cognitive biases • Subjectivity is vulnerable to bias & hunches • Concept of cognitive bias was developed in 1970s by Tversky and Kahneman • Four main groups - Social - Probability/belief - Memory - Decision making
- 24. Social biases • Ascribe positive or negative traits to self, individuals or groups • Loading values or anticipating action based on prior experience or a bias against self, individuals or groups • Academic impact: need to verify information and not rely on own views; important to remember when analysing human subjects
- 27. Memory biases • How you perceive past events • False memory, positive memory, imbalanced memory • E.g. A Photo, a Suggestion, a False Memory • Academic impact: importance of accurate record keeping and note taking
- 28. Memory biases
- 29. Memory biases
- 30. Probability and belief • To disregard or to pay too much attention to probability • Academic impact: need to treat each research finding as distinct and to judge it in its own right
- 33. Decision-making biases • Influences on your decisions by own biases or those of a group • Academic impact: need to be objective and consider all possible routes of enquiry and treat all research findings as valid until proved otherwise e.g. Semmelweis reflex
- 34. Cognitive biases • On your table, group the forty cards into four piles of ten • Social • Memory • Probability • Decision
- 35. • Explored some of the key bias which may subconsciously be impacting upon how your (subjective) measure of „value‟ might effect how you filter information and your evaluative criteria.# Part 2 Summary
- 37. Evaluating information In a literature review you need to evaluate: • Relevance to the topic • Authority of the author, publisher etc • Objectivity • Presentation • Methodology • Currency
- 38. Evaluating information In a literature review you need to evaluate: • Relevance to the topic • Authority of the author, publisher etc • Objectivity • Presentation • Methodology • Currency
- 39. • Before reading the text… • Read the abstract, introduction or summary. • Scan the bibliographic information which may highlight key subject areas not specifically alluded to. • Emphasis may not be clear until you read in full. Relevance to the topic
- 41. • Upon reading the text… • What level is the information at? • Does it contain, and discuss in enough detail the information you are seeking? • Is the research relevant to the subject domain / geographical area / demographic / time period you are interested in? • Three ways of reading: Comprehension, Analysis, Interpretation Relevance to the topic
- 42. “In the course of a series of investigations into various aspects of mental inheritance an intensive study has been made of so-called „identical‟ twins. The cases examined fall into two main groups: first, those reared together in their parents' homes; secondly, those separated in early infancy, and brought up apart. With the latter, despite wide differences in environmental conditions, the correlations for intelligence, unlike those for school attainments, prove to be surprisingly high. It is argued that this implies that „intelligence‟, when adequately assessed, is largely dependent on genetic constitution.” http://10.1111/j.2044-8295.1966.tb01014.x “ Burt‟s study of monozygotic twins reared apart … involved the largest number of separated twin pairs at the time and produced the highest estimate of heritability for IQ” Relevance to the topic
- 43. Comprehension: Burke identified a link between IQ and inherited genes. Analysis: This article compares the evidence for IQ being determined by inherited genes as opposed to IQ being affected by external environmental factors. Interpretation: Evidence for IQ being an inherited trait rather than affected by external factors has potential implications for the development of social and education policy. Relevance to the topic
- 44. • Key topics and ideas • Level of information presented • Relevance in terms of location/subject/scope. • Three ways of reading: Comprehension, Analysis, Interpretation Relevance to the topic Be aware of what is filtering your choices… - Vocabulary and broadness of interpretation. Are you under- estimating the value of a source because it doesn‟t match your choice of keywords precisely?
- 45. “The trouble with quotes from the internet is that you never know if they are genuine.” Abraham Lincoln (1809-1865, President of the United States of America). Authority
- 46. • Are the authors acknowledged experts in the field? - frequently cited? do they have an h-index? - have you or colleagues heard of them? - can you find any profile information where they work? - how well respected is the author, and their work, in their related field of research? • Where is it published? - impact factors for a journal (not always an accurate measure of quality, but potentially one of prestige) - is it peer reviewed? Authority
- 47. • Sir Cyril Burt • Fellow of British Academy • Author of over 350 articles and a number of books. • “pioneer research on the inheritance of mental ability” Authority
- 49. ””the most satisfactory attempt” to estimate hereditability of IQ” and “”the most valuable” of all the separated twin research.” ”the largest of its kind and the only one where “the distribution of children into foster homes was random ” “ Richard J Herrnstein Arthur R Jenson
- 50. ””the most satisfactory attempt” to estimate hereditability of IQ” and “”the most valuable” of all the separated twin research.” • Professor at the University of California, Berkeley. • Author of 400+ peer reviewed papers. • In 2002, was listed in the Review of General Psychology‟s top 50 “most eminent psychologists of the 20th century” ”the largest of its kind and the only one where “the distribution of children into foster homes was random ” “ • Edge Pierce Professor of Psychology at Harvard University. • Author of several peer reviewed papers and books. • Credited with discovering and developing several models and theories as one of the founding researchers in the field of quantitative analysis of behaviour. Richard J Herrnstein Arthur R Jenson
- 51. ”the only one of its kind in which the calculation of heritability had any meaning.” ””the best data“ on separated twins.” William B Shockley Hans Eysenck
- 52. ”the only one of its kind in which the calculation of heritability had any meaning.” • Professor of Psychology at the Institute of Psychiatry, King‟s College, London. • In 2002, was listed in the Review of General Psychology‟s top 100 “most eminent psychologists of the 20th century” as the most cited living psychologist at the time of his death. ””the best data“ on separated twins.” • Alexander M. Poniatoff Professor of Engineering and Applied Science at Stanford University. • Joint awarded Nobel Prize in Physics in 1956. • 1960‟s and 1970‟s moved also into area of hereditary behaviour. William B Shockley Hans Eysenck
- 53. - British Journal of Mathematical and Statistical Psychology - Impact Factor 1.258 - 5th of 13 journals in category “Psychology, Mathematical” - British Journal of Educational Psychology - Impact Factor 2.093 - 11th of 50 journals in category “Psychology, Educational” - British Journal of Psychology - Impact Factor 2.103 - 26th of 126 journals in category “Psychology, Multi-disciplinary” Authority – Impact Factors
- 54. • Are the authors acknowledged experts in the field? - frequently cited? do they have an h-index? - have you or colleagues heard of them? - can you find any profile information where they work? - how well respected is the author, and their work, in their related field of research? • Where is it published? - Impact factors for a journal (not always an accurate measure of quality, but potentially one of prestige) and is it peer reviewed? Authority Be aware of what is filtering your choices y - Is the prestige of the author or the publication impacting on how you evaluate the content?
- 55. • Is the subject controversial? • What is the authors purpose in writing the paper? • If there are differing views on the subject area, does the author consistently fall into one „camp‟? Objectivity
- 56. ** This is an over-simplification…. ** Objectivity IQ is inherited IQ is affected by external factors Burt, C (1943) “Ability and Income” British Journal of Educational Psychology Burt, C.L. (1957) “Heredity and Intelligence; A reply to criticisms” British Journal of Statistical Psychology Burt, C.L. (1958). "The inheritance of mental ability", American Psychologist, Burt, C.L. (1972). "Inheritance of general intelligence", American Psychologist, Burt C (1966) “The Genetic Determination of Differences in Intelligence: A Study of Monozygotic Twins Reared Apart and Together.” British Journal of Psychology
- 57. • Is the subject controversial? • What is the authors purpose in writing the paper? • If there are differing views on the subject area, does the author consistently fall into one „camp‟? Objectivity Be aware of what is filtering your choices - Does the author demonstrate any hidden bias on the topic? - Evaluate yourself? Are you subconsciously over-valuing the resource because it confirms your own prejudices? Are you being objective?
- 58. • are the methodology and sources of data used clearly identified or explained? • what evidence is presented to support the ideas and conclusions expressed? • are the conclusions and assumptions made by the author consistent, logical and justified? • does the research raise any unanswered questions? Methodology
- 59. “ tests of the usual type… as a means of estimating genotypic differences, even the most carefully constructed tests are highly fallible instruments, and … their verdicts are far less trustworthy than the judgments of the pupil‟s own teachers.” (on measuring IQ of subject children] Methodology
- 60. “ tests of the usual type… as a means of estimating genotypic differences, even the most carefully constructed tests are highly fallible instruments, and … their verdicts are far less trustworthy than the judgments of the pupil‟s own teachers.” Burt, C. (1957)British Journal of Statistical Psychology “the unaided judgments even of the most experienced teachers … are nevertheless far less trustworthy in the long run that the results obtained by properly applied intelligence tests.” Burt, C. (1943)British Journal of Educational Psychology Methodology
- 61. “ tests of the usual type… as a means of estimating genotypic differences, even the most carefully constructed tests are highly fallible instruments, and … their verdicts are far less trustworthy than the judgments of the pupil‟s own teachers.” Burt, C. (1957)British Journal of Statistical Psychology “the unaided judgments even of the most experienced teachers … are nevertheless far less trustworthy in the long run that the results obtained by properly applied intelligence tests.” Burt, C. (1943)British Journal of Educational Psychology Methodology
- 63. • Various criteria you can assess a resource by. - a lot more „citation‟ tools available for journal literature. • How much time do you realistically have? Part 3 Summary
- 64. • All quotes and opinions were taken from one article: Tucker, W.H.(1994) “Fact and fiction in the discoevry of Sir Cyril Burt‟s Flaws” Journal of the History of the Behavioral Sciences (30). • Does that change your opinion on some of the previous slides assertions and emphasis? Part 3 Summary
- 65. Bibliography • Kahneman, D. and Amos, A. (1972) „Subjective probability: a judgment of representativeness‟, Cognitive Psychology. 3(3): 430-454. • Luckin, R. (2010) Redesigning learning contexts: technology-rich, learner-centred ecologies. Abingdon: Routledge. • Strange, D., Hayne, H. and Garry, M. (2008) ‟A photo, a suggestion, a false memory‟, Applied Cognitive Psychology.22: 587–603. • Whitworth, D (2010) “The three domains of value: Why IL practitioners must take a holistic approach” Available at: http://prezi.com/rxqnzpoooolb/the-three-domains-of-value-why-il- practitioners-must-take-a-holistic-approach/ • Tucker, W.H.(1994) “Fact and fiction in the discoevry of Sir Cyril Burt‟s Flaws” Journal of the History of the Behavioral Sciences (30). • http://www.informationliteracy.ie/
- 66. Image Credits [Slide 11, 12] Via Flickr Creative Commons, by Martin LaBar. Original available at http://www.flickr.com/photos/32454422@N00/163107859/ [Slide 3, 8, 21, 36] Via Flickr Creative Commons, by Kevin Dooley. Original available at http://www.flickr.com/photos/12836528@N00/2577006675 [Slide 14] Via Flickr Creative Commons, by shellorz. Original available at http://www.flickr.com/photos/59198719@N00/2192821345 [Slide 17] Via Flickr Creative Commons, by Richard Cocks. Original available at http://www.flickr.com/photos/richardland/3999234316/ [Slide 16] Via Flickr Creative Commons, by Photo Extremist. Original available at http://www.flickr.com/photos/thevlue/4839060646/ [Slide 15] Via Flickr Creative Commons, by What Dave Sees. Original available at http://www.flickr.com/photos/whatdavesees/2487875504/
- 67. Image Credits Picture of Professor Arthur Jensen at the 2002 ISIR meeting. Author: Tim Bates. Available at http://en.wikipedia.org/wiki/File:Arthur_Jensen_Vanderbilt_2002.jpg Picture of Hans Eysenck. Author: Sirswindon. Available at https://en.wikipedia.org/wiki/File:Hans.Eysenck.jpg Picture of William B Shockley. Author Chuck Painter / Stanford News Service. Available at http://en.wikipedia.org/wiki/File:William_Shockley,_Stanford_University.jpg [Slide 18] Via Flickr Creative Commons, by vl8189. Original available at http://www.flickr.com/photos/27630470@N03/ [Slide 19] Via Flickr Creative Commons, by opensourceway.Original available at http://www.flickr.com/photos/47691521@N07/4371001458/ [Slide 22] From tdifh.blogspot.com [Slide 28-29] Provided by colleague [Slide 68] „Vitae®, © 2010 Careers Research and Advisory Centre (CRAC) Limited„ Available at www.vitae.ac.uk/rdf
- 68. Measuring Researcher Development Vitae Researcher Development Framework [see image credits]
Editor's Notes
- The focus of this session is how you/we EVALUATE and RANK materials we locate, and how this is often a more complex process than we often appreciate… there are subtle processes going on, you are not just an empty vessel into which information is poured and knowledge is poured out.This is not a session looking at how to avoid bias, but to enable you to recognise it and to be prepared to incorporate it in to your research methodology.It is about taking a step backwards, being aware of the biases in play so you can acknowledge them in your work, especially in any viva or the equivalent. It is about YOUR role and responsibility as a professional researcher, and being able to state your awareness of it.The first half of the session is divided into two parts: Firstly, looking briefly at the importance of evaluation within the context of the research environment. Secondly, looking at ‘forms of value’ in the research context – a Marxist concept looking at how individuals or groups establish what as commodity is worth.
- Image – self awareness, reflection on your role as a researcher.
- Validity: do they come from an authoritive source relevant to the field of knowledgeReliability: are the facts and statements accurate and backed up with evidenceApplicability: does the reader apply the facts and statements appropriately
- Image – self awareness, reflection on your role as a researcher.
- Para 1: Just as a point of reference, for many (not all) first year undergraduates when we say ‘research’, by and large they “just find it”Para 2: But as a postgraduate researcher you must ‘find it’. You must ‘find all of it’ and then you must ‘read it’, ‘evaluate it’ and ‘synthesise it’.Para 3: The importance of critically evaluating information to you as a professional researcher is two-fold. - identifying and appropriate use of information, data and methodologies - awareness of your impact on both your own research, but also the research of others.Para 4: Focus to start is about what defines YOUR evaluative criteria, about an awareness of your role as a researcher in the evaluative process and less about developing a checklist of how to evaluate. - although we will look at this at the end of the session.
- Being aware of your impact as a researcher is important when you consider what Rosemary Luckin, Professor of Learner Centred Design at the University of Sussex, defined as ‘the ecology of resources’. (reference in bibliography) - knowledge is not static - it is constantly changing and evolvingInformation resources are connected to a body of work linked by references and citations… - as we use them… - … we transform them into new knowledge… - … and this impacts upon and shapes what we, and others, go on to create.
- Prior knowledge shapes what we go on to find, how we interpret it and what we create.The chronology of what we learn impacts on our and other future decisions… what we read and where to search.How we evaluate information is the same, not a distinct clinical process. Our evaluation of information is subject to our own and others personal bias.Diagram… looking at the ‘original acorn’, that key text in any area of research. - You find it, and it moves you. - You think about it. You link it to other information and knowledge. And it grows, and changes. You shape it into a new form (the tree)… - others do the same, and it grows new branches… - others also find your work, and develop from your knowledge… and new stems grow - all of these branches and leaves grow and are shaped by us
- Para 1: In theory we can select almost any information to complete a task.Para 2: In practice we filter it by selecting the resources we think are most appropriate.But are we always conscious of how, and why, we filter the resources which shape the knowledge we create? - would a researcher in the same discipline but from a geographic location, or with a different social, cultural or educational background select the same resources, and use them in the same way? - how have the authors of sources, or subjects of our research, filtered information to shape their knowledge and understanding?Its not just about the socio-cultural factors we bring in to our research, or which effect us, but also those we inherit from them.
- Para 3: In turn, motivation impacts upon what we find and how we use it. - can we be bothered to find it? - what about the authors we refer to? Could they? Did they search thoroughly and effectively? How did they filter that informatio they found? - What has influenced your choice of research area, your supervisor and work-style? How will that affect the information you look for, find and use?Image: how far do you sit back and accept what you find as the limits as to how far you will look?
- Even before we make an active choice to filter information, this is already done for us...People – our family, other academics, friends,our supervisor... What do they recommend? Has your supervisor steered you away from any resources or information and towards something else? Could it be sometimes because the direction you might have been heading was slightly out of their comfort zone?
- Technologies - how many of you will find the time to use all of the technologies and tools available to you? - how many of you have tried some of our databases that you aren’t already familiar with or are introduced to in a session or by a colleague? - how many of you will make do with Google Scholar – Google Scholar will not find everything.Also... - if you search on Google you will get different results to yahoo - if two of you search for the same keywords on Google, you won’t necessarily get the same results (location, search history, recently visited sites, browser version)And finally... - you are now the most information rich generation ever... But most of us access this through a 16” screen, maybe three paragraphs at a time. Does this limited window affect how we access, read, interpret and evaluate information?
- Cost - many of the resources you access whilst at Durham are subscriptions we pay for you to access. - how much is your access or ability to search affected by the cost of resources, or even the cost of reaching resources if they are hidden in archives or private collections, need translating or restoring?
- Skills – only as good as your search skills / language skills / social networking skills
- CopyrightWhat can you not access, or use, because of copyright protection?Growth of the open access industry to allow access without restriction by cost – but also to allow use and re-use of both data and created knowledge. But there is still a long way to go.
- In summary.... - you need to be critical and reflect on what you find. - you have a responsibility to do so. - but you must remember that you are using and creating knowledge from existing information which is not static... It has been created by others... And they will have applied their own evaluation process, will have filtered information and will have had different factors impacting upon them which has filtered their access to information. How will that have affected the knowledge you use and create?
- Image – self awareness, reflection on your role as a researcher.
- Purpose of this slide – idea of taking in small amounts of information, our mind naturally filters information to avoid cognitive overload. - our mind is making all sorts of decisions and ‘corrections’ to what we see, think and understand all the time. The reason for a team strip is to make it easy for members to pick out a teammate on the pitch, without having to look for other less obvious details (hair, skin colour, height, face)... But Karl Power took this a little further.Example – Ask if anyone knows what is wrong with this picture ANSWER:12 footballers not 11 Karl Power ran out from the crowd and joined the line up unquestioned by officials and players (tho perhaps Roy Keane has spotted something is up). [against Bayern Munich in Munich – perhaps why officials didn’t stop him]. Karl Power exploits problems of cognitive load. How did he get away with it? Because he wearing the kit – if he had been in his own clothes he would have been stopped. This is why you wear the kit on a pitch – haven’t got time to look at faces – in a high pressure and high speed environment you brain has to prioritise and it can’t cope with looking at detail.“We look for patterns and we look for information which confirms what we want to hear or what we expect to hear.” (DW)
- We, as professionals and researchers are affected by bias... And so are those we may be researching or speaking to as part of our research.Tversky and Kahneman, professors at Stanford and Princeton respectively in the 70’s, demonstrated ways in which our own judgments and decisions differ from rational choices. - arises as a natural mechanism to prevent cognitive overload, eg when asking directions.Has been suggested we are subject to over 200 attempts to influence our opinions each day (adverts, news, newspapers, conversations etc.)I’m just going to quickly summarise a few examples of each of these four main groups... Then there is a short exercise for you to try and match examples from the literature to their respective groups...
- Some examples...Forer effect:- people pick up on little snippets of what is otherwise general or vague information, and mis-apply it to themselves or another over-valuing its accuracy. For example horoscopes...Herd instinct... Following the majority decision without question.Does this happen in the academic community?
- Halo effectHalo effect... See following screenshots. Example from a few years ago of fraudulent research that successfully duped the research establishment and caught media attention . Leading South Korean stem cell research scientist, Dr Hwang Woo-suk, who was found to have "faked all his research on cloned human stem cells". (2006. Cloner's Disgrace. New Scientist. Jan 10. p.4 )Highlighted section demonstrates that he had a number of influences – money and reputation.
- Shock wasn’t so much that he had been fraudulent, but that he had got through the review process seemingly because his research wasnt’ questionned as thoroughly as it should have been, because of his esteem in the related field of cloning where he had been involved in the first successful cloning of a dog embryo.
- Strange, Hayne and Garry conducted research in 2008 on false memories. Showed half a group of children doctored pictures of themselves in a hot air balloon and then later on, questioned them over if they had had a ride in a hot air balloon, and asked them to describe it.And over half of them did, and expanded on the memory... And were also when asked very confident that it had actually happened.
- Childhood memories are often sunny... My summer holiday memories certainly are. But i lived in the north east of england, and my family holidayed in Scotland or the Lake District, not necessarily known for their sun, so can that really be true?
- Gambler’s fallacy – future possibilities are affected by past events...Just because 17 research studies form one conclusion, and then a new study finds differently... Can you then just discount the anomally without evaluating it, and the previous 17 studies, further?Before we talk about the last example of cognitive bias two examples
- How much larger is the yellow circle on the right of the screen compared to the yellow circle on the left of the screen?1) The Ebbinghaus illusion is an optical illusion of relative size perception. In the best-known version of the illusion, two circles of identical size are placed near to each other and one is surrounded by large circles while the other is surrounded by small circles; the first central circle then appears smaller than the second central circle.
- Which of the grey columns is darker?2) White’s illusion is an optical illusion illustrating the fact that the same target luminance can elicit different perceptions of brightness in different contexts. Note, that although the grey rectangles are all of equal luminance, the ones seen in the context with the dark stripes appear brighter than the ones seen in the context with the bright stripes.
- Previous two slides metaphors... As we saw, the circles were the smae size / the grey bars were the same colour. But if we don’t evaluate something based upon its own merit, and allow our decision to be overly affected by other factors, then we may come to an imperfect conclusionhttp://chronicle.com/article/The-Undue-Weight-of-Truth-on/130704/ Haymarket riot and trial of 1886. (Timothy Messer-Cruse - Professor in the School of Cultural and Critical Studies at Bowling Green State University.)Academic, new research, identified earlier research had falsely claimed (in a case of deliberate bias) that there had been no evidence presented at the trial… when in fact over 100 witnesses had been called and gave evidence. He was the leading expert in the field… but was barred from having his correction added to Wikipedia because:-He could not cite reliable sources (his own work and the primary sources were not being considered ‘reliable’ as they were a ‘minority view’, because most work published up to that point had originated with that one source of information)Because each of the other sources had not questioned the original report, they created a body of knowledge that was given almost unquestionned value.Qualifier – he did, as a professor and an academic not approach the wikipedia community on its own terms, and expected it to conform to his own academic credentials. I am not arguing the right or wrong of how the disagreement with Wikiepedia developed, but this is a good example of how one piece of information was taken unquestionningly, and re-used repeatedly without any objective treatment of the research findings or motives of the original author, and created a body of knowledge whose ‘value’ had very shaky foundations.
- After 1 hour (allow 15mins)At end: some are more difficult to assign than others, and may easily fall under more than one heading. Task was more to get you to think and discuss different bias which may or may not affect you and your evaluative criteria.
- Image – pause and reflect on what might be affecting any practical decisions and actions you might take...
- Worksheets handout – an example of different things to look at to inform any ‘value judgment’ you make on an information resource.Practical tips handout – highlight some tools to aid in this, and examples of those criteria mentioned in the worksheet.
- Just going to highlight some tips for some of these…Mention going to use one example as a case study for some of these (authority, objectivity, methodology)
- See handout (images from handout)
- Three ways of reading a passage: Comprehension / Analysis / Interpretation
- Three ways of reading a passage: Comprehension / Analysis / Interpretation
- Quick intro – need to think carefully about the provenance, where you have found the information, is the source reliable.
- Bit of a fudge as these are impact factors for the most recent years data, not from 1970 which is before the JCRs were published.Number of citations can be misleading – these may include positive and negative citations. But gives you a controlled list of references to check to see how others might have evaluated the research.But to give an idea of one measure of ‘authority’ of where the paper has been published, these journals were some of the key titles (or inherited titles) the author published in.
- Highlight the expertise and standing of these commentators, ask class if this would lead to them tending to give the paper a higher or lower value as a source of information?Looking back at cognitive bias…Halo effect? - applying the respect earned by the author for earlier work generally to all of their researchBandwagon effect? - other experts in the field agreed / praised the research.Confirmation bias? - was the research given positive treatment because it confirmed the prejudices or beliefs of others?
- Bit of a fudge as these are impact factors for the most recent years data, not from 1970 which is before the JCRs were published. But to give an idea of one measure of ‘authority’ of where the paper has been published, these journals were some of the key titles (or inherited titles) the author published in.Highlight the “impact factor” of journals the research appeared in, ask class if this would lead to them tending to give the paper a higher or lower value as a source of information?Looking back at cognitive bias…Stereotyping/Ingroup bias? - assumption that because an article is in this journal, it must have the same value as other articles.Halo effect? - ascribing the characteristics of other articles to this paper.Confirmation bias? - was the research given positive treatment because it confirmed the prejudices or beliefs of others?
- Demo H-index and JCRs, comparing a journal within its subject field in a year an article was published. JCRs for journalH-index for author (mention comparison to other authors)H-index of journal, for the same year as article interested in as a comparison to the number of citations received by the article.Remind them – Impact factors are based on number of citations to the publication. The number of citations should not be assumed to indicate a level of quality. But, as many researchers still cling to these as a measure of prestige (wanting to publish in “high impact” journals), it is some measure of ‘perceived level of interest it may generate for the subject community’ at the point of publication.H-index is also a measure of the “no. of citations” and does not provide a clear positive or negative value.
- Authors purpose:- - does the author have a specific message they wish to report - has a contemporary issue, ‘viewpoint’ or policy influenced the authors purpose? - for whom is the material intended?
- Question for class: Does the author consistently fall on to one side of a particular discussion. What other information would you want to look at: - the opposing point of view - criticisms and critiques of both sides of the argumentAuthors purpose:- - does the author have a specific message they wish to report - has a contemporary issue, ‘viewpoint’ or policy influenced the authors purpose? - for whom is the material intended?
- Authors purpose:- - does the author have a specific message they wish to report - has a contemporary issue, ‘viewpoint’ or policy influenced the authors purpose? (refer back to ecology of resources) - for whom is the material intended?
- If the article is peer-reviewed, be aware of differences in peer-review requirements (blind peer review, select your own reviewers, etc.)Editorial policy may give some indication of what influences choices I what is published?
- Ask class to read this description and what they think the two key methods used were (IQ test and interview) and what specific questions they might also want an answer to:e.g. – what specific tests were employed (maybe include an example questionnaire/test if used, or interview schedule)
- Highlight need to evaluate on terms of consistency as well as clarity!
- Highlight need to evaluate on terms of consistency as well as clarity!
- If you can get hold of this… I have not yet been able to.Very nice 3 minute dissection of some Canadian research into the effects of the lunar cycle on levels of hospital admissions: - Relevancy, Objectivity and Methodology: including no clear setting out on what was being measured as part of the lunar cycle: eg luminescence (no evidence in methodology for taking into account cloud cover),
- How much time do you have? Thinking back to the unmotivated cat... Do you have more sympathy now thinking about all the different impacts upon your decision process, and the different things you could explore in great detail?But you do need to be responsible as a researcher... And be aware that others may not have been.
- How much time do you have? Thinking back to the unmotivated cat... Do you have more sympathy now thinking about all the different impacts upon your decision process, and the different things you could explore in great detail?But you do need to be responsible as a researcher... And be aware that others may not have been.
- 30 minutes into session.Give 15 minutes hands on time