Critical Evaluation: Critical Reading & Critical Thinking

10,236 views

Published on

Critical Evaluation (February 2014) slides. Delivered as part of the Durham University Researcher Development Programme. Further Training available at https://www.dur.ac.uk/library/research/training/

Published in: Education, Technology
  • Be the first to comment

Critical Evaluation: Critical Reading & Critical Thinking

  1. 1. Critical Evaluation Critical Reading Critical Thinking James Bisset james.bisset@durham.ac.uk Academic Liaison Librarian (Research Support)
  2. 2. Session outline - What is Critical Reading / Critical Thinking? - Definitions, Three types of reading a text - Approaching a process for critical reading - Scanning/Skimming, Critical Reading, Critical Thinking, SQ3R - Evaluation of Research Information - What to look for as a critical reader when evaluating a text - Recognising your evaluative criteria in your role as a researcher - What you bring to the table, self awareness and cognitive bias
  3. 3. Exercise 1 • Spend 5 minutes to read the short extract on your desks, and make some brief notes which you would find useful to return to later to re-appraise yourself of the text. Via Flickr Creative Commons, by © Stuti Sakhalkar. Original available at http://www.flickr.com/photos/theblackcanvas/2945878325/
  4. 4. Part 1 What do we mean by critical reading & thinking?
  5. 5. The non-critical reader - Reads a text as a source for... - memorising facts & statements - repeating facts & statements - building a narrative around facts & statements without analysing validity, reliability or applicability
  6. 6. The critical reader - Reads a text as... - One interpretation of facts - Recognises the importance of... - what a text says - how the text evidences and portrays the subject matter
  7. 7. Critical Reading “ Critical Reading involves understanding the content of a text as well as how the subject matter is developed. Critical reading takes in the facts, but goes further. “ http://www.rimt.edu.au/studyandlearningcentre/
  8. 8. The critical thinker - Reads a text as... - One interpretation of facts - Recognises what a text says and does… - applies own knowledge & values - to evaluate and interpret a text’s overall meaning.
  9. 9. Critical Thinking “Critical thinking involves reflecting on the validity of what you have read in light of our prior knowledge and understanding of the world.“ http://www.criticalreading.com/critical_reading_thinking.htm
  10. 10. What a text says… (Restatement) What a text does… (Description) (how it says what it says) What a text means… (Interpretation) (what it means to you, as the reader)
  11. 11. What a text says… (Restatement) Restate the same topics and facts. What a text does… (Description) Discuss the topics & facts within the context of how the original argument was made. What a text means… (Interpretation) Interprets an overall meaning within the wider context of the readers prior knowledge and values.
  12. 12. Exercise 2 • Spend 5 minutes to read the following short [edited] extract and think about: - what the text says - what the text does - what the text means Via Flickr Creative Commons, by © Stuti Sakhalkar. Original available at http://www.flickr.com/photos/theblackcanvas/2945878325/
  13. 13. Goals of critical reading - recognise author's purpose - what is within the scope of their writing, and what isn’t - what they are trying to do; does it match what you are looking for - understand tone & persuasive elements of the argument - in contrast to the objective data and evidence - what are they trying to sell you; what are you actually being sold - recognise bias - identifying patterns of choice of content and language (eg negative vs positive language, repeated omission or discounting)
  14. 14. Critical Reading & Thinking - Is not about: - being negative or finding fault. - It is about: - assessing the strength of the evidence and the argument presented
  15. 15. Part 2 An efficient approach to critical reading:
  16. 16. Critical Reading & Thinking - "If we sense that assertations are ridiculous or irresponsible (critical thinking), we examine the text more closely to test our understanding (critical reading)“ https://www.york.ac.uk/media/biology/documents/careers/critical_reading_handout.pdf - Conversely, you can only think about a text critically if you have understood it (critical reading) - to understand why we agree or disagree with an alternative opinion, statement or conclusion. - to understand which issues we agree and/or disagree with in an argument.
  17. 17. Critical Reading: Myth busting - You do not have the time to read everything. - You do not have the time to read everything critically. - You must be selective. - Stay focussed: get the info you need.
  18. 18. Adopt an efficient approach - Start with some basic principles - Quickly scan/skim the material - [Critical Reading] Read more thoroughly and make notes - [Critical Thinking] Consider/Review against your prior knowledge and understanding of the topic
  19. 19. Some basics
  20. 20. Some basics - Most readers have an attention span of 15-20 minutes. - Be clear about why you are reading the text. - Pause for thought - Don’t (always) read in isolation. Discuss authors, publications, ideas and arguments with colleagues. - Usually, reading the text once is not enough.
  21. 21. SQ3R
  22. 22. SQRRR - Survey - Question - Read - Recall - Review Scan / Skim text to provide an overall impression of usefulness, scope, structure and argument.
  23. 23. SQRRR - Survey - Question - Read - Recall - Review Formulate questions you want to answer / points you want to confirm through more in depth reading.
  24. 24. SQRRR - Survey - Question - Read - Recall - Review Read the text in a more focussed way, aiming to answer the questions formulated.
  25. 25. SQRRR - Survey - Question - Read - Recall - Review Pause. Then test your understanding and memory of the text, and if you feel you have addressed all your questions raised.
  26. 26. SQRRR - Survey - Question - Read - Recall - Review You could also ‘test’ your understanding by discussing with a colleague.
  27. 27. SQRRR - Survey - Question - Read - Recall - Review Return to the text. Read in more detail, taking notes and identifying any further questions raised or left unanswered.
  28. 28. Survey Scanning / Skimming the text
  29. 29. Scanning text - Before reading in any detail - “Scan” your eye over the text quickly - to identify specific words or phrases - to get a feel for structure • headings & subheadings • figures, data, images • contents pages • index (for keywords) • reference list • abstract / body text
  30. 30. Scanning text – why? - Note – requires you to have thought about why you are reading / what you are looking for.
  31. 31. Scanning text – why? - Evaluate the relevance and usefulness - Make a judgment on whether you should read further - Can help you to decide what parts of a document you want to focus time on.
  32. 32. Skimming text - Note – scanning the text first can help you decide where to concentrate your time.
  33. 33. Skimming text - Speed read to get an overview of - structure of text - scope and content of the text • Note key points in summaries / abstract • Read 1st and last paragraph / section to get main points • look at 1st sentence of each paragraph to get a feel of content and thrust of argument.
  34. 34. Exercise 3 • From a 30 second view, skim the text (focus on first and last lines of each paragraph) to try and get an impression of the scope and content of the text. • Holmes, P., Fay, R., Andrews, J., Attia, M., (2013) Researching multilingually: New theoretical and methodological directions. International Journal of Applied Linguistics 23, 285–299. Via Flickr Creative Commons, by © Stuti Sakhalkar. Original available at http://www.flickr.com/photos/theblackcanvas/2945878325/
  35. 35. Question Why are you reading the text?
  36. 36. What are you looking for? - background info? - latest developments? - seek evidence to support/refute an idea? - to reinforce your own prejudices? - to examine a methodology or identify how a result was reached? - because you have to?
  37. 37. Read
  38. 38. Tips whilst reading - Identify core arguments - link evidence to any conclusions drawn - identify arguments you feel are under-evidenced / purpose behind arguments + interpretation of author. - identify alternative conclusions which could have been drawn.
  39. 39. Tips whilst reading - Look for repetitions of argument, phrases or words to give clues to authors intentions. - what do they consider crucial? - does this match what you think is crucial?
  40. 40. Recall & Review
  41. 41. Tips for making notes - Make notes as you read... - throw away your highlighter - annotate margins: key issues / questions raised - develop your own symbols... (AP) ** // !!
  42. 42. Tips for making notes An argument should: - explain why the authors considered what they are doing is worthwhile - explain the approach and methodology chosen - explain why the data collected/material selected was most appropriate - how conclusions drawn link to wider context
  43. 43. Tips for making notes Broad definition of evidence: - encompasses what you read, not just the data collected and presented - choice of methodology - context of data collection / creation (eg sources used, scope of study/experiment) - rationale for interpretations and conclusions drawn - relevance of theory underpinning argument
  44. 44. Part 3 Evaluation of Research Information
  45. 45. Evaluating information When reading critically you need to evaluate: • Relevance to the topic • Authority of the author, publisher etc • Purpose / Objectivity • Presentation • Methodology • Currency
  46. 46. Relevance to the topic • Before reading the text… • Read the abstract, introduction or summary. • Scan the bibliographic information which may highlight key subject areas not specifically alluded to. • Emphasis may not be clear until you read in full.
  47. 47. Relevance to the topic • Upon reading the text… • What level is the information at? • Does it contain, and discuss in enough detail the information you are seeking? • Is the research relevant to the subject domain / geographical area / demographic / time period you are interested in?
  48. 48. Relevance to the topic “In the course of a series of investigations into various aspects of mental inheritance an intensive study has been made of so-called ‘identical’ twins. The cases examined fall into two main groups: first, those reared together in their parents' homes; secondly, those separated in early infancy, and brought up apart. With the latter, despite wide differences in environmental conditions, the correlations for intelligence, unlike those for school attainments, prove to be surprisingly high. It is argued that this implies that ‘intelligence’, when adequately assessed, is largely dependent on genetic constitution.” http://10.1111/j.2044-8295.1966.tb01014.x “ Burt’s study of monozygotic twins reared apart … involved the largest number of separated twin pairs at the time and produced the highest estimate of heritability for IQ”
  49. 49. Relevance to the topic Restatement: Burke identified a link between IQ and inherited genes. Description: This article compares the evidence for IQ being determined by inherited genes as opposed to IQ being affected by external environmental factors. Interpretation: Evidence for IQ being an inherited trait rather than affected by external factors has potential implications for the development of social and education policy.
  50. 50. Relevance to the topic • Key topics and ideas. • Level of information presented. • Relevance in terms of location/subject/scope. • Does the content and level match your needs. Be aware of what is filtering your choices… - Vocabulary and broadness of interpretation. Are you under-estimating the value of a source because it doesn’t match your choice of keywords precisely?
  51. 51. Authority “The trouble with quotes from the internet is that you never know if they are genuine.” Abraham Lincoln (1809-1865, President of the United States of America).
  52. 52. Authority • Is it clear where the information comes from? • Are the authors acknowledged experts in the field? - frequently cited? do they have an h-index? - have you or colleagues heard of them? - can you find any profile information where they work? - how well respected is the author, and their work, in their related field of research? • Where is it published? - impact factors for a journal (not always an accurate measure of quality, but potentially one of prestige) - is it peer reviewed?
  53. 53. Authority • Sir Cyril Burt • Fellow of British Academy • Author of over 350 articles and a number of books. • “pioneer research on the inheritance of mental ability”
  54. 54. Authority – Citations
  55. 55. ””the most satisfactory attempt” to estimate hereditability of IQ” and “”the most valuable” of all the separated twin research.” • Professor at the University of California, Berkeley. • Author of 400+ peer reviewed papers. • In 2002, was listed in the Review of General Psychology’s top 50 “most eminent psychologists of the 20th century” ”the largest of its kind and the only one where “the distribution of children into foster homes was random ” “ • Edge Pierce Professor of Psychology at Harvard University. • Author of several peer reviewed papers and books. • Credited with discovering and developing several models and theories as one of the founding researchers in the field of quantitative analysis of behaviour. Richard J Herrnstein Arthur R Jenson
  56. 56. ”the only one of its kind in which the calculation of heritability had any meaning.” • Professor of Psychology at the Institute of Psychiatry, King’s College, London. • In 2002, was listed in the Review of General Psychology’s top 100 “most eminent psychologists of the 20th century” as the most cited living psychologist at the time of his death. ””the best data“ on separated twins.” • Alexander M. Poniatoff Professor of Engineering and Applied Science at Stanford University. • Joint awarded Nobel Prize in Physics in 1956. • 1960’s and 1970’s moved also into area of hereditary behaviour. William B Shockley Hans Eysenck
  57. 57. Authority – Impact Factors - British Journal of Mathematical and Statistical Psychology - Impact Factor 1.258 - 5th of 13 journals in category “Psychology, Mathematical” - British Journal of Educational Psychology - Impact Factor 2.093 - 11th of 50 journals in category “Psychology, Educational” - British Journal of Psychology - Impact Factor 2.103 - 26th of 126 journals in category “Psychology, Multi-disciplinary”
  58. 58. Authority • Are the authors acknowledged experts in the field? - frequently cited? do they have an h-index? - have you or colleagues heard of them? - can you find any profile information where they work? - how well respected is the author, and their work, in their related field of research? • Where is it published? - Impact factors for a journal (not always an accurate measure of quality, but potentially one of prestige) and is it peer reviewed? Be aware of what is filtering your choices y - Is the prestige of the author or the publication impacting on how you evaluate the content?
  59. 59. Objectivity • Is the subject controversial? • Does the author use emotive language? • What is the authors purpose in writing the paper? • If there are differing views on the subject area, does the author consistently fall into one ‘camp’?
  60. 60. Objectivity ** This is an over-simplification…. ** IQ is inherited IQ is affected by external factors Burt, C (1943) “Ability and Income” British Journal of Educational Psychology Burt, C.L. (1957) “Heredity and Intelligence; A reply to criticisms” British Journal of Statistical Psychology Burt, C.L. (1958). "The inheritance of mental ability", American Psychologist, Burt, C.L. (1972). "Inheritance of general intelligence", American Psychologist, Burt C (1966) “The Genetic Determination of Differences in Intelligence: A Study of Monozygotic Twins Reared Apart and Together.” British Journal of Psychology
  61. 61. Objectivity • Is the subject controversial? • Does the author use emotive language? • What is the authors purpose in writing the paper? • If there are differing views on the subject area, does the author consistently fall into one ‘camp’? Be aware of what is filtering your choices - Does the author demonstrate any hidden bias on the topic? - Evaluate yourself? Are you subconsciously over-valuing the resource because it confirms your own prejudices? Are you being objective?
  62. 62. Methodology • are the methodology and sources of data used clearly identified or explained? • what evidence is presented to support the ideas and conclusions expressed? • is the methodology direct or indirect? • are the conclusions and assumptions made by the author consistent, logical and justified? • does the research raise any unanswered questions?
  63. 63. Methodology “ tests of the usual type… as a means of estimating genotypic differences, even the most carefully constructed tests are highly fallible instruments, and … their verdicts are far less trustworthy than the judgments of the pupil’s own teachers.” Burt, C. (1957)British Journal of Statistical Psychology “the unaided judgments even of the most experienced teachers … are nevertheless far less trustworthy in the long run that the results obtained by properly applied intelligence tests.” Burt, C. (1943)British Journal of Educational Psychology
  64. 64. Part 3 Summary • Various criteria you can assess a resource by. - a lot more ‘citation’ tools available for journal literature. • How much time do you realistically have?
  65. 65. Part 3 Summary • Some quotes and opinions were taken from one article: Tucker, W.H.(1994) “Fact and fiction in the discovery of Sir Cyril Burt’s Flaws” Journal of the History of the Behavioral Sciences (30). • Does that change your opinion on some of the previous slides assertions and emphasis?
  66. 66. Part 4 Recognising your evaluative criteria
  67. 67. Critical Thinking “Critical thinking involves reflecting on the validity of what you have read in light of our prior knowledge and understanding of the world.“ http://www.criticalreading.com/critical_reading_thinking.htm
  68. 68. Ecology • Resources are interconnected and they evolve • Information resources are transformed into knowledge • Knowledge becomes a resource • Therefore prior knowledge shapes what we go on to create
  69. 69. Cognitive biases • Subjectivity is vulnerable to bias & hunches • Concept of cognitive bias was developed in 1970s by Tversky and Kahneman • Four main groups - Social - Probability/belief - Memory - Decision making
  70. 70. 1) “Hwang faked all research on human stem cells”
  71. 71. “Hwang faked all research on human stem cells” Halo effect “The tendency for a person's positive or negative traits to "spill over" from one personality area to another in others' perceptions of them”
  72. 72. 2) Strange, Hayne & Garry (2008) ‘A photo, a suggestion, a false memory’ Applied Cognitive Psychology. 22, 587–603.
  73. 73. Strange, Hayne & Garry (2008) ‘A photo, a suggestion, a false memory’ Applied Cognitive Psychology. 22, 587–603. False memory bias “A form of misattribution where imagination, or incorrect recall, is mistaken for a memory.”
  74. 74. Memory biases
  75. 75. 3) “Once people become aware of an event or conclusion, they will usually express a good deal of confidence about the predictability of the outcome.”
  76. 76. Fischhoff, B., and Beyth, R. (1975) ‘"I knew it would happen" Remembered probabilities of once-future things’ Organizational Behaviour and Human Performance. 13, 1-16. Hindsight bias “The inclination to see past events as being more predictable than they actually were”
  77. 77. Exercise 4 • Read through the example “cognitive bias” on the cards, and try to decide which of the four “groups” of cognitive bias each might sit in. - Social (Ascribe positive or negative traits to self, individuals or groups) - Probability/belief (disregard or to pay too much attention to probability) - Memory (How you perceive past events) - Decision making (Influences on your decisions by own or group biases) Via Flickr Creative Commons, by © Stuti Sakhalkar. Original available at http://www.flickr.com/photos/theblackcanvas/2945878325/
  78. 78. Cognitive biases • Shape how we read, interpret and evaluate the information we receive. • Shape how others read, interpret and evaluate the information they receive. • Any published research is just one interpretation of the facts available
  79. 79. Further Reading • Tucker, W.H.(1994) “Fact and fiction in the discovery of Sir Cyril Burt’s Flaws” Journal of the History of the Behavioral Sciences (30). • University of Leicester: What is Critical Reading http://www2.le.ac.uk/offices/ld/resources/writing/writing-resources/critical-reading • University of Newcastle: Think Critically http://www.ncl.ac.uk/library/research-support/informed-researcher/think-critically/ • Open University: Critical Reading Techniques http://www.open.ac.uk/skillsforstudy/critical-reading-techniques.php/ • Dan Kurland’s Critical Reading Website http://www.criticalreading.com/ • The SQ3R method http://www.ic.arizona.edu/ic/wrightr/other/sq3r.html
  80. 80. Image Credits [69] Via Flickr Creative Commons, by Martin LaBar. Original available at http://www.flickr.com/photos/32454422@N00/163107859/ [4,44,67] Via Flickr Creative Commons, by Kevin Dooley. Available at http://www.flickr.com/photos/12836528@N00/2577006675 [75] Via Flickr Creative Commons, and by Brian Yap: Original available at http://www.flickr.com/photos/30265340@N00/465957804 [15] Via Flickr Creative Commons, and by FutUndBeidl: Available at http://www.flickr.com/photos/61423903@N06/7369580478

×