Crit Eval Part 2

759 views
648 views

Published on

This slide show was used as part of a two day institute that walked teachers through instructional supports for fostering reading skills focused on critically evaluating relevance, accuracy, reliability, and author perspective during the online inquiry process. It has a companion website at http://www.lite.iwarp.com/CoiroVT2009.html

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
759
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Crit Eval Part 2

  1. 1. Julie Coiro - University of Rhode Island [email_address] http://www.lite.iwarp.com/CoiroVT2009.html Teaching Critical Evaluation on the Internet: Blending Strategy Instruction with Collaborative Inquiry (Part 2)
  2. 2. Experiencing the Challenges: Learning Strategies for Critically Evaluating Online Information <ul><li>Evaluating understanding : Does it make sense to me? </li></ul><ul><li>Evaluating relevancy : Does it meet my needs? </li></ul><ul><li>Evaluating accuracy : Can I verify it with another reliable source? </li></ul><ul><li>Evaluating reliability : Can I trust it? </li></ul><ul><li>Evaluating bias : How does the author shape it? </li></ul><ul><li>Evaluating choice : Which one is best and why? </li></ul>Our Agenda for Today
  3. 3. Experiencing the Challenges: Learning Strategies for Critically Evaluating Online Information <ul><li>10:30-11:30 (Challenge 1 and 2) </li></ul><ul><ul><li>Evaluating relevancy : Does it meet my needs? </li></ul></ul><ul><li>11:30-12:00 (Challenge 3) </li></ul><ul><ul><li>Evaluating accuracy : Can I verify it with another reliable source? </li></ul></ul><ul><ul><li>LUNCH </li></ul></ul><ul><li>1:15-2:00 (Challenge 4) </li></ul><ul><ul><li>Evaluating reliability (Level 1) : Can I trust the author? </li></ul></ul><ul><li>1:15-2:00 (Challenge 5) </li></ul><ul><ul><li>Evaluating reliability (Level 2) : Can I trust the information? </li></ul></ul><ul><li>2:00-2:45 (Challenge 6) </li></ul><ul><ul><li>Evaluating bias : How does the author shape it? </li></ul></ul><ul><li>Tomorrow morning (Challenge 7) </li></ul><ul><ul><li>“ The whole shebang”! </li></ul></ul>Our Agenda for Today
  4. 4. <ul><li>Evaluating Relevancy </li></ul><ul><ul><li>Does it meet my needs? </li></ul></ul><ul><li>Evaluating Accuracy </li></ul><ul><ul><li>Can I verify it (or the opposite) with another reliable source? </li></ul></ul><ul><li>Evaluating Reliability </li></ul><ul><ul><li>Can I trust it? </li></ul></ul><ul><li>Evaluating Bias (Perspective) </li></ul><ul><ul><li>How does the author shape it? </li></ul></ul>What are the challenges … What can I teach … Research into practice <ul><li>Previewing search results / Previewing websites </li></ul><ul><li>Confirming/disconfirming and Considering the context </li></ul><ul><li>Investigating the author </li></ul><ul><li>What’s there, how is it shaped, what’s missing, and how do I feel about the issues? </li></ul>
  5. 5. So, where are we headed? <ul><li>Review findings from recent research about how readers respond to different aspects of critical evaluation prompted by Internet texts and tasks </li></ul><ul><li>Pair up challenges with strategy lessons </li></ul><ul><ul><li>Evaluating relevancy </li></ul></ul><ul><ul><li>Evaluating accuracy </li></ul></ul><ul><ul><li>Evaluating reliability </li></ul></ul><ul><ul><li>Evaluating bias/point of view </li></ul></ul>
  6. 6. Findings from recent research
  7. 7. <ul><li>87.5% (42 out of 48) reported the site was reliable; </li></ul><ul><li>Slightly more than half of our students (27 out of 48) reported it to be v ery reliable </li></ul><ul><li>6 out of 48 reported the site was unreliable (just participated in a lesson that taught this site, showing students to be suspicious of information online) </li></ul>Study #1:
  8. 8. Online Reading Comprehension Assessment - Scenario I (ORCA-1) Study #2:
  9. 9. N=109 X 2=218 Study #2:
  10. 10. Population: Demographics of School Districts Included In This Study
  11. 11. Population: Ethnicity of Participants (n=118)
  12. 12. Talking about aspects of critical evaluation with students <ul><li>Relevancy : the information’s level of importance to a particular reading purpose or stated information need </li></ul><ul><li>Accuracy : the extent to which information contains factual and updated details that can be verified by consulting alternative and/or primary sources </li></ul><ul><li>Reliability: the information’s level of trustworthiness based on information about the author and the publishing body [ most and least reliable] </li></ul><ul><li>Bias (Perspective): the position or slant toward which an author shapes information </li></ul><ul><li>Commercial bias : the extent to which information appeared to be influenced by commercial interests for or against a certain product. </li></ul>
  13. 13. Evaluating Accuracy of Online Information Total N=109 80% do not know how to evaluate accuracy or did not locate the page. Only 20% have strategies for evaluating accuracy 0 = 54% It seems right but you can never know; The website I N=58 think is always right; It had plenty of pictures; I checked it out with Ask Jeeves; Why would they lie? [misconceptions] 1 = 26% I know this is accurate because it’s made by a corporation N=28 and there is a place to contact them. [implicit trust] 2 = 19% I know this is accurate because I learned it in science class. N=21 [compared with prior knowledge] 3 = .02% I checked this information with www.____.com and they N=2 compared similarly. [checked with 2nd reliable source]
  14. 14. Evaluating Least Reliable Source Total N=109 68% considered only relevancy or interest or did not know. Only 32% correctly identified the least reliable source 0 = 47% This is the least reliable source because the text is a little N=51 confusing to read. [readability, size of page, etc.] 1 = 21% It doesn’t really talk about anything specific - there’s not a N=23 lot of information about the topic [only relevancy or interest] 2 = 18% It’s a .com and they are trying to sell you something at the N=20 very top of the page. [surface procedures] 3 = 14% There are a lot of misspellings, there is no way to contact the people who put out this site, and it’s a .com compared to a N=15 .org or .gov [at least one critical attempt]
  15. 15. Evaluating Most Reliable Source Total N=109 71% considered only relevancy, text length, or did not know. Only 29% correctly identified the most reliable source 0 = 37% It’s really detailed and it has like 10 paragraphs of N=40 information. [readability, size of page, etc.] 1 = 34% I knew more about carbon monoxide than I knew from reading N=37 all of the other pages [only relevancy or interest] 2 = 16% There are no spelling mistakes and the url is a .org. [surface N=18 procedures] 3 = 13% This is most reliable because it is made by doctors from the N=14 American Lung Association [critically consider source]
  16. 16. Evaluating Commercial Bias Total N=109 71% are mildly cautious of the ads [not the info] or think ads make it better! Only 30% see reason to caution information with advertising 0 = 43% It makes me think they must really know what they’re talking about; I think that what they are telling you about those things N=47 is really true [ads make the information better] 1 = 17% I think websites that try to sell you things just want to get a lot of money because the Internet is really expensive [caution, but N=28 not in relation to the information] 2 = 17% If a website is trying to sell you something then it is a bad source and doesn’t have good information [some suspicion or N=20 overgeneralization] 3 = 13% This site is probably one sided info - the information might be N=14 inaccurate or exaggerated [balanced consideration of bias]
  17. 17. Overall, findings suggest… <ul><li>Only 20-32% of these 7th graders (N=109) had at least one effective strategy for evaluating accuracy, reliability, and potential bias of online information </li></ul><ul><li>That means 68-80% of students either did not have effective strategies for evaluating online information (35-45%) or could not even locate the information in the first place (33-35%). </li></ul>
  18. 18. Critically Evaluating Information <ul><li>Evaluating relevancy : Does it meet my needs? </li></ul><ul><li>Evaluating accuracy : Can I verify it with another reliable source? </li></ul><ul><li>Evaluating reliability : Can I trust it? </li></ul><ul><li>Evaluating bias : How does the author shape it? </li></ul>So, let’s get started… Questions/Concerns…
  19. 19. Challenge 1: Evaluating Relevancy - Reading Search Engine Results
  20. 20. Challenge: Evaluating relevancy Which link is most useful? <ul><li>Learning Objective: </li></ul><ul><li>Previewing search results </li></ul><ul><ul><li>What clues do the words after the link give me? </li></ul></ul><ul><ul><li>Are the results in any special order? </li></ul></ul><ul><ul><li>Who sponsors the site? </li></ul></ul><ul><ul><li>What’s missing from this list? </li></ul></ul><ul><ul><li>How do you know and Why does it matter? </li></ul></ul>
  21. 21. Sample Lesson: How do you fare?
  22. 22. Evaluating search results <ul><li>Which site features information about hieroglyphics? </li></ul><ul><li>How many websites were found using this search? </li></ul><ul><li>Which site is sponsored by a California school district? </li></ul><ul><li>Which site is most apt to not be available in three months? </li></ul><ul><li>How can you tell in the results what search terms were used? </li></ul><ul><li>What’s the biggest disadvantage to visiting only these six websites for information about Ancient Egypt for 6th graders? (Hint: what’s missing??) </li></ul>
  23. 23. Challenge 2: Evaluating Relevancy - Reading Within a Website
  24. 24. Challenge: So I’ve found a website…but where do I read first??? How do I know if it has what I need? A book versus a website…
  25. 25. LIST FIVE “BRAIN STEPS” YOU MIGHT TAKE TO PREVIEW THIS WEBSITE.
  26. 26. <ul><ul><li>STOP and THINK! </li></ul></ul><ul><ul><li>Preview left menu and top menu bars </li></ul></ul><ul><ul><li>Anticipate where each link will lead </li></ul></ul><ul><ul><li>Anticipate multiple levels (closer or further) </li></ul></ul><ul><ul><li>Explore interactive mouseover functions </li></ul></ul><ul><ul><li>Note the author/webmaster </li></ul></ul><ul><ul><li>Understand website search features </li></ul></ul> Learning Objective: Previewing a website Challenge: Evaluating relevancy Where do I read first? 1 2 3
  27. 27. 1 2 3 4 5
  28. 29. Challenge 2: Evaluating Relevancy - Reading Within a Website Log onto the website and refer to page 7 in your handouts Activity Imagine... you want your students to get more involved with community service and to network with other youth volunteers around the country. You will have one minute to preview each of the websites using the strategies above and rank each website according to its relevancy to your needs (0=lowest, 3=highest). Provide details to support your decision. Then, identify two parts of the “best” website that you believe would be most relevant for your students. Site 1 Site 2 Site 3 Site 4
  29. 32. Challenge 3: Evaluating Accuracy - Reading To Verify Online Information
  30. 33. Evaluating Accuracy <ul><ul><li>STOP and THINK! </li></ul></ul><ul><ul><li>Cross check factual data with at least three other sources </li></ul></ul><ul><ul><li>Ask probing questions: </li></ul></ul><ul><ul><ul><li>What claims is the author making? </li></ul></ul></ul><ul><ul><ul><li>What evidence do I find elsewhere to support those claims? </li></ul></ul></ul><ul><ul><ul><li>What evidence do I find elsewhere to refute those claims? </li></ul></ul></ul><ul><ul><li>Consider the context of where you find the evidence </li></ul></ul><ul><ul><li>The challenge of confirming vs. disconfirming </li></ul></ul><ul><ul><li>-- Adapted from IMSA 21st Century Information Fluency Project (2007) </li></ul></ul>
  31. 34. Cross-checking author claims <ul><li>What claims is the </li></ul><ul><li>author making? </li></ul><ul><li>Evidence to support ? </li></ul><ul><li>Evidence to refute ? </li></ul>
  32. 35. The complexities of confirming, disconfirming, and considering context <ul><ul><li>What claim(s) is the author making? Look for evidence elsewhere… </li></ul></ul><ul><ul><ul><li>The tree octopus is on the endangered species list. </li></ul></ul></ul><ul><ul><ul><li>The tree octopus lives on the Olympic Peninsula, near the Hood Canal. </li></ul></ul></ul><ul><ul><ul><li>Tree octopuses are amphibious… </li></ul></ul></ul>Consider the context! Who’s the author?
  33. 36. Confirming, disconfirming, and considering context
  34. 37. Challenge 3: Evaluating Accuracy - Reading To Verify Online Information Log onto the website and refer to page 9 in your handouts <ul><li>Activity </li></ul><ul><li>Imagine you are selecting websites for your students to use as part of their online research for their reports on strange animals. Apply the above questioning strategies to the websites below to verify or refute the accuracy of each author's claims. Be prepared to provide evidence for your decisions. </li></ul><ul><ul><li>Rock Nest Monster </li></ul></ul><ul><ul><li>The Elephant Art Gallery (Related Video) </li></ul></ul><ul><ul><li>Armed and Dangerous Dolphins </li></ul></ul>
  35. 38. Verifying Online Information
  36. 39. A real-life scenario… What causes a hurricane?
  37. 40. Investigate the source homepage
  38. 41. Scrolling for “About Us”.. and then ??
  39. 42. Do some investigating at Snopes.com Where else could I look?
  40. 43. Search < dolphins dart guns >
  41. 46. “ As testament to its value and accuracy, this site won more than 20 awards, including the 1998 ‘Cool Site of the Year,’ just during the past twelve months.” AAAS (American Assoc. for the Advancement of Science) Who is the author?
  42. 47. Reliable UK News Source ?? A constant process of reading, thinking, questioning, verifying, and synthesizing - many authentic opportunities await!
  43. 48. Challenge 4: Evaluating Reliability - Investigating the author’s credentials
  44. 49. Surface level evaluations of reliability <ul><li>Does it have an acceptable URL? </li></ul><ul><ul><li>.com, .edu, .org, .net, or .gov </li></ul></ul><ul><li>Is there an author’s name or affiliation/sponsor? </li></ul><ul><li>Is there a way to contact the author/sponsor (email address)? </li></ul><ul><li>But did you really “investigate” (like a detective) or just “view” this information?? </li></ul><ul><li>Thinking about it versus actually doing it </li></ul>
  45. 50. Reading more deeply - Think & Check Activity Coiro (2005) Educational Leadership
  46. 51. Investigating the Author
  47. 52. Think and Check… <ul><li>Does this sound like it makes sense? </li></ul><ul><ul><li>Be skeptical, ask a friend </li></ul></ul><ul><li>Where else can I look? </li></ul><ul><ul><li>Use Google to find another source - “velcro crop” </li></ul></ul><ul><li>Who created the website and for what purpose? </li></ul><ul><ul><li>Explore the “About Us” link </li></ul></ul><ul><li>Who IS the author? </li></ul><ul><ul><li>Do a search for “Ken Umbach” </li></ul></ul><ul><li>Who is linking to this site? </li></ul><ul><ul><li>Type Link:URL in the search box </li></ul></ul>California’s Velcro Crop Under Challenge …
  48. 53. Considering the author(s) and their level of credibility Reading deeply at the “About Us” page
  49. 54. Challenge 4: Evaluating Reliability - Investigating an author’s credentials Log onto the website and refer to page 10 in your handouts <ul><li>Activity (Complete at least two websites) </li></ul><ul><li>Locate the &quot;About Us&quot; link on each website below. Complete the activity in your handout by following these steps: </li></ul><ul><ul><li>(a) identify the name of hyperlink that led you to information about the website's creator; </li></ul></ul><ul><ul><li>(b) tell one thing you learned about the author's background and level of expertise; </li></ul></ul><ul><ul><li>(c) search off the website for the author's name and tell one other thing you learned about the author; </li></ul></ul><ul><ul><li>(d) is there anything about the site that appears to increase or decrease the reliability of the information found there? (please explain your hunch ) </li></ul></ul><ul><ul><li>(e) rate each author's level of expertise from 1 (lowest) to 5 (highest) and be prepared to justify your decision with evidence. </li></ul></ul>
  50. 55. Investigating the author: Who is Patsy Stevens?
  51. 56. Challenge 5: Critically evaluating online information - Level 2 Integrating strategies for evaluating accuracy and reliability
  52. 57. Challenge 5: Integrating strategies for evaluating accuracy & reliability Activity 1. Work with your group to brainstorm strategies for critically evaluating the information at a website. Create a list of these ideas in your handout. 2. See if you can determine if each of these sites is real or not real (there are some of both). Avoid relying only on your prior knowledge - since often your students will not have this knowledge to rely on or it is inaccurate. If you determined that the information is real, what evidence do you have to prove it? If you think it is a hoax, how do you know for sure? 3. Share with the larger group the strategies that helped you determine the quality of information at each website. Add to your own list any new strategies you learned from someone else in the group.
  53. 58. Confirming/disconfirming
  54. 59. The challenge of comparing sources
  55. 61. Challenge 6: Evaluating Bias - Separating fact from opinion, determining purpose, and understanding what influences an author’s perspective
  56. 62. Dogsledding images and questions from assessment Reading Across Multiple Perspectives
  57. 63. Reading Across Multiple Perspectives
  58. 64. Reading Across Multiple Perspectives
  59. 65. Activity Explore each website related to the Iditarod dog sled races as you complete the three tasks described below... * Ray Redington's Dog Care * Racing for the Grave * Scholastic's Is the Iditarod for the Dogs? Level 1: Detecting Fact Versus Opinion Tell which website you think has the STRONGEST opinions about the use of sled dogs in the Iditarod. Tell whether you think the author of the website you chose is for or against racing sled dogs for competition. Select a quote from the website you chose and explain why you think it is an example of the author sharing strong opinions. Detecting Facts, Opinions, Points of View, & Underlying Agendas
  60. 66. <ul><li>Level 2: Detecting Bias and Considering the Author's Affiliation </li></ul><ul><li>Tell which website (Site A, B, or C) gives opinions from more than one side of the issue. Who are the two people whose opinions are given in the website you chose in number 1? What factors make these two people feel the way they do about the treatment of sled dogs ? </li></ul><ul><li>Level 3. Determining Author's Purpose in Relation to Affiliation </li></ul><ul><li>Identify the general and specific purpose(s) of each site and provide at least two reasons to support your answer. </li></ul><ul><li>For example - detecting evidence of bias: </li></ul><ul><ul><li>Does the website provide factual information? </li></ul></ul><ul><ul><li>… try to persuade you to think or feel a certain way? </li></ul></ul><ul><ul><li>… try to sell you something? </li></ul></ul><ul><ul><li>… try to raise money or collect donations? </li></ul></ul>Detecting Facts, Opinions, Points of View, & Underlying Agendas
  61. 67. Reading Across Multiple Perspectives Identify your own perspective Compose a blog entry that tells which side of the issue you support and give at least two reasons from the websites you read to support your ideas. (opportunity to practice/assess authentic use of online communication tools)
  62. 68. Challenge 6: Negotiating Multiple Perspectives How do different authors portray the Japanese Internment Camp Experience to readers?
  63. 69. An Information Challenge For You
  64. 70. How do different authors portray the Japanese Internment Camp Experience to readers?
  65. 71. <ul><li>Scroll to Challenge 6: Evaluating Bias </li></ul><ul><li>Choose a partner & select 1-2 resources. </li></ul><ul><li>Use your Activity Sheet to guide your discussion of the source. </li></ul><ul><li>Discuss and reflect across texts. </li></ul><ul><li>Explore additional resources if time. </li></ul>An Information Challenge For You: How do different authors portray the Japanese Internment Camp Experience to readers?
  66. 72. How do different authors portray the Japanese Internment Camp Experience to readers?
  67. 73. How do different authors portray the Japanese Internment Camp Experience?
  68. 74. Reading Across Multiple Perspectives <ul><li>Your Turn: </li></ul><ul><li>Which topics in your grade level curriculum lend themselves to being seen from more than one perspective? </li></ul><ul><li>Discuss at least two topics with your partner and share ideas about the multiple views of the issue you might collect as part of a set of materials. Be prepared to share with the larger group. </li></ul>www.procon.org
  69. 75. <ul><li>Evaluating Relevancy </li></ul><ul><ul><li>Does it meet my needs? </li></ul></ul><ul><li>Evaluating Accuracy </li></ul><ul><ul><li>Can I verify it (or the opposite) with another reliable source? </li></ul></ul><ul><li>Evaluating Reliability </li></ul><ul><ul><li>Can I trust it? </li></ul></ul><ul><li>Evaluating Bias (Perspective) </li></ul><ul><ul><li>How does the author shape it? </li></ul></ul>What are the challenges … What can I teach … Recapping central ideas from today’s session <ul><li>Previewing search results / Previewing websites </li></ul><ul><li>Confirming/disconfirming and Considering the context </li></ul><ul><li>Investigating the author </li></ul><ul><li>What’s there, how is it shaped, and what’s missing? </li></ul>
  70. 76. Reflections… What did you notice/learn during today’s activities? What implications do these ideas have for your own classroom literacy instruction?
  71. 77. <ul><li>Readings for Homework </li></ul><ul><li>CQ Researcher: Internet Accuracy [link at top of website - p. 625-634] </li></ul><ul><li>Coiro (2003): Critical Evaluation [link on website] </li></ul><ul><li>Optional: Eagleton & Dobler, Ch. 1-3 (New Literacies & QUEST model) and Ch. 6 (Evaluation) </li></ul><ul><li>1. What did you notice/learn from today’s activities/readings? </li></ul><ul><li>2. What implications do these ideas have for your own classroom literacy instruction? </li></ul>Thanks for your participation today!

×