Crit Eval Part 2

Julie Coiro - University of Rhode Island [email_address] http://www.lite.iwarp.com/CoiroVT2009.html Teaching Critical Evaluation on the Internet: Blending Strategy Instruction with Collaborative Inquiry (Part 2)

Experiencing the Challenges: Learning Strategies for Critically Evaluating Online Information

Evaluating understanding : Does it make sense to me?

Evaluating relevancy : Does it meet my needs?

Evaluating accuracy : Can I verify it with another reliable source?

Evaluating reliability : Can I trust it?

Evaluating bias : How does the author shape it?

Evaluating choice : Which one is best and why?

Our Agenda for Today

Experiencing the Challenges: Learning Strategies for Critically Evaluating Online Information

10:30-11:30 (Challenge 1 and 2)

11:30-12:00 (Challenge 3)

LUNCH

1:15-2:00 (Challenge 4)

Evaluating reliability (Level 1) : Can I trust the author?

1:15-2:00 (Challenge 5)

Evaluating reliability (Level 2) : Can I trust the information?

2:00-2:45 (Challenge 6)

Tomorrow morning (Challenge 7)

“ The whole shebang”!

Our Agenda for Today

Evaluating Relevancy

Does it meet my needs?

Evaluating Accuracy

Can I verify it (or the opposite) with another reliable source?

Evaluating Reliability

Can I trust it?

Evaluating Bias (Perspective)

How does the author shape it?

What are the challenges … What can I teach … Research into practice

Previewing search results / Previewing websites

Confirming/disconfirming and Considering the context

Investigating the author

What’s there, how is it shaped, what’s missing, and how do I feel about the issues?

So, where are we headed?

Review findings from recent research about how readers respond to different aspects of critical evaluation prompted by Internet texts and tasks

Pair up challenges with strategy lessons

Evaluating relevancy

Evaluating accuracy

Evaluating reliability

Evaluating bias/point of view

Findings from recent research

87.5% (42 out of 48) reported the site was reliable;

Slightly more than half of our students (27 out of 48) reported it to be v ery reliable

6 out of 48 reported the site was unreliable (just participated in a lesson that taught this site, showing students to be suspicious of information online)

Study #1:

Online Reading Comprehension Assessment - Scenario I (ORCA-1) Study #2:

N=109 X 2=218 Study #2:

Population: Demographics of School Districts Included In This Study

Population: Ethnicity of Participants (n=118)

Talking about aspects of critical evaluation with students

Relevancy : the information’s level of importance to a particular reading purpose or stated information need

Accuracy : the extent to which information contains factual and updated details that can be verified by consulting alternative and/or primary sources

Reliability: the information’s level of trustworthiness based on information about the author and the publishing body [ most and least reliable]

Bias (Perspective): the position or slant toward which an author shapes information

Commercial bias : the extent to which information appeared to be influenced by commercial interests for or against a certain product.

Evaluating Accuracy of Online Information Total N=109 80% do not know how to evaluate accuracy or did not locate the page. Only 20% have strategies for evaluating accuracy 0 = 54% It seems right but you can never know; The website I N=58 think is always right; It had plenty of pictures; I checked it out with Ask Jeeves; Why would they lie? [misconceptions] 1 = 26% I know this is accurate because it’s made by a corporation N=28 and there is a place to contact them. [implicit trust] 2 = 19% I know this is accurate because I learned it in science class. N=21 [compared with prior knowledge] 3 = .02% I checked this information with www.____.com and they N=2 compared similarly. [checked with 2nd reliable source]

Evaluating Least Reliable Source Total N=109 68% considered only relevancy or interest or did not know. Only 32% correctly identified the least reliable source 0 = 47% This is the least reliable source because the text is a little N=51 confusing to read. [readability, size of page, etc.] 1 = 21% It doesn’t really talk about anything specific - there’s not a N=23 lot of information about the topic [only relevancy or interest] 2 = 18% It’s a .com and they are trying to sell you something at the N=20 very top of the page. [surface procedures] 3 = 14% There are a lot of misspellings, there is no way to contact the people who put out this site, and it’s a .com compared to a N=15 .org or .gov [at least one critical attempt]

Evaluating Most Reliable Source Total N=109 71% considered only relevancy, text length, or did not know. Only 29% correctly identified the most reliable source 0 = 37% It’s really detailed and it has like 10 paragraphs of N=40 information. [readability, size of page, etc.] 1 = 34% I knew more about carbon monoxide than I knew from reading N=37 all of the other pages [only relevancy or interest] 2 = 16% There are no spelling mistakes and the url is a .org. [surface N=18 procedures] 3 = 13% This is most reliable because it is made by doctors from the N=14 American Lung Association [critically consider source]

Evaluating Commercial Bias Total N=109 71% are mildly cautious of the ads [not the info] or think ads make it better! Only 30% see reason to caution information with advertising 0 = 43% It makes me think they must really know what they’re talking about; I think that what they are telling you about those things N=47 is really true [ads make the information better] 1 = 17% I think websites that try to sell you things just want to get a lot of money because the Internet is really expensive [caution, but N=28 not in relation to the information] 2 = 17% If a website is trying to sell you something then it is a bad source and doesn’t have good information [some suspicion or N=20 overgeneralization] 3 = 13% This site is probably one sided info - the information might be N=14 inaccurate or exaggerated [balanced consideration of bias]

Overall, findings suggest…

Only 20-32% of these 7th graders (N=109) had at least one effective strategy for evaluating accuracy, reliability, and potential bias of online information

That means 68-80% of students either did not have effective strategies for evaluating online information (35-45%) or could not even locate the information in the first place (33-35%).

Critically Evaluating Information

Evaluating reliability : Can I trust it?

So, let’s get started… Questions/Concerns…

Challenge 1: Evaluating Relevancy - Reading Search Engine Results

Challenge: Evaluating relevancy Which link is most useful?

Learning Objective:

Previewing search results

What clues do the words after the link give me?

Are the results in any special order?

Who sponsors the site?

What’s missing from this list?

How do you know and Why does it matter?

Sample Lesson: How do you fare?

Evaluating search results

Which site features information about hieroglyphics?

How many websites were found using this search?

Which site is sponsored by a California school district?

Which site is most apt to not be available in three months?

How can you tell in the results what search terms were used?

What’s the biggest disadvantage to visiting only these six websites for information about Ancient Egypt for 6th graders? (Hint: what’s missing??)

Challenge 2: Evaluating Relevancy - Reading Within a Website

Challenge: So I’ve found a website…but where do I read first??? How do I know if it has what I need? A book versus a website…

LIST FIVE “BRAIN STEPS” YOU MIGHT TAKE TO PREVIEW THIS WEBSITE.

STOP and THINK!

Preview left menu and top menu bars

Anticipate where each link will lead

Anticipate multiple levels (closer or further)

Explore interactive mouseover functions

Note the author/webmaster

Understand website search features

Learning Objective: Previewing a website Challenge: Evaluating relevancy Where do I read first? 1 2 3

1 2 3 4 5

Challenge 2: Evaluating Relevancy - Reading Within a Website Log onto the website and refer to page 7 in your handouts Activity Imagine... you want your students to get more involved with community service and to network with other youth volunteers around the country. You will have one minute to preview each of the websites using the strategies above and rank each website according to its relevancy to your needs (0=lowest, 3=highest). Provide details to support your decision. Then, identify two parts of the “best” website that you believe would be most relevant for your students. Site 1 Site 2 Site 3 Site 4

Challenge 3: Evaluating Accuracy - Reading To Verify Online Information

Evaluating Accuracy

STOP and THINK!

Cross check factual data with at least three other sources

Ask probing questions:

What claims is the author making?

What evidence do I find elsewhere to support those claims?

What evidence do I find elsewhere to refute those claims?

Consider the context of where you find the evidence

The challenge of confirming vs. disconfirming

-- Adapted from IMSA 21st Century Information Fluency Project (2007)

Cross-checking author claims

What claims is the

author making?

Evidence to support ?

Evidence to refute ?

The complexities of confirming, disconfirming, and considering context

What claim(s) is the author making? Look for evidence elsewhere…

The tree octopus is on the endangered species list.

The tree octopus lives on the Olympic Peninsula, near the Hood Canal.

Tree octopuses are amphibious…

Consider the context! Who’s the author?

Confirming, disconfirming, and considering context

Challenge 3: Evaluating Accuracy - Reading To Verify Online Information Log onto the website and refer to page 9 in your handouts

Activity

Imagine you are selecting websites for your students to use as part of their online research for their reports on strange animals. Apply the above questioning strategies to the websites below to verify or refute the accuracy of each author's claims. Be prepared to provide evidence for your decisions.

Rock Nest Monster

The Elephant Art Gallery (Related Video)

Armed and Dangerous Dolphins

Verifying Online Information

A real-life scenario… What causes a hurricane?

Investigate the source homepage

Scrolling for “About Us”.. and then ??

Do some investigating at Snopes.com Where else could I look?

Search < dolphins dart guns >

“ As testament to its value and accuracy, this site won more than 20 awards, including the 1998 ‘Cool Site of the Year,’ just during the past twelve months.” AAAS (American Assoc. for the Advancement of Science) Who is the author?

Reliable UK News Source ?? A constant process of reading, thinking, questioning, verifying, and synthesizing - many authentic opportunities await!

Challenge 4: Evaluating Reliability - Investigating the author’s credentials

Surface level evaluations of reliability

Does it have an acceptable URL?

.com, .edu, .org, .net, or .gov

Is there an author’s name or affiliation/sponsor?

Is there a way to contact the author/sponsor (email address)?

But did you really “investigate” (like a detective) or just “view” this information??

Thinking about it versus actually doing it

Reading more deeply - Think & Check Activity Coiro (2005) Educational Leadership

Investigating the Author

Think and Check…

Does this sound like it makes sense?

Be skeptical, ask a friend

Where else can I look?

Use Google to find another source - “velcro crop”

Who created the website and for what purpose?

Explore the “About Us” link

Who IS the author?

Do a search for “Ken Umbach”

Who is linking to this site?

Type Link:URL in the search box

California’s Velcro Crop Under Challenge …

Considering the author(s) and their level of credibility Reading deeply at the “About Us” page

Challenge 4: Evaluating Reliability - Investigating an author’s credentials Log onto the website and refer to page 10 in your handouts

Activity (Complete at least two websites)

Locate the "About Us" link on each website below. Complete the activity in your handout by following these steps:

(a) identify the name of hyperlink that led you to information about the website's creator;

(b) tell one thing you learned about the author's background and level of expertise;

(c) search off the website for the author's name and tell one other thing you learned about the author;

(d) is there anything about the site that appears to increase or decrease the reliability of the information found there? (please explain your hunch )

(e) rate each author's level of expertise from 1 (lowest) to 5 (highest) and be prepared to justify your decision with evidence.

Investigating the author: Who is Patsy Stevens?

Challenge 5: Critically evaluating online information - Level 2 Integrating strategies for evaluating accuracy and reliability

Challenge 5: Integrating strategies for evaluating accuracy & reliability Activity 1. Work with your group to brainstorm strategies for critically evaluating the information at a website. Create a list of these ideas in your handout. 2. See if you can determine if each of these sites is real or not real (there are some of both). Avoid relying only on your prior knowledge - since often your students will not have this knowledge to rely on or it is inaccurate. If you determined that the information is real, what evidence do you have to prove it? If you think it is a hoax, how do you know for sure? 3. Share with the larger group the strategies that helped you determine the quality of information at each website. Add to your own list any new strategies you learned from someone else in the group.

Confirming/disconfirming

The challenge of comparing sources

Challenge 6: Evaluating Bias - Separating fact from opinion, determining purpose, and understanding what influences an author’s perspective

Dogsledding images and questions from assessment Reading Across Multiple Perspectives

Reading Across Multiple Perspectives

Activity Explore each website related to the Iditarod dog sled races as you complete the three tasks described below... * Ray Redington's Dog Care * Racing for the Grave * Scholastic's Is the Iditarod for the Dogs? Level 1: Detecting Fact Versus Opinion Tell which website you think has the STRONGEST opinions about the use of sled dogs in the Iditarod. Tell whether you think the author of the website you chose is for or against racing sled dogs for competition. Select a quote from the website you chose and explain why you think it is an example of the author sharing strong opinions. Detecting Facts, Opinions, Points of View, & Underlying Agendas

Level 2: Detecting Bias and Considering the Author's Affiliation

Tell which website (Site A, B, or C) gives opinions from more than one side of the issue. Who are the two people whose opinions are given in the website you chose in number 1? What factors make these two people feel the way they do about the treatment of sled dogs ?

Level 3. Determining Author's Purpose in Relation to Affiliation

Identify the general and specific purpose(s) of each site and provide at least two reasons to support your answer.

For example - detecting evidence of bias:

Does the website provide factual information?

… try to persuade you to think or feel a certain way?

… try to sell you something?

… try to raise money or collect donations?

Detecting Facts, Opinions, Points of View, & Underlying Agendas

Reading Across Multiple Perspectives Identify your own perspective Compose a blog entry that tells which side of the issue you support and give at least two reasons from the websites you read to support your ideas. (opportunity to practice/assess authentic use of online communication tools)

Challenge 6: Negotiating Multiple Perspectives How do different authors portray the Japanese Internment Camp Experience to readers?

An Information Challenge For You

How do different authors portray the Japanese Internment Camp Experience to readers?

Scroll to Challenge 6: Evaluating Bias

Choose a partner & select 1-2 resources.

Use your Activity Sheet to guide your discussion of the source.

Discuss and reflect across texts.

Explore additional resources if time.

An Information Challenge For You: How do different authors portray the Japanese Internment Camp Experience to readers?

How do different authors portray the Japanese Internment Camp Experience to readers?

How do different authors portray the Japanese Internment Camp Experience?

Reading Across Multiple Perspectives

Your Turn:

Which topics in your grade level curriculum lend themselves to being seen from more than one perspective?

Discuss at least two topics with your partner and share ideas about the multiple views of the issue you might collect as part of a set of materials. Be prepared to share with the larger group.

www.procon.org