This document summarizes research on the challenges students face with reading and writing arguments using online sources. It introduces an online inquiry tool designed to scaffold the argumentation process. Key features of the tool include planning perspectives, locating and organizing evidence from multiple sources, evaluating sources, and integrating evidence into an essay. Research found the tool helped organization but did not significantly improve essay quality. Using the tool in pairs versus individually did not impact performance. Students struggled with source evaluation. Future work is needed to determine how to best support students through task design and additional scaffolds.

1. Julie Coiro, University of Rhode
Island
jcoiro@uri.edu
http://uri.academia.edu/JulieCoiro/Papers
Using The Online Inquiry Tool to Scaffold
Argumentation, Deliberation, and Close
Reading

2. Slides available on SLIDE
SHARE

3. Goals for the session
 Highlight features of a newly developed Online
Inquiry Tool designed to scaffold how students plan,
locate, organize, evaluate, and integrate evidence
for and against relevant claims using multiple
sources and build consensus on appropriate
solutions to issues having more than one point of
view.
 Offer examples of how teachers have used the
tool and discuss the potential for use in your setting
 Share research findings about the challenges
students face when asked to read and/or write
about information found in multiple and conflicting
online sources.

4. Challenge 1: Reading Arguments
 Secondary students have difficulty
identifying and analyzing arguments when
they read traditional, linear texts (Larson et al,
2004; Marttunen et al., 2005; Le Bigot & Rouet, 2007; Wiley et
al., 2009).
 Exploring controversial issues (or issues
with more than one solution) on the open
Internet compounds these challenges even
further. Learners require skills in organizing,
evaluating, comparing and contrasting
information from multiple sources (Britt & Rouet,
2012) and in moving beyond their own
perspective (Barzalai & Zohar, 2012)

5. Challenge 2: Writing Arguments
 Today’s secondary students also have
difficulty writing argumentative essays
 Unfamiliar with conventions of written
arguments (Beach, Thein, & Webb, 2012)
 Difficulties considering counterarguments in
their writing (Perskey, Dan, & Jin, 2003).
 They lack knowledge of what an
argumentative essay requires and have
difficulty developing an argument within the
context of the curriculum (Bacha, 2010; Beach, Thein,
& Webb, 2012; Wingate, 2011).

6. Challenge 3: Using the Internet for Online
Research
Monitoring
and regulating
one’s activities
Asking
questions
Locating
information
Evaluating
information
Synthesizing
information
Communicating
information

7. (c) 2014 Carita Kiili, Julie Coiro, Jari
Hämäläinen
How do we develop a digital tool to support elements of
online inquiry AND reading/writing argumentation texts?
(using representational guidance tools and optimizing germane load)

8. What features of a digital tool can
be used to support online inquiry
and the reading and writing of
argumentative texts around an
issue that has more than one side?
Digital Inquiry Tool

9. Kiili, C., Coiro, J., & Hämäläinen, J. (2016). An online inquiry tool to support
the exploration of controversial issues on the Internet. Journal of Literacy

10. Introducing Features Of The Tool
EVALUATE: Rate
trustworthiness and justify
reasoning (show/hide)
SYNTHESIZE/INTEGRATE:
Weigh conflicting evidence and
organize/integrate arguments
INTEGRATE/COMMUNICATE:
Structure essay to form a cohesive
representation that reflects multiple
perspectives
PLAN: Identify a specific claim
PLAN: Ponder the
kinds of perspectives
from which to
approach the issue at
hand
PLAN: Formulate
more specific
questions for guiding
the online inquiry
LOCATE/ORGANIZE: Focus on
one perspective at a time
(search/read)LOCATE/ORGANIZE: Consider
conflicting sides of an issue
LOCATE/ORGANIZE: Record
source for easy return
LOCATE/ORGANIZE: Build
additional arguments & add
evidence

12. (c) 2014 Carita Kiili, Julie Coiro, Jari
Hämäläinen
VIDEO: https://goo.gl/QKMnn5

13. Turn and Talk
 What features of the tool might be most
beneficial for your students?
 If you teach younger students, what
kinds of age-appropriate terms might be
used in the tool?
 Optional: Post your thinking on Google
Docs at https://goo.gl/7mNfVW

14. What types of curriculum-based
scenarios can be used to engage
students with this digital online
inquiry tool?
How have others used this tool?

15. Examples of tasks
Lang. Arts
Romeo and Juliet(Gr. 10)
Video Games (Gr. 9 – Alternative Education
Program)
Science/Social Science
Genetic Engineering or Social Media (Gr.
10-11) (Pairs vs. Individuals)
Digital Literacy (Graduate Students)
History
Atomic Bomb (Gr. 9)
Country Conflicts (Gr. 9)

16. General Lesson Frame (with options)
 Phase 1: Short lesson on how to use
the features of the tool (video tutorial if
useful)
 Phase 2: Search and close reading
about a controversial issue (2 or more
days)
 Read online sources (sometimes given;
sometimes searched for)
 Generate a claim (sometimes given a claim)
 Locate/Identify arguments that support or
refute claim
 Organize information into a digital argument

17. U.S. History (Grade 9)
Average to low performing students

18. U.S. History (Grade 9)
Average to low performing students

19. U.S. History (Grade 9)
Average to low performing students

20. English/Lang Arts (Grade 9)
Alternative School Students

21. English/Lang Arts (Grade 9)
Alternative School Students

22. World History (Grade 9) – Six teams
Average to low performing students

23. World History (Grade 9) – Six teams
Average to low performing students

24. English/Lang Arts (Grade 10)
Average students
Is it morally acceptable to modify the
genetic codes of human beings?

25. English/Lang Arts (Grade 10 Honors)

26. English/Lang Arts (Grade 10 Honors)

27. Controlled Study (2015-2016)
Elva Knight Research Award
 370 Students Gr. 9-10: 250 Finland and
120 U.S.
 Choose one of two tasks stated as a
claim:
 Social media increases people’s quality of life.
 Genetic engineering of plants and animals
should be allowed.
 Students asked to search for information from
online sources (no texts provided), fill in the graph
& write their essay
 Role of the tool? compare tool/no tool groups
Julie Coiro and Carita Kiili – ILA Elva Knight
Award 2016

28. Data Sources
SAMPLE FILLED=IN ARGUMENT GRAP

29. Many researchers believe social media plays a
significant role in teenagers’ lives. It can help people stay
in touch and spread knowledge throughout everyone’s
followers and friends. On the other hand, friends and
followers may know too much because of the lack of
privacy these sites have. Once someone puts his or her
information on a site, it is there forever. In addition, social
media is a great way to make businesses make known
their products. Ads are all over social media
sites…because more people go on social media than
other ways companies try to advertise, like TV and radio.
Also, because social media sites use the Internet, it is
easier for customers to order products straight from the
ads they see. But if the advertisement is faulty, like many
things can be on social media, it can steal your
information, give faulty information or overcharge. I’ve
concluded after researching Site 1 that when social
+1 people stay
in touch
-1 no privacy
+2 businesses
sell products
-2 faulty ads
Social media increases the quality of people’s
lives
Source 1:
Positive and
negative
Researchers’ perspective:

30. Table 1. Average scores on essay variables across four U.S. gro
Question #1: Impact of No Tool Use
vs. Tool Use on Essay Quality
Tool use higher, but not significan

31. Question #2: Individuals vs. Pairs:
Differences on quality of argument graph
 88 US Tool Users
 But, no significant differences between
individuals and pairs on any of five
argument graph variables
INDIVIDUALS
SLIGHTLY HIGHER
PAIRS
SLIGHTLY HIGHER
Planning
perspectives
Argumentative
reasoning
Evaluating sources
Synthesizing
Total Graph Quality
Julie Coiro and Carita Kiili – ILA Elva Knight
Award 2016

32. Question #2: Pairs vs. Individuals
Differences on quality of argument graph
Table 2. Average scores on argument graph variables across all four U.S.

33. Julie Coiro and Carita Kiili – ILA Elva Knight
Award 2016
Question 3. Impact of Tool Use on Source
Evaluations
3% at least one relevant justification across whole sample (much greater than typical
BUT avg. score was 2.7/5

34. Modified Research
Rubric

37. Specific challenges
What Are We Learning?

38. Realizing the challenges: Important role
of..
 The task itself (and directions to clarify task)
 Are claims, arguments, and/or perspectives
given or left to students to generate? (e.g.,
Atomic bomb task vs. US/Finnish tasks)
 Are claims worded positively (easier) or
negatively (harder) or as a question?
(hardest to compare across); Word as
positive claim as much as possible
 Is it an isolated task, end-of-unit
assignment, or a test?
 Is it required or optional to stop and evaluate
quality of claims?

39. Realizing the challenges: Important role
of..
 Students’ and teachers’ prior knowledge
about topic and task
 Familiarity with argumentation terms and
writing structures
 Connection to curriculum and prior units
 Texts selected as part of the task
 Number and difficulty of texts (readability;
complexity of hypertexts and how many
external hyperlinks)
 Genre (Expository vs. Narrative)
 Relevance of the selected texts to the task
 Who selected the texts (teachers or

40. The Challenge of Evaluating Sources
 Most students in our sample did NOT
spontaneously comment on the quality of sources
as part of the research/writing process
 Students may not have been motivated to take
time to do this – most weren’t getting a grade for
the task or essay
 Students were not sure why/what/how to
evaluate quality of sources or quality of content
 Complicated by messages from teachers and
librarians that evaluation is not necessary in
library databases
 Help teachers and students examine the quality of
each author’s arguments (data evidence or their

41. Conclusions from our Research
 Regardless of content area or level of typical
academic performance, many students
struggle with many aspects of these types of
reading and writing tasks.
 Students say the tool helps organize, balance, monitor,
and write – but it’s still VERY HARD
 Teachers struggle to articulate key concepts of
argumentation, perspective, and processes for
evaluating & writing arguments
 We need more professional development and
more controlled studies to determine which
task/context features and additional scaffolds
will best support which kinds of students.

42. Specific interventions
What Are We Learning?

43. Facilitate awareness of diverse
perspectives

44. Emphasize Deliberation Rather Than
Debate
** Social
Deliberation:
Working with a
Partner
www.socialdeliberativeskills.com

45. Provide Explicit Instruction in Evaluation
https://goo.gl/jDZ86A
Coiro, Coscarelli, Maykel, & Forzani (2015). Investigating criteria 7th graders
use to evaluate the quality of online information. Journal of Adolescent and

46. Encourage Use of Multiple & Varied
Indicators of Quality … SCAM?
a. SOURCE: Ask students to elaborate: Who is the author? In
what specific area is his/her expertise? What kind of
company does he/she work for and for how long?
b. CLAIMS: How does the author’s expertise and affiliation
influence claims being made? corroborate with others?
c. ARGUMENTS: Evidence to support and refute
claims?
What is the author’s purpose?
What techniques are used to attract and hold attention?
What lifestyles, values, and points of view are represented?
What is omitted from the message? (Renee Hobbs, Media Education Lab)
d. MAKE A DECISION about the validity of the
claims & arguments in relation to author & affiliation
Coiro, Coscarelli, Maykel, & Forzani, E. (2015). Investigating criteria seventh graders use to evaluate
the quality of online information. Journal of Adolescent and Adult Literacy, 58(7), 546-550.

47. Provide Explicit Instruction in Linking
Ideas
Julie Coiro and Carita Kiili – ILA Elva Knight
Award 2016

48. What issues might lend
themselves to this kind of
thinking?
Questions / Concerns
Ideas for Your Curriculum?

49. Julie Coiro, University of Rhode Island
Literacy for All Conference
October 24, 2017
jcoiro@uri.edu
http://uri.academia.edu/JulieCoiro/Papers
Thank you!

50. Online Inquiry Tool
bit.ly/inquirytool3
Example from Elva Knight Award

51. Consider the following claim:
Using and creating digital texts in
school has a positive influence on
students.
 What perspective does this source
represent?
 What evidence does the author give to
support reasoning for and/or against

52. Time to Explore
1. Insert claim
2. Add perspectives
? ? ?
? ?
Using and creating digital texts in school has a positive influence on students

53. Using and creating digital texts in school has a positive influence on students
SOCIAL
How does using
the computer
impact social
skills?
LEARNING
How does reading
and creating
digital texts
influence
learning?
Example of perspectives & guiding questions

54. Using and creating digital texts in school has a positive influence on studen
Working together to
make sense of digital
texts cultivates
collaboration
SOURCE 1
SOCIAL
Online forums allow
everyone to
participate
SOURCE 2
Social skills may be
under-developed if
working at a
computer
SOURCE 4
Focus on efficiency
may hinder social
interaction
SOURCE 3
How does reading
on the computer
impact social
skills?
Collaborative tasks
and online discussion
may positively
influence social skills.
However, digital
distractions and a
focus on efficiency
may restrict social
interaction among
people.
LEARNING
How does digital
creation influence
learning?
Perspective One: Reasons For & Against; Evaluate &
Synthesize

55. Using and creating digital texts in school has a positive influence on studen
Working together to
make sense of digital
texts cultivates
collaboration
SOURCE 1
SOCIAL
Online forums allow
everyone to
participate
SOURCE 2
Social skills may be
under-developed if
working at a
computer
SOURCE 4
Focus on efficiency
may hinder social
interaction
SOURCE 3
How does reading
on the computer
impact social
skills?
Collaborative tasks
and online discussion
may positively
influence social skills.
However, digital
distractions and
focus on speed my
restrict social
interaction among
people.
LEARNING Creating websites
increases critical
evaluation skills
SOURCE 2
How does digital
creation influence
learning?
Reading on websites
may decrease
comprehension
SOURCE 3
Creativity may
increase but
comprehension
may decrease
SOURCE 4
Start to organize your notes – following down the
Perspective Two: Reasons For & Against; Evaluate &
Synthesize