This presentation outlines several recent areas of my research around these topics.

1. Understanding Online Reading
Comprehension, Collaboration,
& Digital Inquiry: Connecting
Theory, Research, & Practice
Julie Coiro, PhD
School of Education
University of Rhode Island
jcoiro@uri.edu
uri.academia.edu/JulieCoiro/

2. The Plan
 Quick context setting: Student engagement & My Views
 Theoretical Lenses
 Three areas of work
 ORCA Project: Online Reading (Critical Evaluation)
 Digital Scaffolds for Online Inquiry, Skillful
Argumentation, and Social Deliberation
 Personal Digital Inquiry (PDI) Framework [and
productive F2F collaborative dialogue]
 Four promising practices for the future

3. How can we engage learners in today’s digital
society?
 Move beyond industrial model of universal school
toward new era focused on lifelong learning and
individual choice – or lose learners emotionally &
physically (Collins & Halverson, 2009: Rethinking Education in the Age of
Technology)
 If we engage learners with rigorous academic content
and expect them to know why, when, and how to apply
knowledge to answer questions & solve problems
(National Research Council, 2012)
 “Lessen achievement gap; lead to positive adult
outcomes for more young people”
Context

4. High school students engaged in deep &
relevant learning opportunities…
 Demonstrated higher levels of interpersonal and
intrapersonal skills (American Institutes for Research, 2014)
 Achieved better outcomes in every aspect of life,
including academic, career, civic, and health
(Center for Public Education, 2009)
Context
Optimistic Findings

5.  Continuing achievement gaps in literacy and content
area learning among students from diverse
backgrounds (local; NAEP; PISA)
 Keeping up with rapid cultural and technological shifts
connected to our conceptions of reading in a digital
world (e.g., Coiro, Knobel, Lankshear, & Leu, 2008;
Leu et al, 2015, RRQ, ORCA Project)
 Understanding, valuing, and capturing the critical role
of affective variables such as motivation, interest, and
self-efficacy (Gallup, 2012; Center for Educational Policy, 2012)
Context
Important Challenges:

6. Gallup Poll (2012) - 500,000 US students, Gr. 5-12 Center for
Education Policy (2012)

7. Gallup Poll (2015) – 929,000 US students, Gr. 5-12
Percentage who strongly agreed with the statement: “The adults at my
school care about me, “ declined from 67% (Grade 5) to 23% (Grade 11)…
Many students don’t feel individually known or cared for at school.
THIS is the PERSONAL we need to focus on!

8. The Challenge: Connecting Learners in Ways That
Matter
Sense of Belonging – Do I fit in? Am I relevant? Do people care about me?
OECD PISA 2015 Students’ Well Being (April 2017)
Outsider (disagree)
KOREA 91.3%
US 76.2 (lowest)
Belong (agree)
KOREA 79.5%
US 74.2

9.  1 high school, 581 classes, 1,132 students
 “Connective instruction [when teachers help students
make personal connections to a class] predicts
engagement more than seven times as strongly as
academic rigor or lively teaching.”
Why is the “personal” piece so important?

10. My Own Goals
 Mixed methodologist with a background in reading
comprehension, curriculum design, and special education
 Bridge between the numbers (research/policy) and the stories
behind the numbers (what, why, for whom, under what
conditions)
 Pair the stories with classroom instruction and professional
development: “What do we do now?”
 Be aware of but not paralyzed by the challenges
 Implications and ideas for
 Changing mindsets/Building new cultures
 Designing learning spaces (communities and/or texts)
 Designing assessments
 Designing curriculum

11. New Literacies
 The nature of literacy and learning is rapidly
changing & transforming as new technologies
emerge
 Common assumptions (Handbook of Research on New
Literacies. Coiro, Knobel, Lankshear, & Leu, 2008)
 New skills, strategies, dispositions, and social
practices are required by new ICT
 These literacies are central to full participation in a
global community
 These literacies regularly change as their defining
technologies change
 These literacies are multifaceted and benefit from
multiple points of view
Theoretical Lens

12.  Self-directed process of constructing texts
and knowledge to solve problems while
engaged in at least five practices:
1. Generating important questions
2. Reading to locate relevant information
3. Reading to critically evaluate information for
accuracy, reliability, and stance
4. Reading to synthesize findings across
multiple sources
5. Reading & writing to communicate findings to
others
A new literacies perspective of online
reading comprehension
Leu, Kinzer, Coiro, & Cammack, 2004;
Leu, Kinzer, Coiro, Castek & Henry, 2013
Theoretical Lens

13.  RAND + Broadened understandings of…
 TEXT (non-linear hypertext; multiple media;
interactive texts)
 READING ACTIVITY (varied learner-directed
purposes; higher-level critical processes; learner-
generated consequences/products)
 READER (cognitive capabilities; affective elements
such as motivation & self-efficacy)
 SOCIAL CONTEXT (collaboration with real global
audiences; ability to take action)
Coiro, 2003, The Reading Teacher

14. Research…
Coiro & Dobler, 2007
Reading Research Quarterly
Similar to Print Unique to Online
Prior Knowledge Topic; Info. Text
Structures
Info Website Structures;
Search Engines
Inferential
Reasoning
Matching, structural &
context clues
Forward, multilayered
reasoning
Self-Regulated
Reading
Cog. monitor & repair; a
single process
Cognitive & Actions;
Rapid, short cycles
 Plan
 Predict
 Monitor
 Evaluate

15. Four interpretations:
1. Some skills are similar and others
are unique
2. Many skills are simultaneously
similar and more complex
3. Some are similar, some are more
complex, and some are unique
4. Some are beyond what’s captured
by multiple-choice assessments of
comprehension
Research…
Leu, D. Castek, J., Hartman, D., Coiro, J.,
Henry, L., Kulikowich, J., Lyver, S. (2005).
Coiro, 2011 (JLR)

16. Other studies of online reading
 Major shift in our conception of reading comprehension in
terms of complexity & multiplicity
RAND Model (2002):
Tetradic conception of
four interacting elements
Hartman, Morsink, & Zheng (2010):
Hexadic conception of six interacting
elements
(each is multiple as well)
Research…

17. Supporting Online Reading Comprehension,
Collaboration & Digital Inquiry Across the Grade Levels
DIGITAL INQUIRY
Personal Digital
Inquiry (PDI)
(Grades K-12)
Coiro, Castek, & Quinn
(2016); Coiro, Dobler, &
Pelekis (forthcoming)
Productive
Collaborative
Dialogue/Talk
Coiro, Castek,
Sekeres & Guzniczak
CRITICAL
READING/LITERACY
ORCA Project
(2009-2014)
Critically Evaluating
Sources (Grades 5-8)
Coiro, Coscarelli,
Forzani, & Burlingame
(2015)
ARGUMENTATION,
COLLABORATION &
DELIBERATION
(During inquiry)
Online Inquiry Tool:
(Grades 7-12)
Coiro & Kiili, 2014-2016
Online Collaborative
Inquiry (ETS/SAIL)
2016-2018
2003 - 2018

18. 1. Expanding Online Reading with
Critical Literacy Practices
 How can we best measure, support, and encourage students
to take a more active & critical role in their own learning
while interacting with a diverse range of texts, images, digital
sources, and perspectives?
 Working with:
 New Literacies Research Team at Univ. of Connecticut (Don Leu,
Coiro, Jill Castek, Elena Forzani)
 Carita Kiili, University of Jyväskylä in Finland
 Carla Coscarelli, Universidade Federal de Minas Gerais, in Brazil
 Renee Hobbs, University of Rhode Island (media literacy)
Critical Literacy

19. Eight Information Problems
Science (Human Body Systems related to eyes,
ears, heart and lungs)
1. How can energy drinks affect your heart? (heart)
2. How can snacks be heart healthy? (heart)
3. How does the volume level of MP3 players affect hearing?
(ears)
4. How well can adults hear mosquito ring tones? (ears)
5. Does third-hand smoke harm your health? (lungs)
6. Can Chihuahua dogs cure asthma? (lungs)
7. Do cosmetic contact lenses harm your eyes? (eyes)
8. Do video games harm your eyes? (eyes)
Learn more
about
Investigate
conflicting
claims
Leu, Kulikowich, Sedransk, & Coiro (2009-2014)
Institute for Education Sciences (IES) www.orca.uconn.edu

20. 20
Reading to Locate
Information Online

21. Reading to Locate
Information Online

22. Reading to Locate
Information Online

23. Reading to Synthesize
Information Online

24. Reading to Evaluate
Information Online
24

25. Reading and Writing to
Communicate Information Online
25
www.orca.uconn.edu

26. The ORCA Project
73-80% of large Grade 7 stratified random
sample struggled with all three evaluation skills!Almost 18%!
Critical Evaluation
Coiro, Coscarelli, Maykel, & Forzani, E. (2015). Investigating criteria seventh graders use to evaluate the
quality of online information. Journal of Adolescent and Adult Literacy, 58(7), 546-550.

27. Challenges when judging the quality
of online information
1. Judging author’s level of expertise in relation to a
specific topic or area of work
 Shallow criteria to judge expertise
2. Understanding consequences of an author’s affiliation
and point of view
 How do authors position their audiences and decide
whether/how information is shared or represented?
3. Providing reasoned evidence to support judgments
about information quality
 Mostly generalized assumptions about Internet; naïve or single
criteria rather than combining several appropriate indicators
Critical Evaluation

28. Strategies for supporting close
& critical reading
1. Discuss/compare multiple dimensions of critical evaluation
Coiro, J. (April 2014). Teaching adolescents how to evaluate the quality of online
information. Edutopia Blog Post.
Critical Evaluation

29. Encourage Use of Multiple & Varied Indicators
of Quality … SCAM?
a. SOURCE: Ask students to elaborate: Who is the author? In
what specific area is his/her expertise? What kind of
company does he/she work for and for how long?
b. CLAIMS: How does the author’s expertise and affiliation
influence claims being made? corroborate with others?
c. ARGUMENTS: Evidence to support and refute claims?
What is the author’s purpose?
What techniques are used to attract and hold attention?
What lifestyles, values, and points of view are represented?
What is omitted from the message? (Renee Hobbs, Media Education Lab)
d. MAKE A DECISION about the validity of the
claims & arguments in relation to author & affiliation
Coiro, Coscarelli, Maykel, & Forzani, E. (2015). Investigating criteria seventh graders use to evaluate
the quality of online information. Journal of Adolescent and Adult Literacy, 58(7), 546-550.

30. Emphasize Deliberation Rather Than Debate

31. 2. Capturing Dimensions of Collaboration and
Deliberation with Multiple-Source Inquiry Tasks
in the SAIL ELA Virtual World
NAEP Survey Assessments Innovations Lab (SAIL)
Collaborative Problem Solving (CPS) Project
May 2016 – May 2018
PI: Julie Coiro, University of Rhode Island
Co-PI: Jesse R. Sparks, Educational Testing Service
Co-PI: Carita Kiili, University of Jyväskylä, Finland
Co-PI: Jill Castek, University of Arizona

32. NAEP SAIL CPS Project Goals
(2016-2018)
1. Collaborative Task Development
 Create and cog lab task in F2F and remote contexts
2. Collaborative Technology and Development
 Test/refine ETS Collaborative Problem Solving (CPS) Frame;
analyze usability and back-end data outputs
3. Conduct Research on Collaborative Task
 Analyze interactive behaviors and inter-student talk to refine
evidence and task models for collaborative skills in F2F and
remote contexts
 Propose methods for analysis and reporting data

33. SAIL ELA: Virtual World Environment
Scenario-based tasks measure reading, writing, and research/inquiry skills
via digital tools to support inquiry activities
Map-Based Navigation of Resource Locations
Simulated Web Search
Planning Inquiry with Virtual Partner (Guide)
Compose Response to Inquiry Task using Collected
Resources

34. Define Problem & Information Needs
• Recognize and determine what information is needed to answer questions or solve problems;
• Generate research questions and plans for locating sources that answer those questions.
Locate Sources
• Identify and collect relevant resources;
• Use search, browsing, or lookup strategies to obtain relevant sources;
• Refine/iterate searches as appropriate;
• Make predictions about the content of a search result, and use those
predictions to decide whether a source is worth exploring in depth.
Evaluate Sources
• Evaluate the usefulness of resources;
• Evaluate the reliability/credibility of resources (author expertise, bias, publication
venue, document type, currency, accuracy, and completeness);
• Interpret resource content with respect to its source (sourcing);
• Evaluate the reliability of information by comparing it to information obtained
from other sources (corroboration).
Process, Analyze, Synthesize Sources
• Demonstrate understanding of resource content by summarizing main ideas;
• Take notes from sources that extract key ideas, using paraphrase or quotation,
with citations
• Compare and evaluate consistencies and discrepancies across a set of sources;
• Integrate and synthesize information into a coherent mental model of the situation
Communicate Results
• Construct responses which effectively address an inquiry question in one’s own words;
• Clearly state claim and supporting evidence for one’s position
• Integrate information from multiple sources using summary, paraphrase, or quotation;
• Write text with appropriate content and tone given the audience and purpose;
• Embeds multimedia elements to support one’s ideas;
• Includes citations/references to document one’s sources
Note that sources often present different information and perspectives on an issue; students must
make sense of multiple perspectives, as an important part of online inquiry (cf. Leu et al., 2013).
SAIL ELA Online
Inquiry
cognitive
targets
include
reading,
writing,
and
research /
inquiry skills.
Evidence Centered
Design –
Student Model
Online Inquiry ConstructSAIL ELA: Online Inquiry

35. Collaborative Online Inquiry and Deliberation =
the capacity of an individual to effectively engage in a
process whereby two or more agents attempt to work
together to locate, critically evaluate, and construct
meaning from multiple online sources in order to
integrate supporting and competing views into a jointly
constructed coherent synthesis that will inform decision
making.
Defining the construct:
“Collaborative Online Inquiry and Deliberation”

37. ETS CPS Frame Technology
Example of ETS CPS Frame with Online Inquiry Task Facilitation pane:
Gives instructions for collaborative task.
Prompts from Table 4 would go here.
Text chat:
Allows textual
communication
between
partners
Task pane:
Where the SAIL Virtual World for ELA
Online Inquiry task will appear
Audio/video
channel:
Allows verbal
and non-verbal
communication

38. Table 3. Example Probes for Assessing Collaborative Online Inquiry
and Deliberation Competencies
Example Probe Skill Assessed from Table 2 Matrix
What does your partner know about this topic? (A1) Identifying personal and partner’s abilities related
to task goals
What steps will you and your partner perform as part of this
task?
(B1) Describing roles and rules of engagement to
complete multiple source inquiry task
What might you do differently at this point to help your
partner?
(D1) Monitoring, providing feedback, and adapting
task completion and roles
What is the main idea of this text? How do you think it
relates to your research goal?
Is this a reliable source, and how do you know?
(B2) Negotiating a mutual understanding of the issue,
related texts, and their sources to build a shared and
critical representation of knowledge (common
ground)
How might you add to the ideas suggested by your partner? (C2) Communicating with partner about how to
combine and build on each other’s understanding of
the issue and related texts and sources
Work with your partner to prepare an outline of talking
points summarizing reasons for and against the issue
(B3) Building a shared plan for perspective taking and
perspective weighing of the issues and related
texts/ideas
Orally present your mutual understanding and analysis of the
issue from these two perspectives.
Come to an agreement about your own position on the issue.
(C3) Integrating self and partner’s ideas into a shared
representation (oral or written) of knowledge about
the issue
What questions do you and your partner still need to find
out more about?
(D2) Monitoring and repairing their shared
understanding
Which of your partner’s points are you still having trouble
agreeing with? What additional information could be used
to support (or refute) this claim or explanation?
(D3) Monitoring and repairing their shared
representation

39. PISA Collaborative Problem Solving > Online Inquiry

40. 3. Building Personal Relationships and
Deepening Learning with Digital Texts &
Tools
 What role(s) do teachers and learners play in the
inquiry process?
 What informs the purposeful selection and use of
digital texts and tools to promote deeper learning?
 Working with:
 Jill Castek, University of Arizona
 Elizabeth Dobler, Emporia State University (Kansas)
 Karen Pelekis (Grade 1 Teacher, New York)
 Amanda Murphy and Erica DeVoe (High School Teachers, History
and ELA, Rhode Island)
 Summer Institute in Digital Literacy (5 years)

41. Learning is social
and part of a mutually
constructive process that
involves face-to-face talking,
listening, and consensus
building.
Personal Digital Inquiry
(Four sets of core values/practices)
Generating questions
and lived experiences
with real issues is
personally fulfilling;
Inquiry can happen
on several levels.
Creative learners make
personal connections
and take action to build
awareness and/or foster
change. “I belong and I can
make a difference”
True inquiry involves
critical analysis, reflection
& self-monitoring,
which leads to
more questions.

42. Dewey 1997/1938
Bruce & Bishop, 2008
Alberta Learning, 2004
Kühn, 1995; Pink, 2009
Schofield & Honore, 2010
Anderson et al, 2013
Palincsar & Brown, 1984
Guthrie & Wigfield, 2004
Casey, 2013;
Jenkins, 2008
Hobbs & Moore, 2013;
Ito et al, 2013; Zhao, 2009
Schön, 1991;
IDEO, 2011
Thomas &
Brown, 2011
Personal Digital Inquiry
(Four sets of core values/practices)

43. Varied Pedagogies to Support Digital Inquiry
• Modeled inquiry: Students observe models of how
the leader ask questions and makes decisions.
• Structured Inquiry: Students make choices which are
dependent upon guidelines and structure given by the
leader (may vary).
• Guided Inquiry: Students make choices during inquiry
that lead to deeper understanding guided by some
structure given by the leader.
• Open Inquiry: Students make all of the decisions.
There is little to no guidance.
Alberta Inquiry Model of Inquiry Based Learning (2004)
Digital Inquiry

44. What do we mean by culture?
Precise
Adapted from Ron Ritchhart’s (2015) Creating Cultures of Thinking

46. include photos, videos, multimodal
& multi-lingual texts with text-to-
speech capability for building
knowledge, deepening
understanding of key concepts, and
increasing motivation with challenge
and support
Digital Texts
enable you and your students to
organize, analyze, annotate,
collaborate, express, reflect,
create, and share that new
knowledge and ideas with others
Digital Tools
Technology = Digital Texts and Digital Tools
that support these 4 sets of core opportunities

47. Knowledge-Based Learning Outcomes
How will students use their knowledge?
Access
Knowledge
Build
Knowledge
Express
Knowledge
Reflect On
Knowledge
Act On
Knowledge
Learners
passively receive
[digital]
information given
or modeled by
others
Learners [use
technology to]
connect new
information to
prior
knowledge
Learners [use
technology
to] share their
new
knowledge
with others
Learners [use
technology to]
reflect on and
evaluate their
inquiry
processes and
products
Learners [use
technology to]
translate their
knowledge into
action for real-
world purpose
Lower Order
Thinking
Higher Order
Thinking
Purposeful Technology Use
How can technology support or enhance learning?
CONSUME INFORMATION
(teacher-directed)
CREATE / PRODUCE INFORMATION
(student-directed)

48. Cognitive Strategies: Read, question, monitor,
repair, infer, connect, clarify, and interpret
Social Practices: Request & give
information; jointly acknowledge,
evaluate, & build on partner’s
contributions
Collaborative F2F
Dialogue for Meaning Making

49. Coiro, Dobler, & Pelekis (forthcoming)
Four places to start to let go

50. 1 2>
3
>

54. OPEN INQUIRY GRADE 12: Global Issue &
Advocacy

55. Learning Task:
Research a
global issue;
Engage in advocacy
on a local level
Blue Pride:
Collected 500
signatures to ban
plastic bags and
use reusable
shopping bags
Next Steps:
Class has ended but
on to legislators…
OPEN INQUIRY GRADE 9: Global Issue & Advocacy
I belong to this community and I can make a difference!

56. OPEN INQUIRY GRADE 9: Global Issue & Advocacy
I belong to this community and I can make a difference!

57. 4. Engaging Students in Skilled
Argumentation & Social Deliberation
Coiro & Killi( 2014-2016)
Elva Knight Research Award

58. Design Features Informed by Four
Theories
 Guide learners through online research and
comprehension process (Leu at el, 2004; 2013)
 Argumentation as learning (Nussbaum, 2008); supports
transactive reasoning (Kruger, 1993) and social deliberation
(Murray et al, 2011)
 Theory of representational guidance (Suthers, 2003) Tools
and notations can guide learners toward practices
considered beneficial for learning (construct, manipulate,
and connect ideas)
 Cognitive load theory: optimize efforts to construct a
cohesive synthesis (Chipperfield, 2006; Kester, Paas, & van
Merriënboer, 2010)
Online Inquiry Tool
Kiili, C., Coiro, J., & Hämäläinen, J. (in press, Spring 2016). An online inquiry tool to support the
exploration of controversial issues on the Internet. Journal of Literacy and Technology.

59. bit.ly/inquirytool3
Kiili, C., Coiro, J., & Hämäläinen, J. (2016). An online inquiry tool to support the
exploration of controversial issues on the Internet. Journal of Literacy and Technology.
https://goo.gl/uiML9t

60. Pilot study results
 Open framework could be used in different
disciplines and for multiple purposes
 Regardless of content area or level of typical
academic performance, most students struggled
with many aspects of the task [when given no
instruction]
 Some provide relevant reasons but few address
counterarguments and/or integrate ideas from
multiple sources into a cohesive synthesis
 Survey results suggest online inquiry tool helped
students organize and monitor their thinking across
perspectives; but suggests this is hard work!
Online Inquiry Tool
Coiro, Killi, Hämäläinen, Cedillo, Naylor, O’Connell, & Quinn (AERA 2015)
https://goo.gl/i8wyjT

61. Test efficacy of tool (and collaboration) to
support online research and skillful
argumentation (n = 300)
Claim 1: Social media increases the
quality of people’s lives
Claim 2: The genetic manipulation of
plants and animals should be allowed.
TOOL / NO TOOL PARTNER / NO PARTNER
Coiro & Killi, 2014-2016 (Elva Knight Research Award)
Research Study

62. Table 1. Average scores on essay variables across four U.S. groups
Question #1: Impact of No Tool Use
vs. Tool Use on Essay Quality
Tool use higher, but not significan

63. Question #2: Individuals vs. Pairs:
Differences on quality of argument graph
88 US Tool Users
But, no significant differences between individuals
and pairs on any of five argument graph variables
INDIVIDUALS
SLIGHTLY HIGHER
PAIRS
SLIGHTLY HIGHER
Planning perspectives
Argumentative reasoning
Evaluating sources
Synthesizing
Total Graph Quality
Julie Coiro and Carita Kiili – ILA Elva Knight Award 2016

64. Question #2: Pairs vs. Individuals
Differences on quality of argument graph
Table 2. Average scores on argument graph variables across all four U.S. groups

65. Julie Coiro and Carita Kiili – ILA Elva Knight Award 2016
Question 3. Impact of Tool Use on Source Evaluations
83% at least one relevant justification across whole sample (much greater than typical);
BUT avg. score was 2.7/5

66. Supporting Online Reading Comprehension,
Collaboration & Digital Inquiry Across the Grade Levels
DIGITAL INQUIRY
Personal Digital
Inquiry (PDI)
(Grades K-12)
Coiro, Castek, & Quinn
(2016); Coiro, Dobler, &
Pelekis (forthcoming)
Productive
Collaborative
Dialogue/Talk
Coiro, Castek,
Sekeres & Guzniczak
CRITICAL
READING/LITERACY
ORCA Project
(2009-2014)
Critically Evaluating
Sources (Grades 5-8)
Coiro, Coscarelli,
Forzani, & Burlingame
(2015)
ARGUMENTATION,
COLLABORATION &
DELIBERATION
(During inquiry)
Online Inquiry Tool:
(Grades 7-12)
Coiro & Kiili, 2014-2016
Online Collaborative
Inquiry (ETS/SAIL)
2016-2018
2003 - 2018

67. Promising Practices for Supporting Online
Reading Comprehension, Personal Digital
Inquiry, and Collaborative Problem Solving
 Understand the role of classroom culture in promoting inquiry-
based learning
 Provide explicit but flexible learning supports that promote
online reading comprehension skills including critical evaluation
 Value contributions of formative, learner-centered assessments
[e.g., screencast recordings of ORCA Project; completed Online
Inquiry Tool; F2F and remote collaboration to help capture
authentic learning and differentiate instruction]
 Connect with flexible supports for professional development
(e.g., collaborative curriculum design – Amber Walraven et al.,
2011)
A Way Forward