Presented by Antonella Poce, Maria Rosaria Re, and Francesca Amenduni (Roma Tre University, Italy) during the 1st European Summit for Critical Thinking Education

1. How to assess Critical Thinking
Through Writing
Antonella Poce, Maria Rosaria Re, Francesca Amenduni
University of Roma Tre, Rome, Italy
antonella.poce@uniroma3.it mariarosaria.re@uniroma3.it
francesca.amenduni@uniroma3.it

2. The assessment of critical thinking turns out to
be a great challenge due to the complexity of the
concept and its underlying dimensions. The
introduction of the California Critical Thinking
Skills Test (CCTST) (1990, updated to 2013, p.
5) reads as follows: «Early assessment of critical
thinking maximizes workforce efficiency and
increases the potential for learning and
educational effectiveness at all levels».
However how to do it?
LET’S DO IT!

3. 1st Activity:
Read, Write and Think
(critically) (20 minutes)

4. But surpassing all stupendous inventions, what sublimity of mind was his who
dreamed of finding means to communicate his deepest thoughts to any other
person, though distant by mighty intervals of place and time! Of talking with
those who are in India; of speaking to those who are not yet born and will not be
born for a thousand or ten thousand years; and with what facility, by the
different arrangements of twenty characters upon a page!
Galileo Galilei - Dialogue Concerning the Two Chief World Systems

5. Your
observations
2nd activity
10 minutes
● Why a writing activity based on
a literary text?
● Why starting from the
paraphrase and then on
comment and analysis?

6. Grasp the subject, and the words will follow
The Verba Sequentur Model
● Literary text and Critical Thinking
People who think critically routinely read
texts that are significant and thus expand
their worldwide. When reading they strive
to accurately represent in their own
thinking what are they are reading in the
text. Close reading requires active
engagement in reading, by creating an
inner dialog with the text (questioning,
summarizing and connecting ideas) - Paul
& Elder, 2006

7. Paraphrasing
Translating an author’ wording into our
own words. In other words, we put the
words and thoughts of the author into our
words.
Explicating and analysing
Stating, elaborating and exemplifying the
thesis of the extract, the author’ purpose,
the most significant information and
concepts.
Reading, writing and thinking (critically)
Descriptive
Level
Metaphorical
Level

8. “Purposeful, self-regulatory judgment, which results in interpretation, analysis, evaluation, and inference, as
well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations
upon which that judgment is based” (Facione, 1990)
Six central cognitive skills:
1. interpretation;
2. analysis;
3. evaluation;
4. inference;
5. explanation;
6. self-regulation.
Five central disposition:
1. Open-mind;
2. Analytical and judicious;
3. Confident in reasoning;
4. Truth-seeking;
5. Systematic inquisitive.

9. The new dimensions related to citizenship
Jiménez-Aleixandre & Puig (2012) proposed including four types of components or dimensions: two related
to purposeful judgment and two related to citizenship.
The second set of dimensions is connected with a second meaning for critical, drawing from critical
theorists, such as Habermas, Bourdieu or Fairclough, as well as from critical educators, such as Célestin
Freinet or Paulo Freire
1. the capacity to develop independent opinions and to challenge socially and culturally established ideas;
2. the capacity to analyze and criticize discourses that justify inequalities, aligned with critical discourse
analysis;

10. Manifesto on Critical Thinking Education
The Manifesto on Critical Thinking education, produced in the Crithinkedu Erasmus +
Project can be used as a guide in fostering the development of CT in HE. Research
suggests that with regard to the development of CT (skills, disposition or combinations of
both) four categories of intervention can be identified:
● To model: CT development is supported when the institute, the teaching program
and the course show what it is to think critically;
● To induce through open-ended questions, ill-structured tasks, complex problems and
real world issues;
● To declare: declaring and making explicit what strategies can be used and what
criteria are to be met. Declaring should be both explicit and specific;
● To surveil through assessment, monitoring, feedback and orientation.

11. Critical Thinking Assessment Methods (Ku, 2009)
Name of the test Disposition Cognitive skills Self-report Multiple-choice Open-
ended
answers
WGCTA - Watson-Glaser Critical Thinking
Appraisal - 1980
X X
CCTST - California Critical Thinking Skills
Test - Facione, 1990
X X
CCTT - Cornell Critical Thinking Test -
Ennis, Millman, & Tomko, 1985
X X
EWCTET - Ennis–Weir Critical Thinking
Essay Test, 1985
X X X
HCTAES - Halpern Critical Thinking
Assessment Using Everyday Situations,
2007
X X X X
CCTDI - California critical thinking
disposition inventory - Facione & Facione,
1992)
X X

12. Open-ended answers advantages
1. Open-ended answers can be used to assess both skills and dispositions (Ku, 2009);
2. Language skills are important precursors for all high-level thought processes. By
regulating thoughts through internal speech and navigating social situations through
external speech, language help people process information at increasingly sophisticated
levels over time, providing foundation to be engaged in CT (Kuhn & Crowell, 2011). Rem
tene, verba sequentur: language shows the quality of thoughts (Poce, 2017);
3. Despite traditional perspectives on CT refer to an individual working on a problem based
task, recent perspectives suggest that CT may evolve thanks to the important role it
could play in various types of information exchange or symbolic interaction (Byrnes &
Dunbar, 2014).

13. Open-ended answers disadvantages
The use of open-ended questions raises
issues of scoring. The high cost and
rater subjectivity are frequent concerns
for human scoring of open ended
questions. In addition, for teachers could
be difficult to design valid assessment
methods (Poce, 2017).
Automated scoring could be a viable
solution to these concerns. (Liu, 2014).

14. Macro-indicators Indicators Descriptors Marks
Use of language Language ability (punctuation, spelling, morphosyntax, lexicon) a. rich and original
b. appropriate
c. mainly correct
d. not precise
e. not correct and improper
Excellent
Very good
Good
Insufficient
Clearly insufficient
5
4
3
2
1
Justification
Argumentation
Elaboration ability (thesis definition and elements of reasoning) a. rich and articulate
b. clear and ordered
c. too synthetic
d. quite consistent
e. inconsistent
Excellent
Very good
Good
Insufficient
Clearly insufficient
5
4
3
2
1
Relevance Consistency (the topic under issue is mentioned) a. complete, deep and original
b. complete and correct
c. generic
d. partial
e. out of line
Excellent
Very good
Good
Insufficient
Clearly insufficient
5
4
3
2
1
Importance Knowledge of the topic (main issues related to the topic are
mentioned)
a. deep and critical
b. complete
c. appropriate
d. superficial
e. not sufficient
Excellent
Very good
Good
Insufficient
Clearly insufficient
5
4
3
2
1
Critical evaluation Personal and critical elaboration of sources and background a. critical and well sounded
b. wide and adequate
c. essential and simple
d. partial
e. contradictory
Excellent
Very good
Good
Insufficient
Clearly insufficient
5
4
3
2
1
Novelty New information, ideas and solutions are added to discuss the
issues raised in the questions
a. widely, critically and originally;
b. in detail;
c. correctly;
d. simply and or partially
e. no new information and solutions are added
Excellent
Very good
Good Insufficient
Clearly insufficient
5
4
3
2
1

15. Now it is your turn! Design an assessment tool to evaluate
students’ level of Critical Thinking in your discipline.

16. ➔ Define your CT learning goals
Based on Facione and Jiménez-Aleixandre & Puig
definitions, on which CT skills or dispositions do you
want to focus on?
➔ Define your activities
Which methods could help you to support the
development of the CT skills and disposition that
you chose?
➔ Define your assessment method
Which tool do you want to use to assess CT skills
and disposition that you chose?