SOLO1.pptx

MARILYN UBIÑA-BALAGTAS, Ph.D.
University Professor, College of Graduate Studies and Teacher E
d
u
c
a
t
i
o
n
Research, Philippine Normal University
President, Philippine Educational Measurement a
n
d
Evaluation Association. Inc. (PEMEA)
balagtas.mu@pnu.edu.ph
Presented this 6th day of December 2021
at DepEd Bulacan//Via Zoom
Use of the SOLO Framework
in Test Development in the
New Normal

Mood S
e
t
t
i
n
g
Use of the SOLO Framework in Test Development in the New Normal
Dr. Marilyn Ubiña-Balagtas
December 2021 2

SESSION TARGET
The participants are expected to use the SOLO Framework in
test development in the new normal.
Topics to Discuss
1. The SOLO Framework and Use in Item Writing, Scoring and
Communication of Test Results
2. The SOLO Framework Combined with the Revised Bloom’s
Taxonomy for Test Development
3. Preparation of the Table of Specifications for
Test Development
December 2021 3

Pre-
Test
https://forms.office.com/r/5yPqL5GdtM
December 2021 4

What is S
O
L
O
?
SOLO to mean Structure o
f Observed Learning
Outcomes was popularized by Kevin F. Collis
and John B. Biggs in 1982 in their
book “Evaluating the Quality of Learning: The
SOLO Taxonomy.
 It is aframework for understanding with five levels, where
each level embraces the previous level but adds
something more.
December 2021 5

Source of Illustration: Doug Belshaw - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=60807631
Understanding/Competence/Proficiency
What is S
O
L
O
?
December 2021 6

Source of Illustration: Doug Belshaw - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=60807631
Understanding/Competence/Proficiency
What is S
O
L
O
?
December 2021 7

What is S
O
L
O
?
December 2021 8

How is SOLO used in t
e
s
tconstruction?
 Use SOLO in structuring r
e
s
p
o
n
s
e
sto questions in a
multiple-choice test format.
 Use SOLO in structuring r
u
b
r
i
c
s in describing
responses to questions in an open-ended or
constructed-response type of test.
 Use SOLO to describe the degree of c
o
r
r
e
c
t
n
e
s
s
of
responses given to questionsin a test.
 Use SOLO in scoring, i
n
t
e
r
p
r
e
t
i
n
g and
communicating test results.
December 2021 9

Level of
Understanding
SOLO
Code Meaning
Levels of
Thinking
Pre-structural (P) 0 I am not sure about this topic
Surface
Uni-structural (U) 1 I have one idea about this topic
Multi-Structural (M) 2 I have several ideas about this topic
Relational (R) 3 I can link the ideas together to see the big
picture
Deep
Extended Abstract
(EA)
4 I can look at this idea in a new and different
way.
e
s
tconstruction?
December 2021 10

Levels of
Proficiency
Level of
Understanding
Code Meaning
Not Proficient Pre-structural 0 One does not know any idea about the lesson
assessed.
Somewhat Uni-structural 1 One knows an idea about the lesson
assessed.
Proficient
Nearly Proficient Multi-Structural 2 One knows several ideas about the lesson
assessed but could not link them together.
Proficient Relational 3 One knows several ideas about the lesson
assessed and can link them together.
Highly Proficient Extended Abstract 4 One can link several ideas about the lesson
assessed and can represent it in a new and
different way/situation.
e
s
tconstruction?
December 2021 11

Levels of
Proficiency
Level of
Understanding
Code Meaning
Level of
Understanding Test Format
Not
Proficient Pre-structural 0 One does not know any idea
about the lesson assessed.
Surface
Multiple
Choice
Constructed-
Response
Somewhat
Proficient Uni-structural 1 One knows an idea about the
lesson assessed.
Nearly Multi-
2 One knows several ideas
about the lesson assessed
Understanding
Proficient
Structural but could not link them
together.
Proficient Relational
and could link them
together.
Deep
Understanding
Highly
Proficient
Extended
Abstract
4 One could link several ideas
and could represent them in a
new or a different
way/situation.
e
s
tconstruction?
December 2021 12

Levels of
Proficiency
Level of
Understanding
Code Meaning
Level of
Understanding Scoring Options
Not
Proficient Pre-structural 0 One does not know any idea
about the lesson assessed.
Surface
Understanding
0 0
Somewhat
Proficient Uni-structural 1 One knows an idea about the
lesson assessed. 1 1
Nearly
Proficient
Multi-
Structural
but could not link them
together.
2 3
Proficient Relational
and could link them
together.
Deep
Understanding
3 5
Highly
Proficient
Extended
Abstract
4 One could link several ideas
and could represent them in a
new or a different
way/situation.
4 6
e
s
tconstruction?
December 2021 13

Let’s have an example
of questioning
in a test that is in
constructed-response
format.
December 2021 14

As of June 30, 2021
How is SOLO used in test c
o
n
s
t
r
u
c
t
i
o
n
?
Study the table below. What percent of the population of the USA got infected of COVID-19 compared
with that of the Philippines?
December 2021 15

Student 1
Philippines
1, 408,058 = 0.01268%
111,025,935
USA
34, 527, 493= 0.1037%
332,929,050
Student 3
Philippines USA
1, 408,058 = 1.27% 34, 527, 493= 10.37%
111,025,935 332,929,050
Student 2
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
10.37% - 1.27% = 9.1%
Who among the four students gave the correct answer?
Student 4
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
e
s
tconstruction?
December 2021 16
Answer: 10.37% of the population of USA got
infected of COVID-19 compared to the
Philippines with only 1.27%.

Student 1
Philippines
1, 408,058 = 0.01268%
111,025,935
USA
34, 527, 493= 0.1037%
332,929,050
Student 2
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
10.37% - 1.27% = 9.1%
How do the students differ in their understanding of the problem based on their answers?
This student knows how to compute the rate but
does not know how to express this in percent.
This student knows how to compute the rate and how to express it in
percent but misunderstood the problem as it does not require subtraction.
Student 3
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
This student knows how to
to express it in percent but
compute the rate and how
did not provide clearly the
correct answer.
Student 4
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
Answer: 10.37% of the populat
COVID-19 compared to the Phi
ion of USA got infected of
lippines with only 1.27% .
This student knows how to compute the rate and how to
express it in percent to answer the problem.
e
s
tconstruction?
December 2021 17

Student 1
Philippines
1, 408,058 = 0.01268%
111,025,935
USA
34, 527, 493= 0.1037%
332,929,050
Student 3
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
Student 2
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
10.37% - 1.27% = 9.1%
This student knows how to compute the rate but does not
know how to express this in percent. (UNISTRUCTURAL)
percent but did not provide clearly the correct answer.
(MULTISTRUCTURAL)
percent but misunderstood the problem as it does not require
subtraction. (MULTISTRUCTURAL)
Student 4
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
Answer: 10.37% of the population of USA got infected of
COVID-19 compared to the Philippines with only 1.27%.
express it in percent to answer the problem. (RELATIONAL)
Which of the SOLO levels does each response represent?
e
s
tconstruction?
December 2021 18

19
Student 3
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
How could Extended Abstract in the SOLO framework look like?
Student 4
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
COVID-19 compared to the Philippines with only 1.27%.
Student 1
Philippines USA
1, 408,058 = 0.01268% 34, 527, 493= 0.1037%
111,025,935 332,929,050
This student knows how to compute the rate but does not
know how to express this in percent. (UNISTRUCTURAL)
Student 2
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
Answer:
Philippin
10.37% of the population of USA got
es with only 1.27% COVID cases. Th
i
i
s
nfected of COVID-19 compared to th
explains why USA is no.1 in the wor
e
ld in
terms of COVID cases while the Philippines is just ranked 24th.
percent to answer the problem. The student also gave more acceptable
information than what is needed. (EXTENDED ABSTRACT)
express it in percent but did not provide clearly the correct
answer. (MULTISTRUCTURAL)
express it in percent to answer the problem. (RELATIONAL)
e
s
tconstruction?
Dr. Marilyn Ubiña-Balagtas//December 2021

Levels of Quality
Adjectival
Description
Indicator SOLO Level
5 Excellent
The answer given is correct and beyond what was
asked for.
Extended Abstract
4 Very Good
The answer given shows understanding of two or more
processes in solving the problem and it directly answers
the problem.
Relational
3 Good
The answer given shows understanding of two or more
processes in solving the problem but does not directly
answer the problem.
Multistructural
2 Fair
The answer given shows understanding of one process
that leads to answering the problem.
Unistructural
1 Poor
There is NO attempt to answer the problem or there is
an answer but meaningless.
Prestructural
e
s
tconstruction?
How will the rubric be designed using SOLO?
December 2021
20

Levels of
Quality
Adjectival
Description
Indicator SOLO Level Code
Example 1
of Scoring
Example 2
of Scoring
5 Excellent The answer given is correct and beyond
what was asked for.
Extended
Abstract
4 4 Points 10 Points
4 Very Good The answer given shows understanding of
two or more processes in solving the
problem and directly answers the problem.
Relational
3 3 Points 8 Points
3 Good The answer given shows understanding of
two or more processes in solving the
problem but does not directly answer the
problem.
Multistructural
2 2 Points 6 Points
2 Fair The answer given shows understanding of
one process that leads to answering the
problem.
Unistructural
1 1 Point 3 Points
1 Poor There is NO attempt to answer the problem
or there is an answer but meaningless.
Prestructural 0 0 Point 0 Point
e
s
tconstruction?
How are responses scored based on SOLO?
December 2021
21

22
Student 1
Philippines
1, 408,058 = 0.01268%
111,025,935
USA
34, 527, 493= 0.1037%
332,929,050
Student 3
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
Student 2
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
This student knows how to compute the rate but does not know how to
express this in percent. (UNISTRUCTURAL) – 3 Points
Student 4
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
COVID-19 compared to the Philippines with only 1.27%
COVID cases.
percent and how to answer the problem. (RELATIONAL) – 8 Points
Answer: 10.37% of the population of USA got infected of COVID-19 compared to
the Philippines with only 1.27% COVID cases. This explains why USA is no.1 in
the world in terms of COVID cases while the Philippines is just ranked 24th.
This student knows how to compute the rate and how to express it in percent
and how to answer the problem. The student also gave more information than
what is needed. (EXTENDED ABSTRACT) – 10 Points
percent but did not provide the clearly the correct answer.
(MULTISTRUCTURAL) – 6 Points
How are responses scored based on SOLO?
e
s
tconstruction?

How do we transform the
constructed-response
item into a
multiple-choice format?
December 2021 23

A. 10.37% of the population of USA got
B. 0.1037% of the population of USA got
C. 0.1037 of the population of USA got
Philippines with only 0.01268.
D. USA has 9.1% more COVID cases than
the Philippines.
e
s
tconstruction?
How do we transform the
December 2021 24
Study the table below. What percent of the population of the USA got infectedc
o
o
f
n
C
s
O
t
r
V
u
I
c
D
t
-
e
1
d
9
-
c
r
e
o
s
m
p
p
o
a
n
r
s
e
e
d
item into a
multiple-choice format?
As of June 30, 2021

As of June 30, 2021 Options
SOLO
CODE
SOLO Levels
A
10.37% of the population of
USA got infected of COVID-19
compared to the Philippines
with only 1.27%.
3
Relational
B.
with only 0.01268%.
1
Unistructural
C.
0.1037 of the population of
with only 0.01268.
1
D.
USA has 9.1% more COVID
cases than the Philippines.
2
Multistructural
e
s
tconstruction?
December 2021 25

Options SOLO
Explanation/
Feedback
A
with only 1.27%.
3
You know how to
compute the rate and
how to express it in
percent to answer the
problem.
B.
with only 0.01268%.
1
You know how to
compute the rate, but
you do not know how to
express this in percent.
C.
0.1037 of the population of
with only 0.01268.
1
D. USA has 9.1% more COVID
cases than the Philippines.
2 You know how to
compute the rate and
how to express it in
percent, but you
misunderstood the
problem as it does not
require subtraction.
Study the given graph. What percent of the population of the USA got infected of COVID-19 compared
As of June 30, 2021
How do we communicate test results using the SOLO Framework?
December 2021 26

SESSION TARGET
Topics to Discuss
2. The SOLO Framework Combined with t
h
e
Revised Bloom’s
Test Development
December 2021 27

Dimensions Remember Understand Apply Analyze Evaluate Create
Factual
least student-centered
& authentic 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
Metacognitive 4 8 12 16 20
most student-
centered &
authentic
How do we combine SOLO withRBT to make
assessment multidimensional?
Revised Bloom’s T
a
x
o
n
o
m
y
(Krathwohl & A
n
d
e
r
s
o
n
,2001)
December 2021 28

Remember:
Retrieving, recognizing, and recalling relevant know
ledgefrom long-
term memory.
Understand:
Constructing meaning from oral, written, and graphic m
e
s
s
a
g
e
s
through interpreting, exemplifying, classifying, inferring, comparing
and explaining.
Apply:
Carrying out or using a procedure through e
x
e
c
u
t
i
n
g
,
or implementing.
Analyze:
Breaking material into constituent parts, determining h
o
w
the parts relate to one another and to an overall
structure or purpose through differentiating, organizing, and
attributing.
Evaluate:
Making judgments based on criteria and standards
through checking and critiquing.
Create:
Putting elements together to form a coherent or f
u
n
c
t
i
o
n
a
l
whole; reorganizing elements into a new pattern or
structure through generating, planning, or producing.
Cognitive Behaviors in the Revised Bloom’s Taxonomy
If you aim for
authentic
assessment,
which of these
levels should
you target?
December 2021 29

Knowledge Dimensions in the
Revised Bloom’s Taxonomy
Knowledge
Dimension
Description
Factual Terminologies, specific details and elements
Conceptual Knowledge of classifications, principles ,
generalization, theories, models, and structures
Procedural Knowledge of subject-specific skills and
algorithms/procedures
Metacognitive Self-knowledge, strategic knowledge, contextual
and conditional knowledge, knowledge of a
cognitive task
If you aim
for
authentic
assessme
nt, which of
these
levels
should you
target?
December 2021 30

Factual 1 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
Metacognitive 4 8 12 16 20 24
(Krathwohl & A
n
d
e
r
s
o
n
,2001)
Which is the most learner-centered and
authentic cell in this matrix?
December 2021 31

Factual & authentic 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
Metacognitive 4 8 12 16 20 most student-
centered &
authentic
(Krathwohl & A
n
d
e
r
s
o
n
,
2001)
December 2021 32

Factual
& authentic 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Procedural 3 15 19 23
most student-
centered &
authentic
1. W
7
hat is 9 in
11
4 +
5 = 9?
(Krathwohl & A
n
d
e
r
s
o
n
,
2001)
December 2021 33

Factual
& authentic 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
Metacognitive 4 2. Wh8at is addit1io2n? 16 20 most student-
centered &
authentic
(Krathwohl & A
n
d
e
r
s
o
n
,
2001)
December 2021 34

Factual
& authentic 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Metacognitive 4 16 20
most student-
centered &
authentic
3
7. What is1
th
1e 1st
step in adding
n
8umbers?12
(Krathwohl & A
n
d
e
r
s
o
n
,
2001)
December 2021 35

How many ripe
Factual
& authentic 5 9 13 17 21
Conceptual 2 6
4. Maria b
o
u
g
1h
0t4 ripe
14 18 22
Procedural 3 7
and 5 unripe1
m1
angoe s. 15 19 23
Metacognitive 4 8mangoes did1s2hebuy? 16 20
most student-
centered &
authentic
(Krathwohl & A
n
d
e
r
s
o
n
,
2001)
December 2021 36

37
Dimensions Remember Analyze Evaluate Create
Factual
& authentic 13 21
23
1
. 7
Show
Understand A
p
p
l
y
11. W
5
hat is the sum9
of
3. What is the first
step when adding
numbers?
Conceptual 2
4 and
6
5?
10 14
ho
1w
8toadd
22
any number.
Procedural 3 7 11 15 19 23
Metacognitive 4 8 12
15. How does the
16 20
most student-
centered &
authentic
sum of 4 and 5
compare with the
sum of 6 and 3?
19. Is the sum of 4 and
5 10?
7. How will you show
addition of 4 and 5?
(Krathwohl & A
n
d
e
r
s
o
n
,
2001)

38
Factual
& aut hentic 21
Conceptual 2 6 14 18 22
most student-
centered &
authentic
24. You need to buy items
at the store. What are these
and their total?
4.Maria bought 4 ripe and 5
unripe mangoes. How many
ripe mangoes did she buy?
8.Maria bought 4
ripe and 5 unripe
mangoes. Which
is more, ripe or
unripe mangoes?
20.Maria bought 4 ripe and
5 unripe mangoes. She
bought 10 mangoes in all.
Do you agree? Explain.
12.Maria bought 4 ripe5and 5
unripe mangoes. How many
mangoes did she buy?
916.Maria bou
1
g3
ht 4 ripe and 1
5 7
unripe mangoes today. She
10
bought 10 mangoes yesterday.
How many mangoes did she buy
11
in all? 15
(Krathwohl & A
n
d
e
r
s
o
n
,
2001)

Let’s see more
examples of how to
increase the
authenticity of items
guided by the RBT.
December 2021 39

Which is more authentic and targets higher thinking
skill? Explain your answer.
1. What is 15 + 5?
A.10
B. 15
C. 20
2.During their morning break, Romalyn bought pancit canton worth Php15 and
a glass of coconut juice worth Php5. How much did Romalyn spend for her
snacks?
A. Php 10
B. Php15
C. Php 20
December 2021 40

1. What is 15 + 5?
A.10
B. 15
C. 20
2. During their morning break, Romalyn bought pancit canton worth Php15 and
a glass of coconut juice worth Php5. How much did Romalyn spend for her
snacks?
A. Php 10
B. Php15
C. Php 20
Add – Apply - Procedural
Add/Solve
Apply/Analyze
Procedural/Metacognitive
December 2021 41

1.During their morning break, Romalyn bought pancit canton worth Php15 and a glass of
coconut juice worth Php5. How much did Romalyn spend for her snacks?
A. Php 10
B. Php15
C. Php 20
2.During your morning break, you bought a pancit canton worth Php15 and a glass of
coconut juice worth Php5. How much did you spend for your snacks?
A. Php 10
B. Php15
C. Php 20
December 2021 42

1. During their morning break, Romalyn bought pancit canton worth Php15 and a glass of
coconut juice worth Php5. How much did Romalyn spend for her snacks?
A.Php 10
B. Php15
C.Php 20
2. During your morning break, you bought a pancit canton worth Php15 and a glass of
coconut juice worth Php5. How much did you spend for your snacks?
A.Php 10
B. Php15
C. Php20
Add/Solve
Apply/Analyze
Add/Solve
Apply/Analyze
Personal/
authentic
December 2021 43

1.During your morning break, you bought a pancit canton worth Php15 and a
glass of coconut juice worth Php5. How much did you spend for your snacks?
A.Php 10
B. Php15
C. Php20
2. During your morning break, you bought worth Php and
worth Php . How much did you spend for your snacks?
December 2021 44

1. During your morning break, you bought a pancit canton worth Php15 and a
glass of coconut juice worth Php5. How much did you spend for your snacks?
A.Php 10
B. Php15
C. Php20
Add/Solve
Apply/Analyze
Personal/
authentic
Add/Solve/Create
Apply/Analyze/Create
Personal/
authentic
December 2021 45

2. Construct your own word problem involving addition of whole numbers.
December 2021 46

1. During your morning break, you bought
worth Php and worth Php .
How much did you spend for your snacks?
2. Construct your own word problem involving
addition of whole numbers.
Add/Solve/Create
Personal/
Authentic/Restricted
Add/Solve/Create
Personal/
Authentic/Extended
December 2021 47

Why combine
the RBT and SOLO
in test development?
December 2021 48

Why combine the SOLO F
r
a
m
e
w
o
r
k
and the
RBT?
 Combining the SOLO Framework and the RBT makes assessment
multidimensional.
 Multidimensional assessment of learning is an approach of looking at different
facets of learning to identify the strengths and weaknesses of the learner to better
inform instruction and better describe achievement.
 Assessment is multidimensional if it satisfies any of the fo
l
l
o
win
g
:
1. It examines two or more dimensions in one’s learning (e.g., Revised Bloom’s
Taxonomy, PISA).
2. It examines both cognitive and non-cognitive dimensions of learning (
e
.
g
.
,TIMSS)
December 2021 49

Why do we need to p
r
a
c
t
i
c
e
multidimensionala
s
s
e
s
s
m
e
n
t
?
 Multidimensional assessment of l
e
a
r
n
i
n
gis needed in the K to 12
program as the curriculum intends to develop
holistically the learners and equip them with 21st
century skills to make them locally and globally
competitive.
 The curriculum framework of the d
i
f
f
e
r
e
n
t areas in
basic and higher education shows multidimensionality,
hence assessment should be multidimensional as well.
December 2021 50

The curriculum framework shows multidimensionality; hence assessment should be multidimensional as well.
Why do we need to p
r
a
c
t
i
c
e
multidimensionala
s
s
e
s
s
m
e
n
t
?
Example of the Multidimensional Curriculum Framework
December 2021 51

The curriculum framework shows multidimensionality;
hence assessment should be multidimensional as well.
Why do we need to p
r
a
c
t
i
c
e
multidimensionala
s
s
e
s
s
m
e
n
t
?
Example of the Multidimensional Curriculum Framework
December 2021 52

What are the m
o
d
e
l
s
of
multidimensional
assessment?
Some models of multidimensional assessment of l
e
a
r
n
i
n
g
that are practiced in the Philippines are as follows:
Classroom
1. Revised Bloom’s Taxonomy
2. Revised Bloom’s Taxonomy combined with the
SOLO Framework
3. Combination of Different Assessment Approaches that
Appropriately Assess Different Learning Outcomes
December 2021 53

What are the m
o
d
e
l
s
of
multidimensional
assessment?
Some models of multidimensional assessment of learning that are
practiced in the Philippines are as follows:
National
1. Assessment of Competencies and Excellence of Students
(ACES) of Rex Institute for Student Excellence, Inc.
2. Basic Education Exit Assessment (BEEA)Framework
December 2021 54

What are the m
o
d
e
l
s
of
multidimensional
assessment?
Assessment of Competencies and Excellence of Students
(ACES) from Rex Institute for Student
E
x
c
e
l
l
e
n
c
e
,Inc. (RISE)
December 2021 55

What are the m
o
d
e
l
s
of
multidimensional
assessment?
E
x
c
e
l
l
e
n
c
e
,Inc. (RISE)
December 2021 56

What are the m
o
d
e
l
s
of
multidimensional
assessment?
E
x
c
e
l
l
e
n
c
e
,Inc. (RISE)
December 2021
57

v
SHS Exit Assessment
 Language and Communication
 Wika at Komunikasyon
 Humanities
 Media and Information Literacy
 Mathematics
 Science
 Social Science and Personal
Development
 Philosophy
Content
Knowledg
e
Performanc
e
Standard
s
Competencie
s
21st Century Skills
Basic Education Exit Assessment (BEEA) for SHS
What are the m
o
d
e
l
s
of
multidimensional
assessment?
Source: Balagtas et.al (2020). Analysis of the
Exit Assessment for Senior High School
Students: Input to the K to12 Reform

What are the m
o
d
e
l
s
of
multidimensional
assessment?
Some models of multidimensional assessment of learning t
h
a
t
are practiced in the Philippines are as follows:
International
1. Program for International Student Assessment (PISA)
2. Trends in International Mathematics and Science Survey (TIMSS)
3. Southeast Asia Primary Learning Metric (SEA-PLM)
December 2021 59

Knowledge
PISA 2015/2018 Scientific Literacy
Assessment Framework
for Cognitive Dimension
Competencies
What are the m
o
d
e
l
s
of
multidimensional
assessment?
December 2021

PISA 2015/2018
Scientific
Literacy
Assessment
Item Framework
COMPETENCIES
• Explaining
phenomena
scientifically
• Evaluating and
designingscientific
inquiry
• Interpreting data
and evidence
scientifically
KNOWLEDGE
• Content
• Procedural
• Epistemic
COGNITIVE
DEMAND
• Low
• Medium
• High
CONTEXTS
• Personal
• Local/National
• Global
What are the m
o
d
e
l
s
of
multidimensional
assessment?
December 2021

What are the m
o
d
e
l
s
of
multidimensional
assessment?
Figure #: Mathematics Framework of the PISA 2021
PISA 2021 F
r
a
m
e
w
o
r
k
for Mathematics
Literacy
December 2021 62

What are the m
o
d
e
l
s
of
multidimensional
assessment?
December 2021 63

What are the m
o
d
e
l
s
of
multidimensional
assessment?
December 2021 64

What are the m
o
d
e
l
s
of
multidimensional
assessment?
December 2021 65

How do we combine
the RBT and SOLO
December 2021 66

How do we combine the R
B
T
and SOLO in
test development?
Objective: Identify the different cognitive processes in the Revised Bloom’s
Taxonomy
Item
1. What cognitive process in the RBT is involved when a student is required to
organize data into various categories?
A. analyze
B. apply ?
C. classify
• What is the answer to the?
does the item represent?
What levels of
understanding based on
the SOLO Framework do
the choices represent?
67
•
D. understand ?
December 2021
item?
• What cognitive process
and knowledge dimension?

B
T
and SOLO in
test development?
Taxonomy
Item
A. analyze
B. apply ?
C. classify
Factual - Remember
• What is the answer to the?
What levels of
68
•
D. understand ?
December 2021
item?
and knowledge dimensio?n

B
T
and SOLO in
test development?
Taxonomy
Item Factual - Remember
A. analyze ?
B. apply Do the options in the MC item
have the same ?
C. classify degree of correctness? ?
D. understand ?
December 2021 69

B
T
and SOLO in
test development?
Taxonomy
A. analyze How will you describe the ?
B. apply offered responses ?
C. classify in terms of SOLO? ?
D. understand ?
December 2021 70

B
T
and SOLO in
test development?
Taxonomy
A. analyze Relational
B. apply Uni-structural
C. classify Uni-structural
D. understand Uni-structural
December 2021 71

B
T
and SOLO in
test development?
Taxonomy
Item
B. apply Unistructural
C. classify Unistructural
D. understand Unistructural
Factual - Remember
How will you represent
pre-structural in this
December 2021 72
set of responses?

B
T
and SOLO in
test development?
Taxonomy
Item
B. apply Unistructural
C. classify Unistructural
D. characterize Prestructural
Factual - Remember
How will you represent
pre-structural in this
December 2021 73
set of responses?

Let’s have another
example of how to combine
RBT and SOLO
in test development.
December 2021 74

A. Examine the instructional objectives
B. Prepare a table of specifications
C. Construct test items
D. Teach the students
Objective: Identify the steps in developing summative tests
Example:
Which is the first step in constructing an Achievement Test ?
B
T
and SOLO in
test development?
• What is the answer to the
item?
and knowledge dimension
• What levels of
December 2021
75

Example: Procedural – Remember
Which is the first step in constructing an Achievement Test ?
A. Examine the instructional objectives Relational
B. Prepare a table of specifications Multistructural
C. Construct test items Unistructural
D. Teach the students Prestructural
Objective: Identify the steps in developing summative tests
B
T
and SOLO in test
development?
December 2021 76

A. Give a parallel test.
B Teach the lesson again in the same way.
C. Count the frequency of errors to know what to explain again.
D. Determine the items students found difficult that would require
further explanation.
Objective: Use assessment results in making decisions about teaching and learning
Situation:
After teaching the day’s lesson, you gave your students a test. The result shows that almost all of
the students got a low score. Which of the following should you do?
item?
• What levels of
B
T
and SOLO in
test development?
December 2021
77

Situation: Metacognitive - Apply
A. Give a parallel test. Uni-structural
B Teach the lesson again in the same way. Uni-structural
C. Count the frequency of errors to know what to explain again. Multi-structural
D. Determine the items students found difficult that would require
further explanation.
Relational
B
T
and SOLO
December 2021 78

Situation:
Somewhat
Proficient
A. Review its pre-requisite lesson. Uni-structural 1 point
B. Teach the lesson again in the same way. Uni-structural 1 point Somewhat
Proficient
Nearly Proficient
C. Count the frequency of errors to know what Multi-structural 2 points
to explain again.
D. Determine the items students found difficult Relational 3 points
that would require further explanation.
Proficient
Metacognitive - Apply
How do we score test items using the SOLO Framework?
December 2021 79

Let’s have another
example of how to combine
RBT and SOLO
in test development.
December 2021 80

Objective: Justify the appropriateness of an assessment method given the
objective of the lesson (Constructed-Response Format).
Situation:
You want to find out if your students can make a new material out of recyclable
materials. You gave an essay test requiring them to explain what to do out of old
telephone directories. Considering the congruency between the intent and the
assessment method used, did you do what is right? Justify your answer.
B
T
and SOLO in
test development?
December 2021 81

objective of the lesson (Constructed-Response Format).
Situation:
You want to find out if your students can make a new
material out of recyclable materials. You gave an essay
test requiring them to explain what to do out of old
telephone directories. Considering the congruency
between the intent and the assessment method used,
item?
• What levels of
B
T
and SOLO in test
development?
the SOLO Framework
could the answers
represent?
did you do what is right? Justify your answer.
December 2021 82

objective of the lesson (Constructed-Response Format). Evaluate-Metacognitive
Situation:
You want to find out if your students can make a new
material out of recyclable materials. You gave an essay
test requiring them to explain what to do out of old
telephone directories. Considering the congruency
between the intent and the assessment method used,
item?
• What levels of
B
T
and SOLO in test
development?
the SOLO Framework
could the answers
represent?
December 2021 83

objective of the lesson (Constructed-Response Format). Evaluate-Metacognitive
Situation:
You want to find out if your students can
make a new material out of recyclable
materials. You gave an essay test requiring
them to explain what to do out of old
telephone directories. Considering the
congruency between the intent and the
assessment method used, did you do what
is right? Justify your answer.
Sample Answers Given
• Example 1:
• Example 2:
• Example 3:
• Example 4:
B
T
and SOLO in test
development?
December 2021 84

Objective: Justify the appropriateness of an assessment method given the objective of the lesson
(Constructed-Response Format).
Situation: You want to find out if your students can make a new material out of recyclable materials. You gave an
essay test requiring them to explain what to do out of old telephone directories. Considering the congruency
between the intent and the assessment method used, did you do what is right? Justify your answer.
Scoring Rubric
1 Poor You gave no response or showed no effort to respond.
2 Fair You gave a response but with no justification.
3 Good You gave a response with a justification that is partially acceptable or not applicable in the given
situation.
4 Very Good You gave a response with a justification that is totally acceptable or applicable in the given situation.
5 Excellent You gave a response with a very convincing justification that is acceptable in most situations and not
only in the given situation.
B
T
and SOLO in
test development?
December 2021 85

Objective: Justify the appropriateness of an assessment method given the objective of the lesson (Constructed-
Response Format).
Situation: You want to find out if your students can make a new material out of recyclable
materials. You gave an essay test requiring them to explain what to do out of old telephone
directories. Considering the congruency between the intent and the assessment method used,
1 Poor You gave no response or showed no effort to respond. Pre-structural
2 Fair You gave a response but with no justification. Uni-structural
3 Good You gave a response with a justification that is partially acceptable or not applicable
in the given situation.
Multi-structural
4 Very Good You gave a response with a justification that is totally acceptable or applicable in the
given situation.
Relational
5 Excellent You gave a response with a very convincing justification that is acceptable in most
situations and not only in the given situation.
Extended
Abstract
B
T
and SOLO in
test development?
December 2021
86

Response Format).
Situation: You want to find out if your students can make a new material out of recyclable materials. You gave an essay test
requiring them to explain what to do out of old telephone directories. Considering the congruency between the intent and the
1 Poor You gave no response or showed no effort to respond. Pre-structural
2 Fair You gave a response but with no justification. Uni-structural
3 Good You gave a response with a justification that is partially acceptable or not applicable in the given situation. Multi-structural
4 Very Good You gave a response with a justification that is totally acceptable or applicable in the given situation. Relational
5 Excellent You gave a response with a very convincing justification that is acceptable in most situations and not only in
the given situation.
Extended Abstract
B
T
and SOLO in
test development?
Sample Answers Given
• Example 3: Dr. Marilyn Ubiña-Balagtas
December 2021
87
• Example 1:
• Example 2:

88
How do we score test items using the SOLO Framework?
Response Format).
Situation: You want to find out if your students can make a new material out of recyclable materials. You gave an essay test
requiring them to explain what to do out of old telephone directories. Considering the congruency between the intent and the
Meta-cognitive - Evaluate
1 Poor You gave no response or showed no effort to respond. Pre-structural 0 Point Not Proficient
2 Fair You gave a response but with no justification. Uni-structural 2 Points Somewhat Proficient
3 Good You gave a response with a justification that is
partially acceptable or not applicable in the given
situation.
Multi-structural 5 Points Nearly Proficient
4 Very Good You gave a response with a justification that is totally
acceptable or applicable in the given situation.
Relational 8 Points Proficient
5 Excellent You gave a response with a very convincing
justification that is acceptable in most situations and
not only in the given situation.
Extended
Abstract
10 Points Highly Proficient
December 2021 88

Situation: You want to find out if your students can make a new material out of recyclable materials. So you gave
an essay test requiring students to explain what to do out of old telephone directories. Considering the
congruency between the intent and the assessment method used, did you do what is right? Justify your answer.
A. Yes, the use of essay test is just appropriate to what is being measured.
B. No, because it is difficult to check the students’ responses to the task using an essay test.
C. No, it is more appropriate to observe them while they are creating something new out of recyclable
materials
D. Yes, asking the students to explain what to do with the material is already enough to gauge what
they are able to do.
B
T
and SOLO in
test development?
December 2021 89

Situation: You want to find out if your students can make a new material out of
recyclable materials. So you gave an essay test requiring students to explain what to do
out of old telephone directories. Considering the congruency between the intent and
the assessment method used, did you do what is right? Justify your answer.
• What is the answer
to the item?
• What cognitive
process and
knowledge
dimension does the
item represent?
• What levels of
understanding based
on the SOLO
Framework could the
choices represent?
B. No, because it is difficult to check the students’ responses to the task using
an essay test.
C. No, it is more appropriate to observe them while they are creating
something new out of recyclable materials
D. Yes, asking the students to explain what to do with the material is already
enough to gauge what they are able to do.
B
T
and SOLO in
test development?
December 2021 90

Situation: You want to find out if your students can make a new material out of recyclable materials. So you gave
an essay test requiring students to explain what to do out of old telephone directories. Considering the
congruency between the intent and the assessment method used, did you do what is right? Justify your answer.
B. No, because it is difficult to check the students’ responses to the task using an essay test.
C. No, it is more appropriate to observe them while they are creating something new out of recyclable
materials
D. Yes, asking the students to explain what to do with the material is already enough to gauge what
they are able to do.
B
T
and SOLO in
test development?
December 2021 91

92
Situation:
Situation: You want to find out if your students can make a new material out of recyclable materials. So you gave an
essay test requiring students to explain what to do out of old telephone directories. Considering the congruency
between the intent and the assessment method used, did you do what is right? Justify your answer.
A. Yes, the use of essay test is just appropriate to what is being measured. Uni-structural
(1 point)
B. No, because it is difficult to check the students’ responses to the task using an essay
test.
Multi-structural
(2 points)
C. No, it is more appropriate to observe them while they are creating something new
out of recyclable materials
Relational (3 Points)
D. Yes, asking the students to explain what to do with the material is already enough to
gauge what they are able to do.
Uni-structural
(1 Point)
B
T
and SOLO in
test development?

Some Rules in Using SOLO for Test Development
1. Construction of Choices:
2. Scoring of the Responses:
unl
3. Interpretation and Feedback to Responses: lain sufficiently what
makes the response correct/incorrect so that students will learn from
th feedback provided. Indicate the proficiency/competence level of
the le rner from the kind of response given.
December 2021 93

Let’s look at
examples of questioning
in PISA.
December 2021 94

Let’s look at a sample test in PISA Math.
Source: PISA Released
Items in Mathematics,
December 2006, OECD
December 2021
Ans: D
95

Let’s look at a sample test in PISA Math.
Source: PISA Released Items in Mathematics, December 2006, OECD
Choices
SOLO
Level
Score/
Code
A. 6 mg Unistructural 1
B. 12 mg Unistructural 1
C. 26 mg Unistructural 1
D. 32 mg Relational 3
How do we apply SOLO in explaining
students’ responses to the item?
December 2021
Ans: D
96

OECD
Choices
SOLO
Level
Score/
Code
Explanation/
Feedback
A. 6 mg Unistructural 1 You gave one correct
idea that is based on
the given information.
B. 12 mg Unistructural 1
C. 20 mg Unistructural 1
D. 32 mg Relational 3 You interpreted
correctly the graph
that led you to
giving the right
answer to the
problem.
Source: PISA Released Items in Mathematics, December 2006,
December 2021
Ans: D
97

Let’s look
at a
sample
test item
in PISA
Reading.
Items in Reading,
December 2006, OECD
December 2021
Ans:
A
98

Let’s look at a sample test item in PISA Reading.
Source: PISA Released Items in Reading, December 2006, OECD
Choices
SOLO
Level
Score/
Code
A. About two meters R 3
B. About 15 meters U 1
C. About 50 meters P 0
D. It has disappeared
completely.
U 1
E. The information is
not provided.
P 0
December 2021
Ans:
A
99

Choices
SOLO
Level
Score/
Code
Explanation/
Feedback
A. About two
meters
R 3 You interpreted
correctly the graph that
led you to giving the
right answer to the
problem.
B. About 15
meters
U 1 You gave one correct
the information given.
C. About 50
meters
P 0 You gave a response
that has no basis at all.
D. It has
disappeared
completely.
U 1 You gave one correct
the information given.
E. The information
is not provided.
P 0 You gave a response
that has no basis at all.
Source: PISA Released Items in Reading, December 2006, OECD
Ans:
A
December 2021
100

Let’s look
at a
sample
test item
in PISA
Science.
Items in Science,
December 2006, OECD
December 2021
Ans:
A
101

Let’s look at a sample test item
in PISA Science.
Source: PISA Released Items in
Science, December 2006, OECD
Choices
SOLO
Level
Score/
Code
A. The Earth rotates on
its axis.
R 3
B. The Sun rotates on
its axis. U 1
C. The Earth’s axis is
tilted. M 2
D. The Earth revolves
around the Sun M 2
December 2021
Ans:
A
102

Source: PISA Released Items in
Science, December 2006, OECD
Ans
:
103 A
Choices
SOLO
Level
Score/
Code
Explanation/
Feedback
A. The Earth
rotates on its
axis.
R 3 You were able to give the right
explanation why daylight and
darkness occur on Earth.
B. The Sun rotates
on its axis.
U 1 You know that the sun rotates
on its axis but this does not
explain day and night on Earth.
C. The Earth’s axis
is tilted.
M 2 You know that the Earth’s axis
is tilted which was explained in
the information provided but it
does not explain the
occurrence of day and night on
Earth.
D. The Earth
revolves around
the Sun
M 2 You know that the Earth
revolves around the Sun which
is the source of light but it
does not fully explain the
occurrence of day and night as
that is due to Earth’s rotation
on its axis.
December 2021

SESSION TARGET
Topics to Discuss
2. The SOLO Framework Combined with the Revised Bloom’s
Test Development
December 2021 104

Content Competencies
Inst’l Time
(hrs) # of Items % of items
Level of Behavior, Format, No.,& Placement of Items and
the Knowledge Dimension
R U Ap An E C
1.Method of
Assessment
Examine the
good and bad
practices in
assessing
student’s
learning
3 10 20% I. 1
#1 (F)
I. 2
#6-7
(C)
I. 1
#15-16
(M)
1. 3
#17-19
(M)
I. 2
#36-38
(M)
II. 39-
40
(M)
Etc.
Total No. of Items 15 50 100% 5 10 20 10 3 2
Scoring 3 pts - maximum
for each item
4 pts - maximum
for each item
5 pts - maximum
for each item
SOLO Codes P-0; U-1; M-2; R-3 P-0; U-1; M-2; R-4 P-0; U-1; M-3; R-5
TotalNumber
of Points
190 points 45 Points 120 Points 25 Points
B
T
and SOLO in test
development?
Balagtas’ Model for Test Development with RBT and SOLO Combined
December 2021 105

Content Inst’l Time
# of Items
% of items KD
Level of Behavior, Item Format, No.,& Placement
R U Ap An E C
1.Methodso
f
Assessment
3 10 20%
F I. 1
#1
C I .2
#6-7
I. 1
#16
P I. 3 #17
to 19
II. 2
#39-40
M I. 2
#36-37
2. etc.
Total 15 50 100% 5 10 20 10 3 2
Scoring 3 pts - maximum
for each item
4 pts - maximum
for each item
5 pts - maximum
for each item
SOLO Codes P-0; U-1; M-2; R-3 P-0; U-1; M-2; R-4 P-0; U-1; M-3; R-5
Total Number of Points 190 points 45 Points 120 Points 25 Points
B
T
and SOLO in test
development?
December 2021
106

Prepare a Grade Level Achievement Test for the Division
Things to Do
December 2021 107
1. Prepare a Table of Specifications (ToS) for an
Achievement Test of a grade level. The number
of subject areas taught in the grade level will be
the same number of ToS to be prepared.
2. Use the suggested template in the preparation
of the ToS of an Achievement Test for a grade.
Dr. Marilyn Balagtas' Model TOS Template for Test Development with RBT and
SOLO Combined
https://tinyurl.com/pmz4hzyj
3. Exchange ToS for the validation guided by a
rubric.

108
Suggested Template for the Table of Specifications

Post-
Test
https://forms.office.com/r/bNmHPNGhm3
December 2021 109

What is your take-away
from the session?
December 2021 110

“You cannot manage
what you cannot measure”
Peter Drucker
“If you cannot measure,
you can’t improve”
Gray Mac Kenzie
December 2021

No one is perfect and so is
an assessment tool.
Reduce its imperfection
through constant revision.
Marilyn U.
Balagtas
December
2021
December 2021

About the
Speaker a University Professor at
the
DR. MARILYN U
B
I
Ñ
A
-
B
A
L
A
G
T
A
S is
Philippine Normal University teaching
courses
on educational
assessment,
evaluation, and research from undergraduate to doctoral level in face-to-face and
online modality. She has been a Dean of PNU College of Flexible Learning and
ePNU. She finished her Doctor of Philosophy with specialization in Educational
Research and Evaluation at the University of the Philippines. She has been a
regional finalist in Metrobank Search for Outstanding Teacher in the Tertiary Level
and has been awarded a post-doctoral fellowship for Leadership in Research by
the University of New England, Australia for her projects as the Inaugural Director
of the Philippine National Research Center for Teacher Quality (RCTQ) that has
led the development of the Philippine Professional Standards for Teachers
(PPST). She is the founding President of the PATEF-United Professionals for the
Development and Advancement of Teacher Education (UPDATE) and currently the
President of the Philippine Educational Measurement and Evaluation Association
(PEMEA), Inc. She is a consultant in educational assessment of a testing
company in the Philippines. She is a researcher, trainor, and a writer in areas of
assessment and teacher education.
December 2021 113

Thank you.
When interested to use this material,
please cite as:
Balagtas, M. U. (2021). Use of the SOLO
Framework in Test Development in the
New Normal. Philippine Normal
University, Manila
MARILYN UBIÑA-BALAGTAS, Ph.D.
University Professor
College of Graduate Studies and T
e
a
c
h
e
rEducation Research
Philippine Normal University
President, Philippine Educational M
e
a
s
u
r
e
m
e
n
tand Evaluation
Association. I
n
c
.(PEMEA)
balagtas.mu@pnu.edu.ph
December 2021

How to construct writtentests?
(For more details, please refer to this book.)
December 2021 115

For details on performance-based and portfolio
assessments and grading, please refer to this b
o
o
k
.
December 2021 116

For inquiries about Assessment in Learning 1 and 2 Textbooks,
please get in touch with given contact details below:
December 2021 117

For inquiries about Assessment in Learning 1 and 2 Textbooks,
please get in touch with given contact details below:
December 2021 118

SOLO1.pptx

More Related Content

What's hot

Similar to SOLO1.pptx

Recently uploaded

SOLO1.pptx