The document discusses a protocol developed to construct exam questions that effectively assess scientific thinking skills in students, addressing the observed lack of inquiry-based activities in classrooms. It highlights various scientific thinking categories, such as conceptual, procedural, and epistemic skills, and emphasizes the need for sustainable assessment methods. Additionally, it presents a structured approach using sequence starters to help teachers formulate better questions aligned with these skills.

1. Jordi Domènech-Casal
jdomen44@xtec.cat | @jdomenechca
EduWikiLab
https://eduwikilab.wordpress.com/ | @eduwikilab
A protocol to construct exam questions to
assess scientific thinking skills

3. Rocard Report
• Students lose
interest in scientific
disciplines.
• Lack of research or
inquiry activities in
the classrooms.

4. Okada et al, 2015

5. • Conceptual: Understanding and using Scientific
Models to explain phenomena.
PISA 2015 Draft Science Framework
http://www.oecd.org/pisa/pisaproducts/Draft%20PISA 2015 Science Framework.pdf
EduWikiLab https://eduwikilab.wordpress.com/ | @eduwikilab

6. • Conceptual: Understanding and using Scientific
Models to explain phenomena.
• Procedural: Using Scientific Thinking Skills to
analyze and solve problems scientifically.
PISA 2015 Draft Science Framework
http://www.oecd.org/pisa/pisaproducts/Draft%20PISA 2015 Science Framework.pdf
EduWikiLab https://eduwikilab.wordpress.com/ | @eduwikilab

7. • Conceptual: Understanding and using Scientific
Models to explain phenomena.
• Procedural: Using Scientific Thinking Skills to
analyze and solve problems scientifically.
• Epistemic: Understanding and participating in
social dynamics to assess and validate
Scientific Knowledge.
PISA 2015 Draft Science Framework
http://www.oecd.org/pisa/pisaproducts/Draft%20PISA 2015 Science Framework.pdf
EduWikiLab https://eduwikilab.wordpress.com/ | @eduwikilab

8. • Conceptual: Understanding and using Scientific
Models to explain phenomena.
• Procedural: Using Scientific Thinking Skills to
analyze and solve problems scientifically.
• Epistemic: Understanding and participating in
social dynamics to assess and validate
Scientific Knowledge.
PISA 2015 Draft Science Framework
http://www.oecd.org/pisa/pisaproducts/Draft%20PISA 2015 Science Framework.pdf
EduWikiLab https://eduwikilab.wordpress.com/ | @eduwikilab

9. Scientific Thinking Skills have a poor
presence in our classrooms, even if:
• We consider it a key point of Science Education.
• They are present in the syllabus.
• They are significant for our students “Scitizenship”.
• European Union promotes it through several
programs and ressources.

10. If you don’t assess it,
It doesn’t exist?
Is it possible to assess scientific
skills in a sustainable way?

11. PISA Tests

12. Short questions (1-2 per exam).
Easy correction (multiple choice).
Directed to specific skills.
Departing from the idea of the PISA questions, we
proposed ourselves to develop our own questions to
Test Scientific Skills (TSS) in a more simple way:

13. Composing and identifying Scientific Questions
Inductive thinking: Making hypothesis and
constructing explanatory models from evidences
Designing and understanding experiments
Deductive Thinking: Making predictions from a
model

18. We realized the difficulty to construct such questions
and looked for a system to help ourselves and other
teachers to construct TSS questions.

19. Language and reasoning are strongly related.
Sequence starters is a strategy widely used in AICLE
to help students to construct phrases or reasonments
by giving the student the beginning of the sentence.
(The cause of this is... In my opinion... ).
We have developed Sequence Starters associated to
each TSS Category to help teachers to construct exam
questions.
These Sequence Starters are the central part of the
TSS protocol, a process sequenced in next slide.
Follow it to construct your own questions.

20. Composing and identifying scientific questions
• Which of the following questions can be answered through science?
• Which questions arise from these data?
• Select from the list the aspects of the scenario you can solve with an experiment.
• Select the possible explanations that could be tested through research.
• Which if the following statements are scientific questions.
• Complete the sentence “does it make a difference if…” with the possible aspects of interest of
the described situation.
• …

21. Inductive thinking: Making hypothesis and constructing explanatory models from evidences
• Select which of the following statements on the situation are hypothesis.
• Underline the statements constituting a hypothesis.
• Which conclusions arise when comparing A and B? And B and C? And all together?
• Do you think the obtained results confirm what was expected?
• Which of the conclusions of the list correspond to these data?
• Taking in account the behavior of the system, which is the role of the element A?
• Identify and order the processes that have participated to get this result.
• …
• …
• …

22. Designing and understanding experiments
• Will this experiment be useful to know if…? And to know if… ?
• Which of the treatments won’t give any information on the question?
• Which question can be answered when comparing A and B? And B and C?
• Which changes should be done to this experiment to be useful to demonstrate…?
• Which additional treatments should be necessary?
• Which of these experiments will be useful to…?
• …
• …
• …

23. Deductive Thinking: Making predictions from a model
• What will happen if we increase the level of A?
• What would happen if B disappears?
• Taking in account the data, which element of the graph would be representing A? Which
element of the system is represented by the Y slope?
• Taking in account the evolution of the system in steps 1, 2 and 3, what should we expect in
steps 4 and 5?
• Taking in account the behaviour of the system, complete in the graph the approximate values
lacking for these magnitudes.
• Select which of the following graphs correspond to the next/previous step. Which of the graphs
correspond to situation A? And situation B?
• …
• …
• …

24. The first questions designed worked well, but we
realized that we could improve it in two ways:
On the format of the question (to make it easy to
use)
On the essence of the question (pedagogic
aspects)

25. On the format of the question
Using several formats in proposing the question (text,
image, graph)
Using/allowing several formats in student’s answer
(text, image, graph)
Easy and fast to correct and mark
Assessment criteria clear and available
Student is required to justify the answer

26. On the essence of the question
Students need to understand the participating
scientific model to answer
The context/situation is relevant for students
The contexts/situations are different from those
used to learn
It is necessary to interact different knowledges or
models
The [question/skills/scientific model] is aiming a
fundamental part of the syllabus

27. 1.Choosing a Scientific Thinking Skill. (Doc:
TSSQuestionTypesandExamples).
2.Using Sequence Starters to develop it
(Doc: TSSSentenceStarters)
3.Improving the question on its format and
essence. (Doc: TSSReviewList)

28. Original question:
Describe the photosynthesis reaction.

29. Original question:
Describe the photosynthesis reaction.
1.Choosing a Scientific Thinking Skill. (Doc:
TSSQuestionTypesandExamples).
2.Using Sequence Starters to develop it
(Doc: TSSSentenceStarters)
3.Improving the question on its format and
essence. (Doc: TSSReviewList)

30. Original question:
Describe the photosynthesis reaction.
TSS Question types and Examples
• Composing and identifying scientific questions
• Inductive thinking: Making hypothesis and constructing explanatory models
from evidences
• Designing and understanding experiments
• Deductive Thinking: Making predictions from a model

31. Original question:
Describe the photosynthesis reaction.
TSS Question types and Examples
• Composing and identifying scientific questions
• Inductive thinking: Making hypothesis and constructing explanatory models
from evidences
• Designing and understanding experiments
• Deductive Thinking: Making predictions from a model

32. Original question:
Describe the photosynthesis reaction.
1.Choosing a Scientific Thinking Skill. (Doc:
TSSQuestionTypesandExamples).
2.Using Sequence Starters to develop it
(Doc: TSSSentenceStarters)
3.Improving the question on its format and
essence. (Doc: TSSReviewList)

33. Original question:
Describe the photosynthesis reaction.
Deductive Thinking: Making predictions from a model
• What will happen if we increase the level of A?
• What would happen if B disappears?
• Taking in account the data, which element of the graph would be representing A? Which
element of the system is represented by the Y slope?
• Taking in account the evolution of the system in steps 1, 2 and 3, what should we expect in
steps 4 and 5?
• Taking in account the behaviour of the system, complete in the graph the approximate values
lacking for these magnitudes.
• Select which of the following graphs correspond to the next/previous step. Which of the graphs
correspond to situation A? And situation B?
• …
• …
• …

34. Original question:
Describe the photosynthesis reaction.
Deductive Thinking: Making predictions from a model
• What will happen if we increase the level of A?
• What would happen if B disappears?
• Taking in account the data, which element of the graph would be representing A? Which
element of the system is represented by the Y slope?
• Taking in account the evolution of the system in steps 1, 2 and 3, what should we expect in
steps 4 and 5?
• Taking in account the behaviour of the system, complete in the graph the approximate values
lacking for these magnitudes.
• Select which of the following graphs correspond to the next/previous step. Which of the graphs
correspond to situation A? And situation B?
• …
• …
• …

35. Original question:
Describe the photosynthesis reaction.
Ideas...
Describing an experiment related with
photosynthesis and asking students the expectable
results.
Experiment covering light /not covering light?
Measuring? Oxygen? Carbon Dioxyde? Both?

36. Original question:
Describe the photosynthesis reaction.
TSS Question. Answer true of False:
In an experiment, we put two plants each in a different glass bottle,
and we measure the Carbon dioxyde and Oxygen concentrations. We
keep the plant A in the dark, and the plant B exposed to sunlight. What
will happen?
a. In both plants, Oxygen concentration will decrease.
b. Oxygen will decrease in plant A, and increase in plant B.
c. Oxygen will increase in plant B, and increase in plant A.
d. In both plants, Oxygen concentration will increase.
e. Differences will be observable only during the day. At night, there
will be no differences.

37. Original question:
Describe the photosynthesis reaction.
1.Choosing a Scientific Thinking Skill. (Doc:
TSSQuestionTypesandExamples).
2.Using Sequence Starters to develop it
(Doc: TSSSentenceStarters)
3.Improving the question on its format and
essence. (Doc: TSSReviewList)

38. Original question:
Describe the photosynthesis reaction.
TSS Question. Answer true of False:
In an experiment, we put two plants each in a different glass bottle,
and we measure the Carbon dioxyde and Oxygen concentrations. We
keep the plant A in the dark, and the plant B exposed to sunlight. What
will happen?
a. In both plants, Oxygen concentration will decrease.
b. Oxygen will decrease in plant A, and increase in plant B.
c. Oxygen will increase in plant B, and increase in plant A.
d. In both plants, Oxygen concentration will increase.
e. Differences will be observable only during the day. At night, there
will be no differences.

39. On the format of the question
Using several formats in proposing the question (text,
image, graph)
Using/allowing several formats in student’s answer
(text, image, graph)
Easy and fast to correct and mark
Assessment criteria clear and available
Student is required to justify the answer
OK
OK

40. Original question:
Describe the photosynthesis reaction.
TSS Question. Answer true of False:
In an experiment, we put two plants each in a different glass
bottle, and we measure the Carbon dioxyde and Oxygen
concentrations. We keep the plant A in the dark, and the plant B
exposed to sunlight. What will happen?
a. In both plants, Oxygen concentration will decrease.
b. Oxygen will decrease in plant A, and increase in plant B.
c. Oxygen will increase in plant B, and increase in plant A.
d. In both plants, Oxygen concentration will increase.
e. Differences will be observable only during the day. At night,
there will be no differences.

41. On the format of the question
Using several formats in proposing the question (text,
image, graph)
Using/allowing several formats in student’s answer
(text, image, graph)
Easy and fast to correct and mark
Assessment criteria clear and available
Student is required to justify the answer
OK
OK

42. On the essence of the question
Students need to understand the participating
scientific model to answer
The context/situation is relevant for students
The contexts/situations are different from those
used to learn
It is necessary to interact different knowledges or
models
The [question/skills/scientific model] is aiming a
fundamental part of the syllabus
OK
OK
OK

43. Original question:
Describe the photosynthesis reaction.
TSS Question. Answer true of False:
You have maybe heard about the danger of slepping in a room
with plenty of plants. In an experiment, we put two plants each
in a different glass bottle, and we measure the Carbon dioxyde
and Oxygen concentrations. We keep the plant A in the dark, and
the plant B exposed to sunlight. What will happen?
a. In both plants, Oxygen concentration will decrease.
b. Oxygen will decrease in plant A, and increase in plant B.
c. Oxygen will increase in plant B, and increase in plant A.
d. In both plants, Oxygen concentration will increase.
e. Differences will be observable only during the day. At night,
there will be no differences.

44. Some students didn’t like this kind of
questions, and got angry about it.
Some students did like the questions.
We began to discuss about Scientific Skills.

45. We realized that it was not only a matter of
adding the questions to the test, and
implemented “preparatory” measures that
changed our classroom practices.

46. Sharing with students the aim of this change.
Communicating assessment in terms of skills.
Discussing classic experiments of Science History
to practice Scientific Skills.
Transforming our Hands on Lab practices to Minds
On: Research Practices.
Correcting and discussing together in the
classroom the Test Questions.
Modelling in the Classroom.
Using students to inquiry: short informations.

47. Is multi-choice opposite to Thinking?

48. Are Hands-on activities enough?

49. Is TSS preparing for “Scitizenship”?

50. • Conceptual: Understanding and using Scientific
Models to explain phenomena.
• Procedural: Using Scientific Thinking Skills to
analyze and solve problems scientifically.
• Epistemic: Understanding and participating in
social dynamics to assess and validate
Scientific Knowledge.
PISA 2015 Draft Science Framework
http://www.oecd.org/pisa/pisaproducts/Draft%20PISA 2015 Science Framework.pdf
EduWikiLab https://eduwikilab.wordpress.com/ | @eduwikilab

51. Jordi Domènech-Casal
jdomen44@xtec.cat | @jdomenechca
EduWikiLab
https://eduwikilab.wordpress.com/ | @eduwikilab
Thanks!!