Paper Writing Service http://StudyHub.vip/ASSESSMENT-OF-PHYSICS-PROBLEMS-POSED-BY 👈

1. ASSESSMENT OF PHYSICS PROBLEMS POSED BY STUDENTS
H. Ergun, S. Gunduz
Marmara University
Istanbul / Turkey
ergunhayrettin@yahoo.com, sevketgunduz@gmail.com
Abstract
A scoring rubric and its criteria of assessing students’ posed problems were developed in
this study. The research was conducted in the university first year introductory physics
course in the 2008-2009 academic year. Sixty students were participated to the study.
Data of the research were collected, from students’ worksheets and reviews with the
students, to find out the scoring criteria of the rubric. Six criteria were determined which
are fluency of the problem, scientific accuracy of the problem, the number of equations
needed to solve problems, number of questions asked from the problem situation, the
type of problem according to complexity and the solvability of the problem. It was found
that; Inter-rater reliability coefficient r = 0.86 and Intra-rater reliability coefficient r = 0.92.
Keywords - Problem Posing, Analytic Rubric, Performance Assessment.
INTRODUCTION
Recent years, researchers and educators have begun to incorporate problem posing into
teaching and learning. In the literature review, it is seen that especially mathematics educators pay
more attention to the problem posing [1,2,3,4,5].
Problem posing involves the creation of a new problem from a given situation or experience
and can take place before, during and after solving a problem [6]. There is a close relation between
problem solving and problem posing. On the other hand, problem posing takes students beyond the
parameters of the solution processes [7].
Recognizing the importance of problem generation as an integral part of the mathematics
curriculum, the National Council of Teachers of Mathematics (NCTM, 1991) urges teachers to provide
opportunities for students to formulate their own problems [8].
Researchers examined problem-posing abilities ranging from elementary school students to
prospective teachers [9,10,11].
Although the course of physics is quite appropriate for problem posing activities, there is little
research on problem posing in physics. Problem posing is a powerful assessment tool for probing
students’ understanding of the physics concept, as well as their ability to transfer their knowledge to
novel contexts [12]. So, physics teachers can improve their students’ physics knowledge, problem
solving performance and conceptual learning by incorporating problem posing activities into their
classrooms.
Problem posing may be considered as instructional strategy or a goal itself, and allows
students to formulate problems, using their own language, vocabulary, grammar, sentence structure,
context, and syntax for the problem situation [13,14].
Problem posing activities in the classroom improve students' problem-solving abilities,
reinforce and enrich basic concepts, foster more diverse and flexible thinking and alert both teacher
and children to misunderstandings and preconceptions [1,14,19].
Although various aspects of problem posing have been examined, far less attention has been
paid assessment of problem posing which studied by a few researchers (2,9,20].
Proceedings of INTED2010 Conference.
8-10 March 2010, Valencia, Spain.
ISBN:978-84-613-5538-9
002605

2. If problem-posing activity is a regular part of instruction, the impact of this instruction should be
assessed in some way as an object of instruction. Silver and Cai considered problem posing in two
ways; assessment with problem posing and assessment of problem posing [6].
Rubrics are typically the specific form of a scoring instrument and used when evaluating
student performances or products resulting from a performance task. Therefore, they can be used to
evaluate students’ problem posing performance.
Rubrics are popular alternative assessment tools in the educational system and can be used
easily to assess students’ performance and work. Since, they produce assessments that are far more
detailed than a single grade. They have become popular with teachers as a means of communicating
expectations for an assignment, providing focused feedback on works in progress, and grading final
products. When used correctly, they serve the purposes of learning as well as of evaluation and
accountability [21,22, 23,].
Objective of the study
The purpose of this study was to find out assessment criteria of the students’ posed problems
and design a rubric. Therefore, it would be possible to assess students’ problem posing performance
in detail.
Although there is a great deal of effort to improve students’ problem posing performance
throughout the educational system, there is no standard way to evaluate problem posing performance
that is valid, reliable, and easy to use.
The quality of posed problems can be determined by using an assessment tool. If we improve
students’ problem posing performance, we should differentiate the weak and well posed problems. So,
the research was conducted to develop an analytic rubric for assessing problems posed by students.
METHODOLOGY
Participants
The participants of the study were sixty students enrolled in introductory physics course for the
engineering faculty students in a state university in Istanbul during the 2008-2009 academic year.
Procedure and Data Collection
Data, which were collected in two ways to find out assessment criteria of problems posed by
students, included all participants’ problem posing products and interviews with students after
problem posing activity.
Participants were given the opportunity to pose their own problems in a given task through
the instructional treatment. Participants were asked to generate problems from given task. The quality
of problems in which students generated depends on the given task [9].
Three different tasks were given to students during problem posing activities. They posed
problems in the classroom during instructional treatment and also they posed problems as a
homework assignment. Participants had no prior problem posing experience but they were aware of
the well feature of well posed problems, because they had solved many problems during their
academic life.
Researchers read and explained the directions of the problem posing tasks to the
participants. It was explained that they could scan all the problems on their textbook to get experience
002606

3. about the kind of problems related with their topic, but they were not allowed to take any problem
without change. They were forced to pose their own problems.
First task: “Pose a problem which is related with physics topic that you have studied in the
classroom”.
Second task: “Pose a problem from a given problem by using re-formulation strategy”.
Third task: “Pose a problem from a given set of information or a problem statement”.
Firstly, participants posed problems related with the first task and then they posed problems
by using re-formulation strategy related with the second task, lastly they posed problems from given
set of information or problem statement related with the third task. For the first task, they had no
experience on problem posing and also they hadn’t got any information about problem posing
strategies. They posed problems freely. The researchers wanted to determine, problem posing
capacity of the students and properties of the problem posing products.
After the first task, participants were given information about the problem posing strategies
related with the second and the third tasks. The researchers posed some problems as an example.
They helped and guided to the students how to pose physics problems. Then students applied what
they had learned by posing a problem. For the second task; “What if not” problem posing strategy
was used which was developed by Brown and Walter (1983). In this strategy, students pose new
problems from a previously solved problem using a process of varying the conditions or goals of the
original problem [24].
Finally, the students posed problems from a given set of information or problem statement. A
problem statement, which was used in the research, is given in Fig.1.
Figure 1- Problem Statement
You are in the destroyer and there is an enemy air craft coming over you apart from you
about distance x, and height h. Destroyer is able to shut with gun, air craft is able to
release bomb.
• Generate the qualitative or quantitative problems.
• You can add extra data and information into the problem situation.
• Try to construct an original problem different from your friends.
• Scan the similar problems given in your books before posing the problem.
• Deliver the posed problem to the instructor with the solution written on A-4 paper
002607

4. Problem posing activities were performed as a part of instruction. The students were given 40
min to pose problems in the classroom every week. They were allowed to open books and notebooks
during problem posing activity. Also, researchers gave problem-posing tasks as a homework
assignment to the students.
DATA ANALYSIS AND RESULTS
The collected data were analyzed inductively to identify main mistake themes made during
problem posing activity. Problems were posed on worksheets, which were collected from the
participants to determine the problem posing criteria. Interviews were conducted after problem posing
activities. Participants were focused on topics related to beliefs about problem posing and
characteristics of posed problems. Data from the worksheets and interviews were analyzed
qualitatively. Data, which were collected, included all participants’ assignments and interviews.
Students’ posed physics problems were analyzed to determine the characteristics of well-designed
and poor designed problems, which were discussed during the interview. There are two samples
which were posed by students during the instruction related to problem posing task in fig.2 and fig-3.
Samples’ context similar to fig.1 but there is a tank instead of a destroyer.
Figure2. A Problem Posed by a Student
English version of the problem:
Tank fires with an angle 37
o
and Vo=100 m/s, in order to shoot the plane going in air. Plane is over
250 m from the ground and has 50 m/s velocity.
a) How many second later, tank hits the plane?
b) What is the hitting velocity?
c) What is the distance that plane takes?
Weakness of the problem:
• Data is insufficient for the solution (The distance between tank and plane is unknown)
• Diagram and expressions are careless.
002608

5. Figure3. A Problem Posed by a Student
While designing scoring rubric, step-by-step processes were realized as below [21];
• The learning objectives of the problem posing were re-examined in order to match scoring
guide with my objectives.
• Specific observable criteria such as; skills, and common mistakes to be addressed by the
problem-posing task, were identified.
• Brainstorming sessions were participated with other instructors to describe the problem posing
performance and score level for each criterion.
• The highest and lowest levels of performance were described using the descriptors for each
criterion separately. Then, for all intermediate levels of performance for each criterion was
written.
• The draft rubric was introduced to the students and it was implemented.
• In the light of last implementation results, the effectiveness of the rubric was discussed with
experts and the students then the draft rubric was revised.
• The revised rubric was implemented and the reliability correlation coefficients of the rubric
were examined.
English version of the problem:
A military plane (F-16) is flying at a constant 100 m/s horizontally. When the soldiers see the
tank, which is traveling at a constant speed of 20 m/s on a level straight highway that is 180
m below, they drop a bomb and destroy the tank. Find the horizontal distance between the
plane and the tank, when the bomb was dropped.
Weakness of the problem:
• Diagram and text are ill-matched (plane must be behind of tank, not front)
• Improper values with real life (plane has a height of 180 m and, velocity of 100 m/s)
002609

6. When the problem posing products of the students were analyzed, it was determined that most of
the problems were unclear and they had grammatical mistakes. Then the first criterion of the rubric
was determined as “fluency of problem”. This criterion was scored from weak to strong as follows;
0: The text of problem is not explicit and understandable.
1: The problem is partially understandable (Text and graph are ill-matched)
2: The problem is partially understandable (text must be supported with figures)
3: The problem is understandable, but text and figure are careless
4: The problem is clear and understandable.
If a student had got zero point from this criterion, then the problem was discarded and not evaluated
further.
Researchers determined that many problems had no compliance with the Principles of
Physics. So the second criterion of the rubric was determined as “scientific accuracy of the problem
(Compliance with the Principles of Physics)”. This criterion was scored from weak to strong as
follows;
Scientific accuracy of the problem (Compliance with the Principles of Physics)
0: Problem is not appropriate with physics’ principles (it violates the physics)
1: Problem is partially appropriate with physics (improper knowledge and figure)
2: Problem is appropriate with physics’ principles but improper values with real life.
3: Problem is appropriate totally with physics’ principles
Researchers determined that some problems needed an equation or a few equations in order
to solve the problem. So “the number of equations needed to solve problems” was determined a
criterion of posed problems. Also numbers of equations are related with the difficulty of the problem.
This criterion was scored as follows;
1: 1-2 equations needed to solve problem
2: 3-4 equations needed to solve problem
3: 5-6 equations needed to solve problem
4: 7 or more equations needed to solve problem
When researchers analyzed the worksheets, they found that although, some problems had a
question related to problem situation, some had more than one questions. So the number of
questions was determined a criterion of the rubric. The number of questions is also related to the
complexity of the problem. This criterion was scored from weak to strong as follows;
1: Only one question has been asked.
2: Two questions have been asked.
3: Three questions have been asked.
4: Four or more questions have been asked.
“The type of problem according to complexity” was determined the fifth criterion of the rubric. It
was found that most of the participants had posed problems as an example type.
This criterion was scored from weak to strong as follows;
1: It is example-type simple problem.
2: It is a normal problem type (textbook like problem)
3: It is a complex problem type that the solver needs to see critical conditions (challenge problems).
002610

7. Last criterion of the rubric was determined “the solvability of the problem”. Some students’
posed problems were classified as unsolvable.
This criterion was scored from weak to strong as follows;
0: Problem can not be solved because of insufficient data,
1: Problem cannot be solved because of its complexity,
2: Problem can be solved but givens and data are wrong,
3: Problem can be solved but givens incomplete,
4: Problem can be solved and givens are appropriate and complete
The rubric for assessing the problems posed by students is shown in table1.
Validity and reliability of the rubric
Validity refers to the degree to which score interpretations are supported by empirical evidence
and theory. Reliability in this context refers to the agreement of scores from multiple raters.
In view of qualitative research, reliability relates to replication, stability, and consistency.
Validity relates to dependability, predictability, and accuracy. In order to increase the reliability and
validity of this research, the data were arranged and generalized based on topic analysis. After
constant comparison and interpretation, triangulation was used to explain the meaning of the data. All
stage of the research, students and physics instructors expressed their wives about the rubric.
Therefore criteria of the rubric were overviewed during the research. Some criteria were cancelled and
some added to the rubric. Also priorities of some scoring criteria were changed. For example, there
was a criterion related to problems that posed qualitatively. It was determined that the students posed
a few qualitative problems during the research period. So there was no need to put a criterion into
rubric related to qualitative problems. There wasn’t a criterion like “the type of problem related with
complexity” in the draft rubric. After implementation of the draft rubric, the researchers needed to add
this criterion to the rubric in order to differentiate the scores of students. Otherwise, there was no
chance to distinguish the difficulties of the problem.
There are two types of reliability, which are inter-rater reliability and intra-rater reliability. Inter-
rater reliability determines the extent to which two or more raters obtain the same result when using
the same instrument to measure a concept. In addition, Intra-rater reliability determines the same rater
obtains the similar results using the same instrument on two or more occasions.
Criteria of the rubric were consistent with both the research literature and the processes
students engage in while posing problems.
Throughout the initial development of the rubric, twenty students’ posed problems were, legible
and reflected a range of detail and quality, randomly selected and scored (out of approximately 100)
by two researchers.
The raters discussed interpretation of the rubric after independently scoring each problem and a
consensus was reached on the scores. The early rubric draft that was used to score students’ posed
problems during the instructional treatment. Then three experts and the researchers discussed the
criteria and each criterion’s scoring level of the rubric. At the same time students’ views about the
assessment criteria were taken into consideration. After final consensus was reached on the scores,
the rubric was revised and used to score students’ posed problems. Seventy students’ posed
problems were selected randomly and scored by a researcher and an expert (an experience
introductory physics instructor). The inter-rater reliability correlation coefficient was 0.86. Also the
researcher scored the same problems after 20 days later, the intra-rater reliability correlation
coefficient was 0,92. Results were satisfying.
002611

8. CONCLUSION
Educators have recently paid more attention to problem posing; therefore they have
incorporated it into classroom instruction. Various aspects of problem posing were researched, such
as the relation between problem posing and problem solving, effectiveness of problem posing task,
strategies used to pose problems etc. Assessment is an integral part of problem posing activities.
Some researches were conducted to determine assessment criteria of problem posing Findings of
this research support the past researches on assessment of problem posing [2,6,9]. In addition, in
this research a rubric was developed to assess students’ posed problems. By using this rubric,
educators can assess the problem posing performance of their students in detail. They will have
opportunity to determine and give them feedback about their weak performance of problem posing.
Six criteria were determined which are fluency of problem, scientific accuracy of the problem,
the number of equations needed to solve problems, number of questions asked from the problem
situation, the type of problem according to complexity and the solvability of the problem. It was found
that; Inter-rater Reliability Coefficient r = 0.86 and Intra-rater Reliability Coefficient r = 0.92. Although
the main problem posing criteria were determined in university introductory physics course, these
criteria can be used for the assessment of problems in the secondary education. Mathematics and
chemistry instructors or teachers can adapt this rubric into their classrooms.
002612

9. Table1. The Rubric for Assessing the Problems Posed by Students
Criteria
Quotient Score
1
Fluency of the problem
Has the problem expressed clearly, fluently and understandably?
0: The text of problem is not explicit and understandable.
1: The problem is partially understandable (Text and graph are ill-matched)
2: The problem is partially understandable (text must be supported with figures)
3: The problem is understandable, but text and figure are careless
4: The problem is clear and understandable.
5 20
2
Scientific accuracy of the problem (Compliance with the Principles of Physics)
Have the knowledge, concepts, principles and graphs given appropriately with physics
principles?
0: Problem is not appropriate with physics’ principles (it violates the physics)
1: Problem is partially appropriate with physics (improper knowledge and figure)
2: Problem is appropriate with physics’ principles but improper values with real life.
3: Problem is appropriate totally with physics’ principles
6 18
3
The number of equations needed to solve problems
How many equations must be used in order to solve the problem?
1: 1-2 equations needed to solve problem
2: 3-4 equations needed to solve problem
3: 5-6 equations needed to solve problem
4: 7 or more equations needed to solve problem
5 20
4
Number of questions asked from the problem situation
How many questions have been asked in the problem?
1: Only one question has been asked.
2: Two questions have been asked.
3: Three questions have been asked.
4: Four or more questions have been asked.
3 12
5
The type of problem according to complexity
What is the type of problem related with complexity
1: It is example-type simple problem.
2: It is a normal problem type (textbook like problem)
3: It is a complex problem type that the solver needs to see critical conditions
(challenge problem).
5 15
6
The solvability of the problem
Has the problem constructed as a soluble?
0: Problem can not be solved because of insufficient data,
1: Problem can not be solved because of its complexity,
2: Problem can be solved but givens and data are wrong,
3: Problem can be solved but givens incomplete,
4: Problem can be solved and givens are appropriate and complete
4 16
Note: If the posed problem has given zero point from 1,2 and 6 criteria then problem will not
be evaluated further.
002613

10. References
[1] English, L., D. (1997). “The Development of Fifth-Grade Children’s Problem Posing Abilities”,
Educational Studies in Mathematics, 34 (3), ss.183-217
[2] Silver, E. and Cai, J. (1996). “An Analyses of Arithmetic Problem Posing by Middle School
Students”, Journal of Research in Mathematics Education, 27 (5), ss.521-539
[3] Gonzales, N. A. (1994). “Problem Posing: A Neglected Component in Mathematics Courses for
Prospective Elementary and Middle School Teachers”, School Science and Mathematics, 94 (2),
ss.78–84.
[4] Grunmeier, T.A.(2002). “University Students’ Problem Posing Abilities and Attitudes Towards
Mathematics”, PRIMUS, 12(2), ss.122-134.
[5] Stoyanova, E. (2005). “Problem Posing Strategies used by years 8 and 9 students”, Australian
Mathematics Teacher, 61 (3), ss.6-11.
[6] Silver, E. and Chi, J. (1995). “Assessing students’ mathematical problem posing”, Teaching
Children Mathematics, 12 (3), ss.129-135
[7] English, L.,D. (1997b). “Promoting a problem posing classroom”, Teaching Children
Mathematics, 3, pp.172- 179
[8] Gonzales, N. A. (1998). “A Blueprint for Problem Posing”, School Science & Mathematics, 9 (8),
pp. 448-456.
[9] Leung, S. K., & Silver, E. A. (1997). “The role of task format, mathematics knowledge, and
creative thinking on the arithmetic problem posing of prospective elementary school teachers”,
Mathematics Education Research Journal. 9 (1), pp. 5-24
[10] Silver, E., Mamona, J., Leung, S., and Kenney, P.,A. (1996). “Posing Mathematical Problems: An
Exploratory Study”, Journal for Research in Mathematics Education. 27 (3), pp.293-309.
[11] Winograd, K. (1992). “What fifth graders learn when they write their own math problems”,
Educational Leadership. 49 (7). pp. 64-67.
[12] Mestre, J. P. (2002). “Probing adults' conceptual understanding and transfer of learning via
problem posing”, Journal of Applied Developmental Psychology, 23, pp.9-50.
[13] Dickerson, V. M. (1999). “The Impact Of Problem Posing Instruction On The Mathematical
Problem-Solving Achievement Of Seventh Graders”. Unpublished Ph.D Dissertation, University
Of Emory, Atlanta.
[14] Silver, E. A. (1994). “On mathematical problem posing”, For the Learning of Mathematics, 14 (1),
pp.19–28.
[15] English, L D., Cudmore, D, And Tilley, D. (1998) "Problem Posing and Critiquing: How It Can
Happen in Your Classroom”, Mathematics Teaching in the Middle School, October, pp.124–29
[20] Leung, S.S. (1993). “Mathematical Problem Posing: The influence of task formats, mathematics
knowledge, and creative thinking”, In I. Hirabayashi, N. Nohda, K. Shigematsu, & F. Lin (Eds.).
Proceedings of the 17
th
international conference of the international Group for the psychology of
Mathematics Education, Vol.III pp:33-40. Tsukuba, Japan.
[21] Andrade, H,G. (2000). “Using rubrics to promote thinking and learning”, Educational Leadership,
57 (5), pp.13-18.
002614

11. [22] Moskal, B. M. & Leydens, J. A. (2000). “Scoring rubric development: validity and reliability”,
Practical Assessment, Research & Evaluation, 7(10). [Available online:
http://pareonline.net/getvn.asp?v=7&n=10].
[23] Popham, J. W. (1997). “What’s wrong and what’s right with rubric”, Educational Leadership, 55
(2), pp.72-75.
[24] Brown, S. I., & Walter, M. I. (1993). Problem posing: Reflections and applications. Hillsdale,
NJ: Erlbaum.
002615