Paper presentation on utilizing clicker technology to facilitate teaching and learning of software engineering problem solving skills

1. ‘Real World Problems’ as
Assessment of Software
Engineering
Stephen T. Frezza, Ph.D., C.S.D.P.
Erie, Pennsylvania
©2007 S. T. Frezza

2. Real-World Software
Engineering Problems
1)
2)
3)
4)
CSDP preparation
Organized around SE topics
Question + answer format
Useful for classroom assessment?
Style: Multiple-choice,
standardized-examination
©2007 S. T. Frezza

3. Experiment
1)
2)
3)
4)
5)
Approach
Goals/expectations
 Will help stimulate
Pose one ‘real-world’
question related to recent
classroom interaction and
course materials.
student performance.
Use Turning Point™ (PRS)  Time investment will
reinforce student learning,
Allow anonymous answers
rather than detract from it.
Record individual scores
for assessment.
Present: Statistics &
Sample Question
Correct response rationale
©2007 S. T. Frezza

4. Jo is gathering the requirements for a software-controlled furnace. After
interviewing several users, Jo obtained the following requirements list:
R1.
R2.
R3.
R4.
Gas inlet valves should be always open when furnace is heating.
Heating stops when furnace temperature reaches 150 ºC.
Furnace temperature should increase gradually when heating.
The gas inlet valves should be closed when the temperature goes
above 200ºC.
Question Which requirements defects can be identified in Jo’s
40%
40%
3D : requirements list?
a.I
only
b.I and II only
20%
c. II and III only
d.I, III and IV only
nl
y
ly
I,
III
an
d
an
d
II
IV
III
o
on
II
nd
Ia
on
ly
0%
nl
y
Ambiguous
Design dependent
Incomplete
Unverifiable
Io
I.
II.
III.
IV.
©2007 S. T. Frezza

5. Correct Response: d
Explanation
The word “gradually” makes R3 ambiguous and unverifiable.
 An unambiguous, verifiable expression for R3 would be, “Furnace
temperature should increase no more than 1ºC/second when heating.”
 Even better would be, “Furnace temperature should increase between
1ºC/second and 2ºC/second when heating.”
The set of requirements is also incomplete, because they do not explain
what happens when temperature is over 150ºC (heating done) and 200ºC
(valve closing). Consider the implications of R2:
 We suppose the valves remain open, but we cannot be sure about it: R4
may be a safety requirement, and unrelated to R1 or R2.
 In part, the incompleteness is caused because R1 and R2 interact in a very
subtle (and undefined) way.
©2007 S. T. Frezza

6. You are a requirements engineer on a project that will replace the current training
system for a large company's training department. The new orientation process
will be online, with a trainer and a group of new employees. While some courses
will be offered online, others will be offered in a traditional mode, complemented
with computer-based activities. The course registration and billing systems will be
automated, e.g., the employee's department will be automatically billed for
courses.
Your organization employs Use Cases as its requirements modeling tool
40% 40%
Question Which of the following is NOT a
characteristic of the application of use
20B : cases as a modeling tool?
ie
w
Sy
s
te
m
's
po
i
on
e
an
th
or
e
M
nt
o
re
q
fv
to
po
in
fv
ie
u.
..
w
e
is
pw
4.
r' s
3.
St
e
2.
Use cases present the functionality of the system in a stepwise fashion.
20%
Use cases present the system from the user's point of view.
Each use case step can represent more than one requirement.
Use cases present the system from the system's point of view.
0%
U
se
1.
©2007 S. T. Frezza

7. Correct Response: d
(1) Use cases used to model the interactions between external entities,
called “actors”, and the system being modeled. The end user would be
one of the actors in these models, so the user’s point of view (1) is
presented.
(2) The use case notation explicitly embodies the concept of sequence of
interactions, and so presents functionality in a step-wise fashion (2).
(3) The analyst may further elaborate the use case by adding scenarios for
normal interaction as well as for exception conditions, or may represent
further detail in a sequence diagram. Thus, A use case may represent
more than one requirement (3).
(4) The system point of view might be documented as an architectural
model. Other facets of the system point of view might be supported by a
data flow model or an entity-relationship model. Use cases do not,
however, present the system from a system point of view.
©2007 S. T. Frezza

8. Issues with classroom use
1.
2.
3.
Questions are long – significant reading time
Answers are long – significant reading time
Delivery mode (Turning Point™) does not
allow both Q & A displayed
©2007 S. T. Frezza

9. % Respondents Correct
Assessment Results
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
4A
2A
6D
17E
1B
20B 19C 3D
22D
Text Question
©2007 S. T. Frezza

10. Assessment Results
Best:
5/9
Worst: 1/9
Mode: 2/9
1
5
2
4
3
Conclusion
Did not appear to stimulate
Not the best classroom
student performance
assessment tool
©2007 S. T. Frezza

11. Stimulating Discussion
“Comment on how the questions were challenging in a
positive way…”
They
helped you think outside the "norm". Interesting, yet
hard questions.
It was good as the RE tools w[ere] applied in different
situations at various instances of the organizations
Yes, it made us to listen classes carefully to try to answer the
questions correctly
Questions did make you think about multiple view points, and
points
consider things more in-depth.
Overall not difficult, and can provide thought and discussion.
discussion
©2007 S. T. Frezza

12. Stimulating Discussion
“Comment on how the questions were challenging in a
negative way…”
Very
hard for someone [just] learning [the] subject matter
Some questions were ambiguous and subjective. (2x)
Usually questions had some vagueness to them that often led
to missing an answer due to little nit-picky parts.
It takes time to think and answer to the question. Mostly we
will click wrong answers within such short period of time,
which impact[ed] our scores
Getting a "clicker" every day was tedious.
Allow more time questions and discussion.
Topic of the question was completely unrelated to the topic
©2007 S. T. Frezza

13. Conclusion
Can
lead to useful discussion
Application
and various viewpoints interesting
Need good question-topic alignment
In-class
use and discussion qualitatively appears
to support student learning.
May
not fit PRS-style in-class assessment use
Reserved
Q&A sessions probably more effective
Mechanics costly
©2007 S. T. Frezza