HCI masters thesis

1. Alina Goldman
Committee Members:
Benjamin Bederson
June Ahn
Tammy Clegg
Predicting and Motivating
Achievement in Self-Paced Learning
A Formative Design, Study and
Evaluation
Human
Computer
Interaction
Laboratory

3. Online Tradition
al

4. Paths to Computer Science
(CMSC 198 C, D, E)

5. Septembe
r
December………………………

7. In Class Activities
30 minutesMini Lecture
• Short in class lectures and student presentations
90 minutesActivity/HW
• In class programming activity
• Working on homework
30 minutesAssessment
• Students who finish a module take an in class assessment or
midterm

8. Students have trouble succeeding in
self-paced courses

9. Gilovich, Griffin and Kahneman (2002). Heuristics and Biases: The Psychology of
Intuitive Judgment

10. Fishbach and Dhar (2005). A. Journal of Consumer Research
Torres, Gross, and Dadashova (2010). A. Journal of College Student Retention

11. Ariey and Wertenbroch (2002). Psychological Science.
Ferrari (1992). A. Journal of Psychopathology and Behavioral Assessment

12. Dunning, Heath, and Suls (2010). Psychological Science in the Public Interest

13. Bauman (2002). Distance Education Report

14. What factors predict
student success in self-
paced environments?

15. How can we encourage
students to set
realistic goals?

16. How can we motivate
students to work
persistently and meet
course goals?

17.  17 men/14 women (31 total)
 6 colleges across campus
 4 ethnicities
 Mean GPA: 3.5
Participants

18. `
Design Goals
Achievable
goals
Goal
follow-through Meaningful
incentives
Meaningful
feedback
Peer motivation

19. Success Capacity
Calibration
Anonymous and
personalized
Progress Feedback
Course Plan
System Overview
Leaderboard, Badges,
Points and Prizes

20. Pre Course
Survey
Course Plan
Post-Credit survey
Post-Credit survey
Post-Credit survey
Calibration
Observation

21. Computer Science
Background
Self-Efficacy
Time and Goal
Commitments
Calibration

22. Calibration
Composite Score
Credit Suggestion For

23. Calibration
Composite Score
Credit Suggestion For
NumberofStudents

24. Course Plan

25. Anonymous Progress Feedback

26. Personal Feedback and Forecasting

27. Badges, Points & Prizes

28. Progress Leaderboard

29. Evaluation

30. Credits Pursued Vs. CompletedNumberofCredits
= 4 students

31. 0
5
10
15
20
25
4 5 6 7 8 9 10 11 12 13 14
NumberofStudents
Weeks
Not Started
C1.M1
C1.M2
C1.M3
C1.M4
C2.M1
C2.M2
C2.M3
Assignments Viewed ( Credit 2)
Last Module Completed Week 4-14

32. Was the calibration suggestion
a good predictor of credits
students completed?
No,
however…

33. Which background variables
predicted the total number of
credits students completed?
Programming Experience
Procrastination
Self-Efficacy

34. other classes got in
the way and this
[course] got pushed
to the back burner

35. Design Considerations
Some calibration predictors
are significant
 Compare students on
individual predictors

36. Did students follow the
suggested credit goal?
Not explicitly,
but they considered it
When choosing their
credit goal

37. I am not at all familiar
with coding and I don't
know how difficult its
going to be. Neither
am I good at
meeting goals.

38. I have lots of other commitments
this semester, such as GRE,
grad school apps, service [gap],
and research. I picked 2
credits instead of 1
because the course is
really interesting

39. Design Considerations
Students anchored and
adjusted credit suggestion
 Give lower suggestions
expecting that students
adjust upward

40. What did students think of
the course plan
and deadlines?
Most students had trouble
meeting deadlines
but…
found them
useful & necessary

41. I liked setting my own
schedule. I had actually planned
when I should finish each credit
before I got to the first class,
but it helped me to be able to
schedule each module.

42. deadlines were set really
high and after I fell behind
I never saw a reason to
catch up to them.

43. although we didn't use a course
plan in the Spring, we did have a
much harder deadline (May 30th)
to finish all our work. This helped
me manage my time and
finish the work accordingly

44. Design Considerations
Deadlines were not effective
 Deadlines should be broken
into smaller increments and
spaced more evenly
 Students need multiple “hard”
deadlines that shouldn’t be
extended

45. Students paid attention to
aggregate feedback
but…
were demotivated by
stagnation
Were students motivated by
group feedback?

46. watching half of the
class fall behind at the
same time was
definitely a non-
motivating factor

47. Everything is fine for me,
[but] I have been noticing
that a lot of other student
s are having trouble
understanding a lot of the
core concepts needed to
complete the assignments

48. Comparing themselves to the
class can be both motivating
and demotivating
 Rather than showing
everyone’s progress,
compare students in
progress cohorts
Design Considerations

49. Students did not actively
respond to individual
feedback
but enjoyed the process of
tracking their progress
Were students motivated by
individual feedback?

50. I find it motivating to
track my progress
Through each credit
with the little check
marks and such

51. Students don’t pay attention to
personal feedback
 Students should receive
dynamic feedback whenever
they log in to work
Design Considerations

52. Students enjoying marking
progress path
 Create intrinsic motivation
from allowing students to
manually check off
assignments
Design Considerations

53. Some students were
motivated to see
themselves at the top
other students wanted
more meaningful
recognition
Were students motivated by
the leaderboard?

54. once you're in the
top three, you feel
like you want
to keep going

55. I had a chemistry teacher who
would ask those students who
had done an especially nice job
with a problem on a test
to describe what they had
done …in front of the whole
class. It was an honor to be
called up, and a [real] motivator.

56. initially motivated
by badges
Prizes were motivating
only when students
really wanted them
Were students motivated by
Badges, points and prizes?

57. At first, getting badges…
was exciting, but it
did not continue to be
motivating because
it became normal.

58. prizes became
meaningless for
students who
fell behind
Were students motivated by
Badges, points and prizes?

59. once someone is
behind, they might
as well not even try,
as the motivators
are unattainable

60. Some didn’t find the
prizes meaningful
Some didn’t appreciate
their visibility
Were students motivated by
Badges, points and prizes?

61. There were no motivation
practices that could make
up for the difficulty of the
class. Chocolate doesn't
Really matter when
the object is to learn
something

62. I don't really care
about the stickers, and
I don't like other
people seeing how
many I got

63. Students need meaningful
recognition and incentives
 Reward students by letting
them take on new roles (e.g.
TA badge for a specific skill)
Design Considerations

64. Students need to be able
to recover from mistakes
 Only positive reinforcement
 Reward students for weekly
progress instead of overall
progress
Design Considerations

65. Visualize achievement
predictors meaningfully
Help students motivate
each other
Future Work

66. Thanks to…
Ben Bederson
Tammy Clegg
June Ahn
My Family
HCIL

67. Questions?
alina-
goldman.com
alina.goldman@gmail.c
om