Presented by Hillary E. Merzdorf, Nathan M. Hicks and Kerrie A. Douglas of Purdue University at The Open University, Milton Keynes, UK on 15 June 2017. This presentation formed part of the FutureLearn Academic Network section (FLAN Day) of the 38th Computers and Learning Research Group (CALRG) conference. For full details, see http://cloudworks.ac.uk/cloudscape/view/3004

1. +
Psychometric Analyses of the
Expectancy-Value-Cost Scale in
Advanced Nanotechnology
MOOCs
MOOC
Evaluation
Research Group
Hillary E. Merzdorf
Nathan M. Hicks
Kerrie A. Douglas
6/16/2017

2. +
The Expectancy-Value-Cost model of
motivation may provide good
information about learners in advanced
STEM MOOCs, but may require revision
for these populations
How do we know this?
6/16/2017
2

3. +
Why measure motivation?
6/16/2017
3

4. +
How to model motivation?
 Extrinsic-Intrinsic Model, derived from Self-
Determination Theory1,2
 Not helpful with classifying users
 Did not seem to align with MOOC-related factors
 Nearly all learners had high intrinsic motivation
 Extrinsic motivators may work differently in MOOCs than
more traditional learning environments
6/16/2017
4
1. Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior.
New York and London: Plenum.
2. Douglas, K. A., Mihalec-Adkins, B. P., Hicks, N. M., Diefes-Dux, H. A., Bermel, P., & Madhavan,
K. (2016). Learners in advanced nanotechnology MOOCs: Understanding their intentions and
motivation. In American Society for Engineering Education’s 123rd Annual Conference &
Exposition, New Orleans, LA.

5. +
How to model motivation?
 Alternative Model: Expectancy-Value-Cost model3
 Expectancy = one’s belief they can accomplish a task
currently (ability beliefs) or in the future (expectancy beliefs)
 Value = how much one wants to do the task based on
whether its perceived enjoyment (interest), usefulness
(utility), and ability to affirm one’s identity (attainment)
 Cost = what one believes they will have to give up or expend
in order to accomplish the task, including time, effort, or self-
image
6/16/2017
5
3. Barron, K. E., & Hulleman, C. S. (2015). Expectancy-Value-Cost Model of Motivation. In J. D.
Wright (Ed.), International encyclopedia of the social and behavioral sciences, 2nd edition, Vol.
8 (pp. 503–509). Oxford: Elsevier.

6. +
Expectancy-Value-Cost Model
6/16/2017
6
… but what will it
cost me?
A goal I value…
… and a belief I
can make it…

7. +
Why is EVC relevant in a MOOC?
 EVC highlights factors that are more relevant in a learning
environment than the intrinsic-extrinsic model (which is more
relevant in work-related settings)
 MOOCs are voluntary, free, and low-risk, but require time and
effort
 Learners are free to enroll regardless of background or abilities or
familiarity with the subject
 Learners come from many backgrounds and likely have varying
levels of competing obligations
 But…
6/16/2017
7
https://ehtrust.org/wp-content/uploads/2014/07/kids-reading-sm.jpg

8. +
Context of study
 Pre-course surveys from two courses:
 Nanophotonic Modeling
 Principles of Biosensors
 Discover the underlying traits with factor analysis
 Investigate item functioning with item response theory (IRT)
6/16/2017
8

9. +
Demographics
6/16/2017
9
Category n Percentage
Course
Nanophotonic
Modeling
365 56.8
Principles of
Electronic
Biosensors
278 43.2
Gender
Male 302 47.0
Female 67 10.4
Transgender 0 0.0
Prefer not to answer 7 1.1
Non-respondent 267 41.5
Category n Percentage
Education
Less than a four-year degree 75 11.7
Four-year degree 110 17.1
Master’s degree 133 20.7
Doctoral or Professional
degree
68 10.6
Non-respondent 257 40.0
Academic Status
Part-time student (either in-
person or online)
219 34.1
Full-time student (either in-
person or online)
161 25.0
Non-respondent 263 40.9

10. +
Demographics
6/16/2017
10
Category n Percentage
Age
24 or under 131 20.4
25-34 150 23.3
35 or older 101 15.7
Non-respondent 261 40.6
Category n Percentage
Employment Status
Employed part-time,
unemployed, or retired
185 28.8
Employed full-time 196 30.5
Non-respondent 262 40.7

11. +
Distribution of Responses
6/15/2017
11
0%
10%
20%
30%
40%
50%
60%
1 2 3 4 5 6
PERCENTOFLEARNERS
MOTIVATION SCORE (AVERAGE OF LIKERT ITEMS)
Expectancy-Value-Cost Histogram
Expectancy
Value
Cost

12. +
Exploratory Factor Analysis
6/16/2017
12
90% CI for
RMSEA
Model 𝜒2 𝑑𝑓 CFI TLI RMSEA LL UL
1-factor 1391.944* 27 .833 .778 .352 .337 .368
2-factor 539.212* 19 .936 880 .259 .241 .278
3-factor 11.040 12 1.000 1.000 .000 .000 .047
4-factor 2.308 6 1.000 1.003 .000 .000 .030
Item
Factor 1
loadings
Factor 2
loadings
Factor 3
loadings
(1) I know I can learn the material in this course. .879 .514 -.292
(2) I believe that I can be successful in this course. .944 .524 -.350
(3) I am confident that I can understand the
material in this course. .880 .549 -.263
(4) I think this course is or will be important. .545 .940 -.261
(5) I value this course. .551 .928 -.269
(6) I think this course is or will be useful. .496 .919 -.167
(7) Because of other things that I do, I do not
expect to have time to put into this course.
-.168 -.183 .839
(8) I think I will be unable to put in the time
needed to do well in this course.
-.270 -.203 .908
(9) I think I may have to give up too much to do
well in this course. -.173 -.067 .693

13. +
Confirmatory Factor Analysis
6/16/2017
13

14. +
Item Response Theory
6/16/2017
14
Expectancy

15. +
Item Response Theory
6/16/2017
15
Value

16. +
Item Response Theory
6/16/2017
16
Cost

17. +
What does it all mean?
 The instrument does measure expectancy, value, and cost in
this population
 However, slight interpretation variability across sub-
populations
 Advanced STEM MOOC learners have strong confidence in
their abilities, and value the courses highly, but experience
varying influence of cost
 In order for this instrument to provide meaningful information,
items need to be revised to better differentiate levels of
expectancy and value, given the population
6/16/2017
17

18. +
Recommendations and implications
 Given that most participants agree to expectancy and value,
questions should be more nuanced to better explore varying
sources of agreement
 Measuring motivation with EVC may have direct relationship to
learning behaviors
 If we can modify instrument to differentiate each dimension, it
can help us
 Personalize learning experiences
 Give learners greater autonomy
 Deliver more targeted interventions
6/16/207
18

19. +
Thank you
Questions?
19
National Science Foundation
PRIME #1544259 and EEC #1227110
Any opinions, findings, and conclusions or recommendations expressed in this material
are those of the authors and do not necessarily reflect the views of the National Science
Foundation.
Hillary E. Merzdorf (hmerzdor@purdue.edu)
Nathan M. Hicks (hicks80@purdue.edu)
Kerrie A. Douglas (douglask@purdue.edu)
School of Engineering Education,
Purdue University
West Lafayette, Indiana