Understanding "Just Enough" Computer Users: Motivation Style and Proficiency

1. Understanding “Just Enough”
Computer Users:
Motivation Style and Proficiency
By Harriet King
Masters Candidate in Computer Science
1 of 45

2. The Question
Why do some proficient daily
computer users, stumble over the
unfamiliar and others easily adapt?
EXAMPLE: More information and detail in Supplementary Slides More
Introduction Study Design Motivation Observations Future Work Conclusions 2 of 45

3. What Is a Just Enough (JE) User?
• Daily computer user
• Competent
• Extrinsic Motivation
Introduction Study Design Motivation Observations Future Work Conclusions 3 of 45

4. The Hypothesis
We hypothesize that
extrinsically motivated
proficient daily computer users
have difficulty with unfamiliar computer
tasks and skill transfer, whereas
intrinsically motivated daily users
accomplish unfamiliar tasks readily.
Introduction Study Design Motivation Observations Future Work Conclusions 4 of 45

5. Who Cares?
• Software designers
• Human Computer Interactions (HCI)
• Software Users
• Stakeholders for computer literacy
“Lest we wish to change our field’s name to
student-computer interaction we should make
effort to find more representative participants”
(Barkhuus and Rode 2012)
Introduction Study Design Motivation Observations Future Work Conclusions 5 of 45

6. Study Design Overview
OUTPUT
INVENTORY
scores & statistics
group descriptors
OBSERVATIONS
Coded & analyzed
attitudes & actions
Introduction Study Design Motivation Observations Future Work Conclusions 6 of 45

7. Richness of Data for Understanding
• Pre-questionnaire: daily users?
• Quantitative motivation inventory scores
• Demographic and interview questions
• Ethnographic observation methods:
– Think Aloud Protocol
– Observation recordings
– Researcher questions and follow up
• Quantify transcripts with coding
• Post-questionnaire and JE Users questionnaire
(Sim 1999; Rose, Shneiderman, Plaisant. 1995)
Introduction Study Design Motivation Observations Future Work Conclusions 7 of 45

8. MOTIVATION
8 of 45

9. Motivation Background
Motivation Styles, adapted from Ryan and Deci (2000) ‘Taxonomy of Human Motivation’.
Low interest and enjoyment are on the left ranging to high interest and enjoyment on the
right. (Pintrich 2003; Deci and Ryan 1991; Downey and Smith 2011; Martens et al. 2004;
Deci and Ryan 1985; Iyengar and Lepper 2000; Henderlong and Lepper 2002; Ryan and
Deci 2000; Ryan and Deci 2012; Oudeyer et al. 2007)
More
More information and detail in Supplementary Slides
Introduction Study Design Motivation Observations Future Work Conclusions 9 of 45

10. Motivation Inventory
Source Factors
Guay, Vallerand, Blanchard 1. Amotivation
(2000) 2. External Regulation
3. Identified Regulation
Ryan and Deci (IMI 2012) 4. Interest/Enjoyment
5. Perceived Choice
6. Perceived Competence
L to R: Richard Ryan and Edward Deci
(Photo: Adam Fenster, August 2010)
Introduction Study Design Motivation Observations Future Work Conclusions 10 of 45

11. Adapting Questions
Precedent: (Shroff and Vogel 2009). Confirmed Inventory with two pilot studies.
Introduction Study Design Motivation Observations Future Work Conclusions 11 of 45

12. Precedents for Scoring Inventory
Likert scale IS-An ordinal measure of ranking
“We did violate some mathematical assumptions in creating an
interval level of measurement index out of ordinal components,
but as previously indicated, this is common practice in the social
and behavioral sciences.” (Sirkin, R. M., 2006. “Statistics for the
Social Sciences.” 3rd edition, Sage Publications.
Precedent for averaging motivation inventory scores
1. Pavlas, Jentsch, Salas, Fiore, and Sims, 2012
2. Shroff and Vogel, 2009
3. McAuley, Duncan, and Tammen, 1989
Introduction Study Design Motivation Observations Future Work Conclusions 12 of 45

13. Who Took the Inventory? Everybody!
• Ages 13 to 87 from FIVE continents
• 9 countries: USA, China, Turkey, Australia, Sweden, U.K.,
South Africa, India, and France
• 130+ total completed questionnaire
• Used 66 for total respondents
• 16 participants observed (7 intrinsics, 9 extrinsics)
Community
Classmates
Faculty
Internet
Introduction Study Design Motivation Observations Future Work Conclusions 13 of 45

14. Required Correlation
Table 8: Pearson Correlation of
Interest/Enjoyment & Perceived Choice
Correlation of Interest/Enjoyment & Perceived Choice
Factors
n = 66 n =16
All Respondents All Observed
Correlation 0.602 0.815
Significance (2- p < 0.01 p < 0.01
tailed)
Introduction Study Design Motivation Observations Future Work Conclusions 14 of 45

15. Grouping Variables
Venn Diagram is
External Regulation > 4.0
intersecting Total and percent inventory
Interest/Enjoyment > 4.0 responses by group with n=66
Introduction Study Design Motivation Observations Future Work Conclusions 15 of 45

16. Inventory T Test Results
Significant Differences in Inventory
Scores, Age, & Digital Native
* Asterisk indicates non parametric Mann-Whitney U test
All other are Independent Samples T-test
Factor Different Significance
Age* NOT different p=0.396
Digital Native* NOT different p=0.166
Perceived Competence* NOT different p=0.071
Amotivation* Different p=0.012
External Regulation Different p<0.001
Interest/Enjoyment Different p<0.001
Perceived Choice Different p=0.001
More information and detail in Supplementary Slides More
Introduction Study Design Motivation Observations Future Work Conclusions 16 of 45

17. Intrinsics: Digital Native or Not
Side by side comparison of digital non-natives (3) on left and digital
natives (4) on right. Ordered from low to high competence
Perceived
Interest/ Enjoyment Perceived Choice
Digital native Competence
Digital native
Digital native
Non-native
Non-native
Non-native
6.43 5.86 6.33
5.00 6.71 4.71 5.14 2.67 6.67
4.14 6.14 4.14 5.29 3.00 6.83
5.57 6.57 4.29 4.86 5.17 7.00 More
Introduction Study Design Motivation Observations Future Work Conclusions 17 of 45

18. Digital Natives
not significantly different
Digital Natives All
70
66
60
50
Number of People
40
41%
30 27
37%
57% 22%
20
16
10
9
6 7
4
2
0
Observed Participants Extrinsics
Inventory Respondents Intrinsics
Groups
Introduction Study Design Motivation Observations Future Work Conclusions 18 of 45

19. Not Significantly Different
Age, Perceived Competence, & Digital Native or not
Mean Age with error bars for Mean Perceived Competence with
standard deviation error bars for standard deviation
90.00 7.00
Mean Perceived Competence
80.00
6.00
70.00 5.38
60.00 55.67 5.00
50.00 46.57
3.70
Age
4.00
40.00
30.00 3.00
20.00
2.00
10.00
0.00
1.00
9 Extrinsics 7 Intrinsics 9 Extrinsics 7 Intrinsics
Introduction Study Design Motivation Observations Future Work Conclusions 19 of 45

20. Mean Inventory Results
with error bars showing standard deviation
n=66 Respondents n=16 Observed n=9 JE Users n=7 Intrinsics
8
7
6
Likert Scale 1 - 7
with neutral at 4
5
4
3
2
1
Grouping Grouping
Variable Variable
0
More
Introduction Study Design Motivation Observations Future Work Conclusions 20 of 45

21. Data Screening
Extra High Perceived Choice
Mean Perceived Choice with standard
Extrinsic Molly = 5.57!? deviation error bars
2.3 standard deviations above
5.57
More
Introduction Study Design Motivation Observations Future Work Conclusions 21 of 45

22. OBSERVATIONS
22 of 45

23. Observation Phases
Introduction Study Design Motivation Observations Future Work Conclusions 23 of 45

24. Near Skill Transfer
More
Introduction Study Design Motivation Observations Future Work Conclusions 24 of 45

25. Participant Hesitation Wording
“uhhhh” looked in there” promising”
“I’m looking for a way to “I think I can just... click on [sigh]
do...” this here, and... that didn’t “no, that's not it”
“maybe if I go here” work” “maybe this”
“what’s this?” “ok, that didn’t work” “so, we're not doing that”
“I can’t...” “I looked at the bottom but “I wouldn't think it'd be
“ummm” there’s nothing there” under that”
“let’s go back here” “I saw this click to ... but “I'm going to try right click
[giggling] that isn’t it” again”
“aaaaannnnnnnd” “hmmm” “I forgot what you said to
“I could try like..” “contacts....contacts.... do”
“no I can’t drag that..” contacts” “this damn mouse”
“I’ll look in here, no I just “that doesn't look very
Introduction Study Design Motivation Observations Future Work Conclusions 25 of 45

26. For Prompting the Participant
“go ahead and tell me what you’re seeing”
“please tell me what you’re thinking”
“Are you trying to decide something, can you tell me about it?”
“did that work?”
“what seems odd about this?”
“what are you thinking?”
“you’re giggling, …you’re sighing…you sound angry, what are you
feeling?”
Introduction Study Design Motivation Observations Future Work Conclusions 26 of 45

27. Rubric for Coding Observations
CODE RULE
Stumble [action] >= 20 seconds
Fall [action] >= 1 minute
Persist [action] >= 3 minutes
Quit attitude towards a task
Resist attitude towards a task
Introduction Study Design Motivation Observations Future Work Conclusions 27 of 45

28. Transcript Example
stumble
persist
resist
quit
fall
time OLIVIA [action] “quote” (time on video) analysis
b 7:58 [while looking for spam, stumbles across trash 7:58 and says I’ll empty the trash 1 1 1
e 9:08 instead, I say go ahead] Participant: “I have no idea how to do that. It’s already IN the
trash” me: “Look around. ...you can empty the trash.” (8:10) Participant: “It’s already IN
trash. Where do you empty trash to? I’m thinking that I never empty my trash because
there’s no way to empty trash because it’s already trash.” (8:25) me: “no, there is a way
to empty trash.” Participant: “There’s no trash emptying.”
[ask about her agitation] Participant: “I’m not agitated at all. You’re just wrong. There’s
no trash emptying.” [ask what she’s feeling] Participant: “I think it’s dumb that the trash
doesn’t have an empty.” (8:40) me: “It does actually”
Participant: “I don’t see it. If I click on something in my trash, all I can do is trash
something in my trash, which is silly because it’s already in my trash” (9:08) me: “Ok,
we’ll come back to this. Let’s look at your spam” [so resistant that I stop this task on
test. Never does trash]
b 9:10 Participant: “I don’t know if I have spam” (9:10) me: “You do have spam.” “No. Really!? 1 1
e 9:45 I’m looking at all my folders and I do not have one called “spam”” (9:20) me: “Did you
find “more” at the bottom?” “There’s a more. Oh look at that, there’s spam.” (9:45)
b 9:50 [directed to delete all spam at once, (9:50), giving her hints] me: “It’s not that tricky, it 1 1
e 11:10 has words and I can see them, I’m looking at it right now” (10:37) (11:10) found “delete
all messages now”
b 11:20 [11:20 Go to address book] Participant: “I’m not fully sure where my address book is, I 1
e 12:10 think I have to go to my calendar”, then found contacts 12:10
Introduction Study Design Motivation Observations Future Work Conclusions 28 of 45

29. Inter Rater Reliability Results
• First Rater (HK)
• 2 outside raters (SK and PM)
• Outside raters reviewed 30% of transcripts
• Stumble, fall, and persist are time related
Rater 1 Rater 2
Stumble, Fall, Persist 100% agreement 100% agreement
Quit 99.13% agreement 97.73% agreement
Resist 96.52% agreement 97.73% agreement
Introduction Study Design Motivation Observations Future Work Conclusions 29 of 45

30. Occurrences for Each Code
• Asterisk indicates statistically significant difference for this code
between extrinsic and intrinsic. Total occurrences with percent of
total in parentheses.
• There was no significant difference between Unfamiliar Task
compared to Near Skill Transfer for either intrinsics or extrinsics.
Stumble* Fall* Persist* Quit* Resist
JE Users 91 56 15 9 13
(81%) (84%) (88%) (90%) (87%)
Intrinsics 21 11 2 1 2
(19%) (16%) (12%) (10%) (13%)
More
Introduction Study Design Motivation Observations Future Work Conclusions 30 of 45

31. All Occurrences of Stumble & Fall
Intrinsics on left and Extrinsics on right
stumble fall
Extrinsics
Intrinsics
20
15
10
5
0
More
Introduction Study Design Motivation Observations Future Work Conclusions 31 of 45

32. JE User vs. Intrinsic: Marsha & Rebecca
Exter: 4.5 Exter: 4.0
Int/En 2.57 Int/En 5.57
Similar:
1. both Amotivation = 1.0
2. Both digital non-native Different:
3. similar experience level 1. Performance
4. similar self rate and perceived competence 2. Different motivation styles
5. similar age
6. Appeared to cruise through unfamiliar tasks
7. Responsible community leaders More
8. Professional women
Introduction Study Design Motivation Observations Future Work Conclusions 32 of 45

33. Resist
• Only 5 out of 16 resisted 8
• 4 extrinsic & intrinsic Mike 7
• Olivia had 7 resists
Total Occurrences of Resist
6
1. Can’t empty trash
2. there is no spam 5
3. doesn’t “add” to group but 4
insists she did
4. says “check mail” button is 3
broken 2
5. won’t remove attachment,
6. says used wrong address but 1
was sent folder issue 0
7. says did not spell a word Lucy Mike* Miranda Marsha Olivia
correctly when did spell
correctly More
Introduction Study Design Motivation Observations Future Work Conclusions 33 of 45

34. Another Type of Resist
Marsha shares, “I never use the google calendar. I’m not telling them what I’m
doing every day. Forget that!”
“Passionate?...I am. I’m not MAD at them [MS Word], I’m frustrated with them.
… they’re leaving out the average person. And maybe that’s what open office is
for. I don’t know.”
Introduction Study Design Motivation Observations Future Work Conclusions 34 of 45

35. Quit
8 of 16 quit Quit Resist
8
* Asterisk indicate intrinsic
Total Occurrences of Quit
7
6
5
4
3
2
1
0
Screenshot of "Contacts" button
behind "Mail" in Gmail. Doesn't
look like a button with no rectangle More
or color change.
Introduction Study Design Motivation Observations Future Work Conclusions 35 of 45

36. Persist
6
Total Occurrences of
5
4
Persist
3
8 of 16 Persist
2
1
0
Mary Molly Lucy Olivia Mike* Marsha Alice Walter
Ann
Walter spent about 5 minutes (7:40 to 13:10)
using wrong password of “guest” and misspelled username trying
to login to gmail online.
He repeated the same behavior while expecting different results
Introduction Study Design Motivation Observations Future Work Conclusions 36 of 45

37. Lowest Interest & Choice
• Extrinsics Lilly and Olivia
• Opposite attitudes (shame vs. blame)
Lilly Olivia
7.00
6.00
5.00
4.00
3.00
2.00
1.00
0.00
Perceived
Enjoyment
amotivation
Competence
regulation
external
Perceived
Choice
Interest
More
Introduction Study Design Motivation Observations Future Work Conclusions 37 of 45

38. Just Enough User Alice (1/9)
“I don’t do ANYTHING that I’m not taught. And that is a big
drawback in my learning.”
“I know enough to get what I want, most of the time. And it
definitely is not a pleasure for me to try to figure out things on
my own. N-O-T AT A-L-L… Maybe everyone thinks they are a “Just
Enough” user.”
More
Introduction Study Design Motivation Observations Future Work Conclusions 38 of 45

39. Just Enough User Lucy (3/9)
“Why would I Google it? I wouldn’t, because it’s a bunch of
teenagers who can’t spell right, who don’t use punctuation, all
lower case.”
“I am fine using the computer only for what I need. I think they
are ruining the world quite frankly, and am slightly proud I find
them somewhat repulsive machines.”
More
Introduction Study Design Motivation Observations Future Work Conclusions 39 of 45

40. Just Enough User Walter (8/9)
“You are … confronting an unbelievably unfamiliar system, with all the
scariness of being surrounded by REAL fully paid, fully trained, card
carrying life member geeks … I got spooked by the surroundings. I got
intimidated by my high level of geekitude surroundings.”
“People do get on without a computer at all, so perhaps ‘No
Computer’ (or ‘The Computer They Make You Use At Work’) is the true
‘Just Enough Computer’.”
More
Introduction Study Design Motivation Observations Future Work Conclusions 40 of 45

41. Future Work
• So much data!
• Bigger sample
• “Just Enough” term?
• Gender, socioeconomic status, years of experience,
aversion to change?
• Separating work and play in motivation study
• Less frequent users?
• What if a “consequence” element?
• Hand held computers?
Introduction Study Design Motivation Observations Future Work Conclusions 41 of 45

42. CONCLUSIONS
42 of 45

43. Statistical Results
• Confirmed competency of JE users
• Extrinsic proficient daily users stumble, fall,
persist and quit significantly more than intrinsics
• AND it is not explained by age, perceived
competence, or being digital native
• JE users account for over 80% of performance
difficulties in our study
• Just Enough users exist in all age groups and
experience levels (18% in our sample)
Introduction Study Design Motivation Observations Future Work Conclusions 43 of 45

44. Observed Phenomena
• Impossible to differentiate JE user from any
other competent user, until faced with the
unfamiliar
• Just Enough users shed competencies as they
become unnecessary
• Wide range of attitudes and experience
related to exploring and performance
• Sense of “not belonging”
Introduction Study Design Motivation Observations Future Work Conclusions 44 of 45

45. A Haiku
Thank
Just Enough is cool
you! till unfamiliar and new
safe routine un-do.
45 of 45

46. Just Enough User Lilly (2/9)
When asked during the test about her feelings,
Lilly shares, “ohhh, why am I so stupid? How can
I not know how to do this? I dread asking one of
my kids because they have no patience.”
“I really want computers to be as unobtrusive in
my daily life as can be. Just Enough term sounds
a bit lazy.”
More
Introduction Study Design Motivation Observations Future Work Conclusions 46 of 45

47. Just Enough User Marsha (4/9)
Marsha says, “I like to sign out, because then they,
THEORETICALLY, aren’t watching me, but you know
they are because advertisements for something I
just looked at turn up on the *weirdest* pages.”
“My feelings are that I would like to be more than
that [JE user]. I would consider a "just enough" user
to be one who uses only email, or only cruises the
web for news, or only uses one application.”
More
Introduction Study Design Motivation Observations Future Work Conclusions 47 of 45

48. Just Enough User Mary Ann (5/9)
“When I’m at work, I’m so busy, that I don’t have
time to play around... I always have to do things
in the fastest way possible, which doesn’t allow
exploration.”
“My feelings are that I would like to be more
than that. I do not want to be a "dinosaur. I
sometimes can do a little more than just enough
if I get up my courage to try."
More
Introduction Study Design Motivation Observations Future Work Conclusions 48 of 45

49. Just Enough User Miranda (6/9)
“It seems stupid and why should I waste my
time staring at the computer.”
“My feelings are, why would I spend any more
time at the computer? I'd rather read a book or
take a walk. Just enough is a perfect name.”
More
Introduction Study Design Motivation Observations Future Work Conclusions 49 of 45

50. Just Enough User Molly (7/9)
“This all is stupid. This is ridiculous. I don’t know
why anyone uses computers. … I don’t really
care. I can basically do anything I need to do and
I have [IT worker] and if I can’t do anything I just
call [IT worker] and cry.”
“The term "Just Enough" is kind. I don't feel
judged or "less than" (stupid).”
More
Introduction Study Design Motivation Observations Future Work Conclusions 50 of 45

51. Just Enough User Olivia (8/9)
“[it] is really annoying not to be able to find
these things that you’re CLAIMING it’s on here.
And it’s like, how are you supposed to know
where it is.....[I’m] irritated.”
“Very proud that I can do it enuf [sic]. People
should make more things easy for us.”
More
Introduction Study Design Motivation Observations Future Work Conclusions 51 of 45

52. “Just Enough Users”, a poem
Just enough is satisficing, works out fine till new and strange.
Computer changes make life messy,
then it’s struggle stumble quit.
Those interest people cruise along,
probably nothing ever wrong.
Curse you easy flexing user. Why can’t I just find my cursor?
Just Enough left me so helpless,
when the web changed all my favorites.
I just want to stay so lazy, stay low interest, stay low effort.
OK sometimes then I stumble. Just Enough was not effective.
Who to blame and who to curse?
Designers! They must be the worst.
Conclusion
52 of 45

53. Intrinsic Motivation Characteristics
• Deeper involvement in activities; natural activity
• More curiosity; exploration
• Trying out more complex options
• Increased persistence
• Higher achievement of goals; improved
performance
• Less avoidance behavior
• Interest, excitement, and confidence
Back
(Martens et al. 2004; Oudeyer et al. 2007; Deci and Ryan 2000)
Motivation Inventory
53 of 45

54. Intrinsic Motivation: Supports & By Products
• Self-esteem and general well-being
• Competence
• Autonomy
• Adaptable
• Pros/cons of praise Back
• Reduced by external rewards
• Supported by seeing examples; having capability
(Pintrich 2003; Deci and Ryan 1991; Downey and Smith 2011; Martens et al. 2004;
Deci and Ryan 1985; Iyengar and Lepper 2000; Henderlong and Lepper 2002; Ryan and
Deci 2000; Ryan and Deci 2012; Oudeyer et al. 2007)
Motivation Inventory
54 of 45

55. Parametric or Non parametric?
Does data pass the 3 assumptions for parametric
statistical analysis?
1. Independence? Yes! All different humans
2. Homogeneity? (equal variance, Levene’s test)
3. Normality? (skewness & kurtosis < |1.95|)
Back
Does it pass for 66 respondents and 16 participants?
Motivation Inventory
55 of 45

56. Homogeneity of Inventory Factors
Levene’s Test for Levene’s Test for
Equality of Variances Equality of Variances
n = 16 Observed. n = 66 Respondents.
Significance Significance
Amotivation 0.053 0.002
Identified Regulation 0.802 0.546
External Regulation 0.572 0.822
Interest/Enjoyment 0.989 0.842
Perceived Choice 0.492 0.218
Perceived Competence 0.152 0.010
Motivation Inventory
Back
56 of 45

57. Normality of Inventory Factors
Respondents n=66 Observed Participants n=16
Skewness Kurtosis Skewness Kurtosis
Amotivation 1.486 1.986 1.004 -0.557
Identified Regulation -0.063 -1.048 -0.527 -1.157
External Regulation 0.038 -0.781 -0.273 -0.870
Interest/Enjoyment -.0513 -0.050 0.165 -1.358
Perceived Choice -0.213 -0.708 -0.050 -1.565
Perceived Competence -0.246 -0.609 -0.533 -0.988
Back
Motivation Inventory
57 of 45

58. T Test Result Detail
Amotivation (Mann-Whitney U test) (U = 9.50,
p = 0.012).
Perceived Choice (independent Samples T test)
extrinsic (M=2.7, SD=1.3) and intrinsics (M=4.9,
SD=0.6); t(14)=4.306, p=0.001.
Back
58 of 45

59. Intrinsics Descriptive Statistics
N Min Max Mean Std. Dev Skewness Kurtosis
Std. Std.
Statistic Error Statistic Error
age 7 23 87 46.71 27.93 0.59 0.79 -1.96 1.59
amotivation 7 1.00 1.50 1.07 0.19 2.65 0.79 7.00 1.59
external 7 1.00 4.00 2.64 1.02 -0.19 0.79 -0.06 1.59
regulation
Interest/ 7 4.14 6.71 5.80 0.95 -0.96 0.79 -0.11 1.59
Enjoyment
Perceived Choice 7 4.14 5.86 4.90 0.59 0.32 0.79 -0.35 1.59
Perceived 7 2.67 7.00 5.38 1.84 -0.87 0.79 -1.30 1.59
Competence
Amotivation (Mann-Whitney U test) (U = 9.50, p = 0.012). Back
Perceived Choice (independent Samples T test) extrinsic (M=2.7,
Motivation Inventory intrinsics (M=4.9, SD=0.6); t(14)=4.306, p=0.001.
SD=1.3) and
59 of 45

60. Extrinsics Descriptive Statistics
N Min Max Mean Std. Dev Skewness Kurtosis
Std. Std.
Statistic Error Statistic Error
age 9 34 74 55.78 14.17 -0.18 0.72 -1.32 1.40
amotivation 9 1.00 2.75 1.89 0.74 0.08 0.72 -1.82 1.40
external regulation 9 4.25 6.50 5.39 0.89 -0.02 0.72 -1.92 1.40
Interest/ 9 1.57 4.00 2.73 0.89 -0.03 0.72 -1.35 1.40
Enjoyment
Perceived Choice 9 1.57 5.57 2.67 1.26 1.71 0.72 3.43 1.40
Perceived 9 2.50 4.33 3.70 0.72 -0.87 0.72 -0.80 1.40
Competence
Back
Motivation Inventory
60 of 45

61. Summary of Correlations
Back
n=66 Inventory Respondents & n=16 Observed Participants
Relationship Correlation Significance n R^2
External Regulation with - 0.821 p=0.001 16 67.40%
Interest/Enjoyment
- 0.397 p=0.001 66 15.76%
External Regulation with Perceived - 0.879 p=0.001 16 77.26%
Choice
- 0.785 p=0.001 66 61.62%
Amotivation with Perceived - 0.602 p=0.014 16 36.24%
Competence
- 0.339 p=0.005 66 11.49%
Age with Perceived Competence - 0.710 p=0.002 16 50.41%
Motivation Inventory
61 of 45

62. Summary of Correlations
n=66 Inventory Respondents & n=16 Observed Participants
Relationship Correlation Significance n R^2
External Regulation with - 0.821 p=0.001 16 67.40%
Interest/Enjoyment
- 0.397 p=0.001 66 15.76%
External Regulation with Perceived - 0.879 p=0.001 16 77.26%
Choice
- 0.785 p=0.001 66 61.62%
Amotivation with Perceived - 0.602 p=0.014 16 36.24%
Competence
- 0.339 p=0.005 66 11.49%
Age with Perceived Competence - 0.710 p=0.002 16 50.41%
Back
Motivation Inventory
62 of 45

63. Digital Native Correlations
Digital Native Significant Correlations for Observed Participants
Digital Native Correlation Significance n R^2
Relationship with...
...Age 0.536 p<0.001 16 28.73%
...Interest/Enjoyment 0.561 p=0.024 16 31.47%
...Perceived Choice 0.575 p=0.020 16 33.06%
...Perceived Competence 0.647 p=0.007 16 41.86%
...External Regulation -0.534 p=0.033 16 28.52%
Back
Motivation Inventory
63 of 45

64. Mean Occurrences of Codes
Mean Number of Code Occurrences for Extrinsics and Intrinsics
Extrinsics Intrinsics
stumble 10.11 3.00
fall 6.11 1.57
quit 1.00 .29
resist 1.11 .29
persist 1.67 .29
Back
Observations
64 of 45

65. Correlations for Extrinsics
Extrinsic Group Significant Relationships
Relationship To Correlation Significance n R^2
Age Persist 0.667 0.050 9 44.49%
Digital Native -0.728 0.026 9 53.00%
Amotivation -0.713 0.031 9 50.84%
External Perceived -0.699 0.036 9 48.86%
Regulation Choice
Back
Observations
65 of 45

66. Correlations for Intrinsics Back
Relationship To Correlation Significance n R^2
Stumble Fall 0.898 .006 7 80.64%
Age 0.823 .023 7 67.73%
Digital Native -0.832 .020 7 69.22%
Interest -0.861 .013 7 74.13%
Perceived Competence -0.917 .004 7 84.09%
Digital Native Age -0.866 .012 7 75.00%
External Regulation -0.874 .010 7 76.39%
Interest/Enjoyment 0.866 .012 7 75.00%
Perceived Choice 0.866 .012 7 75.00%
Perceived Competence 0.866 .012 7 75.00%
Age External Regulation 0.757 .049 7 57.30%
Perceived Competence -0.929 .003 7 86.30%
Perceived Fall -0.768 .044 7 58.98%
Competence Interest/Enjoyment 0.786 .036 7 61.78%
External -0.883 .008 7 77.97%
Regulation Perceived Choice
Observations
66 of 45

67. Extrinsics Detail
Asterisk denotes digital native Back
Digital native
Competence
amotivation
Enjoyment
regulation
Perceived
Perceived
Self rate
External
stumble
Interest
Choice
persist
exper
resist
quit
age
fall
name
Alice 6 4 0 0 4 71 1 3 16to25 1.00 4.75 4.00 3.29 4.17
Lilly 10 5 1 0 0 48 1 7 6to15 1.50 6.50 1.57 1.57 4.33
Lucy* 3 3 1 1 1 34 2 6 16to25 2.75 6.00 3.00 1.57 4.33
Marsha 10 8 1 3 2 68 1 3 more25 1.00 4.50 2.57 2.43 4.33
Mary
Ann 16 6 0 0 1 60 1 4 16to25 1.25 6.50 3.71 2.57 4.00
Miranda 10 9 1 2 0 58 1 4 6to15 2.00 4.50 2.00 3.14 3.33
Molly* 10 6 2 0 1 40 2 6 more25 2.75 4.25 3.43 5.57 3.67
Olivia 12 5 1 7 1 48 1 4 16to25 2.75 5.75 1.57 2.14 2.50
Walter 14 10 2 0 5 74 1 2 6to15 2.00 5.75 2.71 1.71 2.67
Observations
67 of 45

68. Intrinsics Detail
Asterisk denotes digital native Back
Digital native
Competence
amotivation
Enjoyment
regulation
Perceived
Perceived
Self rate
External
stumble
Interest
Choice
persist
exper
resist
quit
age
fall
Name
Beth* 3 2 0 0 0 26 2 8 16to25 1.00 1.00 6.43 5.86 6.33
Jane* 0 0 0 0 0 27 2 9.5 16to25 1.00 2.50 6.71 5.14 6.67
Mike 8 6 1 2 2 74 1 4 16to25 1.50 2.75 4.14 4.14 3.00
Peter* 0 0 0 0 0 24 2 10 16to25 1.00 2.50 6.57 4.86 7.00
Rebecca 3 0 0 0 0 65 1 5 more25 1.00 4.00 5.57 4.29 5.17
Roger* 0 0 0 0 0 23 2 11 16to25 1.00 2.00 6.14 5.29 6.83
Wilma 7 3 0 0 0 87 1 3 16to25 1.00 3.75 5.00 4.71 2.67
Observations
68 of 45

69. Phase 2 & 3 Extrinsic Stumbles
Back
Total stumble occurrences for each extrinsic participant in phase 2 (blue on left) and
phase 3 (orange on right)
Observations
69 of 45

70. Comparing Intrinsic Digital Native Inventory Scores
Ordered from Lowest Perceived Competence to Highest
Competence
Amotivation
Regulation
Enjoyment
Perceived
Perceived
External
Interest/
Choice
Experience Age
Beth 16-25 years 26 1.00 1.00 6.43 5.86 6.33
Jane 16-25 years 27 1.00 2.50 6.71 5.14 6.67
Roger 16-25 years 23 1.00 2.00 6.14 5.29 6.83
Peter 16-25 years 24 1.00 2.50 6.57 4.86 7.00
Observations Back
70 of 45

71. Comparing Intrinsic Digital Non-Native
Inventory Scores
Ordered from Lowest Perceived Competence to Highest
Competence
Amotivation
Regulation
Enjoyment
Perceived
Perceived
External
Interest/
Choice
Experience Age
Wilma 16 - 25 years 87 1.00 3.75 5.00 4.71 2.67
Mike 16 - 25 years 74 1.50 2.75 4.14 4.14 3.00
Rebecca 25+ years 65 1.00 4.00 5.57 4.29 5.17
Back
Observations
71 of 45

72. Using Help or Not Back
• Many had no experience
• Or old experience from 10 years ago when help was
notoriously bad
• Stumbling of intrinsic digital native Beth had different
quality because used help
“Because it’s going to have 50 pages of text that I have no
desire whatsoever to read about something that I use
rarely, and I don’t really care to know. I don’t read
instruction manuals, generally. And why would I Google
it? I wouldn’t, because it’s a bunch of teenagers who
can’t spell right, who don’t use punctuation, all lower
case,” answers Lucy.
Introduction Study Design Motivation Observations Future Work Conclusions 72 of 45

73. All 66 Respondents
Extrinsics
8.00
7.00
Apathy 8.00
7.00
perceived competence
perceived competence
6.00 6.00
5.00 5.00
4.00 4.00
3.00 3.00
2.00 2.00
1.00 1.00
0.00 0.00
0.00 2.00 4.00 6.00 0.00 2.00 4.00 6.00 8.00
amotivation external regulation
8.00
Intrinsics 8.00
7.00 7.00
perceived competence
perceived competence
6.00 6.00
5.00 5.00
4.00 4.00
3.00 3.00
2.00 2.00
1.00 1.00
0.00
0.00 2.00 4.00 6.00 8.00
0.00
0.00 2.00 4.00 6.00 8.00 More
interest/enjoyment perceived choice
Introduction Study Design Motivation Observations Future Work Conclusions 73 of 45

74. All 16 Observed Participants
8 8
7 7
perceived competence
perceived competence
6 6
5 5
4 4
3 3
2 2
1 1
0 0
0 1 2 3 0 2 4 6 8
amotivation external regulation
8 8
7 7
perceived competence
perceived competence
6 6
5 5
4 4
3 3
2 2
1 1
0 0
Back
0 2 4 6 8 0 2 4 6 8
interest/enjoyment perceived choice
74 of 45

75. Intrinsics
Correlations
Relationship To
Stumble Fall +
Age + Extrinsics
Digital Native - Relationship To
Interest -
Age Persist +
Perceived Competence -
Digital Native Age - Digital Native -
External Regulation - Amotivation -
Interest/Enjoyment + External Perceived -
Perceived Choice + Regulation Choice
Perceived Competence +
Age External Regulation +
Perceived Competence -
Perceived Fall -
Competence Interest/Enjoyment +
External -
Regulation Perceived Choice More
Introduction Study Design Motivation Observations Future Work Conclusions 75 of 45

76. Test Effects
• Lowering of emotions
• Learning without any teaching
– “do it” = “you CAN do it”
– Expect researcher to fix any problems
• Performance hindrances
– Age (Mike, Wilma, Walter, Marsha)
– Eye strain (Walter, Wilma)
– Tiredness (Lucy, Miranda, Wilma, Walter)
– Distraction (Molly’s daughter, Walter)
Introduction Study Design Motivation Observations Future Work Conclusions 76 of 45

77. Test Effects: Stress
Max Stress Self Rating of Participant by intrinsic (left) and extrinsic (right) on
a scale of 1 to 10 with 10 being high and 1 is low. Ordered from low stress to
high for both groups.
Intrinsics max stress JE Users max stress
Beth 1 Lilly 2
Jane 1 Mary Ann 4
Rebecca 1 Molly 5
Roger 1 Olivia 5
Mike 2 Walter 5
Peter 6 Alice 6
Wilma 10 Lucy 7
Marsha 8
Miranda 10
Introduction Study Design Motivation Observations Future Work Conclusions 77 of 45

78. Proposed Solutions
• Limit unfamiliar tasks, software, or systems
(impractical)
• Teach big picture patterns and how they relate
from one situation to another
• Teach visual and vocabulary tools
• Give a sense of belonging
• Generate interest and choice
Introduction Study Design Motivation Observations Future Work Conclusions 78 of 45

79. JE User vs. Intrinsic: Walter & Mike
Similarities:
1. Same age Differences:
2. Both retired professors 1. Performance
3. Both persisting 2. Different motivation style
4. Similar competence
5. Similar experience level
6. Both agitated but say they are “fine” More
Introduction Study Design Motivation Observations Future Work Conclusions 79 of 45

80. Weaknesses
1. Sample was a convenient sample
2. Ordinal Likert scale results should not be
averaged
3. All participants had different tasks so they
are not easily comparable
4. Sample size was small
5. Pros and cons of qualitative ethnographic
techniques
6. Did not measure the quantity, rate, and type
of task success
Introduction Study Design Motivation Observations Future Work Conclusions 80 of 45

81. What could have been…
1. Randomized respondent selection for motivation inventory to
get an evenly distributed sample
2. Standardized tasks assigned to measure rate and type of
stumbling and success
3. Standardized unfamiliar and familiar system and software
4. Give written instructions instead of verbal
5. Keep researcher ignorant of motivation scores before
observations
6. Participant alone in a room with the observer outside the room
7. Possibly observing through one way glass or video camera and
screen capture
8. Eliminating researcher interaction with participants
Introduction Study Design Motivation Observations Future Work Conclusions 81 of 45

82. So Much Data…
Could be re-analyzed with other emphases
• Digital literacy
• Communication patterns
• Misinformation or ignorance of a novice
• Attitudes to life long learning
• Attitudes of a “refuser”
• And more…
Introduction Study Design Motivation Observations Future Work Conclusions 82 of 45

83. Bigger Sample
• What percent are extrinsic? Intrinsic?
• What percent are Low-Low or High-High?
• How to characterize Low-Low or High-High?
• What percent are digital natives and non-
natives?
• Do age, perceived competence, or being
digital native hold no difference across
intrinsic and extrinsic?
Introduction Study Design Motivation Observations Future Work Conclusions 83 of 45

84. The Flaws Were Also Strengths
• Rich and diverse insights into identifying JE users
• Diverse population of daily proficient users
• Successfully quantified failure
• Likert scale average is standard in Social Science
• Captured individual proficiency and tested
unfamiliar tasks, software, & system
• Captured motivation style
Introduction Study Design Motivation Observations Future Work Conclusions 84 of 45

85. Other Questions
• “Just Enough” term?
• Gender, socioeconomic status, years of
experience, aversion to change?
• Separating work and play in motivation study
• Less frequent users?
• What if a “consequence” element?
• Hand held computers?
• Food and sleep deprived?
Introduction Study Design Motivation Observations Future Work Conclusions 85 of 45

86. Statistical Analysis of Inventory
Do 3 assumptions hold for n=66 Respondents and
also for n=16 Participants?
Statistical Analysis Inventory Factors
PARAMETRIC: Passes 3 assumptions 1.external regulation
for parametric analysis 2.interest/enjoyment
3.perceived choice
4.identified regulation
NON-PARAMETRIC: Must be non- 1.amotivation
parametrically analyzed 2.perceived competence
3.age
4.digital native
More information and detail in Supplementary Slides More
Introduction Study Design Motivation Observations Future Work Conclusions 86 of 45

87. Significant Differences
between extrinsic and intrinsic
Extrinsics and Intrinsics Have Significant Differences in Phase 2,
Phase 3, and total occurrences.
Phase Stumble Fall Persist Quit
2 Different Different Not Significant Different
(p=0.003) (p=0.003) (p=0.127) (p=0.041)
3 Different Different Different Not Significant
(p=0.018) (p=0.025) (p=0.023) (p=0.470)
Both Different Different Different Different
Phases (p=0.004) (p=0.005) (p=0.030) (p=0.014)
Introduction Study Design Motivation Observations Future Work Conclusions 87 of 45

88. Task FAMILIAR Difference in UNFAMILIAR UNFAMILIAR
GMAIL webmail Work Flow Obscure Company webmail GMX webmail
compose click “compose” button, top obscure is more difficult to see click pencil/paper icon, top click “compose mail” button,
mail (Fig. left, contrasting color compose but all compose are in left between other icons, top left, same color
6) same area of screen same color, no words
open click “inbox” word on left NO DIFFERENCE click “inbox” word on left top, click “inbox” word on left top,
inbox top, same color same color same color
read mail click “[name of sender or Gmail replaces center panel, click “[name of sender or click “[name of sender or
participant]” of mail in others open side by side with participant]” of mail in center participant]” of mail in left half
center large panel, same inbox list either below or to the top half panel, same color, of center panel, same color,
color, opens by replacing right opens in bottom half of opens in right half of center
same center window center panel panel
reply to click “arrow” icon button on Gmail has two places, both click “reply” button with click “reply” button with
mail right at top of what reading, same color, one is word, one is picture and word, top icon picture and word, top icon
same color, no word, OR icon, one or both can disappear bar, first of 9 buttons with bar, 2nd of 7 buttons with
gray “reply” word link at with medium and bigger emails, words, same color words, same color
bottom, same color in floats on top of mail view so not
separate white box. NOTE: always visible. Both gmx and
if email is medium to large, obscure have one step, button
“arrow” icon button with word and icon, always
disappears into the header visible
and the second choice
disappears into the footer
forward click “drop down” arrow on Gmail has two places, one click “forward” button with click “forward” button with
mail “arrow” for reply to see requires two steps (select from picture and word, top icon picture and word, top icon
more options, same color, drop down), both ways are bar, 3rd of 9 buttons with bar, 3rd of 7 buttons with
then select “forward in drop same color, one is word, one is words, same color words, same color
down menu”, all on center icon, one or both can disappear
right at top of email with medium and bigger emails,
reading, or click gray on floats on top of mail view so not
white words in white box at always visible. Both gmx and
bottom (often not visible if obscure have one step, button
reading anything other than with word and icon, always
shortest email) NOTE: visible, both at top center area
same as for reply 88 of 45

89. Phase 2 & 3 No Difference
There was no significant different between Unfamiliar
Task compared to Near Skill Transfer for either
intrinsics or extrinsics.
Stumble Fall Persist Quit Resist
Extrinsic (p=0.370) (p=0.147) (p=0.738) (p=0.056) (p=0.494)
Intrinsic (p=0.784) (p=0.872) (p=0.317) (p=0.317) (p=0.317)
Introduction Study Design Motivation Observations Future Work Conclusions 89 of 45