Defense 20121130
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Defense 20121130

on

  • 601 views

Understanding "Just Enough" Computer Users: Motivation Style and Proficiency

Understanding "Just Enough" Computer Users: Motivation Style and Proficiency

Statistics

Views

Total Views
601
Views on SlideShare
601
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Defense 20121130 Presentation Transcript

  • 1. Understanding “Just Enough” Computer Users:Motivation Style and Proficiency By Harriet King Masters Candidate in Computer Science 1 of 45
  • 2. The Question Why do some proficient daily computer users, stumble over the unfamiliar and others easily adapt? EXAMPLE: More information and detail in Supplementary Slides MoreIntroduction Study Design Motivation Observations Future Work Conclusions 2 of 45
  • 3. What Is a Just Enough (JE) User? • Daily computer user • Competent • Extrinsic MotivationIntroduction Study Design Motivation Observations Future Work Conclusions 3 of 45
  • 4. The Hypothesis We hypothesize that extrinsically motivated proficient daily computer users have difficulty with unfamiliar computer tasks and skill transfer, whereas intrinsically motivated daily users accomplish unfamiliar tasks readily.Introduction Study Design Motivation Observations Future Work Conclusions 4 of 45
  • 5. Who Cares? • Software designers • Human Computer Interactions (HCI) • Software Users • Stakeholders for computer literacy “Lest we wish to change our field’s name to student-computer interaction we should make effort to find more representative participants” (Barkhuus and Rode 2012)Introduction Study Design Motivation Observations Future Work Conclusions 5 of 45
  • 6. Study Design Overview OUTPUT INVENTORY scores & statistics group descriptors OBSERVATIONS Coded & analyzed attitudes & actionsIntroduction Study Design Motivation Observations Future Work Conclusions 6 of 45
  • 7. Richness of Data for Understanding • Pre-questionnaire: daily users? • Quantitative motivation inventory scores • Demographic and interview questions • Ethnographic observation methods: – Think Aloud Protocol – Observation recordings – Researcher questions and follow up • Quantify transcripts with coding • Post-questionnaire and JE Users questionnaire (Sim 1999; Rose, Shneiderman, Plaisant. 1995)Introduction Study Design Motivation Observations Future Work Conclusions 7 of 45
  • 8. MOTIVATION 8 of 45
  • 9. Motivation Background Motivation Styles, adapted from Ryan and Deci (2000) ‘Taxonomy of Human Motivation’. Low interest and enjoyment are on the left ranging to high interest and enjoyment on the right. (Pintrich 2003; Deci and Ryan 1991; Downey and Smith 2011; Martens et al. 2004; Deci and Ryan 1985; Iyengar and Lepper 2000; Henderlong and Lepper 2002; Ryan and Deci 2000; Ryan and Deci 2012; Oudeyer et al. 2007) More More information and detail in Supplementary SlidesIntroduction Study Design Motivation Observations Future Work Conclusions 9 of 45
  • 10. Motivation Inventory Source Factors Guay, Vallerand, Blanchard 1. Amotivation (2000) 2. External Regulation 3. Identified Regulation Ryan and Deci (IMI 2012) 4. Interest/Enjoyment 5. Perceived Choice 6. Perceived Competence L to R: Richard Ryan and Edward Deci (Photo: Adam Fenster, August 2010)Introduction Study Design Motivation Observations Future Work Conclusions 10 of 45
  • 11. Adapting Questions Precedent: (Shroff and Vogel 2009). Confirmed Inventory with two pilot studies.Introduction Study Design Motivation Observations Future Work Conclusions 11 of 45
  • 12. Precedents for Scoring Inventory Likert scale IS-An ordinal measure of ranking “We did violate some mathematical assumptions in creating an interval level of measurement index out of ordinal components, but as previously indicated, this is common practice in the social and behavioral sciences.” (Sirkin, R. M., 2006. “Statistics for the Social Sciences.” 3rd edition, Sage Publications. Precedent for averaging motivation inventory scores 1. Pavlas, Jentsch, Salas, Fiore, and Sims, 2012 2. Shroff and Vogel, 2009 3. McAuley, Duncan, and Tammen, 1989Introduction Study Design Motivation Observations Future Work Conclusions 12 of 45
  • 13. Who Took the Inventory? Everybody!• Ages 13 to 87 from FIVE continents• 9 countries: USA, China, Turkey, Australia, Sweden, U.K., South Africa, India, and France• 130+ total completed questionnaire• Used 66 for total respondents• 16 participants observed (7 intrinsics, 9 extrinsics) Community Classmates Faculty InternetIntroduction Study Design Motivation Observations Future Work Conclusions 13 of 45
  • 14. Required Correlation Table 8: Pearson Correlation of Interest/Enjoyment & Perceived Choice Correlation of Interest/Enjoyment & Perceived Choice Factors n = 66 n =16 All Respondents All Observed Correlation 0.602 0.815 Significance (2- p < 0.01 p < 0.01 tailed)Introduction Study Design Motivation Observations Future Work Conclusions 14 of 45
  • 15. Grouping Variables Venn Diagram is External Regulation > 4.0 intersecting Total and percent inventory Interest/Enjoyment > 4.0 responses by group with n=66Introduction Study Design Motivation Observations Future Work Conclusions 15 of 45
  • 16. Inventory T Test Results Significant Differences in Inventory Scores, Age, & Digital Native * Asterisk indicates non parametric Mann-Whitney U test All other are Independent Samples T-test Factor Different Significance Age* NOT different p=0.396 Digital Native* NOT different p=0.166 Perceived Competence* NOT different p=0.071 Amotivation* Different p=0.012 External Regulation Different p<0.001 Interest/Enjoyment Different p<0.001 Perceived Choice Different p=0.001 More information and detail in Supplementary Slides MoreIntroduction Study Design Motivation Observations Future Work Conclusions 16 of 45
  • 17. Intrinsics: Digital Native or Not Side by side comparison of digital non-natives (3) on left and digital natives (4) on right. Ordered from low to high competence Perceived Interest/ Enjoyment Perceived Choice Digital native Competence Digital native Digital native Non-native Non-native Non-native 6.43 5.86 6.33 5.00 6.71 4.71 5.14 2.67 6.67 4.14 6.14 4.14 5.29 3.00 6.83 5.57 6.57 4.29 4.86 5.17 7.00 MoreIntroduction Study Design Motivation Observations Future Work Conclusions 17 of 45
  • 18. Digital Natives not significantly different Digital Natives All 70 66 60 50 Number of People 40 41% 30 27 37% 57% 22% 20 16 10 9 6 7 4 2 0 Observed Participants Extrinsics Inventory Respondents Intrinsics GroupsIntroduction Study Design Motivation Observations Future Work Conclusions 18 of 45
  • 19. Not Significantly Different Age, Perceived Competence, & Digital Native or not Mean Age with error bars for Mean Perceived Competence with standard deviation error bars for standard deviation 90.00 7.00 Mean Perceived Competence 80.00 6.00 70.00 5.38 60.00 55.67 5.00 50.00 46.57 3.70Age 4.00 40.00 30.00 3.00 20.00 2.00 10.00 0.00 1.00 9 Extrinsics 7 Intrinsics 9 Extrinsics 7 Intrinsics Introduction Study Design Motivation Observations Future Work Conclusions 19 of 45
  • 20. Mean Inventory Results with error bars showing standard deviation n=66 Respondents n=16 Observed n=9 JE Users n=7 Intrinsics 8 7 6 Likert Scale 1 - 7 with neutral at 4 5 4 3 2 1 Grouping Grouping Variable Variable 0 MoreIntroduction Study Design Motivation Observations Future Work Conclusions 20 of 45
  • 21. Data Screening Extra High Perceived Choice Mean Perceived Choice with standard Extrinsic Molly = 5.57!? deviation error bars 2.3 standard deviations above 5.57 MoreIntroduction Study Design Motivation Observations Future Work Conclusions 21 of 45
  • 22. OBSERVATIONS 22 of 45
  • 23. Observation PhasesIntroduction Study Design Motivation Observations Future Work Conclusions 23 of 45
  • 24. Near Skill Transfer MoreIntroduction Study Design Motivation Observations Future Work Conclusions 24 of 45
  • 25. Participant Hesitation Wording “uhhhh” looked in there” promising” “I’m looking for a way to “I think I can just... click on [sigh] do...” this here, and... that didn’t “no, thats not it” “maybe if I go here” work” “maybe this” “what’s this?” “ok, that didn’t work” “so, were not doing that” “I can’t...” “I looked at the bottom but “I wouldnt think itd be “ummm” there’s nothing there” under that” “let’s go back here” “I saw this click to ... but “Im going to try right click [giggling] that isn’t it” again” “aaaaannnnnnnd” “hmmm” “I forgot what you said to “I could try like..” “contacts....contacts.... do” “no I can’t drag that..” contacts” “this damn mouse” “I’ll look in here, no I just “that doesnt look veryIntroduction Study Design Motivation Observations Future Work Conclusions 25 of 45
  • 26. For Prompting the Participant “go ahead and tell me what you’re seeing” “please tell me what you’re thinking” “Are you trying to decide something, can you tell me about it?” “did that work?” “what seems odd about this?” “what are you thinking?” “you’re giggling, …you’re sighing…you sound angry, what are you feeling?”Introduction Study Design Motivation Observations Future Work Conclusions 26 of 45
  • 27. Rubric for Coding Observations CODE RULE Stumble [action] >= 20 seconds Fall [action] >= 1 minute Persist [action] >= 3 minutes Quit attitude towards a task Resist attitude towards a taskIntroduction Study Design Motivation Observations Future Work Conclusions 27 of 45
  • 28. Transcript Example stumble persist resist quit falltime OLIVIA [action] “quote” (time on video) analysisb 7:58 [while looking for spam, stumbles across trash 7:58 and says I’ll empty the trash 1 1 1e 9:08 instead, I say go ahead] Participant: “I have no idea how to do that. It’s already IN the trash” me: “Look around. ...you can empty the trash.” (8:10) Participant: “It’s already IN trash. Where do you empty trash to? I’m thinking that I never empty my trash because there’s no way to empty trash because it’s already trash.” (8:25) me: “no, there is a way to empty trash.” Participant: “There’s no trash emptying.” [ask about her agitation] Participant: “I’m not agitated at all. You’re just wrong. There’s no trash emptying.” [ask what she’s feeling] Participant: “I think it’s dumb that the trash doesn’t have an empty.” (8:40) me: “It does actually” Participant: “I don’t see it. If I click on something in my trash, all I can do is trash something in my trash, which is silly because it’s already in my trash” (9:08) me: “Ok, we’ll come back to this. Let’s look at your spam” [so resistant that I stop this task on test. Never does trash]b 9:10 Participant: “I don’t know if I have spam” (9:10) me: “You do have spam.” “No. Really!? 1 1e 9:45 I’m looking at all my folders and I do not have one called “spam”” (9:20) me: “Did you find “more” at the bottom?” “There’s a more. Oh look at that, there’s spam.” (9:45)b 9:50 [directed to delete all spam at once, (9:50), giving her hints] me: “It’s not that tricky, it 1 1e 11:10 has words and I can see them, I’m looking at it right now” (10:37) (11:10) found “delete all messages now”b 11:20 [11:20 Go to address book] Participant: “I’m not fully sure where my address book is, I 1e 12:10 think I have to go to my calendar”, then found contacts 12:10Introduction Study Design Motivation Observations Future Work Conclusions 28 of 45
  • 29. Inter Rater Reliability Results • First Rater (HK) • 2 outside raters (SK and PM) • Outside raters reviewed 30% of transcripts • Stumble, fall, and persist are time related Rater 1 Rater 2 Stumble, Fall, Persist 100% agreement 100% agreement Quit 99.13% agreement 97.73% agreement Resist 96.52% agreement 97.73% agreementIntroduction Study Design Motivation Observations Future Work Conclusions 29 of 45
  • 30. Occurrences for Each Code • Asterisk indicates statistically significant difference for this code between extrinsic and intrinsic. Total occurrences with percent of total in parentheses. • There was no significant difference between Unfamiliar Task compared to Near Skill Transfer for either intrinsics or extrinsics. Stumble* Fall* Persist* Quit* Resist JE Users 91 56 15 9 13 (81%) (84%) (88%) (90%) (87%) Intrinsics 21 11 2 1 2 (19%) (16%) (12%) (10%) (13%) MoreIntroduction Study Design Motivation Observations Future Work Conclusions 30 of 45
  • 31. All Occurrences of Stumble & Fall Intrinsics on left and Extrinsics on right stumble fall Extrinsics Intrinsics201510 5 0 MoreIntroduction Study Design Motivation Observations Future Work Conclusions 31 of 45
  • 32. JE User vs. Intrinsic: Marsha & Rebecca Exter: 4.5 Exter: 4.0 Int/En 2.57 Int/En 5.57 Similar: 1. both Amotivation = 1.0 2. Both digital non-native Different: 3. similar experience level 1. Performance 4. similar self rate and perceived competence 2. Different motivation styles 5. similar age 6. Appeared to cruise through unfamiliar tasks 7. Responsible community leaders More 8. Professional womenIntroduction Study Design Motivation Observations Future Work Conclusions 32 of 45
  • 33. Resist • Only 5 out of 16 resisted 8 • 4 extrinsic & intrinsic Mike 7 • Olivia had 7 resists Total Occurrences of Resist 6 1. Can’t empty trash 2. there is no spam 5 3. doesn’t “add” to group but 4 insists she did 4. says “check mail” button is 3 broken 2 5. won’t remove attachment, 6. says used wrong address but 1 was sent folder issue 0 7. says did not spell a word Lucy Mike* Miranda Marsha Olivia correctly when did spell correctly MoreIntroduction Study Design Motivation Observations Future Work Conclusions 33 of 45
  • 34. Another Type of Resist Marsha shares, “I never use the google calendar. I’m not telling them what I’m doing every day. Forget that!” “Passionate?...I am. I’m not MAD at them [MS Word], I’m frustrated with them. … they’re leaving out the average person. And maybe that’s what open office is for. I don’t know.”Introduction Study Design Motivation Observations Future Work Conclusions 34 of 45
  • 35. Quit 8 of 16 quit Quit Resist 8 * Asterisk indicate intrinsic Total Occurrences of Quit 7 6 5 4 3 2 1 0 Screenshot of "Contacts" button behind "Mail" in Gmail. Doesnt look like a button with no rectangle More or color change.Introduction Study Design Motivation Observations Future Work Conclusions 35 of 45
  • 36. Persist 6 Total Occurrences of 5 4 Persist 3 8 of 16 Persist 2 1 0 Mary Molly Lucy Olivia Mike* Marsha Alice Walter Ann Walter spent about 5 minutes (7:40 to 13:10) using wrong password of “guest” and misspelled username trying to login to gmail online. He repeated the same behavior while expecting different resultsIntroduction Study Design Motivation Observations Future Work Conclusions 36 of 45
  • 37. Lowest Interest & Choice • Extrinsics Lilly and Olivia • Opposite attitudes (shame vs. blame) Lilly Olivia 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00 Perceived Enjoyment amotivation Competence regulation external Perceived Choice Interest MoreIntroduction Study Design Motivation Observations Future Work Conclusions 37 of 45
  • 38. Just Enough User Alice (1/9) “I don’t do ANYTHING that I’m not taught. And that is a big drawback in my learning.” “I know enough to get what I want, most of the time. And it definitely is not a pleasure for me to try to figure out things on my own. N-O-T AT A-L-L… Maybe everyone thinks they are a “Just Enough” user.” MoreIntroduction Study Design Motivation Observations Future Work Conclusions 38 of 45
  • 39. Just Enough User Lucy (3/9) “Why would I Google it? I wouldn’t, because it’s a bunch of teenagers who can’t spell right, who don’t use punctuation, all lower case.” “I am fine using the computer only for what I need. I think they are ruining the world quite frankly, and am slightly proud I find them somewhat repulsive machines.” MoreIntroduction Study Design Motivation Observations Future Work Conclusions 39 of 45
  • 40. Just Enough User Walter (8/9) “You are … confronting an unbelievably unfamiliar system, with all the scariness of being surrounded by REAL fully paid, fully trained, card carrying life member geeks … I got spooked by the surroundings. I got intimidated by my high level of geekitude surroundings.” “People do get on without a computer at all, so perhaps ‘No Computer’ (or ‘The Computer They Make You Use At Work’) is the true ‘Just Enough Computer’.” MoreIntroduction Study Design Motivation Observations Future Work Conclusions 40 of 45
  • 41. Future Work • So much data! • Bigger sample • “Just Enough” term? • Gender, socioeconomic status, years of experience, aversion to change? • Separating work and play in motivation study • Less frequent users? • What if a “consequence” element? • Hand held computers?Introduction Study Design Motivation Observations Future Work Conclusions 41 of 45
  • 42. CONCLUSIONS 42 of 45
  • 43. Statistical Results • Confirmed competency of JE users • Extrinsic proficient daily users stumble, fall, persist and quit significantly more than intrinsics • AND it is not explained by age, perceived competence, or being digital native • JE users account for over 80% of performance difficulties in our study • Just Enough users exist in all age groups and experience levels (18% in our sample)Introduction Study Design Motivation Observations Future Work Conclusions 43 of 45
  • 44. Observed Phenomena • Impossible to differentiate JE user from any other competent user, until faced with the unfamiliar • Just Enough users shed competencies as they become unnecessary • Wide range of attitudes and experience related to exploring and performance • Sense of “not belonging”Introduction Study Design Motivation Observations Future Work Conclusions 44 of 45
  • 45. A HaikuThank Just Enough is cool you! till unfamiliar and new safe routine un-do. 45 of 45
  • 46. Just Enough User Lilly (2/9) When asked during the test about her feelings, Lilly shares, “ohhh, why am I so stupid? How can I not know how to do this? I dread asking one of my kids because they have no patience.” “I really want computers to be as unobtrusive in my daily life as can be. Just Enough term sounds a bit lazy.” MoreIntroduction Study Design Motivation Observations Future Work Conclusions 46 of 45
  • 47. Just Enough User Marsha (4/9) Marsha says, “I like to sign out, because then they, THEORETICALLY, aren’t watching me, but you know they are because advertisements for something I just looked at turn up on the *weirdest* pages.” “My feelings are that I would like to be more than that [JE user]. I would consider a "just enough" user to be one who uses only email, or only cruises the web for news, or only uses one application.” MoreIntroduction Study Design Motivation Observations Future Work Conclusions 47 of 45
  • 48. Just Enough User Mary Ann (5/9) “When I’m at work, I’m so busy, that I don’t have time to play around... I always have to do things in the fastest way possible, which doesn’t allow exploration.” “My feelings are that I would like to be more than that. I do not want to be a "dinosaur. I sometimes can do a little more than just enough if I get up my courage to try." MoreIntroduction Study Design Motivation Observations Future Work Conclusions 48 of 45
  • 49. Just Enough User Miranda (6/9) “It seems stupid and why should I waste my time staring at the computer.” “My feelings are, why would I spend any more time at the computer? Id rather read a book or take a walk. Just enough is a perfect name.” MoreIntroduction Study Design Motivation Observations Future Work Conclusions 49 of 45
  • 50. Just Enough User Molly (7/9) “This all is stupid. This is ridiculous. I don’t know why anyone uses computers. … I don’t really care. I can basically do anything I need to do and I have [IT worker] and if I can’t do anything I just call [IT worker] and cry.” “The term "Just Enough" is kind. I dont feel judged or "less than" (stupid).” MoreIntroduction Study Design Motivation Observations Future Work Conclusions 50 of 45
  • 51. Just Enough User Olivia (8/9) “[it] is really annoying not to be able to find these things that you’re CLAIMING it’s on here. And it’s like, how are you supposed to know where it is.....[I’m] irritated.” “Very proud that I can do it enuf [sic]. People should make more things easy for us.” MoreIntroduction Study Design Motivation Observations Future Work Conclusions 51 of 45
  • 52. “Just Enough Users”, a poem Just enough is satisficing, works out fine till new and strange. Computer changes make life messy, then it’s struggle stumble quit. Those interest people cruise along, probably nothing ever wrong. Curse you easy flexing user. Why can’t I just find my cursor? Just Enough left me so helpless, when the web changed all my favorites. I just want to stay so lazy, stay low interest, stay low effort. OK sometimes then I stumble. Just Enough was not effective. Who to blame and who to curse? Designers! They must be the worst.Conclusion 52 of 45
  • 53. Intrinsic Motivation Characteristics • Deeper involvement in activities; natural activity • More curiosity; exploration • Trying out more complex options • Increased persistence • Higher achievement of goals; improved performance • Less avoidance behavior • Interest, excitement, and confidence Back (Martens et al. 2004; Oudeyer et al. 2007; Deci and Ryan 2000)Motivation Inventory 53 of 45
  • 54. Intrinsic Motivation: Supports & By Products • Self-esteem and general well-being • Competence • Autonomy • Adaptable • Pros/cons of praise Back • Reduced by external rewards • Supported by seeing examples; having capability (Pintrich 2003; Deci and Ryan 1991; Downey and Smith 2011; Martens et al. 2004; Deci and Ryan 1985; Iyengar and Lepper 2000; Henderlong and Lepper 2002; Ryan and Deci 2000; Ryan and Deci 2012; Oudeyer et al. 2007)Motivation Inventory 54 of 45
  • 55. Parametric or Non parametric? Does data pass the 3 assumptions for parametric statistical analysis? 1. Independence? Yes! All different humans 2. Homogeneity? (equal variance, Levene’s test) 3. Normality? (skewness & kurtosis < |1.95|) Back Does it pass for 66 respondents and 16 participants?Motivation Inventory 55 of 45
  • 56. Homogeneity of Inventory Factors Levene’s Test for Levene’s Test for Equality of Variances Equality of Variances n = 16 Observed. n = 66 Respondents. Significance SignificanceAmotivation 0.053 0.002Identified Regulation 0.802 0.546External Regulation 0.572 0.822Interest/Enjoyment 0.989 0.842Perceived Choice 0.492 0.218Perceived Competence 0.152 0.010Motivation Inventory Back 56 of 45
  • 57. Normality of Inventory Factors Respondents n=66 Observed Participants n=16 Skewness Kurtosis Skewness Kurtosis Amotivation 1.486 1.986 1.004 -0.557 Identified Regulation -0.063 -1.048 -0.527 -1.157 External Regulation 0.038 -0.781 -0.273 -0.870 Interest/Enjoyment -.0513 -0.050 0.165 -1.358 Perceived Choice -0.213 -0.708 -0.050 -1.565 Perceived Competence -0.246 -0.609 -0.533 -0.988 BackMotivation Inventory 57 of 45
  • 58. T Test Result DetailAmotivation (Mann-Whitney U test) (U = 9.50,p = 0.012).Perceived Choice (independent Samples T test)extrinsic (M=2.7, SD=1.3) and intrinsics (M=4.9,SD=0.6); t(14)=4.306, p=0.001. Back 58 of 45
  • 59. Intrinsics Descriptive Statistics N Min Max Mean Std. Dev Skewness Kurtosis Std. Std. Statistic Error Statistic Errorage 7 23 87 46.71 27.93 0.59 0.79 -1.96 1.59amotivation 7 1.00 1.50 1.07 0.19 2.65 0.79 7.00 1.59external 7 1.00 4.00 2.64 1.02 -0.19 0.79 -0.06 1.59regulationInterest/ 7 4.14 6.71 5.80 0.95 -0.96 0.79 -0.11 1.59EnjoymentPerceived Choice 7 4.14 5.86 4.90 0.59 0.32 0.79 -0.35 1.59Perceived 7 2.67 7.00 5.38 1.84 -0.87 0.79 -1.30 1.59Competence Amotivation (Mann-Whitney U test) (U = 9.50, p = 0.012). Back Perceived Choice (independent Samples T test) extrinsic (M=2.7,Motivation Inventory intrinsics (M=4.9, SD=0.6); t(14)=4.306, p=0.001. SD=1.3) and 59 of 45
  • 60. Extrinsics Descriptive Statistics N Min Max Mean Std. Dev Skewness Kurtosis Std. Std. Statistic Error Statistic Errorage 9 34 74 55.78 14.17 -0.18 0.72 -1.32 1.40amotivation 9 1.00 2.75 1.89 0.74 0.08 0.72 -1.82 1.40external regulation 9 4.25 6.50 5.39 0.89 -0.02 0.72 -1.92 1.40Interest/ 9 1.57 4.00 2.73 0.89 -0.03 0.72 -1.35 1.40EnjoymentPerceived Choice 9 1.57 5.57 2.67 1.26 1.71 0.72 3.43 1.40Perceived 9 2.50 4.33 3.70 0.72 -0.87 0.72 -0.80 1.40Competence BackMotivation Inventory 60 of 45
  • 61. Summary of Correlations Back n=66 Inventory Respondents & n=16 Observed Participants Relationship Correlation Significance n R^2 External Regulation with - 0.821 p=0.001 16 67.40% Interest/Enjoyment - 0.397 p=0.001 66 15.76% External Regulation with Perceived - 0.879 p=0.001 16 77.26% Choice - 0.785 p=0.001 66 61.62% Amotivation with Perceived - 0.602 p=0.014 16 36.24% Competence - 0.339 p=0.005 66 11.49% Age with Perceived Competence - 0.710 p=0.002 16 50.41%Motivation Inventory 61 of 45
  • 62. Summary of Correlations n=66 Inventory Respondents & n=16 Observed Participants Relationship Correlation Significance n R^2 External Regulation with - 0.821 p=0.001 16 67.40% Interest/Enjoyment - 0.397 p=0.001 66 15.76% External Regulation with Perceived - 0.879 p=0.001 16 77.26% Choice - 0.785 p=0.001 66 61.62% Amotivation with Perceived - 0.602 p=0.014 16 36.24% Competence - 0.339 p=0.005 66 11.49% Age with Perceived Competence - 0.710 p=0.002 16 50.41% BackMotivation Inventory 62 of 45
  • 63. Digital Native Correlations Digital Native Significant Correlations for Observed Participants Digital Native Correlation Significance n R^2 Relationship with... ...Age 0.536 p<0.001 16 28.73% ...Interest/Enjoyment 0.561 p=0.024 16 31.47% ...Perceived Choice 0.575 p=0.020 16 33.06% ...Perceived Competence 0.647 p=0.007 16 41.86% ...External Regulation -0.534 p=0.033 16 28.52% BackMotivation Inventory 63 of 45
  • 64. Mean Occurrences of Codes Mean Number of Code Occurrences for Extrinsics and Intrinsics Extrinsics Intrinsics stumble 10.11 3.00 fall 6.11 1.57 quit 1.00 .29 resist 1.11 .29 persist 1.67 .29 BackObservations 64 of 45
  • 65. Correlations for Extrinsics Extrinsic Group Significant Relationships Relationship To Correlation Significance n R^2 Age Persist 0.667 0.050 9 44.49% Digital Native -0.728 0.026 9 53.00% Amotivation -0.713 0.031 9 50.84% External Perceived -0.699 0.036 9 48.86% Regulation Choice BackObservations 65 of 45
  • 66. Correlations for Intrinsics BackRelationship To Correlation Significance n R^2Stumble Fall 0.898 .006 7 80.64% Age 0.823 .023 7 67.73% Digital Native -0.832 .020 7 69.22% Interest -0.861 .013 7 74.13% Perceived Competence -0.917 .004 7 84.09%Digital Native Age -0.866 .012 7 75.00% External Regulation -0.874 .010 7 76.39% Interest/Enjoyment 0.866 .012 7 75.00% Perceived Choice 0.866 .012 7 75.00% Perceived Competence 0.866 .012 7 75.00%Age External Regulation 0.757 .049 7 57.30% Perceived Competence -0.929 .003 7 86.30%Perceived Fall -0.768 .044 7 58.98%Competence Interest/Enjoyment 0.786 .036 7 61.78%External -0.883 .008 7 77.97%Regulation Perceived Choice Observations 66 of 45
  • 67. Extrinsics Detail Asterisk denotes digital native Back Digital native Competence amotivation Enjoyment regulation Perceived Perceived Self rate External stumble Interest Choice persist exper resist quit age fall name Alice 6 4 0 0 4 71 1 3 16to25 1.00 4.75 4.00 3.29 4.17 Lilly 10 5 1 0 0 48 1 7 6to15 1.50 6.50 1.57 1.57 4.33 Lucy* 3 3 1 1 1 34 2 6 16to25 2.75 6.00 3.00 1.57 4.33 Marsha 10 8 1 3 2 68 1 3 more25 1.00 4.50 2.57 2.43 4.33 Mary Ann 16 6 0 0 1 60 1 4 16to25 1.25 6.50 3.71 2.57 4.00 Miranda 10 9 1 2 0 58 1 4 6to15 2.00 4.50 2.00 3.14 3.33 Molly* 10 6 2 0 1 40 2 6 more25 2.75 4.25 3.43 5.57 3.67 Olivia 12 5 1 7 1 48 1 4 16to25 2.75 5.75 1.57 2.14 2.50 Walter 14 10 2 0 5 74 1 2 6to15 2.00 5.75 2.71 1.71 2.67Observations 67 of 45
  • 68. Intrinsics Detail Asterisk denotes digital native Back Digital native Competence amotivation Enjoyment regulation Perceived Perceived Self rate External stumble Interest Choice persist exper resist quit age fall Name Beth* 3 2 0 0 0 26 2 8 16to25 1.00 1.00 6.43 5.86 6.33 Jane* 0 0 0 0 0 27 2 9.5 16to25 1.00 2.50 6.71 5.14 6.67 Mike 8 6 1 2 2 74 1 4 16to25 1.50 2.75 4.14 4.14 3.00 Peter* 0 0 0 0 0 24 2 10 16to25 1.00 2.50 6.57 4.86 7.00 Rebecca 3 0 0 0 0 65 1 5 more25 1.00 4.00 5.57 4.29 5.17 Roger* 0 0 0 0 0 23 2 11 16to25 1.00 2.00 6.14 5.29 6.83 Wilma 7 3 0 0 0 87 1 3 16to25 1.00 3.75 5.00 4.71 2.67Observations 68 of 45
  • 69. Phase 2 & 3 Extrinsic Stumbles Back Total stumble occurrences for each extrinsic participant in phase 2 (blue on left) and phase 3 (orange on right)Observations 69 of 45
  • 70. Comparing Intrinsic Digital Native Inventory Scores Ordered from Lowest Perceived Competence to Highest Competence Amotivation Regulation Enjoyment Perceived Perceived External Interest/ Choice Experience Age Beth 16-25 years 26 1.00 1.00 6.43 5.86 6.33 Jane 16-25 years 27 1.00 2.50 6.71 5.14 6.67 Roger 16-25 years 23 1.00 2.00 6.14 5.29 6.83 Peter 16-25 years 24 1.00 2.50 6.57 4.86 7.00Observations Back 70 of 45
  • 71. Comparing Intrinsic Digital Non-Native Inventory Scores Ordered from Lowest Perceived Competence to Highest Competence Amotivation Regulation Enjoyment Perceived Perceived External Interest/ Choice Experience Age Wilma 16 - 25 years 87 1.00 3.75 5.00 4.71 2.67 Mike 16 - 25 years 74 1.50 2.75 4.14 4.14 3.00 Rebecca 25+ years 65 1.00 4.00 5.57 4.29 5.17 BackObservations 71 of 45
  • 72. Using Help or Not Back • Many had no experience • Or old experience from 10 years ago when help was notoriously bad • Stumbling of intrinsic digital native Beth had different quality because used help “Because it’s going to have 50 pages of text that I have no desire whatsoever to read about something that I use rarely, and I don’t really care to know. I don’t read instruction manuals, generally. And why would I Google it? I wouldn’t, because it’s a bunch of teenagers who can’t spell right, who don’t use punctuation, all lower case,” answers Lucy.Introduction Study Design Motivation Observations Future Work Conclusions 72 of 45
  • 73. All 66 Respondents Extrinsics 8.00 7.00 Apathy 8.00 7.00 perceived competence perceived competence 6.00 6.00 5.00 5.00 4.00 4.00 3.00 3.00 2.00 2.00 1.00 1.00 0.00 0.00 0.00 2.00 4.00 6.00 0.00 2.00 4.00 6.00 8.00 amotivation external regulation 8.00 Intrinsics 8.00 7.00 7.00 perceived competence perceived competence 6.00 6.00 5.00 5.00 4.00 4.00 3.00 3.00 2.00 2.00 1.00 1.00 0.00 0.00 2.00 4.00 6.00 8.00 0.00 0.00 2.00 4.00 6.00 8.00 More interest/enjoyment perceived choiceIntroduction Study Design Motivation Observations Future Work Conclusions 73 of 45
  • 74. All 16 Observed Participants 8 8 7 7 perceived competenceperceived competence 6 6 5 5 4 4 3 3 2 2 1 1 0 0 0 1 2 3 0 2 4 6 8 amotivation external regulation 8 8 7 7 perceived competenceperceived competence 6 6 5 5 4 4 3 3 2 2 1 1 0 0 Back 0 2 4 6 8 0 2 4 6 8 interest/enjoyment perceived choice 74 of 45
  • 75. Intrinsics CorrelationsRelationship ToStumble Fall + Age + Extrinsics Digital Native - Relationship To Interest - Age Persist + Perceived Competence -Digital Native Age - Digital Native - External Regulation - Amotivation - Interest/Enjoyment + External Perceived - Perceived Choice + Regulation Choice Perceived Competence +Age External Regulation + Perceived Competence -Perceived Fall -Competence Interest/Enjoyment +External -Regulation Perceived Choice MoreIntroduction Study Design Motivation Observations Future Work Conclusions 75 of 45
  • 76. Test Effects • Lowering of emotions • Learning without any teaching – “do it” = “you CAN do it” – Expect researcher to fix any problems • Performance hindrances – Age (Mike, Wilma, Walter, Marsha) – Eye strain (Walter, Wilma) – Tiredness (Lucy, Miranda, Wilma, Walter) – Distraction (Molly’s daughter, Walter)Introduction Study Design Motivation Observations Future Work Conclusions 76 of 45
  • 77. Test Effects: Stress Max Stress Self Rating of Participant by intrinsic (left) and extrinsic (right) on a scale of 1 to 10 with 10 being high and 1 is low. Ordered from low stress to high for both groups. Intrinsics max stress JE Users max stress Beth 1 Lilly 2 Jane 1 Mary Ann 4 Rebecca 1 Molly 5 Roger 1 Olivia 5 Mike 2 Walter 5 Peter 6 Alice 6 Wilma 10 Lucy 7 Marsha 8 Miranda 10Introduction Study Design Motivation Observations Future Work Conclusions 77 of 45
  • 78. Proposed Solutions • Limit unfamiliar tasks, software, or systems (impractical) • Teach big picture patterns and how they relate from one situation to another • Teach visual and vocabulary tools • Give a sense of belonging • Generate interest and choiceIntroduction Study Design Motivation Observations Future Work Conclusions 78 of 45
  • 79. JE User vs. Intrinsic: Walter & Mike Similarities: 1. Same age Differences: 2. Both retired professors 1. Performance 3. Both persisting 2. Different motivation style 4. Similar competence 5. Similar experience level 6. Both agitated but say they are “fine” MoreIntroduction Study Design Motivation Observations Future Work Conclusions 79 of 45
  • 80. Weaknesses 1. Sample was a convenient sample 2. Ordinal Likert scale results should not be averaged 3. All participants had different tasks so they are not easily comparable 4. Sample size was small 5. Pros and cons of qualitative ethnographic techniques 6. Did not measure the quantity, rate, and type of task successIntroduction Study Design Motivation Observations Future Work Conclusions 80 of 45
  • 81. What could have been… 1. Randomized respondent selection for motivation inventory to get an evenly distributed sample 2. Standardized tasks assigned to measure rate and type of stumbling and success 3. Standardized unfamiliar and familiar system and software 4. Give written instructions instead of verbal 5. Keep researcher ignorant of motivation scores before observations 6. Participant alone in a room with the observer outside the room 7. Possibly observing through one way glass or video camera and screen capture 8. Eliminating researcher interaction with participantsIntroduction Study Design Motivation Observations Future Work Conclusions 81 of 45
  • 82. So Much Data… Could be re-analyzed with other emphases • Digital literacy • Communication patterns • Misinformation or ignorance of a novice • Attitudes to life long learning • Attitudes of a “refuser” • And more…Introduction Study Design Motivation Observations Future Work Conclusions 82 of 45
  • 83. Bigger Sample • What percent are extrinsic? Intrinsic? • What percent are Low-Low or High-High? • How to characterize Low-Low or High-High? • What percent are digital natives and non- natives? • Do age, perceived competence, or being digital native hold no difference across intrinsic and extrinsic?Introduction Study Design Motivation Observations Future Work Conclusions 83 of 45
  • 84. The Flaws Were Also Strengths • Rich and diverse insights into identifying JE users • Diverse population of daily proficient users • Successfully quantified failure • Likert scale average is standard in Social Science • Captured individual proficiency and tested unfamiliar tasks, software, & system • Captured motivation styleIntroduction Study Design Motivation Observations Future Work Conclusions 84 of 45
  • 85. Other Questions • “Just Enough” term? • Gender, socioeconomic status, years of experience, aversion to change? • Separating work and play in motivation study • Less frequent users? • What if a “consequence” element? • Hand held computers? • Food and sleep deprived?Introduction Study Design Motivation Observations Future Work Conclusions 85 of 45
  • 86. Statistical Analysis of Inventory Do 3 assumptions hold for n=66 Respondents and also for n=16 Participants? Statistical Analysis Inventory Factors PARAMETRIC: Passes 3 assumptions 1.external regulation for parametric analysis 2.interest/enjoyment 3.perceived choice 4.identified regulation NON-PARAMETRIC: Must be non- 1.amotivation parametrically analyzed 2.perceived competence 3.age 4.digital native More information and detail in Supplementary Slides MoreIntroduction Study Design Motivation Observations Future Work Conclusions 86 of 45
  • 87. Significant Differences between extrinsic and intrinsic Extrinsics and Intrinsics Have Significant Differences in Phase 2, Phase 3, and total occurrences. Phase Stumble Fall Persist Quit 2 Different Different Not Significant Different (p=0.003) (p=0.003) (p=0.127) (p=0.041) 3 Different Different Different Not Significant (p=0.018) (p=0.025) (p=0.023) (p=0.470) Both Different Different Different Different Phases (p=0.004) (p=0.005) (p=0.030) (p=0.014)Introduction Study Design Motivation Observations Future Work Conclusions 87 of 45
  • 88. Task FAMILIAR Difference in UNFAMILIAR UNFAMILIAR GMAIL webmail Work Flow Obscure Company webmail GMX webmailcompose click “compose” button, top obscure is more difficult to see click pencil/paper icon, top click “compose mail” button,mail (Fig. left, contrasting color compose but all compose are in left between other icons, top left, same color6) same area of screen same color, no wordsopen click “inbox” word on left NO DIFFERENCE click “inbox” word on left top, click “inbox” word on left top,inbox top, same color same color same colorread mail click “[name of sender or Gmail replaces center panel, click “[name of sender or click “[name of sender or participant]” of mail in others open side by side with participant]” of mail in center participant]” of mail in left half center large panel, same inbox list either below or to the top half panel, same color, of center panel, same color, color, opens by replacing right opens in bottom half of opens in right half of center same center window center panel panelreply to click “arrow” icon button on Gmail has two places, both click “reply” button with click “reply” button withmail right at top of what reading, same color, one is word, one is picture and word, top icon picture and word, top icon same color, no word, OR icon, one or both can disappear bar, first of 9 buttons with bar, 2nd of 7 buttons with gray “reply” word link at with medium and bigger emails, words, same color words, same color bottom, same color in floats on top of mail view so not separate white box. NOTE: always visible. Both gmx and if email is medium to large, obscure have one step, button “arrow” icon button with word and icon, always disappears into the header visible and the second choice disappears into the footerforward click “drop down” arrow on Gmail has two places, one click “forward” button with click “forward” button withmail “arrow” for reply to see requires two steps (select from picture and word, top icon picture and word, top icon more options, same color, drop down), both ways are bar, 3rd of 9 buttons with bar, 3rd of 7 buttons with then select “forward in drop same color, one is word, one is words, same color words, same color down menu”, all on center icon, one or both can disappear right at top of email with medium and bigger emails, reading, or click gray on floats on top of mail view so not white words in white box at always visible. Both gmx and bottom (often not visible if obscure have one step, button reading anything other than with word and icon, always shortest email) NOTE: visible, both at top center area same as for reply 88 of 45
  • 89. Phase 2 & 3 No Difference There was no significant different between Unfamiliar Task compared to Near Skill Transfer for either intrinsics or extrinsics. Stumble Fall Persist Quit Resist Extrinsic (p=0.370) (p=0.147) (p=0.738) (p=0.056) (p=0.494) Intrinsic (p=0.784) (p=0.872) (p=0.317) (p=0.317) (p=0.317)Introduction Study Design Motivation Observations Future Work Conclusions 89 of 45