The Effect of an MIT Engineering Education on User-Centered Design Skills  Li Bian, Blaine Ziegler, Simon Shum
Agenda <ul><li>Introduction </li></ul><ul><li>Motivation/Significance </li></ul><ul><li>Methods </li></ul><ul><li>Results ...
Introduction <ul><li>What is User-Centered Design (“UCD”)? </li></ul><ul><ul><li>Accounts users’ needs, wants, limits </li...
Literature Review <ul><li>User-Centered System Design: New Perspectives on Human-Computer Interaction  (Norman, 1986) </li...
Motivation/Significance <ul><li>MIT Electrical Engineering & Computer Science (“Course 6”) education </li></ul><ul><ul><li...
Objective <ul><li>Research Question:  </li></ul><ul><ul><li>Do Course 6 students put less consideration into users’ intere...
Methods <ul><li>Overview: </li></ul><ul><ul><li>Design scenario, open-ended questions </li></ul></ul><ul><ul><li>Graded su...
Methods Experimental Design Dependent Independent User-Centered  towards Question 1 User-Centered  towards Question 2 User...
Methods <ul><ul><li>Sampling </li></ul></ul><ul><ul><li>Student center </li></ul></ul><ul><ul><li>E-mail </li></ul></ul><u...
Design Scenario <ul><li>Ed Smith is a retired auto mechanic.  He’s in his 70s now, and lives alone in a small, run-down ap...
Questions <ul><li>1. Describe how you would begin the process of  designing a solution to the problem presented. </li></ul...
Grading Model <ul><li>1  – I would explore the various technologies relevant to the field, like network protocols and data...
Grading Examples <ul><li>Question 2:  After coming up with your first working prototype, what steps do you take to make th...
Grading Examples <ul><li>Question 1:  Describe how you would begin the process of designing a solution to the problem pres...
Grading Examples <ul><li>Question 3:  After you have completed your design and it has been implemented, what do you look f...
Grading Examples <ul><li>Question 1:  Describe how you would begin the process of designing a solution to the problem pres...
Results Course 6  (n = 22) Non-Course 6  (n = 27) t-stat p-value Q1 ( 2.47 , 1.18) ( 3.05 , 1.30) 1.619 0.056 Q2 ( 2.47 , ...
Results <ul><li>H 0 : Course 6 students score the same as non Course 6 students </li></ul><ul><li>H A : Course 6 students ...
Discussion <ul><ul><li>Results suggest that the tech-oriented curriculum of a Course 6 education decreases the considerati...
Discussion <ul><li>Limitations & Confounds </li></ul><ul><ul><li>External Validity - Results applied to this case study on...
Conclusion <ul><li>Importance of User-Centered Design Skills  </li></ul><ul><ul><li>Course 6 Students place less emphasis ...
Questions? Thanks
Upcoming SlideShare
Loading in …5
×

UCD Awareness Research

639 views
563 views

Published on

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
639
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
9
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

UCD Awareness Research

  1. 1. The Effect of an MIT Engineering Education on User-Centered Design Skills Li Bian, Blaine Ziegler, Simon Shum
  2. 2. Agenda <ul><li>Introduction </li></ul><ul><li>Motivation/Significance </li></ul><ul><li>Methods </li></ul><ul><li>Results </li></ul><ul><li>Discussion </li></ul><ul><li>Conclusion </li></ul>
  3. 3. Introduction <ul><li>What is User-Centered Design (“UCD”)? </li></ul><ul><ul><li>Accounts users’ needs, wants, limits </li></ul></ul><ul><ul><li>Multi-stage process with user-experimentation </li></ul></ul>
  4. 4. Literature Review <ul><li>User-Centered System Design: New Perspectives on Human-Computer Interaction (Norman, 1986) </li></ul><ul><ul><li>“ stressed the need to fully explore the needs and desires of the users and the intended uses of the product.” </li></ul></ul><ul><li>User-Interface Design for Motorola Vehicle-Navigation System: A Case Study of Product Development (Marcus, 1986) </li></ul><ul><ul><li>simplified navigation systems for safety purposes </li></ul></ul><ul><li>Education of Interface Design – an Interdisciplinary Approach (Joshi, 2004) </li></ul><ul><ul><li>In 2004 - “The HCI practice within these organizations [IT companies] is rapidly becoming mainstream.” </li></ul></ul><ul><ul><li>By 2009 -“The top quarter of Indian IT companies will be well integrated into HCI Design.” </li></ul></ul>
  5. 5. Motivation/Significance <ul><li>MIT Electrical Engineering & Computer Science (“Course 6”) education </li></ul><ul><ul><li>lacks courses that emphasize User-Centered Design </li></ul></ul><ul><li>Help Course 6 students </li></ul><ul><ul><li>recognize shortcomings </li></ul></ul><ul><ul><li>improve overall design philosophy </li></ul></ul>
  6. 6. Objective <ul><li>Research Question: </li></ul><ul><ul><li>Do Course 6 students put less consideration into users’ interests than do other MIT students? </li></ul></ul><ul><li>Hypothesis: </li></ul><ul><ul><li>Experiencing the largely tech-oriented curriculum of a Course 6 education decreases the consideration that designers put into users’ needs </li></ul></ul>
  7. 7. Methods <ul><li>Overview: </li></ul><ul><ul><li>Design scenario, open-ended questions </li></ul></ul><ul><ul><li>Graded subjectively on 1-5 scale </li></ul></ul><ul><ul><li>Results compared for MIT Course 6 students vs. non-MIT Course 6 students </li></ul></ul>
  8. 8. Methods Experimental Design Dependent Independent User-Centered towards Question 1 User-Centered towards Question 2 User-Centered towards Question 3 Average User-Centered towards all questions Course 6 Non Course 6
  9. 9. Methods <ul><ul><li>Sampling </li></ul></ul><ul><ul><li>Student center </li></ul></ul><ul><ul><li>E-mail </li></ul></ul><ul><li>Internal Validity </li></ul><ul><ul><li>responses graded blind </li></ul></ul><ul><ul><li>responses graded independently by 3 team members, averaged </li></ul></ul>
  10. 10. Design Scenario <ul><li>Ed Smith is a retired auto mechanic. He’s in his 70s now, and lives alone in a small, run-down apartment across the alley from the garage he used to work at. He has a son from an ex-wife, but he hasn’t spoken to either in quite some time. It’s not that he’s angry at them, he just hasn’t gotten around to it, and his friends are worried that he’s becoming too shut off from the world. They want him to get email so that he can easily communicate and share photos with other people. </li></ul><ul><li>He has a computer that a friend’s children set up for him, but he hasn’t even turned it on in weeks. And there’s no way he’ll ever be able to use something as complicated as Microsoft Outlook. His friends want you to design a brand new, fully-featured email client to help him stay in touch. </li></ul>
  11. 11. Questions <ul><li>1. Describe how you would begin the process of designing a solution to the problem presented. </li></ul><ul><li>2. After coming up with your first working prototype, what steps do you take to make the second iteration even better? </li></ul><ul><li>3. After you have completed your design and it has been implemented, what do you look for in deciding if your design was a success? </li></ul>
  12. 12. Grading Model <ul><li>1 – I would explore the various technologies relevant to the field, like network protocols and database storage, so that I could find creative ways to improve on them. </li></ul><ul><li>3 – I would use a commercial email client as a template, and try to trim away unnecessary features. </li></ul><ul><li>5 – I would interview Ed and even shadow him for a few hours if he allows it, so that I could learn more about his level of computer expertise, his budgetary concerns, and what he wants to gain from using email. </li></ul>
  13. 13. Grading Examples <ul><li>Question 2: After coming up with your first working prototype, what steps do you take to make the second iteration even better? </li></ul><ul><li>A: Test it until it crashes, fix the bugs, run it again. Iteratively search for and correct mistakes, while checking if the spec is being met. </li></ul><ul><li>1! </li></ul>
  14. 14. Grading Examples <ul><li>Question 1: Describe how you would begin the process of designing a solution to the problem presented. </li></ul><ul><li>A: Find out why he doesn’t keep in touch. Some other emotional reason? Find out how comfortable he is with technology, i.e., computers, typing, mice, etc. </li></ul><ul><li>5! </li></ul>
  15. 15. Grading Examples <ul><li>Question 3: After you have completed your design and it has been implemented, what do you look for in deciding if your design was a success? </li></ul><ul><li>A: If the guy remarries his ex-wife. </li></ul><ul><li>5! </li></ul>
  16. 16. Grading Examples <ul><li>Question 1: Describe how you would begin the process of designing a solution to the problem presented. </li></ul><ul><li>A: Make the application very user friendly, and incorporate features that may make using it more enjoyable for him. </li></ul><ul><li>1,2,3 = 2 </li></ul>
  17. 17. Results Course 6 (n = 22) Non-Course 6 (n = 27) t-stat p-value Q1 ( 2.47 , 1.18) ( 3.05 , 1.30) 1.619 0.056 Q2 ( 2.47 , 1.05) ( 3.44 , 1.08) 3.182 0.00130 Q3 ( 2.56 , 0.95) ( 3.23 , 1.14) 2.229 0.015 Overall ( 2.50 , 0.82) ( 3.24 , 1.00) 2.811 0.00358
  18. 18. Results <ul><li>H 0 : Course 6 students score the same as non Course 6 students </li></ul><ul><li>H A : Course 6 students score lower than non Course 6 students </li></ul><ul><li>Reject H 0 for questions 2, 3, as well as overall, at the 0.05 significance level </li></ul><ul><li>Cannot reject H 0 for question 1 at the 0.05 significance level </li></ul>
  19. 19. Discussion <ul><ul><li>Results suggest that the tech-oriented curriculum of a Course 6 education decreases the consideration that designers put into users’ needs </li></ul></ul><ul><ul><li>Interesting/Surprising Results: </li></ul></ul><ul><ul><ul><li>Higher variability in Non-Course 6 Students </li></ul></ul></ul><ul><ul><ul><li>Question 1 - insignificant at the .05 level: </li></ul></ul></ul><ul><ul><ul><ul><li>Course 6 students might be good at the initial stages, but fail to recognize users’ needs in later stages </li></ul></ul></ul></ul><ul><ul><ul><li>Outliers in the Course 6 student data: Experience in industry? Double major? </li></ul></ul></ul>
  20. 20. Discussion <ul><li>Limitations & Confounds </li></ul><ul><ul><li>External Validity - Results applied to this case study only </li></ul></ul><ul><ul><li>Sampling Bias/Data Collection </li></ul></ul><ul><ul><ul><li>Different forms of data collection </li></ul></ul></ul><ul><ul><ul><li>Getting people to participate - People who fill out surveys are more “people oriented” </li></ul></ul></ul><ul><ul><li>Grading Methodology </li></ul></ul><ul><ul><li>Survey Bias </li></ul></ul><ul><ul><li>Gender representation in Course 6 </li></ul></ul><ul><ul><ul><li>males possibly less user-centered than females (higher % of males in Course 6) </li></ul></ul></ul>
  21. 21. Conclusion <ul><li>Importance of User-Centered Design Skills </li></ul><ul><ul><li>Course 6 Students place less emphasis user’s interests than do other MIT students in a design </li></ul></ul><ul><li>Further Steps & Future Research: </li></ul><ul><ul><li>add more factors: </li></ul></ul><ul><ul><ul><li>gender, school, age group </li></ul></ul></ul><ul><ul><li>diversify case study, grading criteria, and course distribution </li></ul></ul>
  22. 22. Questions? Thanks

×