UCD Awareness Research

The Effect of an MIT Engineering Education on User-Centered Design Skills Li Bian, Blaine Ziegler, Simon Shum

Agenda
Introduction
Motivation/Significance
Methods
Results
Discussion
Conclusion

Introduction
What is User-Centered Design (“UCD”)?
Accounts users’ needs, wants, limits
Multi-stage process with user-experimentation

Literature Review
User-Centered System Design: New Perspectives on Human-Computer Interaction (Norman, 1986)
“ stressed the need to fully explore the needs and desires of the users and the intended uses of the product.”
User-Interface Design for Motorola Vehicle-Navigation System: A Case Study of Product Development (Marcus, 1986)
simplified navigation systems for safety purposes
Education of Interface Design – an Interdisciplinary Approach (Joshi, 2004)
In 2004 - “The HCI practice within these organizations [IT companies] is rapidly becoming mainstream.”
By 2009 -“The top quarter of Indian IT companies will be well integrated into HCI Design.”

Motivation/Significance
MIT Electrical Engineering & Computer Science (“Course 6”) education
lacks courses that emphasize User-Centered Design
Help Course 6 students
recognize shortcomings
improve overall design philosophy

Objective
Research Question:
Do Course 6 students put less consideration into users’ interests than do other MIT students?
Hypothesis:
Experiencing the largely tech-oriented curriculum of a Course 6 education decreases the consideration that designers put into users’ needs

Methods
Overview:
Design scenario, open-ended questions
Graded subjectively on 1-5 scale
Results compared for MIT Course 6 students vs. non-MIT Course 6 students

Methods Experimental Design Dependent Independent User-Centered towards Question 1 User-Centered towards Question 2 User-Centered towards Question 3 Average User-Centered towards all questions Course 6 Non Course 6

Methods
Sampling
Student center
E-mail
Internal Validity
responses graded blind
responses graded independently by 3 team members, averaged

Design Scenario
Ed Smith is a retired auto mechanic. He’s in his 70s now, and lives alone in a small, run-down apartment across the alley from the garage he used to work at. He has a son from an ex-wife, but he hasn’t spoken to either in quite some time. It’s not that he’s angry at them, he just hasn’t gotten around to it, and his friends are worried that he’s becoming too shut off from the world. They want him to get email so that he can easily communicate and share photos with other people.
He has a computer that a friend’s children set up for him, but he hasn’t even turned it on in weeks. And there’s no way he’ll ever be able to use something as complicated as Microsoft Outlook. His friends want you to design a brand new, fully-featured email client to help him stay in touch.

Questions
1. Describe how you would begin the process of designing a solution to the problem presented.
2. After coming up with your first working prototype, what steps do you take to make the second iteration even better?
3. After you have completed your design and it has been implemented, what do you look for in deciding if your design was a success?

Grading Model
1 – I would explore the various technologies relevant to the field, like network protocols and database storage, so that I could find creative ways to improve on them.
3 – I would use a commercial email client as a template, and try to trim away unnecessary features.
5 – I would interview Ed and even shadow him for a few hours if he allows it, so that I could learn more about his level of computer expertise, his budgetary concerns, and what he wants to gain from using email.

Grading Examples
Question 2: After coming up with your first working prototype, what steps do you take to make the second iteration even better?
A: Test it until it crashes, fix the bugs, run it again. Iteratively search for and correct mistakes, while checking if the spec is being met.
1!

Grading Examples
Question 1: Describe how you would begin the process of designing a solution to the problem presented.
A: Find out why he doesn’t keep in touch. Some other emotional reason? Find out how comfortable he is with technology, i.e., computers, typing, mice, etc.
5!

Grading Examples
Question 3: After you have completed your design and it has been implemented, what do you look for in deciding if your design was a success?
A: If the guy remarries his ex-wife.
5!

Grading Examples
Question 1: Describe how you would begin the process of designing a solution to the problem presented.
A: Make the application very user friendly, and incorporate features that may make using it more enjoyable for him.
1,2,3 = 2

Results Course 6 (n = 22) Non-Course 6 (n = 27) t-stat p-value Q1 ( 2.47 , 1.18) ( 3.05 , 1.30) 1.619 0.056 Q2 ( 2.47 , 1.05) ( 3.44 , 1.08) 3.182 0.00130 Q3 ( 2.56 , 0.95) ( 3.23 , 1.14) 2.229 0.015 Overall ( 2.50 , 0.82) ( 3.24 , 1.00) 2.811 0.00358

Results
H 0 : Course 6 students score the same as non Course 6 students
H A : Course 6 students score lower than non Course 6 students
Reject H 0 for questions 2, 3, as well as overall, at the 0.05 significance level
Cannot reject H 0 for question 1 at the 0.05 significance level

Discussion
Results suggest that the tech-oriented curriculum of a Course 6 education decreases the consideration that designers put into users’ needs
Interesting/Surprising Results:
Higher variability in Non-Course 6 Students
Question 1 - insignificant at the .05 level:
Course 6 students might be good at the initial stages, but fail to recognize users’ needs in later stages
Outliers in the Course 6 student data: Experience in industry? Double major?

Discussion
Limitations & Confounds
External Validity - Results applied to this case study only
Sampling Bias/Data Collection
Different forms of data collection
Getting people to participate - People who fill out surveys are more “people oriented”
Grading Methodology
Survey Bias
Gender representation in Course 6
males possibly less user-centered than females (higher % of males in Course 6)

Conclusion
Importance of User-Centered Design Skills
Course 6 Students place less emphasis user’s interests than do other MIT students in a design
Further Steps & Future Research:
add more factors:
gender, school, age group
diversify case study, grading criteria, and course distribution

Questions? Thanks

UCD Awareness Research

by li_chakuli on Mar 17, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

Statistics

UCD Awareness Research — Presentation Transcript