Alarming Usability Follow Up: Usability Testing in Public June 28, 2007

Thank you to Menlo Innovations!

What is the Michigan UPA? Promote and support user-centered design and usability in products, software, and web sites for Michigan companies. Bring user experience designers, usability engineers, and human factors specialists together to network and learn. Organize relevant events for training, awareness, and networking

Promote and support user-centered design and usability in products, software, and web sites for Michigan companies.

Bring user experience designers, usability engineers, and human factors specialists together to network and learn.

Organize relevant events for training, awareness, and networking

What is the Michigan UPA? Events around the state Detroit Oakland County Ann Arbor East Lansing Grand Rapids

Events around the state

Detroit

Oakland County

Ann Arbor

East Lansing

Grand Rapids

What is the Michigan UPA? World Usability Day for Michigan (November)‏ Led by/held at MSU UAC Participation and co-sponsored by MIUPA, STC-SM, MOCHI, and several schools and departments at MSU Significant sponsorship by areas at MSU and outside companies. Internet User Experience 2007 (February)‏ A conference spotlighting methods for dramatically improving today’s web site user experience Co-sponsored by STC-SM, MOCHI, and UXNet

World Usability Day for Michigan (November)‏

Led by/held at MSU UAC

Participation and co-sponsored by MIUPA, STC-SM, MOCHI, and several schools and departments at MSU

Significant sponsorship by areas at MSU and outside companies.

Internet User Experience 2007 (February)‏

A conference spotlighting methods for dramatically improving today’s web site user experience

Co-sponsored by STC-SM, MOCHI, and UXNet

Upcoming Events July 24 in East Lansing Understanding the human/package interface Ongoing work at the school Child resistant AND elderly accessible Making warning labels noticeable Lab tour Eye tracker Label legibility measurement instrument Human/package touch interface measurement

July 24 in East Lansing

Understanding the human/package interface

Ongoing work at the school

Child resistant AND elderly accessible

Making warning labels noticeable

Lab tour

Eye tracker

Label legibility measurement instrument

Human/package touch interface measurement

More Upcoming Events August The Latest and Greatest: Recent Research for Usability and Interface Design (location TBD)‏ Dr. Susan Weinschenk, Chief of Technical Staff, Human Factors International September LIVE Web Site Evaluations: Usability, Branding, and Marketing Message (Location TBD)‏ Dave Mitropoulos-Rundus November 8 World Usability Day for Michigan Michigan State University

August

The Latest and Greatest: Recent Research for Usability and Interface Design (location TBD)‏

Dr. Susan Weinschenk, Chief of Technical Staff, Human Factors International

September

LIVE Web Site Evaluations: Usability, Branding, and Marketing Message (Location TBD)‏

Dave Mitropoulos-Rundus

November 8

World Usability Day for Michigan

Michigan State University

MI UPA’s New Officers Mike Beasley Carissa Demitris Dunrie Greiling Dave Mitropoulos-Rundus

Mike Beasley

Carissa Demitris

Dunrie Greiling

Dave Mitropoulos-Rundus

MI UPA’s New Award UPA Chapter Award for Promoting UPA Increasing its visibility Collaborating with other related chapters universities

UPA Chapter Award for

Promoting UPA

Increasing its visibility

Collaborating with

other related chapters

universities

Dave MR’s New Award Our Immediate Past President has won a Distinguished Service Award From the Usability Professionals’ Association

Another Other Announcements?

Thank You Menlo Innovations Tec-Ed Ann Arbor Hands-On Museum Phil Holmes Laurie Kantner Tim Keirnan Carol Treat Morgan Sumanth Muthyala Paul Ott John Rivard Dave Robinson Barbra Wells Lori Anschuetz Yelena Babin Tom Burton-West Jennifer Carlson Robert Gawecki Wyatt Hazlett Barb Hernandez Lauren Holmes

Menlo Innovations

Tec-Ed

Ann Arbor Hands-On Museum

Phil Holmes

Laurie Kantner

Tim Keirnan

Carol Treat Morgan

Sumanth Muthyala

Paul Ott

John Rivard

Dave Robinson

Barbra Wells

Lori Anschuetz

Yelena Babin

Tom Burton-West

Jennifer Carlson

Robert Gawecki

Wyatt Hazlett

Barb Hernandez

Lauren Holmes

Volunteering is Cool Volunteering is a great experience and a great way to meet new people (some call it “networking”). Interested? Talk to any of the officers Visit http://www.miupa.org/ (specifically, the Contact Us page)‏

Volunteering is a great experience and a great way to meet new people (some call it “networking”).

Interested?

Talk to any of the officers

Visit http://www.miupa.org/ (specifically, the Contact Us page)‏

The Ann Arbor Hands-On Museum Alarming Usability on Sunday afternoon: 5 alarm clocks 5 stop watches 1 task An army of volunteers Lots of children http://www.aahom.org/ http://www.upaboston.org/

Alarming Usability on Sunday afternoon:

5 alarm clocks

5 stop watches

1 task

An army of volunteers

Lots of children

http://www.aahom.org/

http://www.upaboston.org/

Usability Testing in Public: Why? Walk-up-and-use products Limited set of tasks No release forms

Walk-up-and-use products

Limited set of tasks

No release forms

Usability Testing in Public: Why not? Competing with: Other interesting things The participants’ agendas Harder to control the situation

Competing with:

Other interesting things

The participants’ agendas

Harder to control the situation

Testing in Public: Things to Know

Testing in Public: Things to Know Get at least one person comfortable with being a barker No time to train the user—they must know what the task entails right away Keep engaging the participants Know the demographics of the place you’ll be Some people just aren’t going to care about your test

Get at least one person comfortable with being a barker

No time to train the user—they must know what the task entails right away

Keep engaging the participants

Know the demographics of the place you’ll be

Some people just aren’t going to care about your test

Testing in Public at the AAHOM

Testing in Public at the AAHOM Staffed by volunteers (all of who are very cool)‏ Had to adapt the source material for event The people going to the museum can be very young

Staffed by volunteers (all of who are very cool)‏

Had to adapt the source material for event

The people going to the museum can be very young

The Alarm Clock Circuit A 20 minute break to amble about and examine the alarm clocks individually.

Alarm Clock #1 Average predicted difficulty: 3.13 Average actual difficulty: 4.52 Average difference in difficulty (actual – predicted): 1.39 Success rate: 12 out of 21 Average time: 118.59 seconds 20 people found it more difficult to use, 1 found it less

Average predicted difficulty: 3.13

Average actual difficulty: 4.52

Average difference in difficulty (actual – predicted): 1.39

Success rate: 12 out of 21

Average time: 118.59 seconds

20 people found it more difficult to use, 1 found it less

Alarm Clock #2 Average predicted difficulty: 1.88 Average actual difficulty: 2.10 Average difference in difficulty (actual – predicted): -.04 Success rate: 17 out of 24 Average time: 43.59 seconds 6 people found it more difficult to use, 8 found it less

Average predicted difficulty: 1.88

Average actual difficulty: 2.10

Average difference in difficulty (actual – predicted): -.04

Success rate: 17 out of 24

Average time: 43.59 seconds

6 people found it more difficult to use, 8 found it less

Alarm Clock #3 Average predicted difficulty: 3.15 Average actual difficulty: 3.41 Average difference in difficulty (actual – predicted): .26 (.03 when one outlier is removed)‏ Success rate: 10 out of 15 Average time: 86.31 seconds 7 people found it more difficult to use, 6 found it less

Average predicted difficulty: 3.15

Average actual difficulty: 3.41

Average difference in difficulty (actual – predicted): .26 (.03 when one outlier is removed)‏

Success rate: 10 out of 15

Average time: 86.31 seconds

7 people found it more difficult to use, 6 found it less

Alarm Clock #4 Average predicted difficulty: 2.80 Average actual difficulty: 3.20 Average difference in difficulty (actual – predicted): .40 Success rate: 10 out of 14 Average time: 102.20 seconds 5 people found it more difficult to use, 2 found it less

Average predicted difficulty: 2.80

Average actual difficulty: 3.20

Average difference in difficulty (actual – predicted): .40

Success rate: 10 out of 14

Average time: 102.20 seconds

5 people found it more difficult to use, 2 found it less

Alarm Clock #5 Average predicted difficulty: 2.29 Average actual difficulty: 2.13 Average difference in difficulty (actual – predicted): -.38 (-.05 if the outlier is removed)‏ Success rate: 11 out of 14 Average time: 30.62 seconds 4 people found it more difficult to use, 6 found it less

Average predicted difficulty: 2.29

Average actual difficulty: 2.13

Average difference in difficulty (actual – predicted): -.38 (-.05 if the outlier is removed)‏

Success rate: 11 out of 14

Average time: 30.62 seconds

4 people found it more difficult to use, 6 found it less

Comparing all Five Clocks

Comparing all Five Clocks

Comparing all Five Clocks

Questions, Comments, Anecdotes?