This document provides a summary of lessons learned from attending the 2004 UPA Conference in Minneapolis. It outlines 8 key lessons for improving usability at Freddie Mac based on presentations and discussions at the conference: 1) If user tests show a 50-50 split on a design choice, let users decide; 2) Usability labs don't need to be sterile; 3) Join the UTEST community to get input from other practitioners; 4) Speed up design and testing but recognize limitations; 5) Prepare for differences between intranets and enterprise portals; 6) Use Morae software to review user tests; 7) Get useful card sorting results from a small number of users; 8) Create successful online events that feel live through

1. 8 Usability Lessons for Freddie Mac
from the 2004 UPA Conference
Prepared for the Usability Community of Practice
By Mark D. Alves
August 11, 2004

2. 2
Overview of UPA Conference
 The 13th annual Usability Professionals’ Association conference,
Connecting Communities, attracted 470 attendees to Minneapolis,
Minnesota from June 7-11, 2004.
 Some events were major presentations for all attendees while at other
times smaller, concurrent sessions were offered. Usability vendors
were onsite as well.
 This year’s materials are at:
http://upassoc.org/conferences_and_events/upa_conference/2004/
 Next year's conference is in Montreal, Canada from June 27 - July 2.
 Some of the most interesting information comes out of informal
discussions in between sessions or after hours.
 Freddie Mac’s usability experience and challenges are not unique.
Some companies are ahead of us, some behind.

3. 3
Examples of Common Items That Could Use
Improved Usability
Minneapolis Sculpture Garden

4. 4
Examples of Common Items That Could Use
Improved Usability...
Mini-golf hole designed by an artist.

5. 5
Examples of Common Items That Could Use
Improved Usability...
Mini-golf hole designed by an artist.

6. 6
8 Usability Lessons for Freddie Mac
1. If your users are split 50-50 on a design choice, let them decide.
2. Your lab doesn’t have to be sterile.
3. Join UTEST now and participate later in the community.
4. Speed up design and testing, but recognize there’ll be false positives
and missed errors.
5. Prepare for the difference between an intranet and an enterprise
portal.
6. Use TechSmith’s Morae for reviewing test participants’ actions and
results, http://techsmith.com/products/morae/.
7. Get good card sorting results with helpful tools and a handful of
testers.
8. Create a successful online “live” event, even when most of it is pre-
produced.

7. 7
Lessons From Other Participants
Lesson 1: If your users are split 50-50, let them decide.
 When it comes to video games, Microsoft employs usability principles while
striving for “appropriate challenges” instead of minimal challenges. Their testing
showed that gamers were split 50-50 on certain games as to whether pushing
the joystick up should make a character’s gun go down and away (like a pilot) or
move up (like a map). Their solution was to force the user to pick a preference
before the game can begin.
 For Freddie Mac websites: offer information both ways. Perhaps your forms
are listed numerically and by category; or your products are listed alphabetically
and by customer need.
Lesson 2: Your lab doesn’t have to be sterile.
 Bose makes high-end audio speakers so their usability lab is set up like a living
room. Make your test environment similar to what your users’ areas are like if
you can’t actually test them in their natural settings.
 For Homefront: perform some tests at users’ desks.
Lesson 3: Join UTEST and later participate in the community.
 UTEST is a private, non-archived listserv for discussing budding usability ideas
and solving usability problems. To request membership, contact Dr. Tharon
Howard at tharon@clemson.edu after reading about the special policies and
culture of the community: http://people.clemson.edu/~tharon/utest/policies.html

8. 8
Speed Up Design and Testing
Lesson 4: Speed up design and testing, but recognize there’ll be false
positives and missed errors.
 Inspection evaluations are efficient, but not effective
– Miss 25% of problems
– Find more non-problems than problems (for every 2 problems found you’ll
have 1 that you missed)
 User testing is effective, but not efficient
 Mind Design Systems, http://mindd.com, is working on software to improve the
efficiency of human testing, but this won’t be commercially available this year.
One way the software improves efficiency is by stopping a task after a certain
number of clicks; you don’t waste time if a user gets really lost.
 Ask true/false questions if testing comprehension. This cuts down on the user
agonizing over the answer.
 For designing intranets, relying on content analysis can speed up the process.
– Use paired-comparison tests to see preference (if any) between content
– User selects preference or indicates none
– User reviews the hierarchy of preferences that results
– Don’t focus on employee task analysis, but on what content the staff needs.

9. 9
Add an Enterprise Portal to Your Intranet
with Care
Lesson 5: Prepare for the difference between an intranet and an
enterprise portal.
 Boeing is working on adding customizable enterprise portals to its 8
million page intranet.
– Keep user feedback forms simple by skipping the user-selected
categories. (25% picked the wrong category, 25% picked “other” so
50% of the data was useless.
– New users typically “test” a portal by searching for applications that
they know already exist on the old site: make sure the search
engine picks them up.
– Users are receptive to “my pages” and related customization.
– Users are resistant to “subscribing” to a community rather than just
looking at a website: “I don’t want to join a club.”

10. 10
Use TechSmith’s Morae for Reviewing Test
Particpants’ Actions and Results
Lesson 6: Use TechSmith’s Morae for reviewing test participants’
actions and results.
 TechSmith’s Morae software, http://techsmith.com/products/morae/,
captures test participants’ desktop behaviors and can integrate a log of
activity with any sound and video you record. Captured behaviors
include:
– Mouse clicks (left and right)
– New windows opened
– Entered text
– Accessed menus
– Any markers you insert
 You can capture all activity or specify when the recording starts, such
as upon clicking on a certain link or opening a particular application.
 Remote viewer option lets you see the subject’s desktop over a LAN
connection so you can be in another room. No one-way mirrors
required.

11. 11
Card Sorting: Use It Quickly and Easily
Lesson 7: Get good card sorting results with helpful tools and a
handful of testers.
 How Many Users Are Enough for a Card-Sorting Exercise?
Poster presented by Larry Wood of BYU, who developed Web Sort
http://www.websort.net
 Correlations:
– 5 users=0.75
– 15 = 0.90
– 20 = 0.93
– 30 = 0.95
– 60 = 0.98

12. 12
Card Sorting: Use It Quickly and Easily (cont.)
 Card Sorting: How Many Users to Test?
Jakob Nielsen co-opts a UPA poster
http://www.useit.com/alertbox/20040719.html
 A 13-step Approach to Card Sorting
by Gerry McGovern
http://www.gerrymcgovern.com/nt/2002/nt_2002_09_23_card_sorting.htm
 Card sorting tools and technology
– Analyzing Card Sort Results with a Spreadsheet Template
by Joe Lamantia
http://www.boxesandarrows.com/archives/analyzing_card_sort_results_with_a_spreadsh
php or http://tinyurl.com/lcuj
– uzCardSort for Uzilla http://uzilla.mozdev.org/cardsort.html
– IBM’s EZSort http://www-3.ibm.com/ibm/easy/eou_ext.nsf/Publish/410
– CardZort by Jorge A. Toro
http://condor.depaul.edu/%7Ejtoro/cardzort/cardzort.htm
– NIST WebCAT web category analysis tool
http://zing.ncsl.nist.gov/WebTools/WebCAT/overview.html
– “Classified” from Information & Design (Australia)
http://www.infodesign.com.au/usabilityresources/classified/

13. 13
Sponsor a Live (Enough) Event Online
Lesson 8: Is it Live or is it Freddie Mac?
 “We now bring in enough people to fill a stadium for the cost of a barn
dance.” Scott Anderson, director of e-marketing, Sun Microsystem.
Sun’s online “live” events replace the hodgepodge of tactics they
formerly employed for their quarterly R&D announcements.
 Take the best of online news sites, websites for temporary museum
exhibits, ESPN-type sports updates, live chats with
celebrities/newsmakers, and online learning sites.
 Combine them to create the urgency and excitement of a “live” event,
with the certainty of pre-recorded commercials and the shelf-life of
important product support materials.
 It’s cheaper than doing the event in person; it’s more interesting than a
regular website; and the results are measurable.

14. 14
Sponsor a Live (Enough) Event Online cont.
 Use the Tivo philosophy experience: what’s happening now is central,
the past is accessible and the future is only glimpsed at.
 The design implications?
– The past: Content that has already been released is available for
viewing at any time.
– The present: Content is being released linearly.
– The future: Promote future content and encourage users to come
back; whether the content is prepared or not, it is not released until
the pre-determined time.
 Organize prepared content around “live” events.
 Break a usability code in exchange for business results: everything is
not available all of the time.

15. 15
Sponsor a Live (Enough) Event Online cont.
 Implications of using video:
– Promoting “live” aspect draws crowds
– Users will view content for a longer period of time when they are
watching a live event.
– Focus on quality, not quantity.
– People want accompanying text to be scannable, not full
transcripts.
 Have a backup plan
– Users who opt not to install plugins or players
– Prepare your help desk for extra volume and more technical
questions
– Allow extra time for QA to test multiple browsers and versions of
the same browser

16. 16
Sponsor a Live (Enough) Event Online cont.
 Examples of tightly integrated video and website.

17. 17
Sponsor a Live (Enough) Event Online cont.
Post-event, you default to playing the first video (not “now playing”).

18. 18
Sponsor a Live (Enough) Event Online cont.
 Options for applying this at Freddie Mac externally.
– In conjunction with our own customer conferences or analyst
meetings
– Product announcements
– As part of regularly scheduled mission updates
– In support of a job fair
– Integrated with a national event, like homeownership month.
 Internal options
– Division events where staff is distributed across multiple locations
– Diversity events to maximize employee participation
– “All”-employee events where not everyone can attend

19. 19
Websites Are Never Finished, But Keep Evolving
Good design always happens.

20. 20
Websites Are Never Finished, But Keep Evolving
Good design always happens...just like
the patient always stops bleeding.

21. 21
Available Documents from UPA Conference
Wednesday, June 9, 2004
Making an Enterprise Portal Useful for Employees
Ken Becker, The Boeing Company
Voice Recording Tips for the Non-Audiophile
Timothy Keirnan, Tec-Ed, Inc.
The Blind Leading the Blind: Theorizing a Web for the Visually Impaired
Jessica D. Moore, Joseph Matthews, AARP Services
Panel: HCI and Human Factors Master's Degree Programs
Kirsten Robinson, Bentley College; Jessica Webster, Tufts University; Amanda Nance, Georgia Tech; Dana Gelman,
Carnegie Mellon University; Rebecca Tremaglio, University of Michigan
Expediting the Usability Testing Process
Robert Bailey, Computer Psychology; Kent Bailey, Mind Design Systems
How to Predict the Future: Becoming an Agent for Change
Susan Fowler, FAST Consulting
Architecting Time: Designing Online Events
Katrina Friedman, Hot Studio
Site-Level Webapp Navigation Using Breadcrumb and Sibling Menus
James Snell, Concur Technologies
Making a Better Web Form
Caroline Jarrett, Effortmark, LTD; Christopher Minott, LoanBright.com
When Your Group Can't Do It All: Investing UCD Resources Wisely
Lisa Battle, Lockheed Martin
User Experience Issues of IP Telephony: Not Your Plain Old Telephone Anymore
Yihsiu Chen, PhD, AT&T Labs
An Alternative Method for Intranet Design Based on Content Analysis
Regis Magyar, Brian M. Anderson, Nancy Babiarz, James Marion, Panasonic Wireless
Accessibility at an Enterprise Software Company: From Guidelines to Accessible Interfaces
George Hackman, Oracle
How to Ensure Usability Offshore When Doing UI Development
Liam Friedland, Westbridge Technology

22. 22
Available Documents from UPA Conference
Thursday, June 10, 2004
The Value of Student Internships in Usability: Connecting Students with Professional Usability Communities
Lee-Ann Kastman Breuch, Alice de la Cova, University of Minnesota
Task Analysis Tune-Up Paul McInerny, IBM; Interaction Guidelines for Japanese Web Forms Manabu Ueno, Sociomedia
Meeting the Challenges Posed by International Usability Testing: Two Case Studies
Jenny Blackburn, Julianne Bryant, Getty Images
Demographic Differences in Preferred Web Site Content Ann Chadwick-Dias, Donna Tedesco, Thomas Tullis, Fidelity Inv.
A Structured Template for Writing Scenarios Paul McInerny, IBM
Usability Guidelines for 3G Mobile Internet: A Korean Case Se-Hoon Kim, Young-Hee Kim, Mi-Jin Kim, Hey-Jin Chung...
Too Much Usability? Steps and Missteps Ron Vutpakdi, Landmark Graphics Corporation
Achieving Experience Equity and Universal Usable Access: Research Results and an Innovative Future
Mary Theofanos, National Cancer Institute; Ginny Redish, Redish and Associates
Balancing Rigor, Adaptation and Mentoring: Field Study with Customers to Initiate a Usability Program
Laurie Kantner, Tec-Ed, Inc.; Rebecca Shaufl, Manatron, Inc.
Going in Blind Doesn't Help: Cues for Navigation Sarah J. Swierenga, PhD, Laura G. Militello, Megan E. Gorman...
Advanced Topic Seminar: Progress Indication and Responsiveness: User Needs, Design, and Implementation
Jeff Johnson, UI Wizards; Paul McInerney, IBM
Usability of Health Websites: What Have We Learned? American Institutes for Research
Website Usability Questionnaires Thomas Tullis, Fidelity Investments; Jacqueline N. Stetson, Fidelity Investments
How Standards Could Benefit Your Business Nigel Bevan, Serco Usability Services
Advanced Topic Seminar: Using Sounds & Graphics in Application Interfaces Alice Preston; Susan Fowler, FAST Consulting
Panel: Building an Online "Community“ Tharon Howard, Clemson University; Caroline Jarrett, Effortmark...
The Usability Imperative Inherent in the Semantic Web Duane Degler, IP Gems; Renee Lewis, Pensare Group
Evaluating the Usability of Mobile Handheld Devices Regis Magyar, Brian M. Anderson, Nancy A. Babiarz
Achieving Global Competitive Advantage: The Business Benefits of Designing for Usability
Steve Jacobs, Information Technology Technical Assistance & Training Center
Peer Reviewed Paper: Integrating Accessibility into User-Centered Design Process: A Case Study
Kay Lewis, Accessiblity Institute, University of Texas at Austin
Peer Reviewed Paper: Freehand Interactive Design Offline (FIDO): A New Method for Participatory Design
Donna Tedesco, Ann Chadwick-Dias, Thomas Tullis, Fidelity Investments

23. 23
Available Documents from UPA Conference
Friday, June 11, 2004
Invited Speaker: Landscapes of Memory: History for Communities to Live By
Kurt F. Anschuetz, Rio Grande Foundation for Communities and Cultural Landscapes
Design Patterns and Guidelines for Usable and Accessible Web Applications
David Hoffman, Lisa Battle, Lockheed Martin
Justifying Funding and Space for a Usability Lab
Alice de la Cova, University of Minnesota
Peer Reviewed Paper: Participatory Techniques for the Design of a New Input Device
Evelyne Millien, Camille Roux, Centre MultiCom Laboratoire CLIPS-IMAG
We Bombed in New Haven: Lessons from the Theater
Whitney Quesenbery, Whitney Interactive Design, LLC
Peer Reviewed Paper: Participatory Design in Cognitive Walkthroughs
Susan Mings, PhD, Scott Ottaway, PhD, Microsoft Corp.
Posters
ViSA: Video Segmentation and Annotation Afzal Godil, National Inst. of Standards and Technology
New Techniques for Understanding Communities Erika Darling, Emily Leventhal, The MITRE Corportation
Adapting User-Centered Design to the Consumer Electronics Research and Development Process
Joonhwan Kim, SungWoo Kim, Younghwan Pan, Samsung Electronics
Eye-tracking in Usability Testing: What Questions Can It Help Us Answer?
Judith Ramey, Mary Deaton, Rich DeSantis, Elisabeth Cuddihy, Cory King, Quan Zhou, Jana Jones, University of Washington
Because Sometimes More is Less Julie Bzostek, Brett Walters, Micro Analysis and Design
Rules of the Road: A Practical Guide to International Research Kathi Kaiser, Lyman Casey, Centralis Partners, Inc.
Taskflow Charting: An Interactive Validation Method for Task Analysis Kraig Finstad, Intel Corporation
How Many Users Are Enough for a Card-Sorting Study?
Larry Wood, Brigham Young University; Thomas Tullis, Fidelity Investments
Recruit More Test Participants Faster, Cheaper and Happier by Piggybacking on Existing Customer Conferences
Mark Alves, Freddie Mac