1. Bringing user-centered design to
open source software
development
Methods: Deciding What to Design 17-652/752
-- guest lecture --
Claudia Müller-Birn, CMU SCS ISR, 11-18-2009

2. Outline
๏ Motivation
๏ Usability & FOSS - a challenging relationship
๏ Usability feedback is hard
๏ HCI know-how is rare
๏ Usability is difficult to measure
๏ Case study: SONIVIS
๏ Research challenges

4. (Nielsen, 1993)
Five characteristics of usability
๏ Learnability: How easy is it for users to accomplish basic tasks the
first time they encounter the design?
๏ Efficiency: Once users have learned the design, how quickly can
they perform tasks?
๏ Memorability: When users return to the design after a period of
not using it, how easily can they re-establish proficiency?
๏ Errors: How many errors do users make, how severe are these
errors, and how easily can they recover from the errors?
๏ Satisfaction: How pleasant is it to use the design?

5. FOSS projects and usability
๏ In free/open source software (FOSS) development projects
contributors from around the globe collaborate, mostly on a
voluntary basis, to create publicly available software applications
๏ Collaboration and communication in these projects is computer-
mediated
๏ Nature of development activities in FOSS communities greatly
influences the usability of the software applications being
developed

6. FOSS projects and usability (cont.)
๏ Work is organized around implementing discrete functions and
features rather than creating a comprehensive design (Nichols &
Twidale 2003)
๏ More and more users without deep technical knowledge, but with
higher quality expectations (Hedberg et al. 2007)
๏ Practice of “release early, release often” can cause poor design
๏ User needs are typically not considered during the design and
development process
๏ In terms of usability, there is a clear divide between the small OSS
projects and large project with company support (Benson 2004 et
al.)

7. Usability feedback is hard
๏ Text-centric tools hinder discussions about interaction design
(Twidale & Nichols 2005)
๏ Difficulties in reporting usability problems (Nichols & Twidale
2006)
๏ “Closed” discussions about usability issues in developer mailing
lists or IRC

8. (Nichols & Twidale 2006)
Reporting usability problems
๏ Bug tracking systems are often complex with a less intuitive
interface
๏ Difficult to handle for non-technical users
๏ Even screen shots might be insufficient to describe the usability
problem
๏ Sometimes problems occur after a sequence of activities
๏ Possible solutions can be better explained by using mock-ups or
simple sketches

9. One solution: Hendrix
๏ Feedback-collecting mechanism for mozilla.org
๏ Lower the barrier to giving feedback
๏ Designed for quick comments

10. Another solution: Design-by-Blog
Wordle from http://en.wikipedia.org/wiki/Blog

11. (Nichols & Twidale 2006)
Design-by-Blog
๏ Distributed design approach New Alarm Dialog
๏ Blogs are supplements to Worked on creating a mock-up for a
re-design of the alarm notification
Bugzilla-like systems dialog in Evolution today, and after
a few revisions, Ben and I agreed on
the image below. I'm working on the
๏ Blog entry contains code to get this implemented and
into CVS, so it should be popping up
on your desktop soon.
- Problem description
- Design goals
- Contraints or design criteria
- Screenshot
๏ Example: Groupware Client
Evolution

12. (Nichols & Twidale 2006)
Design-by-Blog (cont.)
...thank god New Alarm Dialog Again
someone is finally
doing something about After my previous blog entry about the
the alarm dialog Alarm dialog mock-up, a lot of feedback was
had, and I went off to implement a version.
What I currently have implemented is in the
screenshot below. If anyone knows a
What happens when I standard localizable way to put the time/
have more than one date information into a single line,
reminder? Will I have please, e- mail me. This version has the
multiple dialogs ...? times as currently displayed by Evolution,
as well as the appointment summary in bold,
instead of just an "Appointment" label, and
single-click snoozing.
I still feel that the
basic Gnome alarm+calendar
functionality shouldn't
depend on a whole email
app...

13. (Nichols & Twidale 2006)
Design-by-Blog (cont.)
New Alarm Dialog in CVS
I don't seem to
understand when the Finally, the new alarm dialog is in CVS. A
alarm is set to go screenshot of what was committed is below.
off The icon was moved to the left side, and
the secondary text/widgets were re-aligned
to match the HIG [Human Interface
Guideline] 2.0 Alert specs more precisely.
This cleans up the code quite a bit and
In Gnome, gets rid of the HTML widget that we were
icons usually go on using before.
the left side of
alert windows...
I do agree with the
comment above though that
said the title and time
should be larger

14. (Nichols & Twidale 2006)
Outcome of Design-by-Blog
๏ Incremental redesign of something that has already been
developed
๏ No formal requirements, scenarios or anything similar
๏ Design is more public and accessible
๏ No registration process required
๏ Can be seen as HCI variant of code inspection
๏ Difficulties for users to post graphics as well
๏ Complexity increases with the increasing number of posts

15. HCI know-how is rare
๏ Developers have a software development
background, and they are highly technically
focused (Benson 2004 et al.)
๏ HCI specialists are seldom involved and
efforts to consider usability processes are
only rarely undertaken (Nichols & Twidale
2006)
๏ Usability specialists are employed by free
software vendors such as Sun and skilled
volunteer designers are even harder to
find (Thomas 2008), therefore supported
projects adopting more likely techniques
from HCI mainstream (Nichols & Twidale
2006)
http://www.flickr.com/photos/tobysimkin/
http://www.flickr.com/photos/as737700/

16. The GNOME usability project
๏ Since August 2000 Sun has
contributed usability resources
to GNOME
๏ Usability study of GNOME 1.2
was first major input
๏ GNOME usability project
(GUP) was formed
๏ Sun’s staff responsible for
GNOME human interface
guidelines
http://live.gnome.org/UsabilityProject

17. GNOME human interface guidelines
๏ Help to design and write
applications that are easy to
use and consistent with the
GNOME desktop
๏ Working with maintainers
to find existing interaction
problems through user
testing and surveys
๏ Visual design and
interaction engineering of
new desktop components
http://library.gnome.org/devel/hig-book/stable/index.html.en

18. (Benson 2004 et al.)
Challenges of GNOME Usability Project
๏ Usability mailing lists and IRC channels are available but there are
few useful discussions
๏ GUP is less as a usability “consultancy” for developers and more
as a group of individuals who comment on usability bugs
๏ Usability assistance is usually only sought after interfaces have
been “designed”
๏ There is no shared or documented vision of GNOME’S target
audience
๏ Some developers feel that “Sun will fix that, so we don’t need to
bother”

19. KDE usability project
๏ Educating developers about
usability practices
๏ Conducting usability activities
to help gather feedback from
users
๏ Working with projects and help
design usable user interfaces
๏ 2005-07 over 20 reports on
usability
๏ Defining human interface
guidelines
๏ Providing user research profiles
http://techbase.kde.org/Projects/Usability

20. KDE user research profile
๏ Purpose
- Provide a place to document an application's user research information
for reference during development
- Top level items are information everyone in the project should be aware
of
๏ Top level items
- Who is the application for?
- What kinds of tasks will they complete?
- What are the environment conditions and requirements?
๏ Based on Persona concept

21. What is a Persona?
๏ User model that formalizes the design of software for specific
archetypes of individuals (Cooper 1999)
๏ All necessary data are derived from interviews and observations
of users and potential users
๏ Possible process (Cooper at al. 2007)
1. Identify behavior patterns
2. Derive goals that are directly related to the software being designed
3. Verify the completeness and the redundancies in personas
4. Prioritizing all personas in order to determine most important
persona

22. What does a Persona look like?
๏ Persona typically includes a fictional name and characteristics that
are consistent with one of the main identified user groups
๏ Example:
http://www.usability.gov/analyze/personas.html

23. Usability is difficult to measure
๏ It is harder to measure whether the software is useful, how
responsive it feels, whether it behaves as people expect then
measures such as technical correctness (Thomas 2008)
๏ To measure human-related qualities in user tests is difficult because
the community is distributed
➡ Weak incentives for usability (Thomas 2008)

24. Another way to evaluate usability: ingimp
๏ Invented by HCI research group at the University of Waterloo
๏ Standalone version of the GNU Image Manipulation Program
(GIMP) that collects real-time usability data
๏ Three types of information: command use, document
characteristics, and interaction data
๏ Possible questions:
- When it is used during the day and how frequently?
- What commands people use and which commands are
most frequently undone?
http://www.ingimp.org/

25. Case study SONIVIS:
Bringing user-centered design to
FOSS development

26. SONIVIS - small-sized FOSS project
๏ Development “starts by scratching a developer’s personal itch”
(Raymond 2001)
๏ Need for a tool to analyze
and visualize virtual
communities in Wikis
๏ Initiated in December 2005
as small university
project with three persons;
today the team is
comprised of nine volunteers
๏ First free software release May 2008
๏ Second release soon after tremendous re-engineering of the back-
end
http://sonivis.org/

27. Basic Idea

28. Basic idea: collaboration network
Name: Bryan Derk
First Change: 2004-12-02
Created Articles: 23
Article Changed: 3,565
Amount contribution: 45,923
Degree: 173
Degree Centrality: 0.96

29. Basic idea: Wiki-link network
Article Name: World Wide Web
Created: 2001-12-07
Number of Different Authors: 1,450
Number of Changes: 11,430
Size: 45.345 characters
Number of Wiki Links: 15
Number of Internet Links: 23
Page Rank: 0.8

30. From the first idea to the first release
๏ Focus was on the development of application functionality
๏ Extending Eclipse RCP by using existing functions
๏ Integrating available FLOSS software
- GNU R (statistics software) for network analysis and text mining
measures
- Prefuse for network visualization
๏ Interface was divided into two viewing modes
- Analysis Perspective for overview and detailed automatic analysis with
statistics techniques
- Manipulation Perspective for visual data exploration
http://www.eclipse.org/
http://www.r-project.org/
http://prefuse.org/

31. Community infrastructure
๏ Website: information medium
๏ Forum: user questions
๏ Wiki: project dokumentation and manuals
๏ Different mailing lists
- Developer list (coordination of development work)
- Announce list (general information)
๏ Skype: weekly meetings
๏ Codebeamer: bug- and issue tracking system
๏ SVN: source code management system
๏ Sourceforge: raise visibility

32. Number of external users
http://www.flickr.com/photos/sreejithk2000/

33. GUI: Analysis Perspective

34. GUI: Analysis Perspective

35. GUI: Analysis Perspective

36. GUI: Manipulation Perspective

37. Hmm...what can I do?

38. (Tidwell 2005)
Interaction design patterns
๏ Originated as an
architectural concept by
Christopher Alexander
๏ Define optimal solutions to
common usability problems
in a specific context
๏ Various libraries available
๏ General Structure
- What...
- Use when...
- Why...
- How...
http://designinginterfaces.com/

39. Example: Card stack
๏ What: Put sections of content onto separate panels or "cards"
๏ Use when: There's too much material on the page
๏ Why: Content is now understandable at a glance; most common
form of Card Stack are very familiar to users
๏ How: Split up the content into coherent chunks, and give them
short, memorable titles (one or two words, if possible)

40. Reviewed patterns
Category Design pattern Application
Center Stage X
Interface Layout Card Stack
Movable panels X
Prominent “done” button
Actions and commands Progress indicator
Cancelability
Sortable table
Showing complex data
Jump to item X
Input hints
Forms and controls
Autocompletion

41. Hmmm...what else?

42. Paper prototyping for improving the interface
๏ Method to collect ideas, sketch, design, test and communicate
๏ Definition of relevant tasks
๏ Tests are carried out with real users = SONIVIS developers
๏ Interaction is based on paper and click is similar to real world
๏ Persons “plays” computer

43. Result of paper prototyping
This is the first
time I’ve seen these
functions. Wow!
Ah, now I
understand how
SONIVIS works...

44. Sketches of new graphical user interface
SONIVIS
File Data Mining Search Window Help
Interaction Nework Context Network Global Settings Display Settings Network Node
Filter Export
Metric Value
Betweenness Centrality 0,2542
Closeness Centrality 0,2364
Degree Centrality 0,0694
Average Degree 3
Density 0,6984
Node Count 123
Edge Count 1234
2001 2002 2003 Start Date 02.03.2002
End Date 15.06.2002

45. New wizards
Global Settings
Node Filter
Please select the namespace you want to visualize.
Edge Filter
Namespace Main
Metrics
Time Settings Please select the category/tags you want
to visualize.
Category/
Tags
Please select the articles or ressources you want
to visualize.
Elements
Cancel Done
Network Preferences
Visualization Algorithm Please choose a metric and adapt the size and
the mapping scale of the nodes if requried.
Node Visualization
Node Size
Edge Visualization
Visual Filtering
Mapping Scale
Please choose a metric and adapt the color and
the mapping scale of the nodes if requried.
Node Color
Mapping Scale
Done Cancel

46. Defined key SONIVIS Persona
Richard M. Watkins
27 years
PhD student at the Department of Computer Science at a
middle-sized university
Research in Social Computing, especially evolution of
virtual communities
BA Social Science, M.S. in Computer Science
He is in the middle of his Phd thesis, has already defined his main area of interest
and his general research questions. Richard has different data sets from different
virtual communities. He is mainly interested to explore these datasets in terms of
their existing network structure and topics. For this he is looking for a tool that
should allow him to import his data sets easily but at the same time that allows him to
extend, if necessary, existing measures and visualizations without a huge programming
effort.
Key attribute
- Has limited programming experiences, need for "ready-to-go" tool
- Has no operating system preferences
- Appreciates using FLOSS rather then closed software
- Wants high level of interactivity

47. Outcome...

48. Recent graphical user interface
Realized design patterns:
½

49. Realized “half pattern”: Sortable Table

50. Defined key SONIVIS Persona
Richard M. Watkins
27 years
PhD student at the Department of Computer Science at a
middle-sized university
Research in Social Computing, especially evolution of
virtual communities
Responses of developers: Science
BA Social Science, M.S. in Computer
0
He is in the middle of his Phd thesis, has already defined his main area of interest
and his general research questions. Richard has different data sets from different
virtual communities. He is mainly interested to explore these datasets in terms of
their existing network structure and topics. For this he is looking for a tool that
should allow him to import his data sets easily but at the same time that allows him to
extend, if necessary, existing measures and visualizations without a huge programming
effort.
Key attribute
- Has limited programming experiences, need for "ready-to-go" tool
- Has no operating system preferences
- Appreciates using FLOSS rather then closed software
- Wants high level of interactivity

51. Growing number of usability documents
Active persons:
Usability design area
1

52. ?

53. Recent approaches
๏ Attracting usability expert as new team member
๏ Defining an integrated development-usability process
๏ Rule: “Don’t touch the GUI before talking to a usability person or
having a look at our guidelines.”
๏ Using Balsamiq to create our mockups

54. Research challenges

55. Recent challenges
๏ Need for methods or tools for distributed usability design (Sinha
2005)
๏ Especially lightweight remote online usability testing methods
๏ Imitating design of proprietary software might repeat mistakes,
therefore FOSS can never have a better design than the
proprietary alternatives (Thomas 2008)

56. Thank you for your attention.
Questions?

57. References
๏ Benson, C. & Muller-Prove, M. & Mzourek, J. (2004) Professional usability in open source projects:
GNOME, OpenOffice.org, NetBeans. In: CHI '04: CHI '04 extended abstracts on Human factors in
computing systems. ACM, New York, NY, USA, 1083 - 1084.
๏ Cooper, A. (1999) The Inmates Are Running the Asylum. Indianapolis, Macmillan Publishing Co., Inc.
๏ Cooper, A., Reimann, R. & Cronin, D. (2007) About Face 3.0: The Essentials of Interaction Design.
Indianapolis, Wiley & Sons.
๏ Hedberg, H., Iivari, N., Rajanen, M. & Harjumaa, L. (2007) Assuring Quality and Usability in Open
Source Software Development. International Workshop on Emerging Trends in FLFOSS Research and
Development. IEEE Computer Society, Los Alamitos, CA, USA, 2007.
๏ Nichols, David M. & Twidale, Michael B.: Usability processes in open source projects. Software
Process: Improvement and Practice. 11/2, 2006, 149-162.
๏ Nichols, D. M. & Twidale, M.B. (2003) The Usability of Open Source Software. First Monday. 1(8), 2003.
๏ Nielsen, J. (1993) Usability engineering. Boston, MA.

58. References (cont.)
๏ Raskin, J. (2000) The Humane Interface. New Directions for Designing Interactive Systems. Amster-
dam, Addison-Wesley Longman.
๏ Raymond, E. S. (2001) The Cathedral and the Bazaar. Sebastopol, O'Reilly.
๏ Sinha, R. (2005) URL: http://rashmisinha.com/2005/04/03/open-source-usability-the-birth-of-a-
movement/
๏ SONIVIS (2009) SONIVIS: Social networks in virtual information spaces. http://www.sonivis.org
๏ Thomas, M.P. (2008) URL: http://mpt.net.nz/archive/2008/08/01/free-software-usability
๏ Tidwell, J. (2005) Designing Interfaces. Sebastopol, O'Reilly Media.
๏ Twidale, M.B. & Nichols, D.M. (2005) Exploring Usability Discussions in Open Source Development.
Hawaii International Conference on System Sciences.