Open by Design: How IBM Partnered with the User Community in the Redesign of Lotus Notes Betsy Comstock Mary Beth Raven Sheri Branco Michelle Cooper Deb Maurer Betsy Comstock  User Studies Lead, IBM Lotus Notes IBM Lotus Software Group:

For a huge, 20-year-old, collaboration platform E-mail Calendar Contacts Thousands of other custom applications Suppose you suddenly needed a new, world-class UI …

For a huge, 20-year-old, collaboration platform

E-mail

Calendar

Contacts

Thousands of other custom applications

We’re talking about IBM Lotus Notes Suppose you suddenly needed a new, world-class UI … In mid 2005, IBM decided to completely revamp and modernize the User Experience Notes version 7, looked like this:

IBM Lotus Notes User Experience Team About 8 people full-time on the Notes 8 client team 3+ User Studies Professionals 4+ User Experience Designers 1+ Visual Designer Many colleagues in the larger Lotus UX community

About 8 people full-time on the Notes 8 client team

3+ User Studies Professionals

4+ User Experience Designers

1+ Visual Designer

Many colleagues in the larger Lotus UX community

One of the goals of the redesign was to maintain the loyalty of our large user base 46,000+ medium and large companies worldwide 10,000+ business partners ~150 million very opinionated users What to do?

One of the goals of the redesign was to maintain the loyalty of our large user base

46,000+ medium and large companies worldwide

10,000+ business partners

~150 million very opinionated users

We had great access to this community of users Design Partners & Beta Programs Customers / Partners / Users who follow Lotus blogs & forums Other Customers with whom we have good relationships Vocal internal & external users Volunteer usability participants Users recruited to meet specific use criteria Annual Lotus conferences, briefings

What should we do with all these people?

We decided to deliberately open our design, to partner with these users

By the time Notes 8 shipped, we had incorporated input from…

over 2,000 users

13 User Research Methods in the Design of Notes 8 One-Way Customer Input Surveys Self-guided usability tests Customer feedback during product use Reaction time experiment on icon design Structured Dialog Usability testing Color and texture identification Personas Spec reviews with customers and business partners Open Dialog Contextual interviews / round table discussions Prioritize features with key customers Customer discussion forums IdeaJam Notes Design Blog

One-Way Customer Input

Surveys

Self-guided usability tests

Customer feedback during product use

Reaction time experiment on icon design

Structured Dialog

Usability testing

Color and texture identification

Personas

Spec reviews with customers and business partners

Open Dialog

Contextual interviews / round table discussions

Prioritize features with key customers

Customer discussion forums

IdeaJam

Notes Design Blog

13 User Research Methods in the Design of Notes 8 One-Way Customer Input Surveys Self-guided usability tests Customer feedback during product use Reaction time experiment on icon design Structured Dialog Usability testing Color and texture identification Personas Spec reviews with customers and business partners Open Dialog Contextual interviews / round table discussions Prioritize features with key customers Customer discussion forums IdeaJam Notes Design Blog

One-Way Customer Input

Surveys

Self-guided usability tests

Customer feedback during product use

Reaction time experiment on icon design

Structured Dialog

Usability testing

Color and texture identification

Personas

Spec reviews with customers and business partners

Open Dialog

Contextual interviews / round table discussions

Prioritize features with key customers

Customer discussion forums

IdeaJam

Notes Design Blog

Usability Testing Fairly standard test methods Goal: Identify as many usability issues as possible Think aloud, observers both local and remote From simple prototypes  real-time collaboration in a realistic test environment with 80 fictitious employees of a company we called “Renovations, Inc.” 475+ sessions on Notes 8 Tests conducted in the usability lab, and at Lotus conferences, and best of all…

Fairly standard test methods

Goal: Identify as many usability issues as possible

Think aloud, observers both local and remote

From simple prototypes  real-time collaboration in a realistic test environment with 80 fictitious employees of a company we called “Renovations, Inc.”

475+ sessions on Notes 8

Tests conducted in the usability lab, and at Lotus conferences, and best of all…

“ Renovations on the Road” At customer sites, participants were close matches to our personas Prompted rich dialog about use of Notes for their sites and job roles Added other activities that fostered open dialog, e.g. Demonstrations of new features Participation in local user groups Contextual interviews Usability Testing (at Customer Sites)

“ Renovations on the Road”

At customer sites, participants were close matches to our personas

Prompted rich dialog about use of Notes for their sites and job roles

Added other activities that fostered open dialog, e.g.

Demonstrations of new features

Participation in local user groups

Contextual interviews

Contextual Interviews & Round Table Discussions Visited ~25 customer sites Interviewed people representing our personas and other roles Learned about their jobs and how they use Notes & other technology to support their work No substitute for these rich first-hand experiences with users Unexpected, memorable events Creates dialog that lasts far beyond the visit

Visited ~25 customer sites

Interviewed people representing our personas and other roles

Learned about their jobs and how they use Notes & other technology to support their work

No substitute for these rich first-hand experiences with users

Unexpected, memorable events

Creates dialog that lasts far beyond the visit

Spec Reviews with Customers & Business Partners Tuned UX Specs with input from stakeholders Held 20+ web-conferenced and face-to-face design reviews Included UX team, developers, customers, and business partners (under nondisclosure agreements) Reviewed specs together before any code was available Merged internal and external feedback Built good will with customers Note: Still need to conduct internal reviews Need a strong moderator (in addition to the spec presenter) Need someone to monitor and respond to the group chat.

Tuned UX Specs with input from stakeholders

Held 20+ web-conferenced and face-to-face design reviews

Included UX team, developers, customers, and business partners (under nondisclosure agreements)

Reviewed specs together before any code was available

Merged internal and external feedback

Built good will with customers

Note:

Still need to conduct internal reviews

Need a strong moderator (in addition to the spec presenter)

Need someone to monitor and respond to the group chat.

Prioritize Features with Key Customers Incorporated customer priorities into decisions about the features to include Conducted prioritization exercises at customer sites and conferences Printed “Samantha can” features on “stickies” Small groups sorted items into Priority levels Everyone added red dots to most important items

Incorporated customer priorities into decisions about the features to include

Conducted prioritization exercises at customer sites and conferences

Printed “Samantha can” features on “stickies”

Small groups sorted items into Priority levels

Everyone added red dots to most important items

Prioritize Features with Key Customers Incorporated customer priorities into decisions about the features to include Conducted prioritization exercises at customer sites and conferences Printed “Samantha can” features on “stickies” Small groups sorted items into Priority levels Everyone added red dots to most important items

Incorporated customer priorities into decisions about the features to include

Conducted prioritization exercises at customer sites and conferences

Printed “Samantha can” features on “stickies”

Small groups sorted items into Priority levels

Everyone added red dots to most important items

Customers reported on their small group consensus Important in later, on-going decisions Challenges Allow participants to add items Limit number of items rated high priority Include a “Trash Can” category Provide experts to disambiguate items Group dynamics need managing Results are a bit “sloppy” Be sure to understand the logic, not just priorities Prioritize Features with Key Customers

Customers reported on their small group consensus

Important in later, on-going decisions

Challenges

Allow participants to add items

Limit number of items rated high priority

Include a “Trash Can” category

Provide experts to disambiguate items

Group dynamics need managing

Results are a bit “sloppy”

Be sure to understand the logic, not just priorities

Customer Discussion Forums & IdeaJam Monitored customer forums and blogs to stay in touch with customer issues Lotus developers took this input very seriously We answered UX items and tried posting questions Limitations Too many forums for our group to monitor (different programs, customers) Tedious to sort through and find UX-related items Input was usually way behind what we were working on

Monitored customer forums and blogs to stay in touch with customer issues

Lotus developers took this input very seriously

We answered UX items and tried posting questions

Limitations

Too many forums for our group to monitor (different programs, customers)

Tedious to sort through and find UX-related items

Input was usually way behind what we were working on

Notes Design Blog Gave us quick answers & provided a public face to Lotus UX Mary Beth Raven (et al.) blogged about Notes 8 design ~20,000 hits a month, often dozens of replies within a day Quick feedback to UX questions from people outside IBM who care Visibility with customers and leverage with developers A consensus often emerged among respondents Established a community of customers/partners empowered to influence the design Some limitations Respondents are often not typical end users The blog is public, which limits certain posts Customers cannot initiate topics, only comment & discuss topics we post www.notesdesignblog.com

Gave us quick answers & provided a public face to Lotus UX

Mary Beth Raven (et al.) blogged about Notes 8 design

~20,000 hits a month, often dozens of replies within a day

Quick feedback to UX questions from people outside IBM who care

Visibility with customers and leverage with developers

A consensus often emerged among respondents

Established a community of customers/partners empowered to influence the design

Some limitations

Respondents are often not typical end users

The blog is public, which limits certain posts

Customers cannot initiate topics, only comment & discuss topics we post

Open feedback comes from a biased sample Some customer feedback carries more weight Data are often qualitative, anecdotal, unstructured, uncategorized Amount of data was sometimes overwhelming Need to be prepared for difficult customer situations Depends on energetic and skilled user experience team, knowledgeable about the product area Opening Design to the User Community: Challenges

Open feedback comes from a biased sample

Some customer feedback carries more weight

Data are often qualitative, anecdotal, unstructured, uncategorized

Amount of data was sometimes overwhelming

Need to be prepared for difficult customer situations

Depends on energetic and skilled user experience team, knowledgeable about the product area

Opening Design to the User Community: Benefits Broader knowledge of users and their work contexts Richer design ideas Customer consensus on design direction Professional partnerships with key customers Increased customer loyalty Better professional interconnections inside IBM Fast, flexible methods

Broader knowledge of users and their work contexts

Richer design ideas

Customer consensus on design direction

Professional partnerships with key customers

Increased customer loyalty

Better professional interconnections inside IBM

Fast, flexible methods

So, how did we do???????

Inbox with Preview & Sametime

Calendar, Quickr Sidebar

Great Customer Feedback Extremely positive reactions from customers, partners, and analysts Faster deployment rate than with previous versions of Notes Increased revenue And, perhaps most gratifying for those of us who feel so connected to our user community…

Extremely positive reactions from customers, partners, and analysts

Faster deployment rate than with previous versions of Notes

Increased revenue

And, perhaps most gratifying for those of us who feel so connected to our user community…

Great Customer Feedback Lotus Notes 8 received a standing ovation at Lotusphere 2007

“ Open by Design” continues to characterize our UX methods today For more information, contact me at: b et [email_address] Or see the full paper: Comstock, E. M., Raven, M. B., Branco, S. F., Cooper, M. L., & Maurer, D. E. Open by Design: How IBM Partnered with the User Community in the Redesign of Lotus Notes. CHI 2009 Extended Abstracts , Boston, MA, 2931-2944. Thank You!

For more information, contact me at: b et [email_address]

Or see the full paper:

Comstock, E. M., Raven, M. B., Branco, S. F., Cooper, M. L., & Maurer, D. E. Open by Design: How IBM Partnered with the User Community in the Redesign of Lotus Notes. CHI 2009 Extended Abstracts , Boston, MA, 2931-2944.

© IBM Corporation 2008. All Rights Reserved. The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. All references to Renovations refer to a fictitious company and are used for illustration purposes only. All references to Samantha Daryn, Ted Amado, and Betty Zechman are fictitious. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Sametime, WebSphere, Workplace and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both.

The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.

All references to Renovations refer to a fictitious company and are used for illustration purposes only. All references to Samantha Daryn, Ted Amado, and Betty Zechman are fictitious.

IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Sametime, WebSphere, Workplace and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both.