Making Usable Software - at Agile Indy / UXPA Joint Meeting

@Carologic

Making Usable Software

September 10, 2012
Indiana UXPA and AgileIndy

"The biggest waste of all
is building something
no one wants"

- @ericries #LeanStartupMI in 2011 via @MelBugai

Create a great,
usable, accessible,
and relevant experience

Integrating with Agile

Sprint 1

Sprint 2

Sprint 3
Phase / Sprint 0
User Design U Test U Test
Research for S2 Design Design
UX

Design UR for S3 for S3 for S4
for S1 UR for S4 UR for S5

Increased understanding of Users
Phase / Sprint 0

Sprint 1

Sprint 2

Sprint 3
Pre- Dev Dev Dev
Dev
Dev

Agile Integration
 Use each study to pick up information
 Additional user research done in parallel

User Design for U Test U Test U Test U Test U Test
Sprint 0

Sprint 1

Sprint 2

Sprint 3

Sprint 4

Sprint 5

Sprint 6
Research S2
UX

Design for Design for Design for Design for Design for
Design for UR for S3 S3 S4 S5 S6 S7
S1 UR for S4 UR for S5 UR for S6 UR for S7 UR for S8

User Observations Interviews Survey
Personas
Increased understanding of Users
Sprint 0

Sprint 1

Sprint 2

Sprint 3

Sprint 4

Sprint 5

Sprint 6
Pre-Dev Dev Dev Dev Dev Dev Dev
Dev

Experience Driven Agile:
Developing Up to an Experience,
Not Down to a Feature
Rehash of a Webinar by: Kaleb Walton & Brian Anderson

The “Pitch”
 Quickly conveys background of problem,
Effective Prioritization and
proposed solution and statement of value Assignment of Work Items
 Shirt-size estimates make for easy The problem is that systems
prioritization (story points are fine too) managers spend too much time
prioritizing and assigning their
 Sprinkle in risk and value to make team's daily work efforts.
prioritization even easier Imagine if Systems Manager
Plus offered better prioritization
 Prioritize dozens of experiences, not capabilities and automated
assignment based on definable
hundreds business rules.
 General format: This solution would result in
reduced cost for systems
The problem is <problem>. Imagine if managers by enabling more
<solution>. This solution would result in efficient work assignment,
leading to better response
<value statement>. times.
 Lightweight precursor to...

Copyright © 2012 Kaleb Walton, Brian Anderson, Michael Hughes and Terri Whitt

The “Scenario”

 Borrowed from UX discipline
 Paints a clear picture of an entire
experience
 Extremely versatile and ready for use
outside development
 Ourreal-world example of a person's
“A definition:
experience with a product, describing context
with a problem and a proposed solution.”


Scenarios Are Agile
 Just Barely Good Enough and Just in Time: Fidelity
naturally matches immediate need.
 Ya Ain’t Gonna Need It: Does it enable the scenario?
 Minimum Viable Product: What is
the minimum experience someone
would pay for?
 Lightweight: Low cost to develop,
flexible and quick to communicate.
 Better Contract: More reliable
as it's written in terms of Experience
rather than Features.


Telling a Story


Example Scenario
EFFECTIVE PRIORITIZATION AND ASSIGNMENT OF WORK ITEMS
PROBLEM
Mary, a systems manager at ABC Health, is responsible for a team of 12 system administrators
who handle steady state support of their health care systems and network. One of her biggest
time sinks is prioritizing and assigning her teams daily work efforts. The tool she uses, Systems
Manager Plus, doesn't give her any prioritization features except for the ability to sort on a
'priority' field when reviewing work items.
As she spends half of her time prioritizing she ends up working over time to tend to her other
duties.
SOLUTION
After a major update Mary signs into Systems Manager Plus, heads to the work items area and
is pleasantly surprised to see a number of new prioritization capabilities. There are more fields
available to sort and filter, as well as a “smart assignment” system that enables her to specify
rules that will result in automatic assignment to specific members of her team.
Mary creates a few rules, applies them to existing work items, and is excited to see that over a
quarter of the items were automatically assigned. She proceeds to sort and filter the remaining
work items to prioritize and assign to her team. As more work items trickle in she notices that
many of them are being auto-assigned.
These improvements have enabled Mary to focus less on prioritizing and more on doing.


Easily Pull Out Stories and Epics

Additional sorting capabilities
As a systems manager I want to sort work items by
additional fields such as created date, severity and
platform so that I can more effectively prioritize them.
Additional filtering capabilities
As a systems manager I want to filter work items by
additional fields such as created date, severity and
platform so that I can more effectively prioritize them.

Smart assignment system (epic)
As a systems manager I want to specify assignment rules for the system to use
to automatically assign work items so that I don't have to assign every work item
manually.

Apply new smart assignment rules to existing work items
As a systems manager I want to apply new smart assignment rules to existing
work items so that I can use smart assignment on work items created after the
smart assignment process has executed.

Basic Experience Driven Agile
Involvement Over Time by Role
Scrum Masters, Developers
Product Owners, and Testers
UX Analysts, Architects and Stakeholders
Product Mgt

Product Backlog Iteration Backlog
Activities

Pitches Scenarios Stories

Estimate, Valuate Estimate
Assess, Prioritize


Experience Driven Agile At Scale
Involvement Over Time by Role
Scrum Masters, Developers
Product Owners, and Testers
UX Analysts, Architects and Stakeholders

Portfolio Backlog
Product Mgt

Product Iteration
Activities

Pitches Backlogs Backlogs

Scenarios Stories
Scenarios

Estimate, Valuate Prioritize Estimate
Assess, Prioritize


Contact Us

Kaleb Brian Anderson
user.experience.guy@gmail.com
Walton
kalebwalton@gmail.com

Thanks to Other Experience Driven Agile Contributors

Michael Hughes, Ph.D Terri Whitt
michaelhughesua@gmail.com tw30306@yahoo.com

http://experiencedrivenagile.com

Usability Testing
in Agile Environments

Any Method Can be Adapted
 Quick
 Bare minimum of effort
 Get needed feedback
 Provide recommendations
 Repeatable

Scope Effort
 Consider budget, resources
 Time
 Recruiting
 Facilitating
 Analyzing
 Adding participants increases budget &
time

Paper, Clickable or Real Code?
 Always start with paper
 Guerilla / hallway test
 Users may misunderstand
 Clickable prototypes
 Easier to understand
 Can easily change
 Real Code
 Great if it’s the right solution

Paper or Clickable Prototype
 Rapid Iterative Testing & Evaluation (RITE)
 Traditional Testing
 In-Person
 Remote more challenging

Rapid Iterative Testing & Evaluation
 Qualitative user feedback
 actions + comments
 Series of small usability tests
 3 participants each day
 Minimum of 3 days of testing
 Iteration between testing days
 Total of 5 days

RITE Process

Priority
Test & Level of Effort Update Test
1 High

2 Medium

3 Low

25

Recap Sessions
 End of each day - after the last session
 Room with a whiteboard.
 About 30 minutes.
 Discuss:
 trends seen
 concerns
 recommendations
 prioritize changes for the next round
 list lower priority changes for future iterations

26

RITE Results
 Final prototype
 Vetted with users
 Base for recommendations
 Light Report: “Caterpillar to Butterfly”
 Screenshots show progressions
 What changes were made and why

What Works for RITE
 Best used early in project lifecycle
 Early concepts
 Need to be vetted with users
 Can assist in quickly shaping designs

28

General Testing

 Traditional Testing
 In-Person
 Remote
 Moderated or Unmoderated
 Less users, shorter sessions: analyze at
lunch
 Recommend 3 or more users
 Half hour to 1 hour each

Regular Testing

(Yes, this is an old idea; a great one!)

Bring it On!
 Small focused tests
 Reduce waiting for recruitment
 Once per week or per Sprint
 Same day mid-week (not Monday or
Friday)

User Testing Day!
 Make team aware
 Invite everyone
 Watch remotely
 Recurring meeting invites for stakeholders

What could I test?
 Identify what to test at start of Sprint
 Work in Progress
 Multiple projects
 Prototypes
 Concepts, rough ideas, brainstorming
 Competing designs, (A/B testing)
 Comparative studies across market
 Conduct interviews to inform research
 More…

“Teams should stretch to get
work into that day’s test
and use the cadence to drive
productivity.”

- Jeff Gothelf - http://blog.usabilla.com/5-effective-ways-for-usability-testing-to-play-nice-with-
agile/

Why Regular?
 Team becomes:
 accustomed to steady stream of qualitative
insight
 insight ensures quick decisions…line up with
business and user goals

Adapted from Jeff Gothelf - http://blog.usabilla.com/5-effective-ways-for-
usability-testing-to-play-nice-with-agile/

Include PWD
 People with disabilities
 “We are all only temporarily able-bodied.
Accessibility is good for us all.”
 Get to spirit of the law (Section 508, WCAG
2.0)

-@mollydotcom at #stirtrek 2011 via @carologic

Pre-Book Your Rooms
 Test & Observation Rooms
 Any location will do
 Conference rooms
 Offices
 Quiet corner of cafeteria
 Remote

Create Reusable Templates
 Screener
 Technology use/experience
 Knowledge of topic
 Scripts/Guides
 Consent Forms
 Data Collection

Find Patterns Quickly

Issue P1 P2 P3

Search Used Yes No No

Widget 1 Used N/A Used –
unsure
about
Task 1 Notes 3 – easy 2 – needed 3 – easy
help
Task 2 Notes 2 – needed 2 – easy 2 – needed
help help
Task 3 Notes 2 – needed 3 – easy Ran out of
help time
Task 4 Notes 2 – needed 3 – easy Ran out of
help time

True Statements
 All interfaces have usability problems
 Limited resources to fix them
 More problems than resources
 Less serious problems distract
 Intense focus on fixing most serious
problems first

Adapted from: Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and
Fixing Usability Problems. By Steve Krug

Debrief with Team
 Assumes stakeholders watched tests
 If not, wait for UX analysis
 Quick analysis to quick decisions
 All decision makers MUST be present

Goal
 Identify top 5 or 10 most serious issues
 Top 3 from each list
 Prioritize from lists
 Commit resources for next sprint
 Stop

Adapted from: Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and
Fixing Usability Problems. By Steve Krug

Guidelines
 Stay on Topic
 Be Constructive
 Don’t get distracted by small problems
 Intense focus on fixing most serious
problems first

Make Useful & Usable
Recommendations - Quickly

Transform Data
 Look for patterns
 Read “between the lines”
 Know what you’ve got
 Sort, reorganize, review, repeat
 What refutes your expectations?
 Surprises?
 Outliers?

Short and Direct Communication
 Email or One Pager
 Think about audience
 How will it be used?
 Include
 Goal of study
 What will be fixed and who assigned to
 Tasks attempted
 Who observed
 Future research/enhancements

Tweak, Don’t Redesign
 Small iterative changes
 Make it better now
 Don’t break something else
 Take something away
 Reduce distractions
 Don’t add – question it

Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability
Problems. By Steve Krug

Do UX Early & Often
 Make users visible
 Information radiators
 Test findings
 Artifacts
 Personas
 Word Clouds - IA

Recommended Readings

Gothelf, Jeff. Lean UX: Getting Out of the Deliverables Business. (Anticipated in
Feb. 2013) 5

Contact Carol
@Carologic

Email: Carologic@gmail.com

SlideShare.net/Carologic

SpeakerRate.com/speakers/15585-CarolJSmith

Tool Considerations
• In-person or remote?
• Lab or on-site?
• Prototype limitations (can it be online?, is it a document
or a clickable site?)
• Number of observers, number of participants?
• Number of facilitators?
• Logging and video editing needs (time on task, highlight
video creation)?
• Surveys before or after?
• Eye tracking?

Usability Testing Software
• Morae
• Ovo
• SilverBack (Mac only)
• UserWorks
• Noldus
• Tobii (Eye-tracker)
• SMI (Eye-tracker)
• SurveyMonkey

Screen Sharing Software
 GoToMeeting – http://www.gotomeeting.com
 Lotus Sametime Unyte – http://www.unyte.com
 YuuGuu -- http://www.yuuguu.com
 WebEx – http://www.webex.com
 Yugma -- https://www.yugma.com/

 Trouble Shooting: CoPilot -
https://www.copilot.com/

Recommended Sites
 Usability.gov
 W3C Web Accessibility Initiative
 http://www.w3.org/WAI/
 Accessibility Standards in US (Section 508)
 http://www.access-board.gov/sec508/508standards.htm
 Jakob Nielsen
 http://www.useit.com
 UPA – professional UX association
 http://www.upainternational.org/

References
 Albert, Bill, Tom Tullis, and Donna Tedesco. Beyond the Usability Lab.
 Beyer, Hugh. User-Centered Agile Methods (Synthesis Lectures on Human-
Centered Informatics)
 Gothelf , Jeff. http://blog.usabilla.com/5-effective-ways-for-usability-testing-
to-play-nice-with-agile/
 Henry, S.L. and Martinson, M. Evaluating for Accessibility, Usability Testing
in Diverse Situations. Tutorial, 2003 UPA Conference.
 Krug, Steve. Rocket Surgery Made Easy: The Do-It-Yourself Guide to
Finding and Fixing Usability Problems.
 Ratcliffe, Lindsay and Marc McNeill. Agile Experience Design: A Digital
Designer's Guide to Agile, Lean, and Continuous.
 Rubin, Jeffrey and Dana Chisnell. Handbook of Usability Testing: How to
Plan, Design, and Conduct Effective Tests. John Wiley & Sons, Inc.

Making Usable Software - at Agile Indy / UXPA Joint Meeting

Recommended

Recommended

More Related Content

Similar to Making Usable Software - at Agile Indy / UXPA Joint Meeting

Similar to Making Usable Software - at Agile Indy / UXPA Joint Meeting (20)

More from Carol Smith

More from Carol Smith (20)

Recently uploaded

Recently uploaded (20)

Making Usable Software - at Agile Indy / UXPA Joint Meeting