Keith Miller and Mike McCoy presented to the ESD QA team on the partnership between UX and QA. They observed that UX and QA share the common goal of quality but have different approaches. Great user experiences result from harmony among project disciplines. The presentation covered UX and visual design processes, principles, testing methods and measures. It proposed innovations to the UX-QA partnership like remote usability testing and UX audits to raise awareness of how design impacts testing.

1. Two Sides of the Same Coin
Making Heads or Tails of the UX/QA
Partnership
Presented to ESD QA
Keith Miller & Mike McCoy

2. Who we are
§ Our History
§ Team Structure
§ Our Backgrounds
§ Our Charter to ESD

3. Our Observations
§ UX & QA share a common goal . . . just different approaches
§ What may feel like overlap is actually opportunity
§ Great UX results from harmony amongst the project team
disciplines

5. Usability, Visual Design & User
Experience (UX) Design
§ Overview
§ Our Process
§ Our Guides: principles, rules & testing
§ A case study
§ How we innovate
§ Our partnership

6. What is Usability?
§ A quality attribute that assesses how easy user interfaces
are to use
§ Defined by 5 quality components:
§ Learnability
§ Efficiency
§ Memorability
§ Errors
§ Satisfaction
§ Effectiveness

7. What is Visual Design
§ Visual design is problem solving on many levels using
imagery, color, typography, symbols, and many other design
elements to form a unified message so that people can
understand them in an effective way.
§ Good Visual Design is simple and almost invisible as it
visually communicates these messages.

8. … and then there’s UX
§ A much broader concept that encompasses all aspects of
end-user's interaction with a company, its services, and its
products.
§ Hierarchy of needs:
§ Functional, Usable, Pleasurable
§ Can do, Will do, Want to do
§ An extension of ‘Satisfaction’ quality attribute: explores how
positive emotions evoked in User Delightful, meaningful,
beautiful, playful, seductive, immersive, thoughtful
§ A lot harder to measure quantitatively

9. Our ‘Typical’ Process
UX Visd
Design Sessions Create Mocks
Design / Prototype Solution Team /Owner Review and signoff
Prototype Audit Design Story Audit
Update Prototype Visde Specifications
User Reviews Visde WebDev
UX Audit

10. What Guides Our Design Work
§ Principles
§ Rules or ‘Heuristics’
§ Testing

11. Usability Principles
§ Visibility
The more visible functions are, the more likely users will be able to know what to
do next
§ Feedback
Providing information about what action has been done and what has been
accomplished, allowing the person to continue with the activity.
§ Constraints
Restricting the kind of user interaction that can take place at a given moment
§ Mapping
Clear relationship between controls and their effects in the world.
§ Consistency
Interfaces have similar operations, use similar elements for achieving similar
tasks
§ Affordance
An attribute of an object that allows people to know how to use it

12. Design Principles
§ Balance
Distribution of heavy and light elements on the page.
§ Contrast
Difference between two objects
§ Emphasis
Attention towards something
§ Unity/proximity
Keeping like elements together and diverse elements further
apart

13. Testing
§ Traditional: Facilitated Observation of User, Tool & Context
§ Remote Moderated: Augmented with technology to
widen reach
§ Strategies
§ Formative – Find and Fix
§ Summative – Assess Using Metrics
§ Usability Inspection vs Usability Testing
§ Scope: Formal Testing ßà ‘Discount’ Testing
13

14. 14
Summary: Elaborate usability tests are a waste of resources. The best results come from testing
no more than 5 users and running as many small tests as you can afford (Nielsen & Landauer).

15. What We Measure:
§ Quantitative / Objective:
§ Time on Task
§ Effort (clicks, presses, switches, cogload)
§ Errors (Mistakes, Slips)
§ Success/Failure Rates
§ Qualitative / Subjective:
§ ‘Think-Aloud’ Protocol
§ How a User ‘Experiences’ the Tool
§ Satisfaction, Confusion or Frustration
15

21. Innovating Processes
§ Auditing with Notable as a precursor to ALM
§ Agile Story Golf
§ Remote Unmoderated testing for better sprint integration
§ Working with QA’s Automation Group to raise awareness of
the impact of design change on testing

22. Create the UX Audit task in the
Appropriate story. These are stories
where UX was needed.
When the story moves to “In Test” start the UX
Audit. QA has a slight lead time based on UX
bandwidth and to find the big functional defects.
By the time the story is accepted all
issues must be fixed or placed in
the backlog with a severity and priority.

23. Details
• With the UX spike concept the team agrees to add 1-2 design only stories.
• These stories should require UX design and are usually complex in nature.
• These design stories are a portion of the estimated effort to final implementation.
UX ‘tees-up’ the design and the rest of the team knocks it down (implements).

26. The Evolving UX / QA Partnership
§ Primary Touch points: Initial Design, Testing
§ Common Questions
§ How do we account for overlaps (synergies)
§ Which role tests for which type of issue? (defects vs UX ‘issues’)
§ Who is accountable for recommending a solution?
§ What's the difference? UX = QA= UX
§ Synergies
§ We each reinforce the others’ messages about quality.
§ We have methodologies, tools, artifacts and expertise to share.
§ UX can benefit from the rigor and maturity of QA.
§ UX can help bake QA sooner into the SDLC.
§ Opportunities for Future Partnership & Synergies

27. What to Read?
27
Steve Krug Russ Unger Jakob Nielsen

28. Your Story . . .
§ Ideas
§ Proposals