Poster presentation at UXPA2019 by Dorothy Cummings, Agile Six and Kath Straub, usability.org Today’s Agile Maturity Assessments are not very usable. Some lack scientific validity, others measure without providing actionable outcomes or guidance for improvement, or they leave out critical elements of mature Agile teams (e.g., UX integration.) We discuss scientific, organizational, and cultural challenges that undermine the meaningful assessments design and/or assessment. Then we present a (new) Agile UX maturity assessment tool and method designed to offer teams concrete insights and actionable opportunities to improve. Then we recommend an implementation and progress management strategy to nurture positive growth toward true agile transformation. We couch our recommendations in an evidence-based review of measurement strategies (e.g., self- vs. external evaluation or individual evaluator vs collective discussion), and a discussion of cultural barriers that typically undermine the utility and impact of assessments (e.g., cultural implications of frank self-assessment and the need to report scores up.). By the end of the session, you will better understand the benefits of (agile UX or other) organizational assessments, cultural and organizational characteristics that necessarily limit their impact, and strategies to assess and track change that can drive improvement. You will also have a new tool to assess Agile UX maturity in your organization.

1. Useful aUX maturity metrics
● Look in, not across
● Evolve with the team
● Guide the team toward next steps/next
level
● Drive further growth and maturity
Agile & UX are natural partners Toward aUX aUX Maturity Checklist
Toward aUX: Folding UX into Agile Maturity measures
Dorothy Cummings1 and Kath Straub, PhD2
1Agile Six, Baltimore MD and 2 Usability.org, Tucson AZ
Contact
Barriers to Effective
Measurement
❏ All team members have basic aUX and basic Agile
training
❏ All team members are accountable for, and have a
means to contribute to, Customer Satisfaction
❏ All aUX research and design work is captured and
tracked in the backlog/kanban
❏ All team members attend and contribute to all Agile
framework ceremonies
❏ All team members contribute to success criteria and
evaluation of aUX stories and vice versa
❏ Research findings and Usability test results are
presented in Demos
❏ UX process integration with Agile framework is
discussed at retrospectives
❏ Customer Research is used to identify and prioritize
future features and functions
❏ Design Patterns and Best Practice Interaction
Design are used within and across development
teams
❏ Agile Momentum and UX Momentum are both
tracked and reported
❏ Customer feedback requiring immediate response
(service and support of existing features) is read out
during stand-ups
❏ Architectural and technical planning and solutioning
always consider UX requirements
Cultural Barriers:
● A mistrusting and/or command and control culture
causing respondents to be dishonest - to look
better than they actually are
● Using results comparatively to rank team
performance
● Never or rarely refining the assessment as team
culture and ownership improves
Procedural Barriers:
● Observations and trends do not result in actionable
follow-on goals and actions
● Lack of understanding of UX and/or Agile process
● UX and Development efforts remain separate, with
results, designs and solutions “handed-over”
between them
● Suggest other, more concrete measures for
leadership’s consumptions - “Working software is
the primary measure of success” (Agile Principle #7)
● Your assessment facilitator MUST be trusted by the
team(s) - be certain to choose wisely
● Make the meeting the team’s meeting: Allow and
expect the teams to own their progress and
improvement journey
● Focus first on the application of the Agile Principles;
then on team dynamics, including aUX integration
● Most importantly, create a culture of trust so
results are reliable and actionable
● TAKE ACTION! Small, visible, incremental steps can
lead to meaningful change.
Agile Software Development and Customer
Experience are both
● disciplined repeatable processes
● use customer input to prioritize possible
improvements
● rapidly iterate
● Measure and track operational
improvements
Underlying philosophy: Fail fast. Fix it.
The Agile Manifesto even calls out Human
Centered Design (HCD) as core component to the
philosophy: Customer collaboration over contract
negotiation
Yet, Agile Maturity Assessments fail to integrate
UX team members, activities or UX maturity into
the metrics. When they do, the level is too high to
be meaningful or too abstract to provide
actionable guidance on how to improve.
What’s the point of measuring if there are no
outcomes-based improvement opportunities?
.
Customer Needs
Research
● Identify and
prioritize MVP
modules/funct
ions
● Innovate and
validate
streamlined
workflow
● Prioritize
MVP
roadmap
Rapid design and
prototype testing
● Brainstorm with
developers
● Validate (micro) utility
and usability before
implementation
● Measure efficiency
improvements
● Identify/prioritize
training and customer
support
Always-on Customer Feedback
(Problems, Questions,
Suggestions)
● Discover and remediate
unanticipated customer
issues in real time (Daily! In
standups)
● Document customers desired
functionality
Customer
Satisfaction Surveys
● Benchmark and
track customer
satisfaction
across releases
using Industry
Standard or
consistent
bespoke metrics
For more information or to learn about aUX
training and coaching, contact:
kath@usability.org
dorothy.cummings@agile6.com
Takeaway
● Work with leadership to establish metrics used for their
visibility and understanding of progress made - Maturity
assessments are not the proper yardstick (see barriers ↞ )
● Define specific, desired goals and outcomes as well as
what positive progress looks like; review often and modify
as needed
● Allow the team(s) to agree as to why and when
assessments will be made and how results will be used
● Train everyone in UX and Agile practices - everyone!
● Use the assessment meeting as a discussion rather than a
Q&A, in order to capture nuance and reasoning behind
the scores
● With the team(s), create an action plan based on
results and include the work on your backlog
Measuring for Success Tips from an Agile Coach