Early Signal Testing: Designing Atlassian’s New Look
•
4 likes•9,513 views
You probably have noticed the new look of Atlassian's Cloud products. Our new Design Guidelines took many months to create, and our team had many tough decisions to make. Luckily, we incorporated customer research along the way to guide us. One of our most valuable research tools is called “early signal testing”, and we think it can help you too. Early signal testing can help you gain confidence in a direction, rather than being paralyzed by a choice. It can help assess your design's usability, clarity, comprehension, and more. This talk explains how your team can gather measurable user feedback in as little as a week, for even the very biggest of problems.
Recommended
More Related Content
Viewers also liked
Viewers also liked (18)
Similar to Early Signal Testing: Designing Atlassian’s New Look
Similar to Early Signal Testing: Designing Atlassian’s New Look (20)
More from Atlassian
More from Atlassian (20)
Recently uploaded
Recently uploaded (20)
Early Signal Testing: Designing Atlassian’s New Look
- 1. Early Signal Testing: Designing Atlassian’s New Look ASHLEIGH STERZENBACH | SENIOR DESIGNER | ATLASSIAN BECKY WHITE | SENIOR RESEARCHER | ATLASSIAN
- 2. 35,000
- 8. Decisions can be dauntinglife-changingscary.exhaustingexcitingan upgrade
- 9. Product decisions can be scary too.
- 16. We too made some scary decisions recently.
- 17. N View issue · Add to sprint 1.4-m1 Issue ADG-745 has been created. MOVED NEW IN PROGRESS REMOVED SUCCESS DEFAULT MOVED NEW IN PROGRESS REMOVED SUCCESS DEFAULT 123122 123122 123122 123122 123122 Alana AwareAlana Aware Buttons Forms Tables Tabs Dropdown button Dropdown button StatisticsReopenMoreCommentAssign to meAssignEdit ConfirmInvite usersSelect pluginsConfigure instance Gray background
- 18. Open in JIRA Open in JIRAOpen in JIRA Open in JIRAOpen in JIRA Open in JIRA RECENT PROJECTS SELECTED ALL PROJECTS Issue rewrites Issue Rewrite (IR) JIRA Core Bord (JCB) JIRA Core Board (JCB) Vertigo Incompatibilties (VI) Vertigo Incompatibilties (VI) JIRA Process Advancement (JPA) JIRA Process Advancement (JPA) JIRA OOME (JOOME) list name View issue • Add to next sprint START-85 has been created MM MM MM 99+122 99+2 12 67%Uploading… MOVED NEW IN PROGRESS REMOVED SUCCESS DEFAULT NEW IN PROGRESS REMOVED SUCCESS DEFAULT MOVED 26/11/2015 Due date November 2015 MON TUE WED THU FRI SATSUN 131 87 1514 2221 29 6 2 9 28 5 1 8 6 13 20 27 5 12 19 26 4 11 18 25 3 10 17 24 2 9 16 23 30 7 3 10 4 11 MOVED NEW IN PROGRESS REMOVED SUCCESS DEFAULT NEW IN PROGRESS REMOVED SUCCESS DEFAULT MOVED
- 21. Trust in our skills Our product decisions are made up of… Careful thought Research
- 23. Agenda What it is Early signal methods Getting started
- 24. Agenda Early signal methods Getting started What it is
- 25. Agenda Early signal methods Getting started What it is
- 26. Improve itEnvision it Make it
- 29. Behavioral Attitudinal Qualitative (direct) Ethnographic field studies Usability lab studies Quantitative (indirect) Interviews Focus groups A/B testing Click analysis Concept testing Customer feedback Diary studies Card sorting Unmoderated studies Early signal testing Early signal testing
- 30. Gather quick, actionable feedback from customers EARLY SIGNAL TESTING
- 31. No worries You don’t need to be a research expert Why use it? Guidance Use information, not just opinions Direction Get cues, not binding answers Quick Don't need detailed mockups or working code
- 32. Methods
- 33. Aesthetics test Info test Expectation test Preference test Page design test Value test Task completion test Comprehension test Navigation test Usability test Concept test Click test DIFFERENT METHODS
- 34. Aesthetics test Info test Expectation test Preference test Page design test Value test Task completion test Comprehension test Navigation test Usability test Concept test Click test DIFFERENT METHODS
- 35. Aesthetics test
- 36. See if there’s any unintended reactions Assess which visual connects best Collect first impressions and emotional responses Aesthetics test
- 37. See if there’s any unintended reactions Assess which visual connects best Collect first impressions and emotional responses Aesthetics test
- 38. See if there’s any unintended reactions Assess which visual connects best Collect first impressions and emotional responses Aesthetics test
- 42. Preference test
- 43. Preference test Build off why users prefer one option See if there’s an easy winner Gather users' initial inclinations
- 44. Build off why users prefer one option See if there’s an easy winner Gather users' initial inclinations Preference test
- 45. Build off why users prefer one option See if there’s an easy winner Gather users' initial inclinations Preference test
- 46. Which option do you prefer and why? A site consists of user groups: admins, project admins, and users. Organizations work on projects in a certain area of interest. Users groups work on and complete these projects. User groups consist of users. Admins restrict each user by access or role. Projects work out specific plans or designs An organization consists of people. Organizations work on projects in a certain area of interest. Teams work on and complete these projects. Teams consist of members. Each member fills a role. Projects work out specific plans or designs. Option A Option B
- 48. Which option do you prefer and why? Option A Option B
- 51. Ensure users don’t misunderstand information Ensure what you’ve written is clear Ensure users comprehend complicated information Comprehension test
- 52. Ensure users don’t misunderstand information Ensure what you’ve written is clear Ensure users comprehend complicated information Comprehension test
- 53. Ensure users don’t misunderstand information Ensure what you’ve written is clear Ensure users comprehend complicated information Comprehension test
- 59. Navigation test
- 60. See if alternate navigation paths are needed Assess usability, but don’t have a full flow yet Understand users’ natural wayfinding inclination Navigation test
- 61. See if alternate navigation paths are needed Assess usability, but don’t have a full flow yet Understand users’ natural wayfinding inclination Navigation test
- 62. See if alternate navigation paths are needed Assess usability, but don’t have a full flow yet Understand users’ natural wayfinding inclination Navigation test
- 66. Ash - we need a screenshot of the icons in product
- 67. Concept test
- 68. Learn what’s valuable, what’s not, what’s missing Compare your idea to users’ current process Explore early product and feature ideas Concept test
- 69. Learn what’s valuable, what’s not, what’s missing Compare your idea to users’ current process Explore early product and feature ideas Concept test
- 70. Learn what’s valuable, what’s not, what’s missing Compare your idea to users’ current process Explore early product and feature ideas Concept test
- 74. Aesthetics test Info test Expectation test Preference test Page design test Value test Task completion test Comprehension test Navigation test Usability test Concept test Click test DIFFERENT METHODS
- 75. Getting started
- 76. 5 quick & easy steps Step 1: Define your question & method Step 2: Figure out the “who” and the “how” Step 3: Create tasks, questions & design mocks Step 4: Test with people Step 5: Analyze
- 77. Step 1: Define your question & method
- 78. Does our intent for our new logos match how people will perceive them? What do people think of the new logos? NOT THIS THIS
- 79. Get your team involved Speaking with stakeholders before you run your research will ensure it hits the mark #ProTip
- 80. 5 quick & easy steps Step 1: Define your question & method Step 2: Figure out the “who” and the “how” Step 3: Create tasks, questions & design mocks Step 4: Test with people Step 5: Analyze
- 81. Step 2: Figure out the “who” and the “how”
- 82. Who?
- 83. How?
- 84. Unmoderated Users independently complete tasks, quick turnaround time Moderated Speak with users, gather rich feedback, ask follow-up questions
- 85. #ProTip Unmoderated toolkit Check out Usertesting.com, User Zoom, Loop11, Validately, and others
- 87. 5 quick & easy steps Step 1: Define your question & method Step 2: Figure out the “who” and the “how” Step 3: Create tasks, questions & design mocks Step 4: Test with people Step 5: Analyze
- 88. Step 3: Create tasks, questions & design mocks
- 90. Write scenarios Rather than just creating tasks, create one story that ties together all the tasks. #ProTip
- 92. Keep it simple Only design what you need and don’t overthink it. #ProTip
- 93. 5 quick & easy steps Step 1: Define your question & method Step 2: Figure out the “who” and the “how” Step 3: Create tasks, questions & design mocks Step 4: Test with people Step 5: Analyze
- 94. Step 4: Test with people
- 95. Don’t just listen It’s not just what users say, it’s what they do #ProTip
- 96. 5 quick & easy steps Step 1: Define your question & method Step 2: Figure out the “who” and the “how” Step 3: Create tasks, questions & design mocks Step 4: Test with people Step 5: Analyze
- 97. Step 5: Analyze
- 98. Task completion Time to taskUMUX-lite Time on task Expectation score Happy path clicks 20% Baseline New 100%+80% 74.81 77.90+3.09 1m : 36s 2m : 27s +51s 5.0 / 7.0 6.3 / 7.0 +3.1 6 3 100% Baseline New 100%0% 79.74 82.48+2.74 0m : 47s 1m : 00s +13s 4.6 / 7.0 5.0 / 7.0 +0.4 6 3 Experience Create a UX scorecard Metrics like time on task, task completion or user-reported metrics like ease of use are valuable comparison points. #ProTip
- 99. Task completion Time to taskUMUX-lite Time on task Expectation score Happy path clicks 20% Baseline New 100%+80% 74.81 77.90+3.09 1m : 36s 2m : 27s +51s 5.0 / 7.0 6.3 / 7.0 +3.1 6 3 100% Baseline New 100%0% 79.74 82.48+2.74 0m : 47s 1m : 00s +13s 4.6 / 7.0 5.0 / 7.0 +0.4 6 3 - Many decision points for users and a cognitive overload - General understanding of certain concepts in project create caused confusion for users i.e. project key Baseline - Help and guidance for first time users - Progressive disclosure of template information - Template information that means something to users New experience What we tested Experience Cohort Existing users We tested in cohorts of 10 existing JIRA users (admin and non-admin) per model. New users We tested in cohorts of 30 new users per model. Tasks: You now want to setup this new project. You need to make sure that in this new project you can plan, track and report all of your work. Show how you would setup this new project. We wanted to know if users can Easily create a project. Easily understand where they are. Easily comprehend and understand the differences between project templates. Create a project quickly (no friction points). UX Scorecard - Creating a project
- 100. 5 quick & easy steps Step 1: Question & method Step 2: Who & how Step 3: Mocks & tasks Step 4: Test Step 5: Analyze
- 101. Early signal testing helps inform your decision but it's not the decision. - ASH & BECKY :)
- 102. The next time you’re facing a tough decision…
- 103. Thank you! ASHLEIGH STERZENBACH | SENIOR DESIGNER | ATLASSIAN BECKY WHITE | SENIOR RESEARCHER | ATLASSIAN