Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Early Signal Testing: Designing Atlassian’s New Look

8,039 views

Published on

You probably have noticed the new look of Atlassian's Cloud products. Our new Design Guidelines took many months to create, and our team had many tough decisions to make. Luckily, we incorporated customer research along the way to guide us.

One of our most valuable research tools is called “early signal testing”, and we think it can help you too. Early signal testing can help you gain confidence in a direction, rather than being paralyzed by a choice. It can help assess your design's usability, clarity, comprehension, and more. This talk explains how your team can gather measurable user feedback in as little as a week, for even the very biggest of problems.

Published in: Software
  • Be the first to comment

Early Signal Testing: Designing Atlassian’s New Look

  1. 1. Early Signal Testing: Designing Atlassian’s New Look ASHLEIGH STERZENBACH | SENIOR DESIGNER | ATLASSIAN BECKY WHITE | SENIOR RESEARCHER | ATLASSIAN
  2. 2. 35,000
  3. 3. Decisions can be dauntinglife-changingscary.exhaustingexcitingan upgrade
  4. 4. Product decisions can be scary too.
  5. 5. We too made some scary decisions recently.
  6. 6. N View issue · Add to sprint 1.4-m1 Issue ADG-745 has been created. MOVED NEW IN PROGRESS REMOVED SUCCESS DEFAULT MOVED NEW IN PROGRESS REMOVED SUCCESS DEFAULT 123122 123122 123122 123122 123122 Alana AwareAlana Aware Buttons Forms Tables Tabs Dropdown button Dropdown button StatisticsReopenMoreCommentAssign to meAssignEdit ConfirmInvite usersSelect pluginsConfigure instance Gray background
  7. 7. Open in JIRA Open in JIRAOpen in JIRA Open in JIRAOpen in JIRA Open in JIRA RECENT PROJECTS SELECTED ALL PROJECTS Issue rewrites Issue Rewrite (IR) JIRA Core Bord (JCB) JIRA Core Board (JCB) Vertigo Incompatibilties (VI) Vertigo Incompatibilties (VI) JIRA Process Advancement (JPA) JIRA Process Advancement (JPA) JIRA OOME (JOOME) list name View issue • Add to next sprint START-85 has been created MM MM MM 99+122 99+2 12 67%Uploading… MOVED NEW IN PROGRESS REMOVED SUCCESS DEFAULT NEW IN PROGRESS REMOVED SUCCESS DEFAULT MOVED 26/11/2015 Due date November 2015 MON TUE WED THU FRI SATSUN 131 87 1514 2221 29 6 2 9 28 5 1 8 6 13 20 27 5 12 19 26 4 11 18 25 3 10 17 24 2 9 16 23 30 7 3 10 4 11 MOVED NEW IN PROGRESS REMOVED SUCCESS DEFAULT NEW IN PROGRESS REMOVED SUCCESS DEFAULT MOVED
  8. 8. Trust in our skills Our product decisions are made up of… Careful thought Research
  9. 9. Early signal testing
  10. 10. Agenda What it is Early signal methods Getting started
  11. 11. Agenda Early signal methods Getting started What it is
  12. 12. Agenda Early signal methods Getting started What it is
  13. 13. Improve itEnvision it Make it
  14. 14. Behavioral Attitudinal Qualitative (direct) Ethnographic field studies Usability lab studies Quantitative (indirect) Interviews Focus groups A/B testing Click analysis Concept testing Customer feedback Diary studies Card sorting Unmoderated studies Early signal testing Early signal testing
  15. 15. Gather quick, actionable feedback from customers EARLY SIGNAL TESTING
  16. 16. No worries You don’t need to be a research expert Why use it? Guidance Use information, not just opinions Direction Get cues, not binding answers Quick Don't need detailed mockups or working code
  17. 17. Methods
  18. 18. Aesthetics test Info test Expectation test Preference test Page design test Value test Task completion test Comprehension test Navigation test Usability test Concept test Click test DIFFERENT METHODS
  19. 19. Aesthetics test Info test Expectation test Preference test Page design test Value test Task completion test Comprehension test Navigation test Usability test Concept test Click test DIFFERENT METHODS
  20. 20. Aesthetics test
  21. 21. See if there’s any unintended reactions Assess which visual connects best Collect first impressions and emotional responses Aesthetics test
  22. 22. See if there’s any unintended reactions Assess which visual connects best Collect first impressions and emotional responses Aesthetics test
  23. 23. See if there’s any unintended reactions Assess which visual connects best Collect first impressions and emotional responses Aesthetics test
  24. 24. Preference test
  25. 25. Preference test Build off why users prefer one option See if there’s an easy winner Gather users' initial inclinations
  26. 26. Build off why users prefer one option See if there’s an easy winner Gather users' initial inclinations Preference test
  27. 27. Build off why users prefer one option See if there’s an easy winner Gather users' initial inclinations Preference test
  28. 28. Which option do you prefer and why? A site consists of user groups: admins, project admins, and users. Organizations work on projects in a certain area of interest. Users groups work on and complete these projects. User groups consist of users. Admins restrict each user by access or role. Projects work out specific plans or designs An organization consists of people. Organizations work on projects in a certain area of interest. Teams work on and complete these projects. Teams consist of members. Each member fills a role. Projects work out specific plans or designs. Option A Option B
  29. 29. Which option do you prefer and why? Option A Option B
  30. 30. Comprehension test
  31. 31. Ensure users don’t misunderstand information Ensure what you’ve written is clear Ensure users comprehend complicated information Comprehension test
  32. 32. Ensure users don’t misunderstand information Ensure what you’ve written is clear Ensure users comprehend complicated information Comprehension test
  33. 33. Ensure users don’t misunderstand information Ensure what you’ve written is clear Ensure users comprehend complicated information Comprehension test
  34. 34. Navigation test
  35. 35. See if alternate navigation paths are needed Assess usability, but don’t have a full flow yet Understand users’ natural wayfinding inclination Navigation test
  36. 36. See if alternate navigation paths are needed Assess usability, but don’t have a full flow yet Understand users’ natural wayfinding inclination Navigation test
  37. 37. See if alternate navigation paths are needed Assess usability, but don’t have a full flow yet Understand users’ natural wayfinding inclination Navigation test
  38. 38. Ash - we need a screenshot of the icons in product
  39. 39. Concept test
  40. 40. Learn what’s valuable, what’s not, what’s missing Compare your idea to users’ current process Explore early product and feature ideas Concept test
  41. 41. Learn what’s valuable, what’s not, what’s missing Compare your idea to users’ current process Explore early product and feature ideas Concept test
  42. 42. Learn what’s valuable, what’s not, what’s missing Compare your idea to users’ current process Explore early product and feature ideas Concept test
  43. 43. Aesthetics test Info test Expectation test Preference test Page design test Value test Task completion test Comprehension test Navigation test Usability test Concept test Click test DIFFERENT METHODS
  44. 44. Getting started
  45. 45. 5 quick & easy steps Step 1: Define your question & method Step 2: Figure out the “who” and the “how” Step 3: Create tasks, questions & design mocks Step 4: Test with people Step 5: Analyze
  46. 46. Step 1: Define your question & method
  47. 47. Does our intent for our new logos match how people will perceive them? What do people think of the new logos? NOT THIS THIS
  48. 48. Get your team involved Speaking with stakeholders before you run your research will ensure it hits the mark #ProTip
  49. 49. 5 quick & easy steps Step 1: Define your question & method Step 2: Figure out the “who” and the “how” Step 3: Create tasks, questions & design mocks Step 4: Test with people Step 5: Analyze
  50. 50. Step 2: Figure out the “who” and the “how”
  51. 51. Who?
  52. 52. How?
  53. 53. Unmoderated Users independently complete tasks, quick turnaround time Moderated Speak with users, gather rich feedback, ask follow-up questions
  54. 54. #ProTip Unmoderated toolkit Check out Usertesting.com, User Zoom, Loop11, Validately, and others
  55. 55. 5 quick & easy steps Step 1: Define your question & method Step 2: Figure out the “who” and the “how” Step 3: Create tasks, questions & design mocks Step 4: Test with people Step 5: Analyze
  56. 56. Step 3: Create tasks, questions & design mocks
  57. 57. Tasks & questions
  58. 58. Write scenarios Rather than just creating tasks, create one story that ties together all the tasks. #ProTip
  59. 59. Keep it simple Only design what you need and don’t overthink it. #ProTip
  60. 60. 5 quick & easy steps Step 1: Define your question & method Step 2: Figure out the “who” and the “how” Step 3: Create tasks, questions & design mocks Step 4: Test with people Step 5: Analyze
  61. 61. Step 4: Test with people
  62. 62. Don’t just listen It’s not just what users say, it’s what they do #ProTip
  63. 63. 5 quick & easy steps Step 1: Define your question & method Step 2: Figure out the “who” and the “how” Step 3: Create tasks, questions & design mocks Step 4: Test with people Step 5: Analyze
  64. 64. Step 5: Analyze
  65. 65. Task completion Time to taskUMUX-lite Time on task Expectation score Happy path clicks 20% Baseline New 100%+80% 74.81 77.90+3.09 1m : 36s 2m : 27s +51s 5.0 / 7.0 6.3 / 7.0 +3.1 6 3 100% Baseline New 100%0% 79.74 82.48+2.74 0m : 47s 1m : 00s +13s 4.6 / 7.0 5.0 / 7.0 +0.4 6 3 Experience Create a UX scorecard Metrics like time on task, task completion or user-reported metrics like ease of use are valuable comparison points. #ProTip
  66. 66. Task completion Time to taskUMUX-lite Time on task Expectation score Happy path clicks 20% Baseline New 100%+80% 74.81 77.90+3.09 1m : 36s 2m : 27s +51s 5.0 / 7.0 6.3 / 7.0 +3.1 6 3 100% Baseline New 100%0% 79.74 82.48+2.74 0m : 47s 1m : 00s +13s 4.6 / 7.0 5.0 / 7.0 +0.4 6 3 - Many decision points for users and a cognitive overload - General understanding of certain concepts in project create caused confusion for users i.e. project key Baseline - Help and guidance for first time users - Progressive disclosure of template information - Template information that means something to users New experience What we tested Experience Cohort Existing users We tested in cohorts of 10 existing JIRA users (admin and non-admin) per model. New users We tested in cohorts of 30 new users per model. Tasks: You now want to setup this new project. You need to make sure that in this new project you can plan, track and report all of your work. Show how you would setup this new project. We wanted to know if users can Easily create a project. Easily understand where they are. Easily comprehend and understand the differences between project templates. Create a project quickly (no friction points). UX Scorecard - Creating a project
  67. 67. 5 quick & easy steps Step 1: Question & method Step 2: Who & how Step 3: Mocks & tasks Step 4: Test Step 5: Analyze
  68. 68. Early signal testing helps inform your decision but it's not the decision. - ASH & BECKY :)
  69. 69. The next time you’re facing a tough decision…
  70. 70. Thank you! ASHLEIGH STERZENBACH | SENIOR DESIGNER | ATLASSIAN BECKY WHITE | SENIOR RESEARCHER | ATLASSIAN

×