© 2012 – All rights reserved Fluxx Ltd.
© 2012 – All rights reserved Fluxx Ltd.
Why this event?© 2012 – All rights reserved Fluxx Ltd.
The 5 ages of User Experience                                                       1. Total ignorance                    ...
What is ‘User Testing’ anyway?                                                       Has it evolved?                      ...
Designed a phone                                            that inspired a                                              g...
Designed a phone                                            that inspired a                                              g...
Designed a phone                                            that inspired a                                              g...
© 2012 – All rights reserved Fluxx Ltd.
“If I’d asked                                          people what they                                          wanted, t...
So our questions for tonight are:         • When should I place emphasis on customers’ opinions, and           when should...
Rajiv Arjan                      Paul Dawson   Gavin Gordon         City                             Fluxx         Mintel ...
Importance of user                                               testing                                                Ra...
Importance of user testing          Raj Arjan, City Interaction Lab
Agenda   •   Introduction of City Interaction Lab and Me   •   User Centered design process   •   Usability testing proces...
About Me   • Managing the interaction lab since 2008.   • Responsible for;       – Business development       – Managing U...
City University London            •   Courses Taught                 – MSc Human Centred Systems                 – MSc Cre...
City Interaction Lab   • Facility used for student     and commercial projects     in areas of;       – Usability       – ...
User Centered Design@rajarjan     Source : kitdigital - http://www.kitd.com/what-we-do/services/design/
Usability Testing Inputs               Wireframe         Live                                 Site                        ...
Usability Testing Process                           Morae Usability Testing Apps                           Tobii Studio – ...
Usability Testing Outputs   •   Usability problems   •   User opinions   •   Task completion times   •   Other Metrics (ey...
Paying attention to what users do@rajarjan
“If I had asked            people what they            wanted, they would            have said faster            horses.”@...
@rajarjan
Paying attention to how users react@rajarjan
Paying attention to how users react        • Movement        • Skin Conductance        • Temperature@rajarjan
Paying attention to how users react
Observations            Opinions        Emotional                            Response@rajarjan
Take Away’s   • User Testing is always important   • Observing what users do is often more     important than what they sa...
Thank you                                               Rajiv Arjan                                          City Interact...
Facts, not opinions                                          Paul Dawson                                           Fluxx L...
Fluxx Labs is our methodicalapproach to validating new productand service propositions, using lean        startup techniqu...
Why?In 15 years of designing and delivering digital products   big and small, we’ve seen a lot of great ideas and         ...
1 : BIG BETS                                             Don’t always pay off© 2012 – All rights reserved Fluxx Ltd.
2 : There is no shortage of bigideas in our client base                  labs
3 : Some people are prettyopinionated  And they often spell the end to a strong concept                          labs
http://www.youtube.com/watch?v=E1IarxtP4aA                     labs
labs
labs
insight      idea       feasibility                              design &                                               ma...
d                                                                                                              i    Easy t...
d                                                                                                              i    Easy t...
labs
The MVP is at the heart of Fluxx LabsIn ‘Lean Startup’ land, the fastest route to getting facts is tolaunch a ‘minimum via...
Dim and distant past                            Painful, but we got there….        Dragon’s Den           insight         ...
labs
He believed in it so muchbecause he generated facts byputting his product into thehands of potential customersThe toughnes...
Fortunately…We live in a world where you can make anything seemreal, to make the learnings as real as they can be.We set F...
labs
Basic InsightWe find it really hard to find a whiteboard thesize we want and think that other people willhave the same pro...
Insight leads to a value hypothesis that wecan begin to test“We think that people will value a premiumproduct that makes a...
1. Make a site that                                 calculates the cost and                                 puts customers...
The first MVP   (2 days)                labs
The 2 nd MVP        labs
The Lab’s LearningsThis is a viable, small, localbusiness.It would be a good idea to makethe whiteboards erasable!It is po...
The High Street BankA real life case study                         labs
Amongst others, we created oneexperiment designed to testsomething that had been debatedfor ages at the bank              ...
It was a thorny subject.It involved using customers’personal data to do something wethought they would like and value.    ...
But would they accept it, or would             they go nuts? We put it live in a few weeks in an MVP to several thousand r...
But before we did…                labs
It was usability tested                 It panned!Only   24% of people said they would do it                      labs
How many ACTUALLY did it in              real life?     96% of people just jumped right on in!                        labs
“What people say they will do and                 what they actually do is often               very different and can be g...
Thank you, and see you in the Lab                                          Paul Dawson - @poleydee                        ...
Mintel’s approach to                                               research                                              G...
A typical Mintel Oxygen report contains…       Market Drivers      Size of the Market and   Brands, Companies and         ...
Different elements of research       Desk research                                  Trade research     Company reports    ...
Desk research• Press reports – trade publications and national media• Regulatory/legislative developments• Gathering and a...
Trade research• Adds relevance and understanding, as well as a source of data• Trade interviewing and attending industry e...
Forecasting – Statistical modelling                                Mintel’s own market size     Analysis of relationships ...
Forecasting – Tackling uncertaintyThe Mintel fan chart The fan chart            Equity Release Schemes, 2011-16 illustrate...
Forecasting - Qualitative insightForecast of games consoles and games software sales by value, 2012-17                    ...
Consumer research planning• Deciding what to cover – insight from desk and trade research• Questionnaire design – collabor...
Qualboard:             1. To add context to rigorous quant data             2. To point to emergent trends             3. ...
Consumer research planning  Online                   Face-to-face           Specialist panel • Internet users          • N...
Bringing it all together       Desk research                                  Consumer research      Company reports      ...
Transforming information into actionable insight: What makes it work• Understanding/evolving the role of syndicated resear...
Updating our Model“To ensure Mintel remains timely”  Move away from the 2-year publishing cycle?      Modified approach to...
Communication“Building    relationships and tapping into client knowledge”  Making clients one of the first ports of call:...
Consistent and Insightful“Usage is high, but frequency is not high enough”   Driving frequent usage up:       Grounding In...
Thinking about your needs, why do you typically use Mintel?   When I have been given a brief for a project                ...
Thank you                                          Gavin Gordon                                             Mintel© 2012 –...
© 2012 – All rights reserved Fluxx Ltd.
The fluxx ‘Tested User’ Framework   Idea/Concept     Product/Service      Maturity Level      Partial/Fully       Develope...
The fluxx ‘Tested User’ Framework                                              Will people find the                       ...
The fluxx ‘Tested User’ Framework                                               MVP/Lean                                  ...
Where product is at an early concept stage, and users            are unaware of the test itself.         Questions Trying ...
An early stage concept, and users are aware they are         being asked for their opinions and input         Questions Tr...
Where product is at a later stage, but you don’t make          consumers aware that they are part of a test         Questi...
Where product is well developed and people know they            are being asked to test it         Questions Trying to Ans...
Intelligence and data on consumer activities, behaviours and            opinions in specific sectors, industries and marke...
Appendix                                              of common                                          Testing Technique...
Fluxx Rapid Start         • A multi-day multi-disciplinary hugely intensive exercise           unique to Fluxx, where busi...
Focus Groups         • Definition:                    – A focus group is a form of qualitative research in which a group o...
Usability Testing         • Definition:                    – Usability testing is a technique used in user-centered intera...
Ethnographic Research                    Ethnography is technically about observing a particular society or culture throug...
A/B Testing         • A/B testing or split testing compares the effectiveness of two           versions of a product/servi...
Heuristic Evaluation         • A Heuristic Evaluation, or Usability Audit, is an evaluation of           an interface by o...
‘Depths’         • Structured, one-on-one interviews help researchers learn           about users’ attitudes and beliefs s...
Card Sorting         • Card sorting is a useful tool to determine how users           categorize the information that will...
Electroencephalography (EEG)         • EEG involves measuring electrical activity in different parts of           the brai...
Minimum Viable Product (MVP)         • Minimum Viable Product is a strategy used for fast and           quantitative marke...
Multi-variate testing (MVT)         • Multivariate testing is a technique for testing hypotheses on           complex mult...
Prototype Testing         • Prototypes can be tested for aspects like design flaws and           ease of use, two things t...
Eye Tracking         • Eye tracking measures the point of gaze and eye movement         • It can detect the direction in w...
Quantitative Market Research         Market research is the systematic gathering and interpretation of information about  ...
About Mintel                     Leading global market intelligence company with more than 500 employees worldwide        ...
“What people say they will do and                 what they actually do is often               very different and can be g...
© 2012 – All rights reserved Fluxx Ltd.
Upcoming SlideShare
Loading in …5
×

Fluxx-The Tested User event-011112-main presentation

642 views
560 views

Published on

The main presentation from our "The Tested User" event held at Fluxx Studios on 1st November 2012.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
642
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Fluxx-The Tested User event-011112-main presentation

  1. 1. © 2012 – All rights reserved Fluxx Ltd.
  2. 2. © 2012 – All rights reserved Fluxx Ltd.
  3. 3. Why this event?© 2012 – All rights reserved Fluxx Ltd.
  4. 4. The 5 ages of User Experience 1. Total ignorance 2. Blind obedience (The Jakob years) 3. User-Centred Design & Usability 4. Let’s go crazy – multi-touch is here! 5. Anything goes, so long as users say it’s ok (AKA nobody gets fired for doing what customers ask you to do) And most recently: Let’s test everything. It will make our lives so much easier. We will never have to make a decision again.© 2012 – All rights reserved Fluxx Ltd.
  5. 5. What is ‘User Testing’ anyway? Has it evolved? Have we evolved our understanding and usage of it? Is it always the right thing to do?© 2012 – All rights reserved Fluxx Ltd.
  6. 6. Designed a phone that inspired a generation ? Re-invented a familiar household object for a new era Made the world feel like a smaller place Pioneered a focus on user testing© 2012 – All rights reserved Fluxx Ltd.
  7. 7. Designed a phone that inspired a generation Re-invented a familiar household object for a new era Made the world feel like a smaller place Pioneered a focus on user testing Henry Dreyfuss, 1904-1972© 2012 – All rights reserved Fluxx Ltd.
  8. 8. Designed a phone that inspired a generation Re-invented a familiar household object for a new era Made the world feel like a smaller place Pioneered a focus on user testing Henry Dreyfuss, 1904-1972© 2012 – All rights reserved Fluxx Ltd.
  9. 9. © 2012 – All rights reserved Fluxx Ltd.
  10. 10. “If I’d asked people what they wanted, they would have said faster horses” Henry Ford© 2012 – All rights reserved Fluxx Ltd.
  11. 11. So our questions for tonight are: • When should I place emphasis on customers’ opinions, and when should I put down the shutters and just show them when we’re ready? • What sorts of means do we have at our disposal for establishing what customers will and won’t do? • What is the future of validating our approaches with customers?© 2012 – All rights reserved Fluxx Ltd.
  12. 12. Rajiv Arjan Paul Dawson Gavin Gordon City Fluxx Mintel UK Interaction Lab© 2012 – All rights reserved Fluxx Ltd.
  13. 13. Importance of user testing Rajiv Arjan City Interaction Lab© 2012 – All rights reserved Fluxx Ltd.
  14. 14. Importance of user testing Raj Arjan, City Interaction Lab
  15. 15. Agenda • Introduction of City Interaction Lab and Me • User Centered design process • Usability testing process • Usability testing enhancements • Benefits of user testing • Questions@rajarjan
  16. 16. About Me • Managing the interaction lab since 2008. • Responsible for; – Business development – Managing UX projects – Supporting Academics/Students – Promoting Interaction Lab and HCID@rajarjan
  17. 17. City University London • Courses Taught – MSc Human Centred Systems – MSc Creativity Innovation and Leadership – Can be undertaken Part time/ Full time@rajarjan
  18. 18. City Interaction Lab • Facility used for student and commercial projects in areas of; – Usability – Accessibility – Interaction design – User research@rajarjan
  19. 19. User Centered Design@rajarjan Source : kitdigital - http://www.kitd.com/what-we-do/services/design/
  20. 20. Usability Testing Inputs Wireframe Live Site Powerpoint/Keyn Paper Prototype@rajarjan ote
  21. 21. Usability Testing Process Morae Usability Testing Apps Tobii Studio – eye tracking Silverback - for MAC OS@rajarjan
  22. 22. Usability Testing Outputs • Usability problems • User opinions • Task completion times • Other Metrics (eye tracking, mouse clicks, key presses)@rajarjan
  23. 23. Paying attention to what users do@rajarjan
  24. 24. “If I had asked people what they wanted, they would have said faster horses.”@rajarjan
  25. 25. @rajarjan
  26. 26. Paying attention to how users react@rajarjan
  27. 27. Paying attention to how users react • Movement • Skin Conductance • Temperature@rajarjan
  28. 28. Paying attention to how users react
  29. 29. Observations Opinions Emotional Response@rajarjan
  30. 30. Take Away’s • User Testing is always important • Observing what users do is often more important than what they say. • The user is not the designer. • Maximise the value of user testing by asking the right questions. • What are you trying to test? The interface or the concept?@rajarjan
  31. 31. Thank you Rajiv Arjan City Interaction Lab @rajarjan© 2012 – All rights reserved Fluxx Ltd.
  32. 32. Facts, not opinions Paul Dawson Fluxx Labs© 2012 – All rights reserved Fluxx Ltd.
  33. 33. Fluxx Labs is our methodicalapproach to validating new productand service propositions, using lean startup techniques. labs
  34. 34. Why?In 15 years of designing and delivering digital products big and small, we’ve seen a lot of great ideas and realised a few things. labs
  35. 35. 1 : BIG BETS Don’t always pay off© 2012 – All rights reserved Fluxx Ltd.
  36. 36. 2 : There is no shortage of bigideas in our client base labs
  37. 37. 3 : Some people are prettyopinionated And they often spell the end to a strong concept labs
  38. 38. http://www.youtube.com/watch?v=E1IarxtP4aA labs
  39. 39. labs
  40. 40. labs
  41. 41. insight idea feasibility design & manufacture distribution development© 2012 – All rights reserved Fluxx Ltd.
  42. 42. d i Easy to do, and no risk Starting to invest time & money Hard to do, and a lot of risk s t r i b u man t ufac i design & ture o development n feasibility insight Idea© 2012 – All rights reserved Fluxx Ltd.
  43. 43. d i Easy to do, and no risk Starting to invest time & money Hard to do, and a lot of risk s t r i b u man t ufac i design & ture o development n feasibility insight Idea© 2012 – All rights reserved Fluxx Ltd.
  44. 44. labs
  45. 45. The MVP is at the heart of Fluxx LabsIn ‘Lean Startup’ land, the fastest route to getting facts is tolaunch a ‘minimum viable product’Q: What is the absolute minimum we can do to see if peopleactually do value our product? labs
  46. 46. Dim and distant past Painful, but we got there…. Dragon’s Den insight Bankrupt factory People bought it concept Give it manufacture to the design & press development distribution Licensee Dodgy feasibility liquidated catches© 2012 – All rights reserved Fluxx Ltd.
  47. 47. labs
  48. 48. He believed in it so muchbecause he generated facts byputting his product into thehands of potential customersThe toughness of his journeywas testament to the fact that hisMVP could not be that minimal labs
  49. 49. Fortunately…We live in a world where you can make anything seemreal, to make the learnings as real as they can be.We set Fluxx Labs up to generate evidence-basedlearning, to help our clients get their ideas right andmake good decisions based on facts. labs
  50. 50. labs
  51. 51. Basic InsightWe find it really hard to find a whiteboard thesize we want and think that other people willhave the same problem. labs
  52. 52. Insight leads to a value hypothesis that wecan begin to test“We think that people will value a premiumproduct that makes and installs whiteboards to aprecise set of dimensions.” labs
  53. 53. 1. Make a site that calculates the cost and puts customers in touch with us. 2. Get a phone!If people value thisservice as much as • Volume of people we do. searching for it on Google • Requests for pricing • People who buy labs
  54. 54. The first MVP (2 days) labs
  55. 55. The 2 nd MVP labs
  56. 56. The Lab’s LearningsThis is a viable, small, localbusiness.It would be a good idea to makethe whiteboards erasable!It is possible in a week to proveout a digital & physicalbusiness in a lean way. labs
  57. 57. The High Street BankA real life case study labs
  58. 58. Amongst others, we created oneexperiment designed to testsomething that had been debatedfor ages at the bank labs
  59. 59. It was a thorny subject.It involved using customers’personal data to do something wethought they would like and value. labs
  60. 60. But would they accept it, or would they go nuts? We put it live in a few weeks in an MVP to several thousand real customers who didn’t know it was just an experiment. labs
  61. 61. But before we did… labs
  62. 62. It was usability tested It panned!Only 24% of people said they would do it labs
  63. 63. How many ACTUALLY did it in real life? 96% of people just jumped right on in! labs
  64. 64. “What people say they will do and what they actually do is often very different and can be greatly affected by their level of awareness of the task at hand”© 2012 – All rights reserved Fluxx Ltd.
  65. 65. Thank you, and see you in the Lab Paul Dawson - @poleydee embrace@fluxx.uk.com© 2012 – All rights reserved Fluxx Ltd.
  66. 66. Mintel’s approach to research Gavin Gordon Mintel© 2012 – All rights reserved Fluxx Ltd.
  67. 67. A typical Mintel Oxygen report contains… Market Drivers Size of the Market and Brands, Companies and Forecast Innovations Advertising and Distribution Trends Consumer Insight Marketing Activity 67 mintel.com
  68. 68. Different elements of research Desk research Trade research Company reports Trade input – formal and Press releases informal Media reports Legislative and regulatory Understanding a Industry events developments market Statistical forecasting Consumer research Mintel and analyst experience Mintel’s market size Mintel’s own consumer database research Other data ONS, Treasury, TGI data Bank of England etc. Qualitative research 68 mintel.com Trade
  69. 69. Desk research• Press reports – trade publications and national media• Regulatory/legislative developments• Gathering and analysing reliable and trusted data sources• Typical sources include ONS, Ofcom, trade associations and other industry data• Company reports and press releases 69 mintel.com
  70. 70. Trade research• Adds relevance and understanding, as well as a source of data• Trade interviewing and attending industry events• How is the market performing? Checking market size and our forecasts.• What challenges and issues is the market facing?• Future trends, innovations, possible developments• What consumer insight do industry players want? 70 mintel.com
  71. 71. Forecasting – Statistical modelling Mintel’s own market size Analysis of relationships 5-year forecasts based on database is supplemented between, Mintel market an advanced statistical by macro- and socio- sizes and a selection of technique known as economic data sourced key economic and ‘multivariate time series from organisations such as demographic factors to auto-regression’ using the the Office for National identify those which have statistical software Statistics, HM Treasury and the most influence on the package SPSS the Bank of England. market. 71 mintel.com
  72. 72. Forecasting – Tackling uncertaintyThe Mintel fan chart The fan chart Equity Release Schemes, 2011-16 illustrates the probability of various outcomes for the market value/volume over the next five years. At a 95% confidence interval, we are saying that 95 out of 100 times, the forecast will fall within these outer limits. 72 mintel.com
  73. 73. Forecasting - Qualitative insightForecast of games consoles and games software sales by value, 2012-17 Slimline, cheaper 4,000 models Sony/ Best case (£m) 3,792 3,500 Microsoft next gen 3,000 Market value (£m) Mintel forecast (£m) 2,811 2,500 Wii U Worst case (£m) 2,000 2,074 1,500 (£m) 1,651 1,000 Confidence intervals 500 95% 90% 0 0 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 70% Est. 50% Actual Forecast 73 mintel.com
  74. 74. Consumer research planning• Deciding what to cover – insight from desk and trade research• Questionnaire design – collaborate with Research Services Team• Looking at ways to provide interesting analysis• Qualitative consumer research74 mintel.com
  75. 75. Qualboard: 1. To add context to rigorous quant data 2. To point to emergent trends 3. To provoke additional quant analysis75 mintel.com
  76. 76. Consumer research planning Online Face-to-face Specialist panel • Internet users • Nationally • IFA research representative • Increasingly popular • HNW groups form of research • Reaching those (Affluent Banking or groups not engaged SIPPs) • More detailed with the internet exploration of issues • Youth surveys • Annuities / Equity Release Schemes • Travel Agents76 mintel.com
  77. 77. Bringing it all together Desk research Consumer research Company reports Mintel’s own consumer Press releases research Media reports TGI data Legislative and regulatory Understanding a developments market Qualitative research Statistical Forecasting Trade research Mintel and analyst Mintel’s market size experience database Trade input – formal and informal Other data ONS, Treasury, Bank of England etc. Industry events 77 mintel.com
  78. 78. Transforming information into actionable insight: What makes it work• Understanding/evolving the role of syndicated research• Faster• More accessible• More consultative• More disruptive. 78 mintel.com
  79. 79. Updating our Model“To ensure Mintel remains timely” Move away from the 2-year publishing cycle? Modified approach to report writing and publishing? Analyst updates (market data/company information/consumer) Auto updates (background data and existing databases) “Found relevant report easily but not Consistency with US? recent enough” “Theres a real lag between some reports being updated” “The reports were too out-of-date - I wanted something more current and relevant” 79 mintel.com
  80. 80. Communication“Building relationships and tapping into client knowledge” Making clients one of the first ports of call: Modifications to the current system? Developing the analyst’s role to become more than a picture on the website Listening to clients on a regular, on-going basis “More consultation Pro-active development led by research. with clients for their specific needs” ‘I didn’t realise I could talk to the analysts’ – 46% 80 mintel.com
  81. 81. Consistent and Insightful“Usage is high, but frequency is not high enough” Driving frequent usage up: Grounding Inspire in more ‘facts’ and making What’s Hot to be more than just a visual treat We’re guilty of ‘hiding away’ insight (reports too long, too much text) More flexible approach to report scheduling (hot topic reports) Consistent approach across Oxygen (both within UK and UK/US). “Keep the same level of detail and analysis but just make it clearer and easier to find what you are looking for” “More raw data “Opinions/viewpoints always where possible. interesting/thought provoking” Let me make decisions!” 81 mintel.com
  82. 82. Thinking about your needs, why do you typically use Mintel? When I have been given a brief for a project 67.6 As evidence to back up our ideas 63.0 To prep for a client meeting/ presentations 51.5 For a strategy meeting 40.4 When I need a quick answer 30.2When somebody recommends a report internally 21.2 When I cannot find data elsewhere 20.1 Other 3.1 82 Base: 646 Mintel Oxygen clients mintel.com
  83. 83. Thank you Gavin Gordon Mintel© 2012 – All rights reserved Fluxx Ltd.
  84. 84. © 2012 – All rights reserved Fluxx Ltd.
  85. 85. The fluxx ‘Tested User’ Framework Idea/Concept Product/Service Maturity Level Partial/Fully Developed Is the User Unaware Aware aware of the test?© 2012 – All rights reserved Fluxx Ltd.
  86. 86. The fluxx ‘Tested User’ Framework Will people find the What are people’s opinions, Idea/Concept product/service of value? expectations and ideas? Product/Service Market Maturity Level Intelligence How well does our Partial/Fully Which is the best way to user interface implement this? Developed support our product? Is the User Unaware Aware aware of the test?© 2012 – All rights reserved Fluxx Ltd.
  87. 87. The fluxx ‘Tested User’ Framework MVP/Lean Market Research Idea/Concept e.g. Fluxx Labs companies Product/Service Market Intelligence Maturity Level e.g. Mintel Partial/Fully Product Development Usability Testing e.g. Fluxx e.g. City Interaction Lab Developed Is the User Unaware Aware aware of the test?© 2012 – All rights reserved Fluxx Ltd.
  88. 88. Where product is at an early concept stage, and users are unaware of the test itself. Questions Trying to Answer: Will people find the product of value? Lab-like When to use: validation When there is a perceived barrier or high degree of uncertainty about willingness of consumers to adopt Where the product paradigm is substantially different from the norm Technique(s) Used: Minimum Viable Product (MVP)© 2012 – All rights reserved Fluxx Ltd.
  89. 89. An early stage concept, and users are aware they are being asked for their opinions and input Questions Trying to Answer: What are people’s opinions, expectations and ideas? Opinions When to use: & ideas When there is a wide range of possible concepts that you want to understand more Where you want early stage insight into how specific consumer groups behave and think You want consumer ideas and input Technique(s) Used: Focus Groups, Market Research© 2012 – All rights reserved Fluxx Ltd.
  90. 90. Where product is at a later stage, but you don’t make consumers aware that they are part of a test Question Trying to Answer: Which is the optimal way to implement this? When to use: Where there are clearly defined potential solutions with no strong supporting evidence as to which is the A/B best approach & MVT Technique(s) Used: A/B Testing, Multi-variate testing (MVT)© 2012 – All rights reserved Fluxx Ltd.
  91. 91. Where product is well developed and people know they are being asked to test it Questions Trying to Answer: How well does our user interface support our product? Can we make it better? When to use: When you are clear on your proposed design solution and want to uncover any User potential barriers to adoption Technique(s) Used: Testing Usability Testing, prototype testing, eye tracking© 2012 – All rights reserved Fluxx Ltd.
  92. 92. Intelligence and data on consumer activities, behaviours and opinions in specific sectors, industries and markets Questions Trying to Answer: Is this the right thing to do? Is there a market for what we want to do? When to use: When you need clarity around target Market segments or market To provide external evidence of key Intelligence adoption assumptions To inform and validate the business case Technique(s) Used: Surveys, Polls, Intelligence Gathering© 2012 – All rights reserved Fluxx Ltd.
  93. 93. Appendix of common Testing Techniques© 2012 – All rights reserved Fluxx Ltd.
  94. 94. Fluxx Rapid Start • A multi-day multi-disciplinary hugely intensive exercise unique to Fluxx, where business teams are taken from early stage ideas to launchable products ready for circulating to consumers • Typically this is done in two days and key side effects include – Rebuilding teams’ belief in their ability to deliver – Promotes collaborative design and ways of working – Incorporating the voice of the customer into the build process – Most rapid way to get something into the hands of customers© 2012 – All rights reserved Fluxx Ltd.
  95. 95. Focus Groups • Definition: – A focus group is a form of qualitative research in which a group of people are asked about their perceptions, opinions, beliefs, and attitudes towards a product, service, concept, advertisement, idea, or packaging. Questions are asked in an interactive group setting where participants are free to talk with other group members.© 2012 – All rights reserved Fluxx Ltd.
  96. 96. Usability Testing • Definition: – Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. This can be seen as an irreplaceable usability practice, since it gives direct input on how real users use the system.© 2012 – All rights reserved Fluxx Ltd.
  97. 97. Ethnographic Research Ethnography is technically about observing a particular society or culture through a passive participation in it, and without impacting on the society itself, and this is still the strictest definition of it. However, nowadays, it is more widely used to categorise techniques that are about trying to understand how people live their lives. Typically this involves visiting people in their home or work environments using a series of highly practical questions designed to uncover what is happening and why, Their goal is to see people’s behaviour on their terms, not in an artificial way. The latter approach is often confused or used interfchangeably to mean what is known as a ‘contextual inquiry’ or ‘contextual analysis’ , the definition of which is observivng someone’s interactions with your product in their usual context or surroundings.© 2012 – All rights reserved Fluxx Ltd.
  98. 98. A/B Testing • A/B testing or split testing compares the effectiveness of two versions of a product/service in order to discover which has better response rate or better sales conversion rate • Two alternate designs or approaches are randomly served to people, and the resultant outcomes of those differently served customer bases compared to each other, and a control group.© 2012 – All rights reserved Fluxx Ltd.
  99. 99. Heuristic Evaluation • A Heuristic Evaluation, or Usability Audit, is an evaluation of an interface by one or more Human Factors experts. Evaluators measure the usability, efficiency, and effectiveness of the interface based on 10 usability heuristics. • There are many heuristics available, but a popular one was created by Jakob Nielsen in 1994. Many argue that this heuristic is now outdated by modern interaction techniques, and takes no account of user attitudes.© 2012 – All rights reserved Fluxx Ltd.
  100. 100. ‘Depths’ • Structured, one-on-one interviews help researchers learn about users’ attitudes and beliefs surrounding a website or application and specific tasks that the website or application supports. In order to prevent the researcher from introducing bias and to ensure that every participant is asked the same questions, the researcher prepares a Discussion Guide, which is a list of questions that will be asked of participants in a particular order.© 2012 – All rights reserved Fluxx Ltd.
  101. 101. Card Sorting • Card sorting is a useful tool to determine how users categorize the information that will appear on a website. The name “card sort” comes from the original exercise, which used index cards with a word or phrase written on one side, and in some cases, a definition or additional information on the reverse side© 2012 – All rights reserved Fluxx Ltd.
  102. 102. Electroencephalography (EEG) • EEG involves measuring electrical activity in different parts of the brain in response to certain stimuli • It can now used for the assessment of user experiences to see how people react at an emotional level to stimulus or a particular experience • Detailed and specific design recommendations can be made by recording reactions at different stages of interaction with a product/service© 2012 – All rights reserved Fluxx Ltd.
  103. 103. Minimum Viable Product (MVP) • Minimum Viable Product is a strategy used for fast and quantitative market testing of a product or product feature • The product is typically deployed to a subset of possible customers, such as early adopters that are thought to be more forgiving, more likely to give feedback, and able to grasp a product vision from an early prototype or marketing information • This is the technique employed by the Fluxx Labs method© 2012 – All rights reserved Fluxx Ltd.
  104. 104. Multi-variate testing (MVT) • Multivariate testing is a technique for testing hypotheses on complex multi-variable systems, especially used in testing market perceptions • It is a process by which more than one component of a product/service may be tested in a live environment • When automated it can be very powerful in finding the optimal combination of multiple component parts© 2012 – All rights reserved Fluxx Ltd.
  105. 105. Prototype Testing • Prototypes can be tested for aspects like design flaws and ease of use, two things that are critical if your product is going to be a success • You need to make sure everything works the way it should - and that your customers can figure out how to make it work too© 2012 – All rights reserved Fluxx Ltd.
  106. 106. Eye Tracking • Eye tracking measures the point of gaze and eye movement • It can detect the direction in which users are looking and the duration for which they are focusing on a specific part of a product e.g. web page • This is helpful knowledge if you are trying to find out if users are actively focusing on particular content on your page or if they miss it all together© 2012 – All rights reserved Fluxx Ltd.
  107. 107. Quantitative Market Research Market research is the systematic gathering and interpretation of information about individuals or organisations using the statistical and analytical methods and techniques of the applied sciences to gain insight or support decision making For example: • Mass participation surveys – Usually aimed at specifically targeted segments. This type of research relies on volume of participants to give its results significance. Questions usually have binary or multiple choice answers. • Omnibus Survey: – A regular survey designed to determine what the general population thinks on a range of topics and issues. Brands can pay for questions on this survey alongside the other questions. Thought to have derived from the phrase “The Man on the Clapham Omnibus’ which was used as a euphemism for ‘the common man’.© 2012 – All rights reserved Fluxx Ltd.
  108. 108. About Mintel Leading global market intelligence company with more than 500 employees worldwide Guide for leaders in marketing, product development and research for more than 35 years Clients rely on our authoritative offering to drive strategic decisions What we do Provide fresh insight into your marketplace and across industries Identify opportunities for profit and growth Formulate insights into consumer behavior, product innovation and competitive marketing strategies Create high-quality data, evocative analysis and tailored consulting© 2012 – All rights reserved Fluxx Ltd.
  109. 109. “What people say they will do and what they actually do is often very different and can be greatly affected by their level of awareness of the task at hand”© 2012 – All rights reserved Fluxx Ltd.
  110. 110. © 2012 – All rights reserved Fluxx Ltd.

×