Recommended
More Related Content
What's hot
Similar to Measuring Intuitiveness and Emotion in User Experience
Similar to Measuring Intuitiveness and Emotion in User Experience (20)
Recently uploaded
Recently uploaded (20)
Measuring Intuitiveness and Emotion in User Experience
- 1. Building on the ! Usability Study! Two Explorations on How to! Better Understand an Interface Anshu Agarwal Madhu Prabaker Usability Analysts HCI International 2009 San Diego, California
- 2. Traditional Usability Measure User Experience
- 3. Traditional ??? Usability Measure Must be Valuable & Practical User Experience
- 4. Traditional Usability Measure Intuitiveness User Experience
- 5. Emotion Traditional Usability Measure Intuitiveness User Experience
- 6. Emotion Traditional Usability Measure Intuitiveness User Experience
- 8. What does it mean to be Intuitive? “Easy to use and understand” “A technical system is, in the context of a certain task, intuitively usable while the particular user is able to interact effectively, not consciously using previous knowledge” ‐ [Naumann et al]
- 9. Project Goals" Measuring Intuitiveness Traditional Usability Measure Intuitiveness User Experience
- 10. Measuring Novice Performance on an Interface
- 11. Measuring Expert Performance on an Interface
- 12. Combining Expert and Novice Performance “… interact effectively, not consciously using previous knowledge.”
- 13. Combining Expert and Novice Performance An intuitive interface can be thought of as one that “achieves novice task performance as close as possible to expert task performance”
- 14. Combining Expert and Novice Performance
- 15. Combining Expert and Novice Performance
- 17. Project Goal" Understanding Emotional Impact Emotion Traditional Usability Measure User Experience
- 18. Existing Ways to Measure Emotion" The Common Tools Physiological Verbal Non‐Verbal
- 19. Our Approach" Combine the Best from Both Worlds Physiological Verbal Non‐Verbal
- 20. Our Verbal Measure" PAD Semantic Differential Scale PAD Dimension Bipolar Pairs Annoyed ‐ Pleased Unsatisfied ‐ Satisfied Pleasure, Arousal, and Pleasure Despairing ‐ Hopefully Dominance (PAD) Tense ‐ Relaxed Semantic Differential Scale Friendly ‐ Unfriendly Relaxed ‐ Stimulated [Mehrabian & Russell] Calm ‐ Excited Arousal Sleepy – Wide Awake Unaroused ‐ Aroused Controlled ‐ Controlling Influenced ‐ Influential Dominance Submissive ‐ Dominant Guided ‐ Autonomous
- 21. Our Non-Verbal Measure" Emo-Card + Arousal Excited Unpleasant Excited Pleasant ‐ Pleasure + Pleasure Calm Unpleasant Calm Pleasant ‐ Arousal
- 22. Validating the Technique" Empirical Study Design 22 Participants (13 male, 9 female) Interface A Interface B 7 Tasks (Randomized) Standard Measures Emotional Measures Time on Task Emocard # of Errors PAD Scale Post‐Task Interview
- 23. Validating the Technique" Empirical Study Results Interface A Interface B Neither Time on Task nor Number of Errors were significantly different between Interface A and Interface B PAD Scale and Emocard showed some clear differences in how users experienced the interfaces
- 24. Validating the Technique" Empirical Study Results Interface A Interface B Neither Time on Task nor Number of Errors were significantly different between Interface A and Interface B PAD Scale and Emocard showed some clear differences in how users experienced the CRM Systems
- 25. Validating the Technique" Empirical Study Results Interface A Interface B Neither Time on Task nor Number of Errors were significantly different between Interface A and Interface B PAD Scale and Emocard showed some clear differences in how users experienced the interfaces
- 26. Conclusion Emotion Traditional Usability Measure Intuitiveness User Experience
- 27. Conclusion Traditional ??? What are you missing that’s Usability Measure Valuable & Easy User Experience
- 28. Anshu Agarwal aagarwal@salesforce.com Madhu Prabaker mprabaker@salesforce.com Agarwal, A. and Meyer, A. 2009. Beyond usability: evaluating emotional response as an integral part of the user experience. In Proceedings of the 27th international Conference Extended Abstracts on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009).
Editor's Notes
- I’ll be presenting some exploratory work that we call “Building on the Usability Study: Two Explorations on How to Better Understand an Interface”. This work was done by my colleague AnshuAgarwaland myselfat salesforce.com. Since salesforce.com makes enterprise software, I should emphasize that this is a talk really aimed at industry application of usability evaluations.
- Both the explorations I’m going to briefly talk about came from our assumption that were important aspects of the User Experience that we were missing through our reliance on more traditional usability assessments. [click]I think we can all agree that Usability is only one facet of our product’s user experience. So the question for us was, what are we missing?
- What we wanted to do was not just investigate all other aspects of the overall user experience, but focus specifically those that shared two important characteristics: 1) they had to be valuable (i.e. it needed to produce insights that would lead to a better product) 2) they had to be practical – the analysis had to be able to be performed at the speed of industry, which was an especially important constraint because we are an agile company with ~3 month release cycles.
- In particular, we were interested in furthering our understanding of our product’s intuitiveness
- … and emotional impact
- However, since I have 15 minutes, I’m giving you the Cliff Notes version…Leaving out prior literatureGoing light on the empirical validationMainly just describing our techniques and findings
- Why is “Intuitive” Important in HCI? - Well, the word “intuitive” is a term that has become increasingly common among user experience professionals, technology journalists, and everyday consumers. -It is used in a manner that suggests it is, at most, a requisite for a good user experience and, at least, a strongly desired characteristic.
- So what does “intuitive” really mean? -The more common definition provided by Oxford American Dictionary is... - Naumann et al did a more formal analysis of the term and came to define it as follows…It’s this 2nd definition that will be more important when deriving our measure…
- Create a technique for “easily” measuring intuitivenessValidate that it leads to design insights that provide real benefits over our standard usability measures
- To build the definition of intuitive, I’m going to start with understanding novice user performance. By measuring the average task completion time across a group of representative, novice users, we can understand how quickly these new users can complete a task using a particular design.To provide more meaning to these task time numbers, we often visualize the task times across a set of tasks, as shown on the page.In addition to gaining understanding of how quickly a new user can perform a task using our product, we are more grounded in that we can make comparisons between tasks; [click] for example we can say, “it took an average novice user x seconds more to complete Task 3 than Task 4”. Although this is a more meaningful statement it is often difficult to understand how long a novice user should take on the task.
- If we turn our attention to Expert user performance, we achieve an even more complete understanding of our interface…As a practical note we could find experts, but they are often hard to find for a new interface, or train novices to be experts, but that’s very expensive– fortunately, we can estimate their performance using a human performance modeling method such as Keystroke-Level Modeling.If we plot these results (in this case, simply shown on top of the novice graph), we can see how long it is expected that an expert can complete a set of tasks using an interface. Another way to look at this is that these values represent the efficiency limit of a particular design.Using this we can make statements about the minimum task times imposed by the design; [click] for example we can say, “we expect that Task 5 will take at minimum x seconds”.
- So what does measuring novice and expert performance have to do with Intuitiveness? Well, earlier I mentioned the more formal definition of what it means for something to be intuitive…[click] Mapping these two on each other yields a more measurable definition – “an intuitive interface is one that achieves novice task performance as close as possible to expert task performance”. When an expert is using the system, they are not consciously thinking about how to use the system, rather, how to solve the task at hand. In this way, the closer the novice user performance resembles expert performance, the more intuitive the interface can be regarded.
- So what does measuring novice and expert performance have to do with Intuitiveness? Well, earlier I mentioned the more formal definition of what it means for something to be intuitive…[click] Mapping these two on each other yields a more measurable definition – “an intuitive interface is one that achieves novice task performance as close as possible to expert task performance”. When an expert is using the system, they are not consciously thinking about how to use the system, rather, how to solve the task at hand. In this way, the closer the novice user performance resembles expert performance, the more intuitive the interface can be regarded.
- This is a more meaningful metric than the novice or expert measures alone because it enables us to make statements like, “Novice users took x seconds longer on Task 8 than our design called for.” It’s important to not underestimate how much more powerful this statement is in driving design changes than the ones we were previously able to make just based on independent analysis of novice or expert performance data. It allows us to more explicitly recognize and dissociate the limits imposed by the design (the efficiency limit) from the observed performance data.
- Additionally, because this visualization quantitatively takes into account the inherent difficulty differences between tasks, it enables us to notice phenomenon that’s hidden in the novice performance data. [click] For example, although it initially appeared that Tasks 2, 5, 8, and 9 may be problematic because of their relatively long task performance times, [click] Tasks2, 8, and 9 are the ones that really demand attention – the design actually performed well for Task 5.We utilized this technique on a study where we compared two equivalent interfaces, and we found that this technique did indeed provide some further insights and didn’t take that much additional effort to perform. I’ll direct you to the paper for more information on that.
- Why is “Emotion” Important in HCI?Tied to human performance and may impact perceived usabilityHappy people solve problems better compared to those in a neutral or negative emotional state (Norman, 2002).As well as impact problem solving strategiesabout a product, they also effect purchase intention and consumption
- Develop an emotion measure that would be quick to utilize, easy to understand, deployable remotely, and easy to incorporate into an empirical studyValidate that it captured important aspects of the User Experience that we were missing using our traditional usability measuresI should also mention that Anshu really led this effort, so I’ll do my best to represent her work…
- Physiological: Sensors on the body: EEG orskin conductionImpractical in a software company and difficult to perform remotelyVerbal: Self Reporting (LikertScale)While it is more well-validated and traditional in the field, it only captures conscious emotional states and is often lengthly – which places users outside the immediacy of the emotionNon-Verbal: Visual Representations of EmotionWhile, regarded are more “fuzzy”, it captures non-conscious emotional states and has been shown to work cross-culturally
- Our approach was to combine a well-validated verbal scale with an interesting non-verbal one.
- For the verbal component, we chose to utilize a Pleasure Arousal and Dominance Semantic Differential Scale. This scale included a set of bipolar adjective pairs for each dimension that were rated along a nine point scale.
- We selected the Emocard tool by Desmet for the nonverbal component. This tool plots emotions along two dimensions: arousal and pleasure, [click] which produces four quadrants. [click] Sixteen cartoon-like faces, half male and half female, are arranged along these dimensions. each representing distinct emotions.
- Employed a between-subjects design22 Participants shown one of two comparable interfaces and asked to perform 7 standard tasks using themFor each task we collected standard Usability metrics (time on task, errors) and we employed theEmo-card measure, then Pleasure, Arousal, and Dominance scale, then conducted a post-task interview.
- Just to give you one example, here you can see a radar graph that summarizes how users responded using the emo card scale. For one task, Interface A has a much more profound Pleasant affect than Interface B, which is pretty spread. In looking at verbal responses users provided we found rich qualitative evidence that supported these differences in emotional impact.
- If we had utilized only the usability metrics as measures of user experience we would have concluded that the quality of the user experience for both interfaces were nearly identical. This conclusion, however, would have been incorrect, and, at the very least, incomplete.
- Studying dimensions of the user experience not directly measured through the traditional usability study yielded significant insight into our understanding of our users’ experience with marginal additional effort.The research efforts discussed here were only initial exploratory studies that merit further research.The intuitiveness measure still demands more empirical testing to validate its ongoing value and accuracy.Now that we can assess differences in the emotional response, we need to further understand how this can lead to specific design changes.
- In the end, we will continue to look for practical ways to further our understanding of our users but we hope toencouraging industry practitioners to extend their everyday usability research in search of greater insights.