Laddering with Young Children in User eXperience Evaluations: Theoretical Groundings and a Practical Case  Vero Vanden Abe...
example <ul><li>WHY? </li></ul><ul><li>  </li></ul><ul><li>Immersion / Fantasy  [V] </li></ul><ul><li>“ it’s like I become...
Testing these findings... |a case study|
|Introduction to the case|
|Research Questions| <ul><li>1   consistent preference?  </li></ul><ul><li>2   full ladders A-C-V? </li></ul><ul><li>3   N...
|Method| <ul><li>46   preschoolers  </li></ul><ul><li>Average  Age  about 5 </li></ul><ul><ul><li>2 to 7 years </li></ul><...
|Procedure| <ul><li>Explorative game play  x   3 (counterbalancing) </li></ul><ul><li>Preference distinction  (ranking) </...
|Procedure| <ul><li>Example preference distinction via ranking </li></ul>
|Results| <ul><li>1   Consistent preference? </li></ul><ul><li>Yes!  Consistent preferences between  answers on preference...
|Results| <ul><li>2   Full ladders A-C-V? </li></ul><ul><li>No  Values </li></ul><ul><li>9  (/46) : preferences >  [stop] ...
|Results| <ul><li>Example preferences > attributes >  [stop] </li></ul>
|Results| <ul><li>3   Number of elements - ladders? </li></ul><ul><li>Average= 1.72 ladder; 2.81 elements </li></ul><ul><l...
|Discussion| <ul><li>Laddering in HCI </li></ul><ul><ul><li>How many elements do you need? </li></ul></ul><ul><ul><li>Stri...
Vero Vanden Abeele [email_address] Bieke Zaman [email_address]
Upcoming SlideShare
Loading in …5

Laddering with young children in User eXperience evaluations


Published on

presentation at IDC 2010 conference on Interaction Design and Children in Barcelona, Spain

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Laddering = Special interview method (attribute eliciations + laddering) Data analysis method (qualitative + quantitative) =&gt; Qualitative analysis Derive core elements =&gt; Quantitative Enter the ladders in a score matrix Count the number of links between elements in the implication matrix Graph these dominant links in a hierarchical value map (HVM) a on line tool to aid with the quantitative analysis, Entering elements into the Score Matrix - Constructing the Implication Matrix Drawing the Hierachical Value Map (HVM) Origins: - means-end theory: product choice ~belief in attributes , having consequences, serving values
  • Wii Maa game and interface Kracht: preferences + reasons + suggesties voor verbeteringen Preferences: preference-consumption differences technique or ‘ranking’ A-C-V sequence or ladder.
  • ’ behavioural and cognitive characteristics ~Preoperational phase - Jean Piaget ~two to seven years old. Language : sufficient level for the adequate use of words to describe actions or represent objects [8,32]. ability to talk about experiences in general and media or technology experiences in particular, skills to think and talk beyond the here and now. This implies that they do not need the object or the experience to be recent and apparent to be able to discuss these - ability to deal with the complexities of open-ended questions Pragmatics “ children’s practical ability to use language to communicate with others in a variety of social contexts” BUT condition: low-demanding situations - in which they can see the interviewer’s reactions or rely on typical conversational aids as gestures, verbal prompts or cues and objects to talk about best = are given tasks that are simplified and relevant to their everyday lives ~technologies evaluated are meaningful objects used in a familiar context-of-use.
  • Ability to infer pscychological causes Preference for internal or external causes Tendency to use a particular type of cause because of its salience, simplicity and accessibility!! Opmerking: preschoolers kunnen en doen ook vaak meer dan 1 cause geven. Preschoolers kunnen meer bepaald twee pieces of information combineren en hen zelfs een verschillend gewicht geven! Spreken over person versus situation Eerder dan internal versus external Ability _________________________________________________________________________________ Rudimentary awareness of mental states and their understanding of psychological causes of behaviour -1. growing ability to speak about their internal, mental states not limited to their own states but also states they are not actually experiencing or mental states of others not limited to the immediate, concrete situation (past present future) - 2. Growing ability to think of events and behaviours in terms of causal relations - refer to cold aspects of mind such as thought, memory, attention and perception - hot aspects such as emotions, motivation and personality Often not realized: young children’s causal expressions of young children are even reflected upon things that they intend to do next instead of things that they have just experienced. Belangrijk model p 278: beter te spreken over person versus situation dichotomy dan internal versus external De psychological causes of someone else (their preferences etc) belong to situation and thus external! Preference___________________________________________________________________________________________ No such thing as one general preference for internal versus external causes Variables that influence the types of attribution made (internal versus external) … whose behaviour they are considering: A) self versus others (for own or object behaviours: rather situational attributions, esp. for negative behaviours of the self) versus other’s behaviours (internal causes) B) teachers and peers versus parents (peers or teachers: rather external attri) versus parents (rather internal attr) whether they consider nonsocial behaviours (entity bias more likely) versus prosocial behaviours salience of a cause salient stimuli are considered causal (doen ook volwassenen domain specifity see further valence of the behaviour positive versus negatives (more likely to make internal attributions concerning positive behaviours) complexity of the information net zoals volw, als men te veel info krijgt en het wordt te complex, dan is men eerder geneigd om gewoon een beschrijving te geven van het voordehandliggende dan te zoeken naar moeilijke causale redenen. type of causes, eg situation based they are more likely to attribute entity causes than situation-environment causes. eg person based more liley to attribute pscychological states (temporal, inentions, motive,s, emotions) than stable traits (physiological states) Cf. p. 278 Als er bias optreedt dan is het bias toward external causes in two forms, depending upon the domain !!! (e.g. choosing toys, helping others, obeying adults, achieving acadmeically) : Actor – observer effect: other’s behavior is explained in terms of internal attributions opinions, likes and needs 272 attribute intentions as causality (unless there is compelling evidence to the contrary Thus rather social causal reasoning (in terms of needs, intentions and motives) than reasoning about the physical world - is actually a preference of temporary states over stable states ie characteristic of person (the first are often more salient , the latter are not understood by children to be stable) - compared to adults their understanding of psychological causes (eg intention) is not that abstract and ‘truly psychological’ as that of adults (dit zijn dan eignl ‘external ‘ vanuit perspectief kind want psychological causes of other people) 2. Entity bias -but in a laddering study and more generally in play material choice situations or considering nonsocial behaviors (e.g. liking a toy) you ask to make a choice between games and the wording of the questions “what is it you like about the game” emphasizes the characteristics of the pictures, thus directing away the attention from the person making the choice!!! Laddering set up thus favours external reasons over internal! This is called entity bias !! Choosing an entity Wording and explicitly relating to the entity The object’s characteristics are more stable than those of the person/situation. The child may feel more certain about the first, it is also less difficult to understand The attractiveness of the toy are perceived as a characteristic of the/inherent to the toy and not as something seen through the eyes of the person. (dit mechanisme wordt generative transmission cause genoemd, children do not understand that the mind mediates and thus may interpret distort … the perception of an object) as such in choosing between play materials it SEEMS that children prefer external to internal reasons Implications for laddering: 1. external reasons ~attributes ~ knowledge needed for product improvements cf. start: ranking, perceived differences in terms of product differences  focus on product cf. why question specification ‘why do you like that game’  focus on product 2. when probing you want to reach further than attribute related explanations But when evaluating the self’s characteristics they do so often in a concrete, observable and often exaggerated positive way (“I am happy when I play that game”, “I am the best in playing that game”) ~ no revelation of general needs but instead context dependent needs
  • What is UX? ISO “ a person’s perceptions and responses that result from the use or anticipated use of a product, system or service” What kind of responses? &lt; Hassenzahl Judgements *** Emotions Behaviours
  • Judgment? Affect driven evaluative responses to an interaction with a stimulus &lt;Hassenzahl How are judgments formed? - Classification and comparison processes &lt; Hassenzahl Bipolar constructs product (good-bad), comparing product features versus features of alternative products (or personal standards / expectations) Result of judgment? “ expression of opinion about the dominance (importance, preference or likelihood) of one element over another”
  • Two types of judgments Overall UX judgments ~preference ~ laddering: preference distinction Attribute related UX judgments ~reasons why ~ laddering: probing for the reasons for preference the design’s locus of attention =product attributes. In need for practical recommendations to improve the product’s attributes. The motivations for these improvements, then refer to the corresponding consequences or values
  • RQ 1 Would our preschoolers express a relatively consistent preference for one of the three prototypes (RQ1). RQ2 RQ3 Number of ladders and accompanying elements? + impact of age (RQ3).
  • Youngest: 2.75 years Oldest: 7 years
  • Likelihood Ratio values = alternative to Pearson’s Chi Squares, to be used when small data sets are involved and when expected values reach frequencies below five. Dummy tables = instead of working with 3 x 3 contingency tables in which the three prototypes are compared simultaneously, dummy tables : focusing at the preference of one game versus the preference of ‘one of the two other games’. Why? increase the expected cell frequencies. Results : we compared the preference uttered via the This-or-That questions and the free play option, and again we found consistent patterns of relations that correspond more than one would expect by accident (when focusing on penguin selection versus the other games: L χ 2 (4)=(30.5), p&lt;0.01; when focusing on the kangaroo preference versus preference for one of the other games: L χ 2 (4)=(25.7), p&lt;0.01 or when focusing at the preference for the bird game versus one of the other two games L χ 2 (4)=(25.4), p&lt;0.01)
  • Why not reaching to values Cognitive limitations due to the nature of the question: “Which did you find most fun?”, conceptually every ladder links from the prototype to the terminal value of ‘fun’. Therefore, even those children that only pointed to the prototype generated a ladder as well. Specifying the questions and focusing to the games in question (making it more concrete, and thus product oriented) When probing further why the child started pointing out attributes again towards more consequences or even values. = climbing down instead of climbing up the ladder children seemed to interpret the repeated why-questions of the interviewer as indicative of not having understood the attributes. This research finding is in line with the emphasis on external causes as reasons for preference
  • If one aims for an interview that generates on average five elements per preschooler, linear regression analysis predicts an age of 59,57 months or almost five years (r. 469, bo=-1.314, b1=.106, p&lt;0.01).
  • Laddering &amp; HCI Reynold and Olson [31] , a traditional Laddering study should aim for approximately 500 elements. This number includes value elements as well. Intentions rather than reflections, cf. miller &amp; A. Studie van Hood and Bloom waarin ze vonden dat kinderen spontaneous causal utterances usaually refer to something they intend to do next than a reflection upon an event in the immediate past!!  further review may frame the laddering questions otherwise as preceding the free play and asking which they are intended to play again!! Individual technique Why? – originally designed as such UX as individual perception, even if it arises in social contexts (&lt; Law, &lt; Hassenzahl) Pros: more comfortable utter more valuable expressions, Cons: transform the data effort of double recruiting and analysis Hand puppet Pros &lt; Paus-hasebrink Cons &lt; barendregt : artificiality of this kind of ‘mediated’ conversations
  • Theory = 2-7 year olds have the cognitive capabilities to perform as Laddering interviewees Case Study= Only preschoolers aged 5 and older can form ladders of preferences, attributes and consequences
  • Laddering with young children in User eXperience evaluations

    1. 1. Laddering with Young Children in User eXperience Evaluations: Theoretical Groundings and a Practical Case Vero Vanden Abeele e-Media Lab, GROUP T - Leuven Leuven, Belgium [email_address] Bieke Zaman CUO/IBBT Katholieke Universiteit Leuven, Leuven, Belgium [email_address]
    2. 3. SAM
    3. 11. example <ul><li>WHY? </li></ul><ul><li> </li></ul><ul><li>Immersion / Fantasy [V] </li></ul><ul><li>“ it’s like I become the cuddle toy in the game” [C] </li></ul><ul><li>“ It’s soft and cute” [A] </li></ul><ul><li>“ I like the cuddle toy” [preference] </li></ul>
    4. 12. 1
    5. 15. 2
    6. 19. Testing these findings... |a case study|
    7. 20. |Introduction to the case|
    8. 21. |Research Questions| <ul><li>1 consistent preference? </li></ul><ul><li>2 full ladders A-C-V? </li></ul><ul><li>3 Number of ladders? Number of elements? </li></ul>
    9. 22. |Method| <ul><li>46 preschoolers </li></ul><ul><li>Average Age about 5 </li></ul><ul><ul><li>2 to 7 years </li></ul></ul><ul><li>Gender: 19 girls and 27 boys </li></ul><ul><li>School </li></ul>
    10. 23. |Procedure| <ul><li>Explorative game play x 3 (counterbalancing) </li></ul><ul><li>Preference distinction (ranking) </li></ul><ul><li>Laddering interview </li></ul><ul><li>Free play option </li></ul>
    11. 24. |Procedure| <ul><li>Example preference distinction via ranking </li></ul>
    12. 25. |Results| <ul><li>1 Consistent preference? </li></ul><ul><li>Yes! Consistent preferences between answers on preference distinction questions & free play option (Likelihood Ratios,, all p<0.01) </li></ul>
    13. 26. |Results| <ul><li>2 Full ladders A-C-V? </li></ul><ul><li>No Values </li></ul><ul><li>9 (/46) : preferences > [stop] </li></ul><ul><li>19 (/46) : preferences > attributes > [stop] </li></ul><ul><li>24 (/46) : preferences > attributes > conseq. > [stop] </li></ul><ul><li>probing further = more external causes </li></ul><ul><li>Starting to climb down the ladder again... </li></ul>
    14. 27. |Results| <ul><li>Example preferences > attributes > [stop] </li></ul>
    15. 28. |Results| <ul><li>3 Number of elements - ladders? </li></ul><ul><li>Average= 1.72 ladder; 2.81 elements </li></ul><ul><li>Mean age respondents not producing ladders </li></ul><ul><ul><ul><li>+/- 3.5 years old </li></ul></ul></ul><ul><li>Mean age respondents producing ladders </li></ul><ul><ul><ul><li>+/- 5 years old </li></ul></ul></ul><ul><li>The older they are, the higher the number of ladders and elements </li></ul><ul><li>Significant (all p<0.01) and positive correlations </li></ul>
    16. 29. |Discussion| <ul><li>Laddering in HCI </li></ul><ul><ul><li>How many elements do you need? </li></ul></ul><ul><ul><li>Striving for values per se? </li></ul></ul><ul><ul><li>Asking for intentions rather than reflections </li></ul></ul><ul><li>Individual technique </li></ul><ul><ul><li>Inviting pairs of children? </li></ul></ul><ul><li>Adult interviewer vs hand puppet? </li></ul>
    17. 30. |Conclusion|
    18. 31. Vero Vanden Abeele [email_address] Bieke Zaman [email_address]