Evaluation of Prototypes and the Problem of Possible Futures
•
1 like•598 views
CHI2017 talk in Denver 9 May 2017. PDF of the paper: http://bit.ly/2q1mJ3B
Recommended
More Related Content
Similar to Evaluation of Prototypes and the Problem of Possible Futures
Similar to Evaluation of Prototypes and the Problem of Possible Futures (20)
More from Antti Salovaara
More from Antti Salovaara (7)
Recently uploaded
Recently uploaded (20)
Evaluation of Prototypes and the Problem of Possible Futures
- 1. Evaluation of Prototypes and the Problem of Possible Futures Antti Salovaara, University of Helsinki Antti Oulasvirta, Aalto University Giulio Jacucci, University of Helsinki UNIVERSITY OF HELSINKI AALTO UNIVERSITY
- 2. Evaluations are about what the prototype might become.
- 3. Picture from p. 518 in Abowd (1999) Classroom 2000: An Experiment with the Instrumentation of a Living Educational Environment. IBM Systems Journal 38(4), 508–530. Classroom of the future at Georgia Tech Benefit of note-taking support 1997 Wireless handheld devices
- 4. 2008 “Please Keep Your Laptops in an Upright and Unlocked Position” by Alan Levine. https://www.flickr.com/photos/cogdog/2871019373/ . CC0 1.0 Universal Social media Distracted attention The future as it became
- 5. Study context (1997) Actual context (future) Automated capture of learning material Wireless note sharingUpload after lecture Social media Overhead projectors Present– future gap
- 6. Study context Actual context Design brief, Requirement specification New way of work New software Functional prototype The old way of work Usability evaluation Present– future gap
- 7. A carelessly designed evaluation may inadvertently evaluate wrong futures, contexts, or user groups, thereby leading to false conclusions and expensive design failures.
- 8. Evaluation as a “time machine” Envisioning Concretisation Projection
- 9. The non-issue of the present–future gap Scenario-building, co-design, PD Future orientation, artifacts, empirical approach Experimental method Future orientation, artifacts, empirical approach Ethnographic methods, participant observation Future orientation, artifacts, empirical approach Engineering and programming Future orientation, artifacts, empirical approach Field trials, Usability evaluations
- 11. Every methodological choice changes the enacted future Concretisation
- 12. CONTROL Removal of present-day features STAGING Creation of futuristic features Wireless note sharing Overhead projectors Every methodological choice changes the enacted future
- 14. INHIBITION in an urban navigation study concretization Free walking Embedding information in urban environment for AR Limit use of AR to a small area
- 15. PROPPING in an in-the-wild AR study concretization Hi-fi prototype Human actors Embedding information in urban environment for AR
- 16. GAMIFICATION in an AR experiment concretization Excitement helps users forget their hesitation
- 18. Projective validity From imperfect data… …to a valid conclusion inferential leap Internal validity Construct validity Conclusion validity External validity Projective validity
- 19. Near future Far future Precise range of futures Singular futures Margin of tolerance
- 20. New mindset for evaluations 1. Evaluate with the future in mind 2. Consider the future-contingency of every methodological choice 3. Consider how evaluation must be controlled & staged 4. Report methodological choices transparently 5. Define the margins for projective validity
- 21. Thank you! Acknowledgments: Academy of Finland (259281 and 298879), European Research Council (637991) David McGookin, Barry Brown, Duncan Brumby, Kasper Hornbæk, Mika Jokiniemi, Stuart Reeves, Baris Serim, John Zimmerman Evaluations are about what the prototype might become.