Applying Iterative Design to the Eco-Feedback Design Process

1,584 views

Published on

Although randomized controlled trials are the gold standard in evaluating the effectiveness of eco-feedback systems on reducing consumption behaviors, such trials are resource intensive and costly. As such, it is crucial that the intervention—the eco-feedback artifact—is well designed before effort is invested in a longitudinal study.

In this talk, I will discuss the application of iterative design to eco-feedback systems. Iterative design is a design methodology based on a cyclic process of prototyping, user testing, and analysis, the results of which are then used to inform a new round of prototyping (and the cycle continues). Through an 18-month design process of a prototype eco-feedback display (Froehlich, 2011), I will describe how iterative design was used to evaluate and refine the aesthetic, usability, understandability, and educational potential of an eco-feedback system before a field deployment. I will highlight the role of massive online surveys in evaluating early eco-feedback design ideas and the role of in-home interviews in evaluating higher-fidelity (more refined) designs. Finally, I will close the talk with a discussion of low-cost methods to deploy and test eco-feedback designs in the field even when underlying resource sensing systems (e.g., smart meters) are unavailable. These methods can be used to evaluate how the eco-feedback system may fit into domestic space, explore differences in perspective and preference across household members, and evaluate how the system affects household dynamics (e.g., if the design provokes privacy concerns) before behavioral trials are conducted in earnest.

Froehlich, J. (2011). Sensing and Feedback of Everyday Activities to Promote Environmental Behaviors. University of Washington Doctoral Dissertation 2011. http://www.cs.umd.edu/~jonf/publications.html

Published in: Technology
1 Comment
5 Likes
Statistics
Notes
No Downloads
Views
Total views
1,584
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
27
Comments
1
Likes
5
Embeds 0
No embeds

No notes for slide

Applying Iterative Design to the Eco-Feedback Design Process

  1. 1. Behavior, Energy, & Climate Change Conference Nov 13, 2012Applying Iterative Design to the Eco-Feedback Design Process Human Computer Interaction @jonfroehlich Laboratory Assistant Professor Computer Science
  2. 2. eco-feedbacksensing and visualizing behavior to reduce environmental impact
  3. 3. eco-feedbacksensing and visualizing behavior to reduce environmental impact
  4. 4. “Getting the design right and the right design.” — Bill Buxton Sketching User Experiences
  5. 5. Before moving forward, I want to ask a question…
  6. 6. The following eco-feedback paper is missing something.What is it?
  7. 7. Brandon et al., Reducing Household Energy Consumption: A Qualitative & Quantitative Field Study, Environmental Psychology 1999
  8. 8. ½ paragraph description of eco-feedback interface and no screenshots in 11 page paperBrandon et al., Reducing Household Energy Consumption: A Qualitative & Quantitative Field Study, Environmental Psychology 1999
  9. 9. Brandon et al., Reducing Household Energy Consumption: A Qualitative & Quantitative Field Study, Environmental Psychology 1999
  10. 10. Well, but that was 1999.
  11. 11. Grønhøj & Thøgersen, Feedback on Household Electricity Consumption: Learning & Social Influence Processes, IJCS2011
  12. 12. 2 paragraph description of eco-feedback interface and no screenshots in 8 page paperGrønhøj & Thøgersen, Feedback on Household Electricity Consumption: Learning & Social Influence Processes, IJCS2011
  13. 13. Where’s Waldo?
  14. 14. So, clearly a disciplinary divide… psychologists designers engineers economists building scientists others?
  15. 15. This oversight seems to reflect a lack of recognitionabout the critical role that particular designchoices play in affecting behavior.
  16. 16. Jain et al., Assessing Eco-Feedback Interface Usage and Design to Drive Energy Efficiency in Buildings, Energy and Buildings 2012
  17. 17. This is abig problemPerhaps because of the design de-emphasis, very fewpapers discuss the design process that led to theultimate design artifact that was created and studied
  18. 18. Overarching QuestionHow can we structure and support the design process to create andidentify the most promising (and potentially most influential) aspects ofan eco-feedback design robustly and in a cost-efficient manner?
  19. 19. An Eco-Feedback Iterative Design Data Ideation / Pilot FormativeIdeation Refinement Gathering Sketch Studies Evaluation
  20. 20. Gunel, A. The Halo Effect: Using Behavior to Upgrade Technology,BECC2012
  21. 21. Design Process Large A/BFormative Small Field Refinement Refinement Randomized IdEvaluation Deployment(s) Control Trial(s)
  22. 22. A/B Testing is Ultimate Playground Large A/B Design A/B Testing Pilot Refinementment Randomized Ideas RCTs Studies Control Trial(s)
  23. 23. Gunel, A. The Halo Effect: Using Behavior to Upgrade Technology,BECC2012
  24. 24. Evaluating early design ideas to prepare for field deployments Large Data Ideation / Pilot Formative Small Field A/B Design Pilot A/B TestingIdeation Refinement Refinement Refinement Randomized Refinement Gathering Sketch Studies Evaluation Deployment(s) Ideas Studies RCTs Control Trial(s)
  25. 25. Froehlich et al., The Design & Evaluation of Prototype Eco-Feedback Displays for Fixture-Level Water Usage Data, CHI2012
  26. 26. An Eco-Feedback Iterative Design Process Data Ideation / Pilot FormativeIdeation Refinement Gathering Sketch Studies EvaluationGoal: gather formative data and use as basis to create a set of early, promising designsInquiry Methods: ethnography, interviews, surveys, literature reviews
  27. 27. An Eco-Feedback Iterative Design ProcessInformal interviews with water experts (e.g., SPU, Amy Vickers)Literature review of water resource management, environmental psychologyOur own online survey of water usage attitudes & knowledge (N=656 respondents) Data Ideation / Pilot FormativeIdeation Refinement Gathering Sketch Studies EvaluationGoal: gather formative data and use as basis to create a set of early, promising designsInquiry Methods: ethnography, interviews, surveys, literature reviews
  28. 28. Respondents (N=651) dramatically underestimated the amount of water used in common everyday activities. underestimate toilet : by 15% shower : by 30% bath : by 55% low-flow shower : by 60% outdoor yard watering : by 83% to 95%[Froehlich, UW PhD Dissertation, 2011]
  29. 29. Challenge: how can we use gathered data to inform our designs?
  30. 30. Eco-Feedback Design SpaceINFORMATION ACCESS DATA REPRESENTATION COMPARISON update aesthetic comparison frequency real-time monthly or less user poll pragmatic artistic target self social goal spatial proximity time comparison by to behavior co-located remote window <hour >year time past projected attentional temporal social-comp. demand glanceable high attention grouping ≤sec by hour by day by week by month ≥year geographically demographically selected target proximal similar social network effort to data goal-setting access low high granularity coarse-grain fine-grain strategy self-set system-set externally-setINTERACTIVITY visual difficulty to reach complexity simple complex comparison target easy hard degree of interactivity none high primary visual comparison variables statistic computation encoding textual graphical time window interface raw value @ this time [yest, last wk, mo, yr] customizability none high measurement time granularity average [hrly, daily, wkly, monthly, yrly] unit resource cost environmental activity time metaphor data grouping median over past [X] days impact mode user data granularity this day type [weekday, weekend] additions user annotations user corrections primary other this day of week (e.g., mondays) measurement unit view temporal spatial categoricalDISPLAY MEDIUM manifestation data grouping by by by by by by consumption SOCIAL ASPECTS webpage mobile wearable custom in-home resource person time space activity category phone app interface display display target person household community state country ambience MOTIVATIONAL/PERSUASIVE STRATEGIES not-ambient ambient private/ persuasive tactics from persuasive tactics include: public private public size psychology and applied social rewards goal-setting small large psychology disciplines: data punishment narrativeACTIONABILITY/UTILITY persuasive design public commitment likeability sharing none everyone written commitment reputation persuasive technology social- degree of loss aversion competition (see COMPARISON) behavioral science/economics comparison available unavailable actionability low high kairos social proof environmental psychology encouragement authority decision game design suggests suggests anomaly descriptive norms emotional appeals support actions social marketing purchase decisions alerts scarcity principle door-in-face health behavior change personal- framing unlock features ization no personalization highly personalized anchoring bias endowment effect defaults collection building information intent Informs one action informs many actions automation/ control no control system controls resource use [Froehlich et al., HCIC2009; CHI2010; UW PhD Dissertation 2011]
  31. 31. My own experiences Existing frameworks Psychology (e.g., persuasive tech, health informatics, (e.g., environmental psychology, social other eco-feedback work, and infovis) psychology, behavioral economics)
  32. 32. Eco-Feedback Design SpaceINFORMATION ACCESS DATA REPRESENTATION COMPARISON update aesthetic comparison frequency real-time monthly or less user poll pragmatic artistic target self social goal spatial proximity time comparison by to behavior co-located remote window <hour >year time past projected attentional temporal social-comp. demand glanceable high attention grouping ≤sec by hour by day by week by month ≥year geographically demographically selected target proximal similar social network effort to data goal-setting access low high granularity coarse-grain fine-grain strategy self-set system-set externally-setINTERACTIVITY visual difficulty to reach complexity simple complex comparison target easy hard degree of interactivity none high primary visual comparison variables statistic computation encoding textual graphical time window interface raw value @ this time [yest, last wk, mo, yr] customizability none high measurement time granularity average [hrly, daily, wkly, monthly, yrly] unit resource cost environmental activity time metaphor data grouping median over past [X] days impact mode user data granularity this day type [weekday, weekend] additions user annotations user corrections primary other this day of week (e.g., mondays) measurement unit view temporal spatial categoricalDISPLAY MEDIUM manifestation data grouping by by by by by by consumption SOCIAL ASPECTS webpage mobile wearable custom in-home resource person time space activity category phone app interface display display target person household community state country ambience MOTIVATIONAL/PERSUASIVE STRATEGIES not-ambient ambient private/ persuasive tactics from persuasive tactics include: public private public size psychology and applied social rewards goal-setting small large psychology disciplines: data punishment narrativeACTIONABILITY/UTILITY persuasive design public commitment likeability sharing none everyone written commitment reputation persuasive technology social- degree of loss aversion competition (see COMPARISON) behavioral science/economics comparison available unavailable actionability low high kairos social proof environmental psychology encouragement authority decision game design suggests suggests anomaly descriptive norms emotional appeals support actions social marketing purchase decisions alerts scarcity principle door-in-face health behavior change personal- framing unlock features ization no personalization highly personalized anchoring bias endowment effect defaults collection building information intent Informs one action informs many actions automation/ control no control system controls resource use [Froehlich et al., HCIC2009; CHI2010; UW PhD Dissertation 2011]
  33. 33. data information representation access inputs display medium thebehavioral eco-feedback design space comparison models social actionability motivational strategies
  34. 34. data information representation access inputs display medium thebehavioral eco-feedback design space comparison models social actionability motivational strategies
  35. 35. data information representation access inputs display medium thebehavioral eco-feedback design space comparison models social actionability motivational strategies
  36. 36. data information representation access inputs display medium thebehavioral eco-feedback design space comparison models social actionability motivational strategies
  37. 37. data information representation access inputs display medium thebehavioral eco-feedback design space comparison models social actionability motivational strategies
  38. 38. data information representation access inputs display medium thebehavioral eco-feedback design space comparison models social actionability motivational strategies
  39. 39. data representationtions aesthetic pragmatic artistic visual complexity simple complex time granularity < hour > year data granularity coarse-grain fine-grain measurement unit resource cost environmental time activity metaphor impact primary view temporal spatial categorical
  40. 40. data representationtions aesthetic pragmatic artistic visual complexity simple complex time granularity < hour > year data granularity coarse-grain fine-grain measurement unit resource cost environmental time activity metaphor impact primary view temporal spatial categorical
  41. 41. Data Granularitycoarse-grain fine-grain ≥neighbor- home room activity fixture fixture ≤ valve hood category
  42. 42. Prototype and Evaluate Across a Fidelity SpectrumSketch Lo-to-Mid Fidelity Higher Fidelity Mockup Mockup
  43. 43. An Eco-Feedback Iterative Design Process Data Ideation / Pilot FormativeIdeation Refinement Gathering Sketch Studies EvaluationGoal: gather formative data and use as basis to create a set of early, promising designsInquiry Methods: ethnography, interviews, surveys, literature reviews
  44. 44. An Eco-Feedback Iterative Design Process Data Ideation / Pilot FormativeIdeation Refinement Gathering Sketch Studies Evaluation Goal: test early design ideas, improve promising ideas, improve usability / aesthetic Evaluation Methods: online interactive surveys, design probe-based interviews, lab studies Challenge: the ultimate goal is to create a design that informs and, possibly, motivates behavior. The former is easier to evaluate with these methods than the latter.
  45. 45. Online Survey Recruitment o Online postings and word-of-mouth Survey Design o 63 questions (10 optional) o Question and answer order randomized when possible Collected Data o 712 completed surveys (651 from US or Canada) o Nearly 6,000 qualitative responses
  46. 46. Our online interactive survey allowed us tostudy a large N and gather both quantitativeand qualitative data
  47. 47. Comparisons were the most uniformly desired pieces ofinformation of all the dimensions
  48. 48. Self-comparisonwas most preferred91%
  49. 49. Our in-home, design-probe interviews allowed usto explore how the display was received by familiesand how (and where) it fit in a domestic setting
  50. 50. In-Home Interviews Recruitment o Online postings and word-of-mouth o Specifically recruited families Interview Method o Semi-structured with two researchers o 90-minutes, 3-phases o Data coded by two researchers into themes Participants o 10 households (20 adults) o 11 female/9 male o Diff. socio-economic backgrounds & occupations o 18 had college degrees
  51. 51. Display Location Preferences kitchen near thermostat high traffic areas accessiblewhen needed
  52. 52. Behavioral Lab Study
  53. 53. Integrate Findings & Revise Designs Data Ideation / Pilot FormativeIdeation Refinement Gathering Sketch Studies Evaluation
  54. 54. A Call1 Place more emphasis on describing eco-feedback designs and how design choices may affect behavior in our research papers / white papers2 Help generate reusable design knowledge by including information not just on the final eco- feedback design but the process used to achieve it
  55. 55. Froehlich et al., The Design & Evaluation of Prototype Eco-Feedback Displays for Fixture-Level Water Usage Data, CHI2012
  56. 56. Jon Froehlich, Sensing and Feedback of Everyday Activities to Promote Environmental Behaviors, PhD Dissertation 2011
  57. 57. Jon Froehlich, Moving Beyond Line Graphs, BECC2010, http://www.cs.umd.edu/~jonf/talks.html
  58. 58. jonf@umd.edu http://bit.ly/jonUMDApplying Iterative Design to the Eco-Feedback Design Process Human Computer Interaction @jonfroehlich Laboratory Assistant Professor Computer Science

×