Download as PDF, PPTX
Uploaded byLeonardo ENERGY
How to design, implement and evaluate behaviour change interventions in hospitals
The document discusses the design, implementation, and evaluation of behavior change interventions focused on energy efficiency in hospitals, highlighting the importance of connecting people and scientific practices to achieve successful outcomes. It emphasizes the complexity of energy behaviors, the need for collaboration across sectors, and the challenges in evaluating such interventions. The Energy Connect program at Carolinas Healthcare System is presented as a case study, showcasing strategies for fostering a culture of energy efficiency among staff and improving patient experiences through effective energy management.
More Related Content
Similar to How to design, implement and evaluate behaviour change interventions in hospitals
How to design, implement and evaluate behaviour change interventions in hospitals
- 1. How to design,implement and evaluate behaviour change interventions in a sector that is often overlooked but has huge energy efficiency potentials: Hospitals Dr. Sea Rotmann SEA - Sustainable Energy Advice, New Zealand Dr. Reuven Sussman American Council for Energy Efficient Economy Kady Cowan Carolinas HealthCare System IEA Webinar - December 21, 2017 1
- 2. Agenda Simon Sinek, 2009;www.startwithwhy.com • Why (Sea) • Focus on people & behaviour • Connect science and practice • Learn and share what works • Serve as a global model • How (Kady) • Background • Objectives • Research process • What (Reuven) • Evaluation • Learnings • Next steps
- 4. WHY are wefocusing on behaviour change? 1. Focus on people 2. Connect science and practice 3. Learn and share what works 4. Serve as a global research model
- 5. WHY? What isbehaviour in our context? Energy behaviour refers to all human actions that affect the way that fuels (electricity, gas, petroleum, coal, etc.) are used to achieve desired services, including the acquisition or disposal of energy-related technologies and materials, the ways in which these are used, and the mental processes that relate to these actions. Behaviour Change in the context of this Task thus refers to any changes in said human actions which were directly or indirectly influenced by a variety of interventions (e.g. legislation, regulation, incentives, subsidies, information campaigns, peer pressure etc.) aimed at fulfilling specific behaviour change outcomes. These outcomes can include any changes in energy efficiency, total energy consumption, energy technology uptake or demand management but should be identified and specified by the Behaviour Changer designing the intervention for the purpose of outcome evaluation.
- 6. WHY? Understanding energybehaviour persistence “unfrozen” half-yearlyyearly Conscious, or well-considered action Once in a lifetime Active information-seeking monthlyrarely Little information-seeking Hardly thinking – taking action Habitualised routinesOnce-off “frozen” consciousness frequency weekly daily cookinggroceriesholidayingChoosing energy supplier Buying a carBuying a house
- 7. Click to addtext Click to add text
- 8. WHY are wedoing Task 24? 2. Connect science and practice 4. Serve as a global research model 1. Focus on people Hub Broad and systemic perspective Focus on replicability and scalability Geographically inclusive, recognise energy access 3. Learn and share what works
- 9. WHY? What’s themoral of the story of Task 24? • There is no silver bullet model for behaviour change • Homo economicus doesn’t exist • Habits are the most difficult thing to break, though • It’s easiest during moments of change • There is no such thing as individual energy use • We need to look at whole-system, societal change • This can’t be done in isolation by one sector - collaboration is key • We need to facilitate shared learning and collaboration in multiple stakeholders • We also need a common language based on narratives It’s all about the people!
- 10. WHY are wedoing field research pilots like this one? 1. Focus on people 2. Connect science and practice 4. Serve as a global research model 3. Learn and share what works
- 11. HOW? Expert Network(Subtask 5) Participating countries, contributing experts 350 experts 21 countries 7 main sectors
- 12. HOW? Task 24– Phase II How it all fits together (with Phase I) What? Subtask 6 ‘The Issues’ Who? Subtask 7 ‘The People’ How? Subtask 8 ‘The Tools’ Why? Subtask 9 ‘The Measure’ So what? Subtask 10 ‘The Story’ Subtask 1 Subtask 2 Subtask 4 Subtask 5 Subtask 1 Subtask 4 Subtask 3 Subtask 11 – Real-life examples
- 13. HOW? Task 24Multi-stakeholder facilitation (Subtasks 6-7) Kania & Kramer (2011): https://ssir.org/articles/entry/collective_impact
- 14. HOW: Task 24Workshops (Subtask 7)
- 18. HOW? Task 24Dissemination (Subtasks 8, 10 and 11) http://www.ieadsm.org/wp/files/IE A-DSM-Task-24-Subtask- 11_CHS-case-study_FONTS.pdf
- 19. Designing Energy Behaviour LeadingChange in Complex Buildings Kady Cowan How we did it
- 20. Carolinas HealthCare System •940 care locations • 62,000+ staff • 7500 beds • 17.5 M ft2 • 12.5 M patient encounters/year • Utilities $37 M • Energy Management born in 2012 Charlotte, North Carolina 20
- 21. Why Hospitals? 8% ofthe energy resources in the U.S. are consumed by healthcare 8% Source: U.S. energy information administration www.eia.gov 21
- 22. Energy Connect will helplink human actions to energy savings, natural resource conservation and patient experience
- 23. Timeline of EnergyConnect CHS Energy Leadership Council was convened Energy Connect programme starts with Building Retuning Training Task 24 workshop focusing on End Users, Behaviour Changers and behavioural issue Energy Summit: Task 24 workshop on intervention design & expert evaluation w/s Roll-out of five interventions at six CHS target facilities Second Energy Summit and final Task 24 workshop: focusing on results and learnings 2015 early 2016 late 2016 early 2017 2017 early 2018 Ongoingmonthlyexpertcalls 23
- 24. identify target close thegap between values and action INSIGHTS Move from imprecise phases of hunch to proof 24
- 25.
- 26.
- 27. We started tocollect our energy stories • Hierarchical culture “building operators are not part of corporate decision making” • Willingness to fix problems is much stronger than willingness to get it right from the start “we can only consider first costs and if the payback is more than five years it can’t be done” • We don’t value the skills required to run the buildings “we don’t hire technically skilled people to run the complex systems installed” • High tolerance for short term solutions “we are always putting out fires and juggling grenades” 27
- 28. Energy Connect for BuildingMechanics 1. Make data visible 2. Select and support a site based energy champion 3. Develop a hot/cold call response process flow 4. Document adjustments in the BAS 5. Promote conversation between occupants and facilities with regard to energy savings Interventions 28
- 29. Medical Office BuildingCase Study 0.00 5000.00 10000.00 15000.00 20000.00 25000.00 30000.00 12/02/2016 12/04/2016 12/06/2016 12/08/2016 12/10/2016 12/12/2016 12/14/2016 12/16/2016 12/18/2016 12/20/2016 12/22/2016 12/24/2016 12/26/2016 12/28/2016 12/30/2016 01/01/2017 01/03/2017 01/05/2017 01/07/2017 01/09/2017 01/11/2017 01/13/2017 01/15/2017 01/17/2017 01/19/2017 01/21/2017 01/23/2017 01/25/2017 01/27/2017 01/29/2017 01/31/2017 02/02/2017 02/04/2017 DailyEnergy(kWh) Estimated 30% energy savings in the building Intervention 29
- 30. Monthly Energy Consumption2015-2017 1 1.2 1.4 1.6 1.8 2 2.2 2.4 Jan-15 Mar-15 May-15 Jul-15 Sep-15 Nov-15 Jan-16 Mar-16 May-16 Jul-16 Sep-16 Nov-16 Jan-17 Mar-17 May-17 Jul-17 Sep-17 Nov-17 KbtuMillions Intervention January 2017 30
- 31. Medical Office Building 2017Energy and Dollar Trajectory 2017 Energy Trend Dollar Trend January to June 4% $9,971 July to December Projected 3% saving 16% $25,609 January to December Projected 3% savings 10% $35,580 31
- 32. Energy Stories… BringEnergy to Life Everyone has a role to play… Once upon a time... there was a pediatric building that was always cold during the winter months. Every day... the peds nurse would set the thermostat to 90F and over all this time, the suite temperature never increased and always stayed freezing. But, one day... the engineer took a look at the discharge air temperature from the vents and found that the temperature was very cold even though the thermostat was set high. He then checked the attic and found that outside air dampers were left open. He fixed the dampers to operate properly and closed them to the minimum level. Because of that... the building site began to warm up and make all the nurses and children happy. Until finally... the engineer and his team now check the dampers every day during the winter. And ever since then... everyone is warm and happy and thankful to the smart engineer. 32
- 33. Energy Connect Evaluation Dr. ReuvenSussman Behavior and Human Dimensions Program American Council for an Energy-Efficient Economy rsussman@aceee.org – 202-507-4746 December 21, 2017: International Energy Agency Webinar Washington, DC, October 7-10, 2018
- 34. What Have WeLearned? Planning for Evaluation 34
- 35. Outline • The challenge •A novel solution • Energy Connect Evaluation Panel • First Energy Connect Summit • Putting the evaluation plan into action • Next steps • Energy champions • Summit 2018 35
- 36. The Challenge • Difficultto evaluate a "living lab" • Unconventional and complex intervention • Rolling timelines • A variety of sites • Multiple overlapping interventions • A variety of important outcomes • Unique challenge = unique solution 36
- 37. Unique Solution: Expert EvaluationPanel • 13 volunteer advisors, with expertise in: • Hospital energy efficiency • Program evaluation • Social science research • Behavior change interventions • Plus local facilities staff and management • Provide • Different perspectives on evaluation • Experience in actual implementation 37
- 38. Energy Connect EvaluationTeam • Recruitment • Started in 2013 via conferences • Two primary components • Monthly calls • On-site workshop: Energy Connect Summit 38
- 39. Evaluation Workshop Energy ConnectSummit 2017 39
- 40. Evaluation Workshop atthe Summit • Purpose • Recommend outcome measures • Recommend evaluation designs (control groups, etc.) 40
- 41. Workshop Agenda • Day1: • Tour of a facility • Briefing on intervention that was designed the previous day • Start discussion of potential outcome measures • Day 2: • Finish discussion of outcome measures • Discuss potential research designs 41
- 42. Outcome Measures • Outcomemeasures • Evidence that intervention achieved program’s goals • Step 1: Brainstorm • Large number of potential measures • Small breakout groups • Measures AND frequency/time of measurement • Step 2: Reduce number of measures • Feasibility • Usefulness • Credibility • Overall preference (voting) 42
- 43. Example: The InitialList Goal Potential Outcome Measure Positively Impact Patient Experience Track number of complaints Monitor spaces staying within comfort zone standards Patient satisfaction surveys Get Well Network complaints correlated with adjustments A collection of staff anecdotes about patient satisfaction with operators' adjustments Save Money and Energy Logbook with Number of adjustments in the system EUI/ENERGY STAR Scores Measure of money spent on maintenance, equipment labor Measure of money spent on vendor calls Monthly billing data Hospital staff sick days Length of patient stay Good vs. bad system overrides (fault detection software) Staff survey about job satisfaction and knowledge of energy efficiency savings Create a culture of Energy Efficiency and Empower Operators Survey of operators Retention of facilities staff Interviews with energy champions Survey of non-facility staff Interview with facility directors and senior managers Attendance at energy management meeting/training 43
- 44. EaseofAcquiring Usefulness Patient satisfaction surveys Get well network complaints correlated with adjustments Numberof comfort complaints Spaces staying within comfort zone standards after defining standards Staff anecdotes about patient satisfaction with operators' adjustments EUI/ENERGY STAR Facilities staff retention Patient length of stay Number of adjustments to systems** Good vs bad overrides (fault detection software)*** Staff survey about job satisfaction and knowledge of EE savings Measure of money spent on maintenance, equipment and labor Money spent on vendor calls Survey of operators Interviews with energy champions Interviews with facility directors Survey of non- facility staff Attendance at energy management meetings/training Staff sick days Monthly billing data* *Monthly billing data should be normalized and adjusted. Thus, even though it is already collected, it may be slightly more difficult to prepare than other data **Definition of “adjustments” and “overrides” should be determined. This will be collected using a paper log book until BAS can be automatically programmed to do so (and possibly afterward as well). ***This requires computer programming, therefore it could be difficult Saving Money and Energy Improving Patient Experience Creating Culture of Energy Efficiency and Empowering Building Operators Goals Usefulness vs Ease of Acquiring 44
- 45. Example: Final List 1.A logbook documenting adjustments made to building systems 2. Monthly billing data 3. Energy Use Intensity and Energy Star Scores 4. Measures of money spent on maintenance, equipment, labor and vendor calls 5. A collection of staff anecdotes about patient satisfaction with operators' building adjustments 6. Survey of operators to reflect on the Energy Connect program 7. Survey of facilities staff about job satisfaction and knowledge of energy efficiency savings 8. Survey of non-facilities staff 9. Interviews with energy champions, facilities directors, and senior managers 10. Attendance numbers at energy management meetings and training sessions 45
- 46. Research Designs • Goal:Provide evidence that Energy Connect causes: • Energy savings, cost savings, positive patient experiences, and empowerment of building operators to make energy efficiency adjustments • Difficult to demonstrate causation • Complex • Overlapping interventions • Long timeframe 46
- 47. Qualitative and QuantitativeData 47
- 48. Research Design Discussion •Energy Use and Cost Data • Matched controls • USA-wide Energy Star Data • Historical Data • Interviews and Surveys • Within and between subjects 48
- 49. After the Summit:Implementation and Evaluation • Ideas from Summit are implemented by CHS teams on site (Kady Cowan) • Evaluation team is consulted on monthly calls • Further in-depth evaluation and design assistance from several evaluators • Dr. Sussman, Dr. Cox, Dr. Mazzi, and Dr. Moezzi 49
- 50. Next Steps • Evaluation •Evaluating energy savings over time • Evaluating baseline culture of energy efficiency in Year 1 • Implementation • Designing and Implementing Energy Champions program • Building investment via the Energy Leadership Council • Summit 2018 • Looking back at 2017 • How to expand and improve 50
- 51.
- 52. Four Key Take-Aways •Having an energy leader • A person doing both strategy and implementation • Including relevant staff and management in the programme design process • A panel of outside experts • Having good stories 52
- 53. Thank you! • Downloadthe full report here: http://www.ieadsm.org/wp/files/I EA-DSM-Task-24-Subtask- 11_CHS-case-study_FONTS.pdf • More information on BECC 2018: http://beccconference.org Reuven Sussman, Ph.D., Sr. Manager, Behavior and Human Dimensions Program rsussman@aceee.org 53
- 54. behavior, energy &climate change WASHINGTON, DCOCTOBER 7-10, 2018 The premier international conference dedicated to understanding human behavior and decision making to accelerate the transition to an energy- efficient and low-carbon future. Hyatt Regency Capitol Hill Pre-conference Workshops: October 7 beccconference.org 54
- 55. Example: How theInterventions Have Evolved Original Intervention Successes Barriers Revised Intervention Create a system for tracking relevant adjustments/overrides in buildings - Universally agreed very important - Successfully implemented at a limited number of sites - Diversity in knowledge about what to track - Diversity in common tools used for tracking (print, online) No revisions Create a manual describing best practices/process for addressing most common maintenance issues - Process flow in place at all test locations - Seen as generally useful for helping to remember all steps - Getting feedback from the frontline on the tool - Frontline perceive they are doing all the steps all the time Simplify manual to just standardized hot/cold call process flow Recruit Building Automation System (BAS) champion(s) who are knowledgeable about key performance indicators - Frontline and their supervisors love this idea - Hosted first champion event in October 2017 - Getting the programme developed quickly to capitalise on the frontline enthusiasm Recruit Energy Connect Champions who have an interest in energy savings and coaching mindset Make energy data visible to site level staff with dashboards - First time many actors have seen energy data for their facility on a regular basis - Frontline staff are getting familiar with the data and know what to look for - Getting the energy data in a usable (simple) format - Getting the energy data in a timely fashion Energy data sharing with a monthly flyer Educate non-facilities staff about the role of front line staff - An effective pathway to open dialogue between operators and occupants to solve energy problems - Challenging to get the occupants attention on energy topics Energy conversations and feedback between operators and occupants to support the perception of Operators as energy experts 55
- 56. CredibilitytoOperators Credibility to Management(or Other Readers of the Evaluation) Patient satisfaction surveys Get well network complaints correlated with adjustments Number of comfort complaints Staff anecdotes about patient satisfaction with operators' adjustments EUI/ENERG Y STAR Facilities staff retention Patient length of stay Number of adjustments to systems** Good vs bad overrides (fault detection software)*** Staff survey about job satisfaction and knowledge of EE savings Measure of money spent on maintenance, equipment and labor Money spent on vendor calls Interviews with energy champions Interviews with facility directors and senior managers Survey of non- facility staff Attendance at energy management meetings/training Staff sick days Monthly billing data* Saving Money and Energy Improving Patient Experience Creating Culture of Energy Efficiency and Empowering Building Operators Goals Spaces staying within comfort zone standards after defining standards Survey of operators Credibility to Operators and Managers 56
- 57. aceee.org @ACEEEdc The AmericanCouncil for an Energy-Efficient Economy is a nonprofit 501(c)(3) founded in 1980. We act as a catalyst to advance energy efficiency policies, programs, technologies, investments, & behaviors. Our research explores economic impacts, financing options, behavior changes, program design, and utility planning, as well as US national, state, & local policy. Our work is made possible by foundation funding, contracts, government grants, and conference revenue. 57
- 58. The top convenerin energy efficiency. aceee.org/conferences Upcoming ACEEE Conferences Hot Water Forum February 26 Portland, OR National Symposium on Market Transformation April 2 Arlington, VA Energy Efficiency Finance Forum May 21 Chicago flastname@aceee.org Slides at slideshare.net/firstlastnames/cool-presentation-i-did
- 59. Thank you foryour consideration! Questions? Comments? Report Dr. Sea Rotmann drsea@orcon.net.nz Kady Cowan kady.cowan@carolinas.org 59