The Carolina Healthcare System (CHS) in the Carolinas is among the leading, and largest healthcare organisations in the U.S., employing 62,000 people in 940 care locations. The system has 7,500 beds and over 12 million patient encounters every year. In its commitment to energy management, efficiency and conservation, the organisation is pursuing strategies to decrease its energy use. One such strategy is implementing programmes that encourage building facilities staff to change their behaviour. The first phase in the CHS behaviour change program, Energy Connect, is an intervention that encourages operators to detect and act on energy inefficiencies within the buildings they are responsible for. Building operators account for a small percentage of people in each building, but have a disproportionally high impact on energy use. Therefore, if they were to change their behaviours, they could dramatically reduce overall energy use. IEA DSM Task 24 and ACEEE’s Behavior and Human Dimensions of Energy Efficiency program helped the Sustainability Director of CHS to co-create a highly collaborative behaviour change field trial.

1. How to design, implement and evaluate
behaviour change interventions in a sector that
is often overlooked but has huge energy
efficiency potentials: Hospitals
Dr. Sea Rotmann
SEA - Sustainable
Energy Advice,
New Zealand
Dr. Reuven Sussman
American Council for
Energy Efficient
Economy
Kady Cowan
Carolinas
HealthCare System
IEA Webinar - December 21, 2017
1

2. Agenda
Simon Sinek, 2009; www.startwithwhy.com
• Why (Sea)
• Focus on people & behaviour
• Connect science and practice
• Learn and share what works
• Serve as a global model
• How (Kady)
• Background
• Objectives
• Research process
• What (Reuven)
• Evaluation
• Learnings
• Next steps

4. WHY are we focusing on behaviour change?
1. Focus on
people
2. Connect
science and
practice
3. Learn and share
what works
4. Serve as a
global research
model

5. WHY? What is behaviour in our context?
Energy behaviour refers to all human actions that affect the way that fuels
(electricity, gas, petroleum, coal, etc.) are used to achieve desired services,
including the acquisition or disposal of energy-related technologies and
materials, the ways in which these are used, and the mental processes that
relate to these actions.
Behaviour Change in the context of this Task thus refers to any changes in
said human actions which were directly or indirectly influenced by a variety
of interventions (e.g. legislation, regulation, incentives, subsidies,
information campaigns, peer pressure etc.) aimed at fulfilling specific
behaviour change outcomes. These outcomes can include any changes in
energy efficiency, total energy consumption, energy technology uptake or
demand management but should be identified and specified by the
Behaviour Changer designing the intervention for the purpose of outcome
evaluation.

6. WHY? Understanding energy behaviour
persistence
“unfrozen”
half-yearlyyearly
Conscious, or well-considered action
Once in a lifetime
Active information-seeking
monthlyrarely
Little information-seeking
Hardly thinking – taking action
Habitualised routinesOnce-off
“frozen”
consciousness
frequency
weekly daily
cookinggroceriesholidayingChoosing
energy supplier
Buying a carBuying a
house

7. Click to add text
Click to add text

8. WHY are we doing Task 24?
2. Connect
science and
practice
4. Serve as a
global research
model
1. Focus on
people
Hub
Broad and systemic
perspective
Focus on replicability
and scalability
Geographically
inclusive, recognise
energy access
3. Learn and share
what works

9. WHY? What’s the moral of the story of Task 24?
• There is no silver bullet model for behaviour change
• Homo economicus doesn’t exist
• Habits are the most difficult thing to break, though
• It’s easiest during moments of change
• There is no such thing as individual energy use
• We need to look at whole-system, societal change
• This can’t be done in isolation by one sector - collaboration is key
• We need to facilitate shared learning and collaboration in multiple
stakeholders
• We also need a common language based on narratives
 It’s all about the people!

10. WHY are we doing field research pilots like this one?
1. Focus on
people
2. Connect
science and
practice
4. Serve as a
global research
model
3. Learn and share
what works

11. HOW? Expert Network (Subtask 5)
Participating countries, contributing experts
350 experts
21 countries
7 main sectors

12. HOW? Task 24 – Phase II
How it all fits together (with Phase I)
What?
Subtask 6
‘The Issues’
Who?
Subtask 7
‘The People’
How?
Subtask 8
‘The Tools’
Why?
Subtask 9
‘The Measure’
So
what?
Subtask 10
‘The Story’
Subtask 1
Subtask 2
Subtask 4 Subtask 5
Subtask 1
Subtask 4 Subtask 3
Subtask 11 – Real-life examples

13. HOW? Task 24 Multi-stakeholder facilitation
(Subtasks 6-7)
Kania & Kramer (2011): https://ssir.org/articles/entry/collective_impact

14. HOW: Task 24 Workshops (Subtask 7)

18. HOW? Task 24 Dissemination (Subtasks 8, 10 and 11)
http://www.ieadsm.org/wp/files/IE
A-DSM-Task-24-Subtask-
11_CHS-case-study_FONTS.pdf

19. Designing Energy Behaviour
Leading Change in Complex Buildings
Kady Cowan
How we did it

20. Carolinas HealthCare System
• 940 care locations
• 62,000+ staff
• 7500 beds
• 17.5 M ft2
• 12.5 M patient
encounters/year
• Utilities $37 M
• Energy Management
born in 2012
Charlotte, North Carolina
20

21. Why Hospitals?
8% of the energy resources in the
U.S. are consumed by healthcare
8%
Source: U.S. energy information administration www.eia.gov
21

22. Energy Connect will
help link human
actions to energy
savings, natural
resource
conservation and
patient experience

23. Timeline of Energy Connect
CHS Energy Leadership Council was convened
Energy Connect programme starts with Building Retuning Training
Task 24 workshop focusing on End Users, Behaviour Changers and behavioural issue
Energy Summit: Task 24 workshop on intervention design & expert evaluation w/s
Roll-out of five interventions at six CHS target facilities
Second Energy Summit and final Task 24 workshop: focusing on results and learnings
2015
early
2016
late
2016
early
2017
2017
early
2018
Ongoingmonthlyexpertcalls
23

24. identify target
close the gap
between values and
action
INSIGHTS
Move from imprecise phases of hunch to proof
24

25. Thinking In Systems
25

26. energy
savings
energy
leaders frontline
facilities
staff
project
construction
smart
building
software
site
manager
vendor &
consultant
occupant
energy
champion
evaluator
sustainability
Actors
in the
Energy
Ecosystem
26

27. We started to collect our energy stories
• Hierarchical culture
“building operators are not part of corporate decision making”
• Willingness to fix problems is much stronger than willingness to get
it right from the start
“we can only consider first costs and if the payback is more than
five years it can’t be done”
• We don’t value the skills required to run the buildings
“we don’t hire technically skilled people to run the complex
systems installed”
• High tolerance for short term solutions
“we are always putting out fires and juggling grenades”
27

28. Energy Connect for
Building Mechanics
1. Make data visible
2. Select and support a site
based energy champion
3. Develop a hot/cold call
response process flow
4. Document adjustments in the
BAS
5. Promote conversation
between occupants and
facilities with regard to energy
savings
Interventions
28

29. Medical Office Building Case Study
0.00
5000.00
10000.00
15000.00
20000.00
25000.00
30000.00
12/02/2016
12/04/2016
12/06/2016
12/08/2016
12/10/2016
12/12/2016
12/14/2016
12/16/2016
12/18/2016
12/20/2016
12/22/2016
12/24/2016
12/26/2016
12/28/2016
12/30/2016
01/01/2017
01/03/2017
01/05/2017
01/07/2017
01/09/2017
01/11/2017
01/13/2017
01/15/2017
01/17/2017
01/19/2017
01/21/2017
01/23/2017
01/25/2017
01/27/2017
01/29/2017
01/31/2017
02/02/2017
02/04/2017
DailyEnergy(kWh)
Estimated 30%
energy savings in
the building
Intervention
29

30. Monthly Energy Consumption 2015-2017
1
1.2
1.4
1.6
1.8
2
2.2
2.4
Jan-15
Mar-15
May-15
Jul-15
Sep-15
Nov-15
Jan-16
Mar-16
May-16
Jul-16
Sep-16
Nov-16
Jan-17
Mar-17
May-17
Jul-17
Sep-17
Nov-17
KbtuMillions
Intervention
January 2017
30

31. Medical Office Building
2017 Energy and Dollar Trajectory
2017 Energy Trend Dollar Trend
January to June 4% $9,971
July to December
Projected 3% saving
16% $25,609
January to December
Projected 3% savings
10% $35,580
31

32. Energy Stories… Bring Energy to Life
Everyone has a role to play…
Once upon a time... there was a pediatric building that was always cold during the winter
months.
Every day... the peds nurse would set the thermostat to 90F and over all this time, the
suite temperature never increased and always stayed freezing.
But, one day... the engineer took a look at the discharge air temperature from the vents
and found that the temperature was very cold even though the thermostat was set high.
He then checked the attic and found that outside air dampers were left open. He fixed
the dampers to operate properly and closed them to the minimum level.
Because of that... the building site began to warm up and make all the nurses and
children happy.
Until finally... the engineer and his team now check the dampers every day during the
winter.
And ever since then... everyone is warm and happy and thankful to the smart engineer.
32

33. Energy Connect
Evaluation
Dr. Reuven Sussman
Behavior and Human Dimensions Program
American Council for an Energy-Efficient Economy
rsussman@aceee.org – 202-507-4746
December 21, 2017:
International Energy Agency Webinar
Washington, DC, October 7-10, 2018

34. What Have We Learned?
Planning for Evaluation
34

35. Outline
• The challenge
• A novel solution
• Energy Connect Evaluation Panel
• First Energy Connect Summit
• Putting the evaluation plan into action
• Next steps
• Energy champions
• Summit 2018
35

36. The Challenge
• Difficult to evaluate a "living
lab"
• Unconventional and complex
intervention
• Rolling timelines
• A variety of sites
• Multiple overlapping interventions
• A variety of important outcomes
• Unique challenge = unique
solution
36

37. Unique Solution:
Expert Evaluation Panel
• 13 volunteer advisors, with
expertise in:
• Hospital energy efficiency
• Program evaluation
• Social science research
• Behavior change interventions
• Plus local facilities staff and
management
• Provide
• Different perspectives on evaluation
• Experience in actual implementation
37

38. Energy Connect Evaluation Team
• Recruitment
• Started in 2013 via
conferences
• Two primary components
• Monthly calls
• On-site workshop: Energy
Connect Summit
38

39. Evaluation Workshop
Energy Connect Summit 2017
39

40. Evaluation Workshop at the Summit
• Purpose
• Recommend outcome
measures
• Recommend evaluation
designs (control groups, etc.)
40

41. Workshop Agenda
• Day 1:
• Tour of a facility
• Briefing on intervention that was designed the previous day
• Start discussion of potential outcome measures
• Day 2:
• Finish discussion of outcome measures
• Discuss potential research designs
41

42. Outcome Measures
• Outcome measures
• Evidence that intervention achieved program’s
goals
• Step 1: Brainstorm
• Large number of potential measures
• Small breakout groups
• Measures AND frequency/time of measurement
• Step 2: Reduce number of measures
• Feasibility
• Usefulness
• Credibility
• Overall preference (voting)
42

43. Example: The Initial List
Goal Potential Outcome Measure
Positively Impact
Patient Experience
Track number of complaints
Monitor spaces staying within comfort zone standards
Patient satisfaction surveys
Get Well Network complaints correlated with adjustments
A collection of staff anecdotes about patient satisfaction with operators' adjustments
Save Money and
Energy
Logbook with Number of adjustments in the system
EUI/ENERGY STAR Scores
Measure of money spent on maintenance, equipment labor
Measure of money spent on vendor calls
Monthly billing data
Hospital staff sick days
Length of patient stay
Good vs. bad system overrides (fault detection software)
Staff survey about job satisfaction and knowledge of energy efficiency savings
Create a culture of
Energy Efficiency and
Empower Operators
Survey of operators
Retention of facilities staff
Interviews with energy champions
Survey of non-facility staff
Interview with facility directors and senior managers
Attendance at energy management meeting/training
43

44. EaseofAcquiring
Usefulness
Patient
satisfaction
surveys
Get well
network
complaints
correlated with
adjustments
Number of
comfort
complaints
Spaces staying within
comfort zone standards
after defining standards
Staff anecdotes
about patient
satisfaction with
operators'
adjustments
EUI/ENERGY
STAR
Facilities staff
retention
Patient length
of stay
Number of
adjustments
to systems**
Good vs bad
overrides (fault
detection
software)***
Staff survey
about job
satisfaction
and knowledge
of EE savings
Measure of money
spent on
maintenance,
equipment and
labor
Money spent
on vendor
calls
Survey of
operators
Interviews
with energy
champions
Interviews with
facility directors
Survey of
non-
facility
staff
Attendance at energy
management
meetings/training
Staff sick days
Monthly
billing
data*
*Monthly billing data should be normalized and adjusted. Thus, even
though it is already collected, it may be slightly more difficult to
prepare than other data
**Definition of “adjustments” and “overrides” should be determined.
This will be collected using a paper log book until BAS can be
automatically programmed to do so (and possibly afterward as well).
***This requires computer programming, therefore it could be difficult
Saving Money and Energy
Improving Patient Experience
Creating Culture of Energy Efficiency and
Empowering Building Operators
Goals
Usefulness vs Ease of Acquiring
44

45. Example: Final List
1. A logbook documenting adjustments made to building systems
2. Monthly billing data
3. Energy Use Intensity and Energy Star Scores
4. Measures of money spent on maintenance, equipment, labor and
vendor calls
5. A collection of staff anecdotes about patient satisfaction with
operators' building adjustments
6. Survey of operators to reflect on the Energy Connect program
7. Survey of facilities staff about job satisfaction and knowledge of
energy efficiency savings
8. Survey of non-facilities staff
9. Interviews with energy champions, facilities directors, and senior
managers
10. Attendance numbers at energy management meetings and training
sessions
45

46. Research Designs
• Goal: Provide evidence that Energy Connect
causes:
• Energy savings, cost savings, positive patient experiences,
and empowerment of building operators to make energy
efficiency adjustments
• Difficult to demonstrate causation
• Complex
• Overlapping interventions
• Long timeframe
46

47. Qualitative and Quantitative Data
47

48. Research Design Discussion
• Energy Use and Cost Data
• Matched controls
• USA-wide Energy Star Data
• Historical Data
• Interviews and Surveys
• Within and between subjects
48

49. After the Summit: Implementation
and Evaluation
• Ideas from Summit are
implemented by CHS teams on
site (Kady Cowan)
• Evaluation team is consulted on
monthly calls
• Further in-depth evaluation and
design assistance from several
evaluators
• Dr. Sussman, Dr. Cox, Dr. Mazzi, and Dr.
Moezzi
49

50. Next Steps
• Evaluation
• Evaluating energy savings over time
• Evaluating baseline culture of energy efficiency in Year 1
• Implementation
• Designing and Implementing Energy Champions program
• Building investment via the Energy Leadership Council
• Summit 2018
• Looking back at 2017
• How to expand and improve
50

51. Key Take-aways
51

52. Four Key Take-Aways
• Having an energy leader
• A person doing both strategy and implementation
• Including relevant staff and management in the
programme design process
• A panel of outside experts
• Having good stories
52

53. Thank you!
• Download the full report here:
http://www.ieadsm.org/wp/files/I
EA-DSM-Task-24-Subtask-
11_CHS-case-study_FONTS.pdf
• More information on BECC 2018:
http://beccconference.org
Reuven Sussman, Ph.D., Sr. Manager, Behavior and Human Dimensions Program rsussman@aceee.org
53

54. behavior, energy & climate change
WASHINGTON, DCOCTOBER 7-10, 2018
The premier international conference dedicated to understanding human
behavior and decision making to accelerate the transition to an energy-
efficient and low-carbon future.
Hyatt Regency Capitol Hill
Pre-conference Workshops: October 7
beccconference.org
54

55. Example: How the Interventions Have
Evolved
Original Intervention Successes Barriers Revised Intervention
Create a system for tracking relevant
adjustments/overrides in buildings
- Universally agreed very
important
- Successfully implemented at a
limited number of sites
- Diversity in knowledge about what to
track
- Diversity in common tools used for
tracking
(print, online)
No revisions
Create a manual describing best
practices/process for addressing most
common maintenance issues
- Process flow in place at all test
locations
- Seen as generally useful for
helping to remember all steps
- Getting feedback from the frontline
on the tool
- Frontline perceive they are doing all
the steps all the time
Simplify manual to just
standardized hot/cold call
process flow
Recruit Building Automation System
(BAS) champion(s) who are
knowledgeable about key
performance indicators
- Frontline and their supervisors
love this idea
- Hosted first champion event in
October 2017
- Getting the programme developed
quickly to capitalise on the frontline
enthusiasm
Recruit Energy Connect
Champions who have an
interest in energy savings
and coaching mindset
Make energy data visible to site level
staff with dashboards
- First time many actors have
seen energy data for their facility
on a regular basis
- Frontline staff are getting
familiar with the data and know
what to look for
- Getting the energy data in a usable
(simple) format
- Getting the energy data in a timely
fashion
Energy data sharing with a
monthly flyer
Educate non-facilities staff about the
role of front line staff
- An effective pathway to open
dialogue between operators and
occupants to solve energy
problems
- Challenging to get the occupants
attention on energy topics
Energy conversations and
feedback between operators
and occupants to support
the perception of Operators
as energy experts
55

56. CredibilitytoOperators
Credibility to Management (or Other Readers of the Evaluation)
Patient
satisfaction
surveys
Get well
network
complaints
correlated
with
adjustments
Number of
comfort
complaints
Staff anecdotes
about patient
satisfaction with
operators'
adjustments
EUI/ENERG
Y STAR
Facilities staff
retention
Patient length
of stay
Number of
adjustments
to systems**
Good vs bad
overrides (fault
detection
software)***
Staff survey
about job
satisfaction
and
knowledge of
EE savings
Measure of
money spent on
maintenance,
equipment and
labor
Money spent
on vendor
calls
Interviews
with energy
champions
Interviews with
facility directors
and senior
managers
Survey
of non-
facility
staff
Attendance at energy
management
meetings/training
Staff sick days
Monthly
billing
data*
Saving Money and Energy
Improving Patient Experience
Creating Culture of Energy Efficiency and Empowering
Building Operators
Goals
Spaces staying within
comfort zone standards
after defining standards
Survey of
operators
Credibility to Operators and Managers
56

57. aceee.org @ACEEEdc
The American Council for an Energy-Efficient Economy is a nonprofit 501(c)(3)
founded in 1980. We act as a catalyst to advance energy efficiency policies,
programs, technologies, investments, & behaviors.
Our research explores economic impacts, financing options, behavior changes,
program design, and utility planning, as well as US national, state, & local policy.
Our work is made possible by foundation funding, contracts, government grants,
and conference revenue.
57

58. The top convener in energy efficiency.
aceee.org/conferences
Upcoming ACEEE Conferences
Hot Water Forum February 26 Portland, OR
National Symposium on Market Transformation April 2 Arlington, VA
Energy Efficiency Finance Forum May 21 Chicago
flastname@aceee.org
Slides at slideshare.net/firstlastnames/cool-presentation-i-did

59. Thank you for your consideration!
Questions? Comments?
Report
Dr. Sea Rotmann
drsea@orcon.net.nz
Kady Cowan
kady.cowan@carolinas.org
59