12 March 2020
Using human-centred design to improve energy
efficiency programs

• David Pryor, Senior Team Leader in the Energy Markets
team, NSW Department of Planning Industry and Environment
(DPIE).
• 14 years experience in development and delivery of energy
savings programs in NSW.
• Leading a small team responsible for:
• developing rules for the Energy Savings Scheme (ESS)
• linking the ESS to other NSW programs to help remove
barriers to take-up, and
• investigating opportunities to modernise and scale
scheme delivery to meet future targets.
• Formerly at the Independent Pricing and Regulatory Tribunal
(IPART) working on administration and compliance of the
Greenhouse Gas Abatement Scheme (GGAS) and the ESS.
• I mostly talk about cycling, surfing, space opera and kids
Introduction
• Introductions
• The Energy Security Safeguard
• The problem we are trying to solve
• ‘Digital Landscape’ research and
its outcomes
• What we learned
Agenda

The Energy Savings Scheme (ESS) is a statutory scheme working
towards building a sustainable energy-efficiency market.
The scheme is coordinated by the NSW Department of Planning, Industry and
Environment (DPIE) and is administered by the Independent Pricing and
Regulatory Tribunal (IPART). It provides incentives to encourage investment in
energy efficiency to reduce energy use and lower costs within households,
businesses and industry.
How it works:
• Energy users work with an accredited provider on energy saving projects.
• The provider claims Energy Savings Certificates for energy savings.
• Certificates are sold to energy retailers and others to help them meet
legislated energy savings targets of 8.5% of NSW electricity sales.
Energy Savings Scheme
Impact: Over 14,000 GWh has been saved between 2009 and
2018. 17,000 GWh of savings have been locked in for the ten
years from 2019 to 2028. In 2018 (calendar year), nearly
2,700 GWh electricity is estimated to have been saved.

The new Energy Security Safeguard

The Energy Security Safeguard
Detailed NSW Electricity Strategy (p.28)
https://energy.nsw.gov.au/government-and-regulation/electricity-strategy
1. The first component of the Safeguard will involve
• expanding the existing Energy Savings Scheme to 2050,
• with targets increasing gradually up to 13 per cent by 2030, and
• expanded set of activities which reduce demand on electricity and gas
networks, including substituting gas for biomass.
2. The second component of the Safeguard will involve
• establishing a new certificate scheme for the deployment of peak
demand reduction technologies,
• such as batteries, smart pool pumps and electric vehicle chargers
• that enable electricity demand to be shifted away from peak periods.

The problem we are trying to solve

ESS Rule Change Consultation Forum – August 2019
What are the three words you would use to
describe the ESS as it is right now?
Where do you see the ESS five years
from now?

‘Digital Landscape’ research
and what we are doing with the outcomes

Context
The following work has been
co-created in collaboration with DPIE,
IPART, certificate creators and
Auditors working within the ESS.
It is reseach that does not represent the NSW Government's view

The Vision

Accelerate verifiable energy
savings, and future-proof the
ESS for a changing energy
landscape.
Our vision:

Approach

Project Context – Brief
Over 4 weeks, Mentally Friendly conducted
research to understand the ESS, and identify
opportunities to improve its operation.
To do this we explored;
Research with scheme
participants
The role of each stakeholder,
how they currently operate and
their current perspectives on the
scheme.
Current state tools
& processes
The range state of tools and
processes used by stakeholders,
where they fall down and how we
can improve them.
Impact of rule
change process
The impact that rule changes
have on stakeholders and how
we can better prepare
stakeholders for this.

Research Approach
72 hours of deep dive
interviews & workshops
with scheme participants
Great care has been taken to involve a diverse range of:
• Roles including auditors, registry managers,
directors and frontline staff.
• Digital-readiness including paper-based to fully
automated/digital certificate providers.
• Business models used by certificate providers
including aggregators, end-to-end and whole-of-
cycle.
• Locations including NSW, VIC and
QLD (with considerable experience in the ESS)

Opportunities to improve
Together we co-designed and assessed six opportunity areas:

Research Approach
Roles in the scheme
Administrators
IPART
Legislators
DPIE
Certificate Providers
ACPs
• Rules are fit for purpose
• Scheme works as a whole
•
•
Commercial success and
confidence in the scheme
Constructive improvements to
the scheme
• Regulate through building an
effective compliance culture
• Ensure a level playing field
Quality Assurance
Documentation
Installer
Auditors

Key Insights and
opportunities
-

Tools and
systems are
fragmented
02.
Six key
insights:
01.
Significant time is
spent on manual tasks
04.
Rule changes are
missing consistent and
effective feedback loops
Expansion of the
scheme is prevented by
siloed data
05.
03.
Certificate providers
aren’t equipped to
interpret rule changes
Complexity prevents
end users from
engaging
06.

Opportunity 01
Streamline manual data
processes
Opportunity 04
A more transparent and
collaborative rule
change process
Opportunity 02
A single source of truth
platform to view and maintain
verified energy savings
Opportunity 05
More effective measurement
and verification of
energy savings
Opportunity 03
Smart legislation that is human
readable and can be modelled
with stakeholders
Opportunity 06
Bring energy savings closer to
the end user
Six Opportunity
Areas:

“Interaction with the online portals is time-
consuming; it’s incredibly manual.”
– ACP
“It’s up to the ACP to fill in the [evidence
pack] sheet correctly.”
– ACP
Key insight: 1 of 6
Significant time is spent
on manual tasks
Manual tasks compromise the accuracy of data, as
many steps make room for error.
Excessive time spent on manual tasks takes away
focus from actually delivering energy efficiency.
Evidence of energy savings is required for the
creation of certificates and the requires
significant manual effort (and transcription).

“We rely entirely on these [Registry] emails,
they’re really the only way we know that
certificates have been registered.”
So what you’ll find is [the Portal] status here
is slow to change, and is currently saying
that these certificates aren’t registered — but
we've got the email confirmation showing
proving that they are.
– ACP
“Now we have to get a whole bunch of
systems all syncing together. And that's what
makes using the registry as a source of truth,
such a high stakes thing for us because we
want to just make sure everything matches
up.”
– ACP
Key insight: 2 of 6
Tools and systems
are fragmented
There’s no unified view that connects to a job site
to an energy savings certificate, making it difficult
to track the entire lifecycle of a project.
There’s confusion and ambiguity among on the role
of the different administrative systems
[ESS Portal and Registry].
Certificate providers want a single source of truth.

Key Benefits:
Opportunity Area 01
Streamline manual data
processes
Reduce the overall effort for multiple stakeholders to
assess and verify data by digitising and automating
manual processes.
Supporting Evidence:
“Double handling of data inputs creates
inefficiencies”
– ACP
“A document might travel from an installer, to
an ACP then used in an audit - but data is
degraded”
– IPART
“We have to consider how much less energy
savings occur in a paper-based system”
– DPIE
•
•
Legislator - Improve the
accuracy and overall quality of
data
Administrator - Automation of
data verification reduces
manual overhead of ACPs to
verify
• Implementer - Improve the
consistency of reporting on
implementation data
Key Risks:
• Standardised process doesn’t
meet the needs of everyone
• Increased manual verification
of automated data processes
• New processes doesn’t
deliver better quality
outcomes - value /effort

Key Benefits:
Opportunity Area 02
A single source of truth platform
to view and maintain verified
energy savings
Create a unified platform for energy savings, to support
future expansion of the scheme.
Supporting Evidence:
“There is no single source of truth”
– ACP
“[We could use the ESS Portal to] validate a
lot more things before they actually become
a problem.
– IPART
“We use 13 or 14 different systems just to
manage commercial lighting certificates”
– ACP
•
•
Enables the ability to target
new markets for ESS to
expand Drives efficiencies -
reduces manual data
processes and systems to
maintain
• Faster & more easily
verified energy savings
Key Risks:
• If functionality is
expanded, there is a risk
of not meeting everyone's
need
• Cost of developing a
solution outweighs the
return
• New system makes ACP
technology redundant

2. Single source of verified energy savings
• Develop a user-centred prototype platform to view
and maintain verified energy savings.
1. Streamline manual data processes
• Work with IPART to develop an upgraded
portal and registry, designed to support the
new Safeguard and increased certificate
creation.
Opportunities
Testing these proto-type opportunities will
inform the business case for future platforms
to support certificate creation!
What happens next?

“Complexity of scheme prohibits obvious
interpretation, (this means) it’s difficult to
interpret rule changes and the impact on
business
– ACP
“Without going into plain English there
may be more meaningful ways of
explaining legislation … Perhaps giving
an example [of how this affects ESCs].”
– Auditor
Key insight: 3 of 6
Certificate providers aren’t
equipped to interpret rule
changes
Rule changes are difficult to interpret for people
with a technical rather than a legal background.
People may not fully understandng the impact of
rule changes, but commercial reality means they
must proceed with their best guess.

“Trying to get feedback in one day is not
adequate given the complexity”
– ACP
“I wouldn’t want any more updates to the
rule, it’s difficult enough as it is to
keep across.”
– ACP
“More frequent updates to the rule create
more admin work for everyone… there can
be delays due to other agencies or data not
being available.”
– DPIE
Key insight: 4 of 6
Rule changes are missing
consistent and effective
feedback loops
It is time-consuming and often difficult to determine
how rule changes will impact those on the ground.
Low involvement from end-users and tradespeople
makes assessment of frontline impacts difficulte.
The frequency of minor rule changes increases
administrative effort.

Key Benefits:
Opportunity Area 03
Smart legislation that is human
readable and can be modelled
with stakeholders
Develop legislation that is clear to understand and model
the impact of rule changes on the scheme
Supporting Evidence:
“If they make that up in isolation without
stakeholders, you’re like — are you kidding?
That means my guy has to be up on a
ladder, in the ceiling, then he has to let go to
take a picture…”
– ACP
“Installers are a long way away from the
space regulated by IPART. Certificate
creation is not their job, but they are often
responsible for collecting and passing on all
required information.”
– DPIE
• Modelling the impact of a
rule change on stakeholders
before it’s rolled out
• Reduced ambiguity &
interpretation issues leads
to lower effort & cost to
manage
• Better fit of rules to real
world problems
Key Risks:
• Is it possible for legislation to
be human readable?
• Resources required to
establish & maintain this
system
• Still requires human
intervention to review code &
data
• May require greater
responsiveness & resources

“Changes are made to policy that we can’t
anticipate how this will impact ACPs”
– DPIE
“I don’t feel like our voice is heard”
– ACP
“There is minimal interaction with front-line
staff - Installers”
– DPIE
“There isn’t guidance material available on
the day of the rule change… so it's up to us
to interpret the rule.”
– ACP
Key Benefits:
Opportunity Area 04
Transparent and collaborative
rule change process
Create consistent feedback loops throughout the rule change
process, so that all scheme participants understand their
impact
Supporting Evidence:
• Shorter timeframe between
development of policy intent
& implementation
• Deeper understanding of
the problems & how they
can be solved
• Create feedback loops with
stakeholders so that they
feel heard & better
understand their impact
Key Risks:
• We aren’t considering all
perspectives feedbacks - its
not possible to involve
everyone
• Require more time &
resources from stakeholders
to contribute
• Key learnings aren’t scaled
the network - other ACPs are
aware

4. A more transparent and collaborative rule
change process
• Investigate the Better Rules framework (New
Zealand) – could it work here?
• Improve the process of consulting and drafting rules
so they’re more human and machine readable.
3. Smart legislation that is human
readable
• Work with the NSW Department of Customer
Service on Rules as Code.
• Our first step is to develop a digital version of
small parts of the ESS rules (the NABERS
method) and test it with users. What happens next?
We are building in more opportunity for
collaboration as we prepare the next rule
changes. We are also extending our ambition to
code the whole rule and make APIs available
Opportunities

“We just don’t have access to that data, it
doesn’t exist. We can’t link the number of
ESCs with a particular site.”
– DPIE
“[Lighting] Energy savings will hit a ceiling,
and we need to be able to transfer that
process.”
– DPIE
Key insight: 5 of 6
Expansion of the scheme is
prevented by siloed data
Lack of data is a barrier to identifying new sectors
Information captured today isn’t ‘clean’ — it can be
missing key information, preventing meaningful insights
from being extracted.
Without a way to analyse existing and emerging
scheme activities, it is harder impacts.

“We’ve come up against an issue where
a customer, on their own, sought
reimbursement from their landlord; and
had no clue that would contravene that
reimbursement clause. And that has
resulted in a significant number of
certificates being up for forfeit.”
– ACP
“Calculations have become so complex,
it’s tricky to give a customer an answer
on the spot.”
– ACP
“Complexity in the scheme prevent small
businesses from understanding and
participating”
– DPIE
Key insight: 6 of 6
Complexity prevents end
users from engaging
The scheme is complex to explain and understand
Reduces the ability for end users to understand
their role and what’s required of them.
The requirements of certificate providers are also
complex, and are difficult to convey to an end user.

“Better data allows us to test policy in real
world scenarios”
- DPIE
“We can directly measure our year on year
savings and compare this to our initial
estimates”
- ACP
Key Benefits:
Opportunity Area 05
More effective measurement &
verification of energy savings
Remove redundant and unnecessarily complex
requirements to verify energy savings, while ensuring
that the scheme remains fair and effective.
Supporting Evidence:
• Testing policy & method
changes with real data could
allow for better
implementation
• Further savings for the end-
user leveraging data from
metered products
• Builds a case for funding &
expansion of the scheme
Key Risks:
• Increasing accuracy often
means increasing complexity;
more room for error
• More rules and requirements
mean greater responsibility &
risk of non compliance
• Depending on design, simply
more M&V may be more
prohibitive

“The objective of the scheme is to help
people save energy”
- ACP
“We need greater education & awareness
of the scheme”
- ACP
Key Benefits:
Opportunity Area 06
Bring energy savings closer
to the end user
Lower the barrier to entry for end users and reduce
the unnecessary complexity, to encourage more
participants and drive behaviour change.
Supporting Evidence:
• Increased awareness &
involvement from the end-
user will drive more energy
savings
• Real-time savings drive
behavior change if it is linked
to ongoing incentives
• More awareness and
education of the ESS could
increase uptake
Key Risks:
• No incentive for energy
providers to identify
opportunities for energy
savings
• End-user is just not interested
in ESS and only wants the
saving
• New system designed to
support customers introduces
new problems

6. Bring energy savings closer to the end user
• Investigate Pay 4 Performance (P4P) programs
in collaboration with other Australian and
International EEO schemes.
5. Better measurement and verification
(M&V) of energy savings
• Test advanced M&V2.0 methods, and
demonstrate use of hourly energy profiles.
• Use human centred design to explore M&V
and simplified methods for calculating
energy savings and ensuring compliance.
Opportunities
For M&V2.0 and P4P to succeed, we need;
more (sub)metered data, the right mix of
energy savings and demand savings, data
privacy, easy finance, and simple
compliance!
What happens next?

Three Horizon
Roadmap

Three Horizon Roadmap
The Three Horizons plan breaks down our vision
for the Energy Savings Scheme into executable
phases of work:
Pilot: 2019-20
Pilot a modern
energy
savings
platform
Horizon 1: 2022
Accelerate and
modernise
energy savings
Horizon 2: 2025
Transparent and
accessible
energy savings
Horizon 3: 2030
Energy
savings are
BAU

- Validated prototype with scheme participants
- Evidence-based recommendations for technology
decisions (eg. smart legislation and platform)
- Investigation into data security and privacy
- Streamlined manual data processes
Pilot: 2019 – 2020
Pilot a modern energy
savings platform
Prototype and validate a unified platform to
streamline energy savings and provide
greater clarity on rule changes.
Outcome
What Success Looks Like
•
•
Stakeholders are aligned on and
support improvements to the scheme
Streamlining removes inefficiencies
leading to acceleration in savings
• Reduction in non-compliance
• Evidence from working prototypes
builds confidence in the scheme’s
expansion

What Success Looks Like
• Rate of non-compliance continues to fall
•
•
ESS increases share of NSW annual
energy savings targets (above 20%)
— and can better measure our impact
100% utilisation of ESS portal data to
registry data
Horizon 1: 2020 – 2022
Accelerate and
modernise energy
savings
Develop a modern, unified platform to
view, maintain and verify energy savings
Outcome
- Unified platform is fully functioning (MVP features)
- Smart legislation expansion
- Improved rule change process
- M&V 2.0

What Success Looks Like
• 10% of NSW residences and businesses
are engaged with the Energy Savings
Scheme
•
•
Energy Savings Scheme approaches
$20 billion of energy savings
80% of businesses and households
have accessed the ESS platform
Horizon 2: 2022 – 2025
Transparent and
accessible energy
savings
Leverage energy efficiency data to
expand energy savings into new sectors.
Outcome
- End-to end energy savings platform
- API integration
- Prototype public access (energy bill / dashboard)
- Data simplification & analysis

What Success Looks Like
• Energy savings scheme drives continuous
improvement in energy efficiency
• M&V is applied to new sectors and schemes
• NSW hits 0 emissions by 2050
Horizon 3: 2025 – 2030
Energy savings
are business as usual
Drive behaviour change through a digitally
integrated state/federal energy efficiency
scheme.
Outcome
- Nationally & internationally connected ESS
- Integrated smart energy system
- Sophisticated energy efficiency modelling to inform
behaviour change

Horizon 1: 2022
Accelerate and
modernise
energy savings
Horizon 2: 2025
Transparent and
accessible
energy savings
Horizon 3: 2030
Energy savings
are BAU
Develop a modern, unified
platform to view, maintain
and verify energy savings.
Leverage energy efficiency
data to expand energy
savings into new sectors.
Drive behaviour change
through a digitally
integrated state or federal
energy efficiency scheme.
Outcome:
MVP of unified platform, smart
legislation, M&V 2.0
Three Horizon Roadmap
Pilot: 2019-20
Pilot a modern
energy savings
platform
Prototype and validate a
unified platform to
streamline energy savings
and provide greater clarity
on rule changes.
Outcome:
Validated prototype with scheme
participants, investigation into
data security and privacy,
streamlined manual
data processes
Outcome:
End to end energy savings
platform, data simplification &
analysis, API integrations
Outcome:
Nationally & internationally
connected ESS, integrated smart
energy system, sophisticated
energy efficiency modelling to
inform behaviour change

What we learned

We need empathy and to focus on all users
1. Ensure we solve the core, root issues, not just the problem presented
 often the problem presented is a symptom, not the cause
2. Focus on people
3. Take a systems point of view
 most complications result from the interdependencies of the multiple parts
4. Continually test and refine our proposals
 ensure they truly meet the needs of the people for whom they are intended
Don Norman “The Four Principles of Human-Centered Design”
https://jnd.org/the-four-fundamental-principles-ofhuman-centered-design/

david.pryor@environment.nsw.gov.au
David Pryor