More Related Content Similar to Optimise Energy Usage Using Amazon SageMaker Reinforcement Learning and Publish Your Model in AWS Marketplace - AWS Summit Sydney (20) More from Amazon Web Services (20) Optimise Energy Usage Using Amazon SageMaker Reinforcement Learning and Publish Your Model in AWS Marketplace - AWS Summit Sydney2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Optimise energy usage using Amazon
SageMaker reinforcement learning and publish
your model in AWS Marketplace
Aparna Elangovan
Solutions Architect
Amazon Web Services
3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Agenda
• The problem of energy optimisation for heating, ventilation, and air
conditioning (HVAC)
• Introduce reinforcement learning
• Reinforcement learning on Amazon SageMaker to optimise energy
usage in HVAC
• AWS Marketplace
4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Business objective: Optimise energy in HVAC
HVAC consumes 40% of building
energy
1. Decrease energy usage
2. Improve occupant’s comfort
5. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Reinforcement Learning (RL)
7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How agents learn - A simplistic intuition
So what can we do?
• Randomly try different actions
• Compute average reward
• Assign higher transition probabilities to high
value actions
R =
-10000
Do
nothing
Reality mostly model free!
• State transitions, intermediate rewards,
probabilities are not known
8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How to simulate building energy consumption?
• Building type
• Building materials
• …..
• Human activity
• Lights, bulbs,…..
Input
• EnergyPlus
Simulator
(Environment)
• Bulb1 300 watts
per hour
• Cooler temp
• …
Output – Energy
consumed +
Current State
Environment
9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What about action and state?
Action
Space (At)
• Cooler set point
• Heater set point
Observation
Space (St)
• Zone temp
• Outside temp
• Relative humidity
10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Reward function
Designing a reward function requires domain knowledge
Aim to maximise total reward in the long run
Environment
11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
The agent
Learns the right action based on state and reward
Map transition probability from state to action (aka policy)
How? Using reinforcement algorithms
Environment
12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
The coding- RL frameworks on Amazon SageMaker
Coach
• Agent RL algorithms
• Compatible with OpenAI
Gym Environments
OpenAI Gym
• Provides unified interface
for the environment
• Your own agent
RLlib
• Scalable Reinforcement
Learning
Supported on SageMaker
RL deep learning - TensorFlow, MxNet
Preconfigured RL libraries
Docker based - built-in or BYO
Hyperparameter tuning
Support distributed training
13. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Challenges using RL
Simulate
real world
Model the
environment
Represent the
observable
environment
State
Align reward
with real
world
Long term vs
short term
rewards
Reward
function
15. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What we heard from our customers
RE IN VENTING T HE WHE E L
Significant time and valuable
resources are spent in developing
ML solutions to problems that are
already solved by others
POOR S E L E CTION
It is hard to find, evaluate, and qualify
trustworthy algorithms
and models for enterprise use cases
L OS T OPPORT UNITY
Difficulty in deployment, version
management, and reproducibility
led to delay in time to market
L ACK OF DAT A/IP S E CURIT Y
No way to ensure data security,
compliance needs, or regulatory
standards for customer’s data
and intellectual property
17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS ML Marketplace - What can you buy or sell?
Algorithms let you
train a custom model
using your own data;
You can also tune
your algorithm using
hyperparameters
Model packages
are pretrained
by your seller
and ready-to-use;
you can run an
inference in
real-time or
batch mode
18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Machine Learning on AWS algorithm/model
pricing and licensing
Hourly
Free trials
Free
Metered
Paid
19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Summary
Energy optimisation has significant financial and environmental reward
Assessing the implementation complexity is key to successful AI implementation
AWS SageMaker reduces some of the technical challenges and supports open source frameworks
Before you build look for what is out there , such as AWS Marketplace AI Application Services,
built-in Algorithms and models
Quick wins are well studied, well implemented AI solutions
20. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Aparna Elangovan