Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Optimise Energy Usage Using Amazon SageMaker Reinforcement Learning and Publish Your Model in AWS Marketplace - AWS Summit Sydney

60 views

Published on

Heating, Ventilation and Air Conditioning (HVAC) is responsible for keeping us warm and comfortable indoors. In a modern building, data such as weather, occupancy, and equipment use are collected routinely and can be used to optimise energy use. Reinforcement learning (RL) is a good fit as it can learn patterns in the data and identify strategies to control the system so as to reduce energy consumption to up to 20%. In this session we will be training a reinforcement learning algorithm with Amazon SageMaker using an open source simulator, EnergyPlus, to train an agent to optimise energy usage. We will then publish this model in AWS Marketplace for other customers to reuse your machine learning solution.

  • Be the first to comment

  • Be the first to like this

Optimise Energy Usage Using Amazon SageMaker Reinforcement Learning and Publish Your Model in AWS Marketplace - AWS Summit Sydney

  1. 1. S U M M I T SYDNEY
  2. 2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Optimise energy usage using Amazon SageMaker reinforcement learning and publish your model in AWS Marketplace Aparna Elangovan Solutions Architect Amazon Web Services
  3. 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Agenda • The problem of energy optimisation for heating, ventilation, and air conditioning (HVAC) • Introduce reinforcement learning • Reinforcement learning on Amazon SageMaker to optimise energy usage in HVAC • AWS Marketplace
  4. 4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Business objective: Optimise energy in HVAC HVAC consumes 40% of building energy 1. Decrease energy usage 2. Improve occupant’s comfort
  5. 5. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  6. 6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Reinforcement Learning (RL)
  7. 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T How agents learn - A simplistic intuition So what can we do? • Randomly try different actions • Compute average reward • Assign higher transition probabilities to high value actions R = -10000 Do nothing  Reality mostly model free! • State transitions, intermediate rewards, probabilities are not known
  8. 8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T How to simulate building energy consumption? • Building type • Building materials • ….. • Human activity • Lights, bulbs,….. Input • EnergyPlus Simulator (Environment) • Bulb1 300 watts per hour • Cooler temp • … Output – Energy consumed + Current State Environment
  9. 9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T What about action and state? Action Space (At) • Cooler set point • Heater set point Observation Space (St) • Zone temp • Outside temp • Relative humidity
  10. 10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Reward function Designing a reward function requires domain knowledge Aim to maximise total reward in the long run Environment
  11. 11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T The agent Learns the right action based on state and reward Map transition probability from state to action (aka policy) How? Using reinforcement algorithms Environment
  12. 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T The coding- RL frameworks on Amazon SageMaker Coach • Agent RL algorithms • Compatible with OpenAI Gym Environments OpenAI Gym • Provides unified interface for the environment • Your own agent RLlib • Scalable Reinforcement Learning Supported on SageMaker RL deep learning - TensorFlow, MxNet Preconfigured RL libraries Docker based - built-in or BYO Hyperparameter tuning Support distributed training
  13. 13. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  14. 14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Challenges using RL Simulate real world Model the environment Represent the observable environment State Align reward with real world Long term vs short term rewards Reward function
  15. 15. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  16. 16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T What we heard from our customers RE IN VENTING T HE WHE E L Significant time and valuable resources are spent in developing ML solutions to problems that are already solved by others POOR S E L E CTION It is hard to find, evaluate, and qualify trustworthy algorithms and models for enterprise use cases L OS T OPPORT UNITY Difficulty in deployment, version management, and reproducibility led to delay in time to market L ACK OF DAT A/IP S E CURIT Y No way to ensure data security, compliance needs, or regulatory standards for customer’s data and intellectual property
  17. 17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T AWS ML Marketplace - What can you buy or sell? Algorithms let you train a custom model using your own data; You can also tune your algorithm using hyperparameters Model packages are pretrained by your seller and ready-to-use; you can run an inference in real-time or batch mode
  18. 18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Machine Learning on AWS algorithm/model pricing and licensing Hourly Free trials Free Metered Paid
  19. 19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T Summary Energy optimisation has significant financial and environmental reward Assessing the implementation complexity is key to successful AI implementation AWS SageMaker reduces some of the technical challenges and supports open source frameworks Before you build look for what is out there , such as AWS Marketplace AI Application Services, built-in Algorithms and models Quick wins are well studied, well implemented AI solutions
  20. 20. Thank you! S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Aparna Elangovan

×