Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Continuous Adapation via Meta Learning

35 views

Published on

Jiaxu Miao

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Continuous Adapation via Meta Learning

  1. 1. Continuous Adaptation via Meta-learning 苗嘉旭
  2. 2. Introduction to Meta-learning  Learning to learn  Artificial agents should be able to learn and adapt quickly from only a few examples, and continuing to adapt as more data becomes available.  Multi-task adaptation
  3. 3. MAML  Model-Agnostic Meta-Learning Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks ICML2017
  4. 4. MAML  Model-Agnostic Meta-Learning Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks ICML2017
  5. 5. Continuous Adaptation Via Meta-learning In Nonstationary And Competitive Environments  Reinforcement Learning algorithms for solving many tasks are designed to deal with stationary environments.  real-world is often nonstationary : changes in the dynamics presence of multiple learning actors  A nonstationary environment can be seen as a sequence of stationary tasks, and hence we propose to tackle it as a multi-task learning problem Continuous Adaptation Via Meta-learning In Nonstationary And Competitive Environments ICLR 2018 Best Paper
  6. 6. A Probabilistic View of MAML  a distribution over tasks, D(T) Continuous Adaptation Via Meta-learning In Nonstationary And Competitive Environments ICLR 2018 Best Paper
  7. 7. A Probabilistic View of MAML  MAML constructs parameters of the task-specific policy, φ, using gradient of LT: Continuous Adaptation Via Meta-learning In Nonstationary And Competitive Environments ICLR 2018 Best Paper
  8. 8. A Probabilistic View of MAML In general, we can think of the task, trajectories, and policies, as random variables Continuous Adaptation Via Meta-learning In Nonstationary And Competitive Environments ICLR 2018 Best Paper
  9. 9. Continuous Adaptation via Meta-learning  In the classical multi-task setting, we make no assumptions about the distribution of tasks, D(T).  When the environment is nonstationary, we can see it as a sequence of stationary tasks on a certain timescale where the tasks correspond to different dynamics of the environment. Then, D(T) is defined by the environment changes, and the tasks become sequentially dependent. Continuous Adaptation Via Meta-learning In Nonstationary And Competitive Environments ICLR 2018 Best Paper
  10. 10. Continuous Adaptation via Meta-learning  In the probabilistic language, our nonstationary environment is equivalent to a distribution of tasks represented by a Markov chain Continuous Adaptation Via Meta-learning In Nonstationary And Competitive Environments ICLR 2018 Best Paper
  11. 11. Continuous Adaptation via Meta-learning  The goal is to minimize the expected loss over the chain of tasks of some length L  the meta-loss on a pair of consecutive tasks Continuous Adaptation Via Meta-learning In Nonstationary And Competitive Environments ICLR 2018 Best Paper
  12. 12. Continuous Adaptation via Meta-learning Continuous Adaptation Via Meta-learning In Nonstationary And Competitive Environments ICLR 2018 Best Paper
  13. 13. Continuous Adaptation via Meta-learning Continuous Adaptation Via Meta-learning In Nonstationary And Competitive Environments ICLR 2018 Best Paper
  14. 14. Continuous Adaptation via Meta-learning Continuous Adaptation Via Meta-learning In Nonstationary And Competitive Environments ICLR 2018 Best Paper
  15. 15. Continuous Adaptation via Meta-learning Continuous Adaptation Via Meta-learning In Nonstationary And Competitive Environments ICLR 2018 Best Paper
  16. 16. Continuous Adaptation via Meta-learning  Adaptation at execution time. Continuous Adaptation Via Meta-learning In Nonstationary And Competitive Environments ICLR 2018 Best Paper
  17. 17. Experiments  Setup Continuous Adaptation Via Meta-learning In Nonstationary And Competitive Environments ICLR 2018 Best Paper
  18. 18. Experiments  Policy Continuous Adaptation Via Meta-learning In Nonstationary And Competitive Environments ICLR 2018 Best Paper

×