Successfully reported this slideshow.

Projectivities and continuous action spaces.

135 views

Published on

  • Be the first to comment

  • Be the first to like this

Projectivities and continuous action spaces.

  1. 1. Projectivities and continuous action spaces feat. Lena Reinaldo Uribe M Sept 30, 2013
  2. 2. w − l space as projective transformation from policy value/cost space Policy Value D 1 −D Episode Length
  3. 3. w − l space as projective transformation from policy value/cost space −D Policy Value D l 1 −D Episode Length w D
  4. 4. w − l space as projective transformation from policy value/cost space Policy Value D 1 −D Episode Length
  5. 5. w − l space as projective transformation from policy value/cost space −D Policy Value D l 1 −D Episode Length w D
  6. 6. Extension to continuous spaces Sample task: two states, continuous actions s1 a1 ∈ [0, 1] r1 = 1 + (a1 − 0.5)2 c1 = 1 + a1
  7. 7. Extension to continuous spaces Sample task: two states, continuous actions s1 a1 ∈ [0, 1] r1 = 1 + (a1 − 0.5)2 c1 = 1 + a1 s2 a2 ∈ [0, 1] r2 = 1 + a2 c2 = 1 + (a2 − 0.5)2
  8. 8. Extension to continuous spaces Sample task: two states, continuous actions Policy Space (Actions) 1 a2 0 0 a1 1
  9. 9. Extension to continuous spaces Sample task: two states, continuous actions Policy Values and Costs Policy value 4 Policy cost 4
  10. 10. Extension to continuous spaces Sample task: two states, continuous actions Policy Manifold in w − l l D/2 w D/2

×