CoRL 2019 Spotlight

1. Learning value functions
with relational state representations
for task-and-motion planning
Beomjoon Kim
Luke Shimanuki

2. Geometric task-and-motion planning

3. Geometric task-and-motion planning

4. Geometric task-and-motion planning
Complex, hybrid search problems

5. Discrete decisions - which object?

6. Continuous decisions - which base pose?

7. Pick(obj, grasp, base_pose)
Parameterized actions
Push(obj, push_dir, base_pose)
continuousdiscrete

8. Planning: sampling-based forward search
Pick(obj1, ?grasp, ?base) Pick(obj1, ?grasp, ?base)

9. Deciding the abstract action
Pick(obj1, ?grasp, ?base)

10. Sampling the associated continuous
parameters
g, b ~ Uniform(G,B)

11. Simulating the concrete action
Pick(obj1, g, b)

12. Guide the search by learning
an action-value function
Q(s,Pick(obj1, g1, b1))
vs
Q(s,Pick(obj2, g2, b2))

13. Guide the search by learning
an action-value function
Q(s,Pick(obj1, g1, b1))
vs
Q(s,Pick(obj2, g2, b2))

14. But planners are applied to
various environments and goals
Q should be
applicable to

15. Values for detailed actions including
base poses and grasps are difficult to
generalize across such variability

16. Q(s, Pick(obj1))
vs
Q(s, Pick(obj2)
We propose to guide planning by learning
an abstract-action value function

17. Q(s, Pick(obj1))
vs
Q(s, Pick(obj2)
Pick(obj1)
Pick(obj1, ?g, ?b) Pick(obj2, ?g, ?b)
Pick(obj2)
We propose to guide planning by learning
an abstract-action value function

18. Abstract Q function approximates
which object should be moved
Which object
should be moved?
Q(s, Pick(?obj))

19. This abstract, high-level knowledge is
generally applicable
Which object
should be moved?
Q(s, Pick(?obj))

20. How do we design a
state-representation
that affords
aggressive generalization?

21. We use a relational state representation

22. OccludesManip(obj1, obj2, home)
OccludesPre(obj1, obj2)
IsReachable(obj1)
ManipFree(obj1, home)
Predicates based on
reachability and occlusions

23. OccludesManip(obj1, obj2, home)
OccludesPre(obj1, obj2)
IsReachable(obj1)
ManipFree(obj1, home)
We encode a relational state using a
graph neural network (GNN)

24. IsReachable(obj1)
IsObj(obj1)
IsRegion(obj1)
IsTargetObj(obj1)
Node - predicates on object or region

25. OccludesPre(obj1, obj2)
Edges - relations on multiple entities

26. Value as the output of the GNN
Q(s, Pick(obj1))

27. Generalization to harder problems
without retraining

28. Generalization to harder problems
without retraining

30. Generalization to harder problems
without retraining

31. Generalization to different environments
without retraining

32. Generalization to different environments
without retraining