1. The document discusses implicit behavioral cloning, which was presented in a 2021 Conference on Robot Learning (CoRL) paper. 2. Implicit behavioral cloning uses an implicit model rather than an explicit model to map observations to actions. The implicit model is trained using an InfoNCE loss function to discriminate positive observation-action pairs from negatively sampled pairs. 3. Experiments showed that the implicit model outperformed explicit models on several manipulation tasks like bi-manual sweeping, insertion, and sorting. The implicit approach was able to generalize better than explicit behavioral cloning.

1. 1
DEEP LEARNING JP
[DL Papers]
http://deeplearning.jp/
Implicit Behavioral Cloning (CoRL 2021)
Koki Yamane, University of Tsukuba

2. 2022/5/13 2
◼ Implicit Behavioral Cloning
◼ CoRL 2021
◼ Robotics at Google
 Pete Florence, Corey Lynch, Andy Zeng, Oscar Ramirez, Ayzaan Wahid,
Laura Downs, Adrian Wong, Johnny Lee, Igor Mordatch, Jonathan Tompson
◼ https://implicitbc.github.io/

3. (BC) EBM
2022/5/13 3

4. ◼

◼

◼
2022/5/13 4
ෝ
𝒂 = 𝑭𝜽 𝒐
Behavior Cloning

5. BC
2022/5/13 5
ෝ
𝒂 = 𝑭𝜽 𝒐
Explicit Model ( )
ෝ
𝒂 = argmin 𝑬𝜽 𝒐, 𝒂
Implicit Model ( )

6. 2022/5/13 6
ෝ
𝒂 = argmin 𝑬𝜽 𝒐, 𝒂
𝐿InfoNCE = ෍
𝑖=0
𝑁
−log ෦
𝑝𝜃 𝑦𝑖|𝑥, ෤
𝑦𝑖
𝑗
𝑗=1
𝑁𝑛𝑒𝑔
෦
𝑝𝜃 𝑦𝑖|𝑥, ෤
𝑦𝑖
𝑗
𝑗=1
𝑁𝑛𝑒𝑔
=
𝑒−𝐸𝜃 𝑥𝑖,𝑦𝑖
𝑒−𝐸𝜃 𝑥𝑖,𝑦𝑖 + σ𝑗=1
𝑁𝑛𝑒𝑔
𝑒
−𝐸𝜃 𝑥𝑖, ෤
𝑦𝑖
𝑗

7. (CNN+) MLP
2022/5/13 7

8. Implicit Model
2022/5/13 8

9. Implicit Model
2022/5/13 9

10. Implicit Model
2022/5/13 10

11. 2022/5/13 11

12. Bi-Manual Sweeping Task
Explicit Model ( ) Implicit Model ( )
2022/5/13 12

13. Insertion Task
Explicit Model ( ) Implicit Model ( )
2022/5/13 13

14. Sorting Task
Explicit Model ( ) Implicit Model ( )
2022/5/13 14

15. 2022/5/13 15
◼ BC EBM
◼
◼
◼

 RNN