제1회 RLKorea 프로젝트 HomeNavi 팀 발표자료 입니다. 3D Environment에서 Navigation Task를 RL로 접근하는 분야 및 논문 소개, 구현에 대한 발표자료 입니다.

1. Look, Listen and Act
Written by 이의령, 김예찬, 양홍선
보고 듣고 행동하는 에이전트
3D Environment HomeNavi

2. Your
Logo
Here
Project Introduction
2www.website.com01/01/2018

3. Project Introduction
3www.website.com
3D Environment 기반 Home Navigation
• House(Indoor) 3D Dataset
• Reinforcement Learning Environment
• ‘Go to Kitchen’ 과 같은 Instruction 기반 Task 수행

4. Project Introduction – House3D
4

5. Project Introduction
5
Language
ActionsVision
• Image / video
understanding
• 3D environment
perception
• Camera motion
• Robotics /
Manipulation
• APIs
• Instruction following
• Question answering
• Dialog
‘Complete’
Agent

6. Project Introduction
6
3D Environment 기반 Home Navigation
• House(Indoor) 3D Dataset
• Reinforcement Learning Environment
• ‘Go to Kitchen’ 과 같은 Instruction 기반 Task 수행

7. Motivation
7
Target-driven Visual Navigation Model using Deep Reinforcement Learning
Y Zhu , ICRA 2017

8. Motivation
8
모바일 로봇에 적용할 수 있지 않을까?

9. Your
Logo
Here
Mobile Robot & Navigation
9

10. Mobile Robot
10
A mobile robot is a robot that is capable of locomotion.
- wikipedia-
중분류 소분류 기술내용
Navigation
Driving
Path Planning
Obstacle Avoidance
Recognizing the surroundings
Localization
&
Mapping
Dead Reckoning
LandMark
SLAM
Credit : Machine Learning & Robotics / Geonhee Lee

11. Path Planning
11
• 현재 위치에서부터 지도상에 지정받은 목표 지점까지 이동 궤적(Trajectory)을 생성
• Map상의 Global Path Planning과 Local Path Planning으로 나누어
로봇의 이동 경로를 생성
• Algorithm: A*, D*, RRT(Rapidly-exploring random tree), Probabilistic Roadmap 등
Credit : Machine Learning & Robotics / Geonhee Lee

12. SLAM
12
Simultaneous Localization and Mapping
• Computational problem of constructing a map of an environment
while simultaneously keeping track of a robot’s location
Credit : Fast Campus SLAM Workshop 2018 / Dong-Won Shin

13. SLAM
13
Visual Localization
• Under the inaccurate GPS
• GPS-denied environment
Credit : Fast Campus SLAM Workshop 2018 / Dong-Won Shin

14. SLAM
14
Mapping
• Scenarios in which a prior map is not available and needs to be built.
• Map can inform path planning or provide an intuitive visualization
for a human or robot.
Credit : Fast Campus SLAM Workshop 2018 / Dong-Won Shin

15. Your
Logo
Here
Navigation
via Reinforcement Learning
15

16. Your
Logo
Here
Vision – Language – Navigation
Deep RL based Navigation
16www.website.com01/01/2018

17. Vision - Language
17www.website.com
Vision + Language Application
• Image Captioning
Input:
The man at bat
readies to swing at
the pitch while the
umpire looks on.
Desired
Output:
A large bus sitting
next to a very tall
building.

18. Vision - Language
18www.website.com
Vision + Language Deep Learning Architecture
• Image Captioning
Credit : https://www.analyticsvidhya.com/blog/2018/04/solving-an-image-captioning-task-using-deep-learning/

19. Vision - Language
19
Vision + Language Application
• Visual Question Answering(VQA)
Input:
Q: What is the
Musache made of?
Q: Is this a
Vegetarian Pizza?
Desired
Output:
A: Bananas A: No

20. Vision - Language
20www.website.com
Vision + Language Deep Learning Architecture
• Visual Question Answering(VQA)
Credit : https://arxiv.org/pdf/1505.00468v6.pdf

21. Vision - Language Navigation
21
Evolution of Language and Vision datasets towards Actions
Credit : https://lvatutorial.github.io/

22. Vision - Language Navigation
22
Evolution of Language and Vision datasets towards Actions

23. Vision - Language Navigation
23
Evolution of Language and Vision datasets towards Actions

24. Vision - Language Navigation
24
Evolution of Language and Vision datasets towards Actions

25. Vision - Language Navigation
25
Evolution of Language and Vision datasets towards Actions

26. Vision - Language Navigation
26
Evolution of Language and Vision datasets towards Actions

27. Vision - Language Navigation
27
Evolution of Language and Vision datasets towards Actions

28. Vision - Language Navigation
28
Evolution of Language and Vision datasets towards Actions

29. Vision - Language Navigation
29
Evolution of Language and Vision datasets towards Actions

30. Vision - Language Navigation
30
Evolution of Language and Vision datasets towards Actions

31. Vision - Language Navigation
31
Evolution of Language and Vision datasets towards Actions

32. Vision - Language Navigation
32
Language
ActionsVision
• Image / video
understanding
• 3D environment
perception
• Camera motion
• Robotics /
Manipulation
• APIs
• Instruction following
• Question answering
• Dialog
‘Complete’
Agent

33. 3D Environment
33
!X
Datasets
Environments
Tasks & Metrics
Credit : Connecting Language and Vision to Actions ACL2018 Tutorial / Abhishek Das

34. 3D Environment
34
!X
SUNCG (Song et al., 2017)
Datasets
Environments
Tasks & Metrics
Matterport3D (Chang et al., 2017) Stanford 2D-3D-S (Armeni et al., 2017)
Credit : Connecting Language and Vision to Actions ACL2018 Tutorial / Abhishek Das

35. 3D Environment
35
!X
SUNCG (Song et al., 2017)
Datasets
Environments
Tasks & Metrics
Matterport3D (Chang et al., 2017)
AI2-THOR
(Kolve et al., 2017)
MINOS
(Savva et al., 2017)
Gibson
(Zamir et al., 2018)
Stanford 2D-3D-S (Armeni et al., 2017)
CHALET
(Yan et al., 2018)
House3D
(Wu et al., 2017)
HoME (Brodeur et al., 2018)
VirtualHome
(Puig et al., 2018)
AdobeIndoorNav
(Mo et al., 2018)
Matterport3DSim
(Anderson et al., 2018)
Credit : Connecting Language and Vision to Actions ACL2018 Tutorial / Abhishek Das

36. 3D Environment
36Credit : Connecting Language and Vision to Actions ACL2018 Tutorial / Abhishek Das
!X
EmbodiedQA
SUNCG (Song et al., 2017)
Datasets
Environments
Tasks & Metrics
Matterport3D (Chang et al., 2017)
AI2-THOR
(Kolve et al., 2017)
MINOS
(Savva et al., 2017)
Gibson
(Zamir et al., 2018)
Stanford 2D-3D-S (Armeni et al., 2017)
CHALET
(Yan et al., 2018)
House3D
(Wu et al., 2017)
Interactive QA
(Gordon et al., 2018)
Vision-Language Navigation
(Anderson et al., 2018)
Language grounding
(Chaplot et al., 2017,
Hermann & Hill et al., 2017)
Visual Navigation
(Zhu & Gordon et al., 2017,
Savva et al., 2017,
Wu et al., 2017)
HoME (Brodeur et al., 2018)
VirtualHome
(Puig et al., 2018)
AdobeIndoorNav
(Mo et al., 2018)
Matterport3DSim
(Anderson et al., 2018)

37. 3D Environment
37Credit : Connecting Language and Vision to Actions ACL2018 Tutorial / Abhishek Das
!X
EmbodiedQA
SUNCG (Song et al., 2017)
Datasets
Environments
Tasks & Metrics
Matterport3D (Chang et al., 2017)
AI2-THOR
(Kolve et al., 2017)
MINOS
(Savva et al., 2017)
Gibson
(Zamir et al., 2018)
Stanford 2D-3D-S (Armeni et al., 2017)
CHALET
(Yan et al., 2018)
House3D
(Wu et al., 2017)
Interactive QA
(Gordon et al., 2018)
Vision-Language Navigation
(Anderson et al., 2018)
Language grounding
(Chaplot et al., 2017,
Hermann & Hill et al., 2017)
Visual Navigation
(Zhu & Gordon et al., 2017,
Savva et al., 2017,
Wu et al., 2017)
HoME (Brodeur et al., 2018)
VirtualHome
(Puig et al., 2018)
AdobeIndoorNav
(Mo et al., 2018)
Matterport3DSim
(Anderson et al., 2018)
>= 2017 (!)

38. Paper (in project)
38
l House3D Environment 구축
l RoomNav 학습 모델
House3D
Yi Wu et, al(2017)
Gated
Attention
Chaplot et, al(2017)
l Gated Attention Module
l House3D RoomNav의
레퍼런스 모델
Embodied QA
Abhishek et, al(2017)
l 최초 VQA + RL 접근
l Embodied QA Dataset 구축
l Hirarchical Model
l PACMAN 학습 모델
l CVPR 2018
FollowNet
P Shah et, al(2017)
l Conditioned Attention 모형
l Long Instruction(Language)
사용
l ICRA 2018
Arxiv Link Arxiv Link Arxiv Link Arxiv Link
Code Code Code

39. Paper
39
l Target Driven Visual
Navigation in Indoor Scene
l Siamese 형태의
RL기반 Navigation 학습 모델
l ICRA 2017
Target Driven
Visual Navi
Yuke Zhu et, al(2017)
CMP
Gupta et, al(2017)
Arxiv Link
l Cognitive Mapping and
Planning for visual Navigation
l Value Iteration Network
l CVPR 2017
Arxiv Link
l Visual Question Answering
in Interactive Environment
l CVPR 2018
Arxiv Link
CodeCodeCode
l Vision and Language
Navigation
l CVPR 2018 spotlight
Arxiv Link

40. Paper
40
Vision Language Navigation 이란 제목으로
2017년부터 지속적으로
Paper & Environment이 나오고 있는 추세

41. Your
Logo
Here
Project Experiment
41

42. Dataset
+
Environment
+
Task

43. Dataset: SUNCG

44. Dataset: SUNCG
Bedroom Toilet,
Bathroom
Garage
Bedroom BedroomRoom type

45. Dataset: SUNCG
사람이 디자인한 45,622 개의 3D 씬
평균적으로 8.9개의 방과 1.3층으로 구성됨
20개의 방 종류 (bedroom, living room, …)
80개의 개체 유형 (cup, chair, …)

46. Environment: House3D

47. Tasks: RoomNav, Embodied QA
RoomNav

48. Tasks: RoomNav, Embodied QA
Embodied QA

49. RoomNav

50. Models

51. Models
1
2

53. Gated LSTM
Gated LSTM
(Vizdoom)

54. Gated LSTM

56. Look / Listen /Act

57. Look / Listen /Act
보고

58. Look / Listen /Act
보고
듣고

59. Look / Listen /Act
보고
듣고
행동

67. House3DVizdoom

68. House3DVizdoom 10% ?

69. Experimental Results of
RoomNav Paper

70. Return to Vizdoom !

71. Difficulty Level

73. 30시간

74. More time!

75. Easy에서 학습된 모델을 à Hard에 적용

76. Visual input과 instruction과의 관계는 알고 있으나..

77. 학습

78. Exploration을 하기 시작

79. Navigation + House3D = Ultra-hard

80. 단계적으로 학습시켜 보기

81. Representation을 좋게

82. Questions?