SlideShare a Scribd company logo
1 of 21
Horizon
Robotics
Language Learning in an Interactive
and Embodied Setting
11/2018
Wei Xu
1
Horizon
Robotics
Horizon
RoboticsA Developmental Approach to Machine Intelligence
1. It might be easier than solving all the tasks a human adult can do
2. Learn skills and knowledges unspecified at design time
3. Gradually proceed from easy tasks to difficult tasks
2
“Instead of trying to produce a program to simulate the adult mind, why
not rather try to produce one which simulates the child's? If this were then
subjected to an appropriate course of education one would obtain the adult
brain.” - Alan Turing (1950)
Language learning in an interactive and embodied setting
Horizon
RoboticsWhy Embodied?
 Learn from the experiences coming from the
machine’s interactions with its environment
 Learn commonsense through the observation
and interaction with the environment
 Meaning emerges by “grounding” language in
modalities in our environment
3Language learning in an interactive and embodied setting
Human driving: < 1000 miles
Self-driving: >10 million miles
Horizon
RoboticsWhy Interactive?
 A useful robot needs to be able to understand
and communicate effectively
 It is easier for human to teach machines directly
using language than writing code
 Humans are great teachers
 Learn the effects of speaking by observing
feedbacks from conversational partner
 Learn human value through the interaction
4Language learning in an interactive and embodied setting
Horizon
RoboticsAnswering Questions and Following Commands
1. Is it possible to learn to follow commands using
end-to-end reinforcement learning without any
pretraining for vision or language?
2. Whether learning question answering can help
learning command
3. Can the machine understand words under new
context not seen in training?
5
Haonan Yu, Haichao. Zhang, Wei Xu “Interactive Grounded Language
Acquisition and Generalization in a 2D World” ICLR 2018
Horizon
RoboticsProblem Setup
6Answering questions and following commands
east and avocado never
appears together in training
Watermelon only appears in
answers during training
Horizon
RoboticsModel architecture
7Answering questions and following commands
answer
action
value
Horizon
RoboticsExperiments
8Answering questions and following commands
No QA training
Horizon
RoboticsGeneralization Ability
9
We can generalize to word combinations
never seen in training
We can generalize to questions containing
words never seen in training
Answering questions and following commands
Held out X(%): %X of word/combinations are held out from training
Horizon
Robotics
Challenges:
 Partially observed
 Much longer delay of reward
 More visual variations
“Navigate to the dog!”Navigation in a 3D Environment
10
Horizon
RoboticsGuided Feature Transformation
Haonan Yu, Xiaochen Lian, Haichao Zhang, Wei. Xu “Guided Feature Transformation (GFT):
A Neural Language Grounding Module for Embodied Agents” CoRL 2018
11Navigation in 3D environment
action
value
Horizon
RoboticsExperimental Results
12Navigation in 3D environment
Horizon
RoboticsDemo the object besides candle is your target .
please move to the object that is front of the basketball
.
can you reach the object right of toilet ?go to the object to the right of bike please .reach the location between car and trampoline please.please navigate to the grid between gift and tower .please navigate to the grid between bucket and chair .please move to the object that is front of basketball .
13Navigation in 3D environment
Horizon
RoboticsLearning to Speak and Remember
1. How to learn to speak by talking with other people?
2. What information should be remembered?
3. How to utilize knowledge in memory?
14
Haichao Zhang, Haonan Yu, Wei Xu “Interactive Language Acquisition with One-Shot
Visual Concept Learning through a Conversation Game” ACL 2018
Horizon
RoboticsProblem Setup
Rewards are given for each learner response based on its
appropriateness
15Learning to speak and remember
Horizon
RoboticsMemory Augmented Imitation + Behavior Shaping
Through RL
Interpreter Speaker
Vision Memory
What is this? It is a bird.
Reward
16Learning to speak and remember
Horizon
RoboticsModel Detail
17
Trained end-to-end using gradient descent over Imitation Cost + Reinforce Cost
Learning to speak and remember
Horizon
RoboticsExample Dialogs
18Learning to speak and remember
T: Virtual teacher
L: Learner (machine)
T: i see grape
L: watermelon grape watermelon
T: tell what you see
L: see see see see see
T: there is grape
L: grape grape watermelon
T: i can observe coconut
L: fox watermelon watermelon
-------------------------------------------------
_________________________________
-------------------------------------------------
-------------------------------------------------
_________________________________
Before learning
After learning
Horizon
RoboticsSummary
 What we have now:
 Learning to understand and use simple
language, memorize useful information, and
execute simple commands from the
interactions with a virtual teacher in virtual
environments
 What we will do in the future:
 Simple → complex
 Virtual → real
19
Horizon
RoboticsAI Research at Horizon Robotics
 About the company
 A leading technology powerhouse of edge AI platform
 Provide algorithms, processors and hardware jointly optimized for high-performance, low-
power and low-cost edge AI capabilities
 CES 2019 Innovation Reward
 General AI Lab @ Silicon Valley
 Research towards the company’s long term vision for artificial general intelligence
 Build machines that can learn skills and knowledges unspecified at design time
 Applied AI Lab @ Silicon Valley
 Applied research focusing on near term needs
 Developing novel AI technologies that are critical to our current products
Job: bit.ly/general-ai-lab
bit.ly/applied-ai-lab
20
Horizon
Robotics
THANKS!
21

More Related Content

What's hot

State of the art in Natural Language Processing (March 2019)
State of the art in Natural Language Processing (March 2019)State of the art in Natural Language Processing (March 2019)
State of the art in Natural Language Processing (March 2019)Liad Magen
 
Li Deng at AI Frontiers: Three Generations of Spoken Dialogue Systems (Bots)
Li Deng at AI Frontiers: Three Generations of Spoken Dialogue Systems (Bots)Li Deng at AI Frontiers: Three Generations of Spoken Dialogue Systems (Bots)
Li Deng at AI Frontiers: Three Generations of Spoken Dialogue Systems (Bots)AI Frontiers
 
Deep learning for natural language embeddings
Deep learning for natural language embeddingsDeep learning for natural language embeddings
Deep learning for natural language embeddingsRoelof Pieters
 
Deep Learning for Natural Language Processing
Deep Learning for Natural Language ProcessingDeep Learning for Natural Language Processing
Deep Learning for Natural Language ProcessingDevashish Shanker
 
Multi modal retrieval and generation with deep distributed models
Multi modal retrieval and generation with deep distributed modelsMulti modal retrieval and generation with deep distributed models
Multi modal retrieval and generation with deep distributed modelsRoelof Pieters
 
Adam Coates at AI Frontiers: AI for 100 Million People with Deep Learning
Adam Coates at AI Frontiers: AI for 100 Million People with Deep LearningAdam Coates at AI Frontiers: AI for 100 Million People with Deep Learning
Adam Coates at AI Frontiers: AI for 100 Million People with Deep LearningAI Frontiers
 
Recent Advances in NLP
  Recent Advances in NLP  Recent Advances in NLP
Recent Advances in NLPAnuj Gupta
 
Deep Learning in real world @Deep Learning Tokyo
Deep Learning in real world @Deep Learning TokyoDeep Learning in real world @Deep Learning Tokyo
Deep Learning in real world @Deep Learning TokyoPreferred Networks
 
The Unreasonable Benefits of Deep Learning
The Unreasonable Benefits of Deep LearningThe Unreasonable Benefits of Deep Learning
The Unreasonable Benefits of Deep Learningindico data
 
Introduction to Artificial Intelligence
Introduction to Artificial IntelligenceIntroduction to Artificial Intelligence
Introduction to Artificial Intelligenceananth
 
MixTaiwan 20170222 清大電機 孫民 AI The Next Big Thing
MixTaiwan 20170222 清大電機 孫民 AI The Next Big ThingMixTaiwan 20170222 清大電機 孫民 AI The Next Big Thing
MixTaiwan 20170222 清大電機 孫民 AI The Next Big ThingMix Taiwan
 
Nikko Ström at AI Frontiers: Deep Learning in Alexa
Nikko Ström at AI Frontiers: Deep Learning in AlexaNikko Ström at AI Frontiers: Deep Learning in Alexa
Nikko Ström at AI Frontiers: Deep Learning in AlexaAI Frontiers
 

What's hot (12)

State of the art in Natural Language Processing (March 2019)
State of the art in Natural Language Processing (March 2019)State of the art in Natural Language Processing (March 2019)
State of the art in Natural Language Processing (March 2019)
 
Li Deng at AI Frontiers: Three Generations of Spoken Dialogue Systems (Bots)
Li Deng at AI Frontiers: Three Generations of Spoken Dialogue Systems (Bots)Li Deng at AI Frontiers: Three Generations of Spoken Dialogue Systems (Bots)
Li Deng at AI Frontiers: Three Generations of Spoken Dialogue Systems (Bots)
 
Deep learning for natural language embeddings
Deep learning for natural language embeddingsDeep learning for natural language embeddings
Deep learning for natural language embeddings
 
Deep Learning for Natural Language Processing
Deep Learning for Natural Language ProcessingDeep Learning for Natural Language Processing
Deep Learning for Natural Language Processing
 
Multi modal retrieval and generation with deep distributed models
Multi modal retrieval and generation with deep distributed modelsMulti modal retrieval and generation with deep distributed models
Multi modal retrieval and generation with deep distributed models
 
Adam Coates at AI Frontiers: AI for 100 Million People with Deep Learning
Adam Coates at AI Frontiers: AI for 100 Million People with Deep LearningAdam Coates at AI Frontiers: AI for 100 Million People with Deep Learning
Adam Coates at AI Frontiers: AI for 100 Million People with Deep Learning
 
Recent Advances in NLP
  Recent Advances in NLP  Recent Advances in NLP
Recent Advances in NLP
 
Deep Learning in real world @Deep Learning Tokyo
Deep Learning in real world @Deep Learning TokyoDeep Learning in real world @Deep Learning Tokyo
Deep Learning in real world @Deep Learning Tokyo
 
The Unreasonable Benefits of Deep Learning
The Unreasonable Benefits of Deep LearningThe Unreasonable Benefits of Deep Learning
The Unreasonable Benefits of Deep Learning
 
Introduction to Artificial Intelligence
Introduction to Artificial IntelligenceIntroduction to Artificial Intelligence
Introduction to Artificial Intelligence
 
MixTaiwan 20170222 清大電機 孫民 AI The Next Big Thing
MixTaiwan 20170222 清大電機 孫民 AI The Next Big ThingMixTaiwan 20170222 清大電機 孫民 AI The Next Big Thing
MixTaiwan 20170222 清大電機 孫民 AI The Next Big Thing
 
Nikko Ström at AI Frontiers: Deep Learning in Alexa
Nikko Ström at AI Frontiers: Deep Learning in AlexaNikko Ström at AI Frontiers: Deep Learning in Alexa
Nikko Ström at AI Frontiers: Deep Learning in Alexa
 

Similar to Wei Xu at AI Frontiers : Language Learning in an Interactive and Embodied Setting

[244]로봇이 현실 세계에 대해 학습하도록 만들기
[244]로봇이 현실 세계에 대해 학습하도록 만들기[244]로봇이 현실 세계에 대해 학습하도록 만들기
[244]로봇이 현실 세계에 대해 학습하도록 만들기NAVER D2
 
Sam Spaulding - Emotion AI Developer Day 2016
Sam Spaulding - Emotion AI Developer Day 2016Sam Spaulding - Emotion AI Developer Day 2016
Sam Spaulding - Emotion AI Developer Day 2016Affectiva
 
Antimo Musone - Vocal Assistant - build natural and rich conversational exper...
Antimo Musone - Vocal Assistant - build natural and rich conversational exper...Antimo Musone - Vocal Assistant - build natural and rich conversational exper...
Antimo Musone - Vocal Assistant - build natural and rich conversational exper...Codemotion
 
Command, Goal Disambiguation, Introspection, and Instruction in Gesture-Free ...
Command, Goal Disambiguation, Introspection, and Instruction in Gesture-Free ...Command, Goal Disambiguation, Introspection, and Instruction in Gesture-Free ...
Command, Goal Disambiguation, Introspection, and Instruction in Gesture-Free ...Vladimir Kulyukin
 
Introduction to Software Engineering Lecture 0
Introduction to Software Engineering  Lecture 0Introduction to Software Engineering  Lecture 0
Introduction to Software Engineering Lecture 0Mohamed Essam
 
[GTC 2019] Bringing Personal Robots Home: Integrating Computer Vision and Hum...
[GTC 2019] Bringing Personal Robots Home: Integrating Computer Vision and Hum...[GTC 2019] Bringing Personal Robots Home: Integrating Computer Vision and Hum...
[GTC 2019] Bringing Personal Robots Home: Integrating Computer Vision and Hum...Preferred Networks
 
Meetup 6/3/2017 - Artificiële Intelligentie: over chatbots & robots
Meetup 6/3/2017 - Artificiële Intelligentie: over chatbots & robotsMeetup 6/3/2017 - Artificiële Intelligentie: over chatbots & robots
Meetup 6/3/2017 - Artificiële Intelligentie: over chatbots & robotsDigipolis Antwerpen
 
Botbeans CISTI 2011
Botbeans CISTI 2011Botbeans CISTI 2011
Botbeans CISTI 2011Pedro Dias
 
Wireless Pick & Place Robot
Wireless Pick & Place RobotWireless Pick & Place Robot
Wireless Pick & Place RobotMarmik Kothari
 
From Natural Language Processing to Artificial Intelligence
From Natural Language Processing to Artificial IntelligenceFrom Natural Language Processing to Artificial Intelligence
From Natural Language Processing to Artificial IntelligenceJonathan Mugan
 
Key Features Of The Pseudo Code
Key Features Of The Pseudo CodeKey Features Of The Pseudo Code
Key Features Of The Pseudo CodeAngilina Jones
 
420_421.ppt
420_421.ppt420_421.ppt
420_421.pptbutest
 
Arti Languages Pre Seed Pitchdeck 2024.pdf
Arti Languages Pre Seed Pitchdeck 2024.pdfArti Languages Pre Seed Pitchdeck 2024.pdf
Arti Languages Pre Seed Pitchdeck 2024.pdfwill854175
 
Understanding Artificial Intelligence - Major concepts for enterprise applica...
Understanding Artificial Intelligence - Major concepts for enterprise applica...Understanding Artificial Intelligence - Major concepts for enterprise applica...
Understanding Artificial Intelligence - Major concepts for enterprise applica...APPANION
 
Designing the Workflow of a Language Interpretation Device Using Artificial I...
Designing the Workflow of a Language Interpretation Device Using Artificial I...Designing the Workflow of a Language Interpretation Device Using Artificial I...
Designing the Workflow of a Language Interpretation Device Using Artificial I...IOSR Journals
 
The Four Principles Of Object Oriented Programming
The Four Principles Of Object Oriented ProgrammingThe Four Principles Of Object Oriented Programming
The Four Principles Of Object Oriented ProgrammingDiane Allen
 
Arti Languages Pre Seed Send Ahead Pitchdeck 2024.pdf
Arti Languages Pre Seed Send Ahead Pitchdeck 2024.pdfArti Languages Pre Seed Send Ahead Pitchdeck 2024.pdf
Arti Languages Pre Seed Send Ahead Pitchdeck 2024.pdfwill854175
 
Programming: Past, Present, Future
Programming: Past, Present, FutureProgramming: Past, Present, Future
Programming: Past, Present, FutureJohn Thomas
 

Similar to Wei Xu at AI Frontiers : Language Learning in an Interactive and Embodied Setting (20)

[244]로봇이 현실 세계에 대해 학습하도록 만들기
[244]로봇이 현실 세계에 대해 학습하도록 만들기[244]로봇이 현실 세계에 대해 학습하도록 만들기
[244]로봇이 현실 세계에 대해 학습하도록 만들기
 
Sam Spaulding - Emotion AI Developer Day 2016
Sam Spaulding - Emotion AI Developer Day 2016Sam Spaulding - Emotion AI Developer Day 2016
Sam Spaulding - Emotion AI Developer Day 2016
 
Antimo Musone - Vocal Assistant - build natural and rich conversational exper...
Antimo Musone - Vocal Assistant - build natural and rich conversational exper...Antimo Musone - Vocal Assistant - build natural and rich conversational exper...
Antimo Musone - Vocal Assistant - build natural and rich conversational exper...
 
TotalSynch-PitchDeck
TotalSynch-PitchDeckTotalSynch-PitchDeck
TotalSynch-PitchDeck
 
Command, Goal Disambiguation, Introspection, and Instruction in Gesture-Free ...
Command, Goal Disambiguation, Introspection, and Instruction in Gesture-Free ...Command, Goal Disambiguation, Introspection, and Instruction in Gesture-Free ...
Command, Goal Disambiguation, Introspection, and Instruction in Gesture-Free ...
 
Introduction to Software Engineering Lecture 0
Introduction to Software Engineering  Lecture 0Introduction to Software Engineering  Lecture 0
Introduction to Software Engineering Lecture 0
 
[GTC 2019] Bringing Personal Robots Home: Integrating Computer Vision and Hum...
[GTC 2019] Bringing Personal Robots Home: Integrating Computer Vision and Hum...[GTC 2019] Bringing Personal Robots Home: Integrating Computer Vision and Hum...
[GTC 2019] Bringing Personal Robots Home: Integrating Computer Vision and Hum...
 
Meetup 6/3/2017 - Artificiële Intelligentie: over chatbots & robots
Meetup 6/3/2017 - Artificiële Intelligentie: over chatbots & robotsMeetup 6/3/2017 - Artificiële Intelligentie: over chatbots & robots
Meetup 6/3/2017 - Artificiële Intelligentie: over chatbots & robots
 
Botbeans CISTI 2011
Botbeans CISTI 2011Botbeans CISTI 2011
Botbeans CISTI 2011
 
Wireless Pick & Place Robot
Wireless Pick & Place RobotWireless Pick & Place Robot
Wireless Pick & Place Robot
 
From Natural Language Processing to Artificial Intelligence
From Natural Language Processing to Artificial IntelligenceFrom Natural Language Processing to Artificial Intelligence
From Natural Language Processing to Artificial Intelligence
 
Key Features Of The Pseudo Code
Key Features Of The Pseudo CodeKey Features Of The Pseudo Code
Key Features Of The Pseudo Code
 
420_421.ppt
420_421.ppt420_421.ppt
420_421.ppt
 
ROBOTICS.pptx
ROBOTICS.pptxROBOTICS.pptx
ROBOTICS.pptx
 
Arti Languages Pre Seed Pitchdeck 2024.pdf
Arti Languages Pre Seed Pitchdeck 2024.pdfArti Languages Pre Seed Pitchdeck 2024.pdf
Arti Languages Pre Seed Pitchdeck 2024.pdf
 
Understanding Artificial Intelligence - Major concepts for enterprise applica...
Understanding Artificial Intelligence - Major concepts for enterprise applica...Understanding Artificial Intelligence - Major concepts for enterprise applica...
Understanding Artificial Intelligence - Major concepts for enterprise applica...
 
Designing the Workflow of a Language Interpretation Device Using Artificial I...
Designing the Workflow of a Language Interpretation Device Using Artificial I...Designing the Workflow of a Language Interpretation Device Using Artificial I...
Designing the Workflow of a Language Interpretation Device Using Artificial I...
 
The Four Principles Of Object Oriented Programming
The Four Principles Of Object Oriented ProgrammingThe Four Principles Of Object Oriented Programming
The Four Principles Of Object Oriented Programming
 
Arti Languages Pre Seed Send Ahead Pitchdeck 2024.pdf
Arti Languages Pre Seed Send Ahead Pitchdeck 2024.pdfArti Languages Pre Seed Send Ahead Pitchdeck 2024.pdf
Arti Languages Pre Seed Send Ahead Pitchdeck 2024.pdf
 
Programming: Past, Present, Future
Programming: Past, Present, FutureProgramming: Past, Present, Future
Programming: Past, Present, Future
 

More from AI Frontiers

Divya Jain at AI Frontiers : Video Summarization
Divya Jain at AI Frontiers : Video SummarizationDivya Jain at AI Frontiers : Video Summarization
Divya Jain at AI Frontiers : Video SummarizationAI Frontiers
 
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI AI Frontiers
 
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 1: Heuristi...
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 1: Heuristi...Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 1: Heuristi...
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 1: Heuristi...AI Frontiers
 
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-lecture 2: Incremen...
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-lecture 2: Incremen...Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-lecture 2: Incremen...
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-lecture 2: Incremen...AI Frontiers
 
Training at AI Frontiers 2018 - Udacity: Enhancing NLP with Deep Neural Networks
Training at AI Frontiers 2018 - Udacity: Enhancing NLP with Deep Neural NetworksTraining at AI Frontiers 2018 - Udacity: Enhancing NLP with Deep Neural Networks
Training at AI Frontiers 2018 - Udacity: Enhancing NLP with Deep Neural NetworksAI Frontiers
 
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 3: Any-Angl...
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 3: Any-Angl...Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 3: Any-Angl...
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 3: Any-Angl...AI Frontiers
 
Percy Liang at AI Frontiers : Pushing the Limits of Machine Learning
Percy Liang at AI Frontiers : Pushing the Limits of Machine LearningPercy Liang at AI Frontiers : Pushing the Limits of Machine Learning
Percy Liang at AI Frontiers : Pushing the Limits of Machine LearningAI Frontiers
 
Mark Moore at AI Frontiers : Uber Elevate
Mark Moore at AI Frontiers : Uber ElevateMark Moore at AI Frontiers : Uber Elevate
Mark Moore at AI Frontiers : Uber ElevateAI Frontiers
 
Mario Munich at AI Frontiers : Consumer robotics: embedding affordable AI in ...
Mario Munich at AI Frontiers : Consumer robotics: embedding affordable AI in ...Mario Munich at AI Frontiers : Consumer robotics: embedding affordable AI in ...
Mario Munich at AI Frontiers : Consumer robotics: embedding affordable AI in ...AI Frontiers
 
Arnaud Thiercelin at AI Frontiers : AI in the Sky
Arnaud Thiercelin at AI Frontiers : AI in the SkyArnaud Thiercelin at AI Frontiers : AI in the Sky
Arnaud Thiercelin at AI Frontiers : AI in the SkyAI Frontiers
 
Anima Anandkumar at AI Frontiers : Modern ML : Deep, distributed, Multi-dimen...
Anima Anandkumar at AI Frontiers : Modern ML : Deep, distributed, Multi-dimen...Anima Anandkumar at AI Frontiers : Modern ML : Deep, distributed, Multi-dimen...
Anima Anandkumar at AI Frontiers : Modern ML : Deep, distributed, Multi-dimen...AI Frontiers
 
Sumit Gupta at AI Frontiers : AI for Enterprise
Sumit Gupta at AI Frontiers : AI for EnterpriseSumit Gupta at AI Frontiers : AI for Enterprise
Sumit Gupta at AI Frontiers : AI for EnterpriseAI Frontiers
 
Yuandong Tian at AI Frontiers : Planning in Reinforcement Learning
Yuandong Tian at AI Frontiers : Planning in Reinforcement LearningYuandong Tian at AI Frontiers : Planning in Reinforcement Learning
Yuandong Tian at AI Frontiers : Planning in Reinforcement LearningAI Frontiers
 
Alex Ermolaev at AI Frontiers : Major Applications of AI in Healthcare
Alex Ermolaev at AI Frontiers : Major Applications of AI in HealthcareAlex Ermolaev at AI Frontiers : Major Applications of AI in Healthcare
Alex Ermolaev at AI Frontiers : Major Applications of AI in HealthcareAI Frontiers
 
Long Lin at AI Frontiers : AI in Gaming
Long Lin at AI Frontiers : AI in GamingLong Lin at AI Frontiers : AI in Gaming
Long Lin at AI Frontiers : AI in GamingAI Frontiers
 
Melissa Goldman at AI Frontiers : AI & Finance
Melissa Goldman at AI Frontiers : AI & FinanceMelissa Goldman at AI Frontiers : AI & Finance
Melissa Goldman at AI Frontiers : AI & FinanceAI Frontiers
 
Li Deng at AI Frontiers : From Modeling Speech/Language to Modeling Financial...
Li Deng at AI Frontiers : From Modeling Speech/Language to Modeling Financial...Li Deng at AI Frontiers : From Modeling Speech/Language to Modeling Financial...
Li Deng at AI Frontiers : From Modeling Speech/Language to Modeling Financial...AI Frontiers
 
Ashok Srivastava at AI Frontiers : Using AI to Solve Complex Economic Problems
Ashok Srivastava at AI Frontiers : Using AI to Solve Complex Economic ProblemsAshok Srivastava at AI Frontiers : Using AI to Solve Complex Economic Problems
Ashok Srivastava at AI Frontiers : Using AI to Solve Complex Economic ProblemsAI Frontiers
 
Rohit Tripathi at AI Frontiers : Using intelligent connectivity and AI to tra...
Rohit Tripathi at AI Frontiers : Using intelligent connectivity and AI to tra...Rohit Tripathi at AI Frontiers : Using intelligent connectivity and AI to tra...
Rohit Tripathi at AI Frontiers : Using intelligent connectivity and AI to tra...AI Frontiers
 
Kai-Fu Lee at AI Frontiers : The Era of Artificial Intelligence
Kai-Fu Lee at AI Frontiers : The Era of Artificial IntelligenceKai-Fu Lee at AI Frontiers : The Era of Artificial Intelligence
Kai-Fu Lee at AI Frontiers : The Era of Artificial IntelligenceAI Frontiers
 

More from AI Frontiers (20)

Divya Jain at AI Frontiers : Video Summarization
Divya Jain at AI Frontiers : Video SummarizationDivya Jain at AI Frontiers : Video Summarization
Divya Jain at AI Frontiers : Video Summarization
 
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
Training at AI Frontiers 2018 - LaiOffer Data Session: How Spark Speedup AI
 
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 1: Heuristi...
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 1: Heuristi...Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 1: Heuristi...
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 1: Heuristi...
 
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-lecture 2: Incremen...
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-lecture 2: Incremen...Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-lecture 2: Incremen...
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-lecture 2: Incremen...
 
Training at AI Frontiers 2018 - Udacity: Enhancing NLP with Deep Neural Networks
Training at AI Frontiers 2018 - Udacity: Enhancing NLP with Deep Neural NetworksTraining at AI Frontiers 2018 - Udacity: Enhancing NLP with Deep Neural Networks
Training at AI Frontiers 2018 - Udacity: Enhancing NLP with Deep Neural Networks
 
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 3: Any-Angl...
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 3: Any-Angl...Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 3: Any-Angl...
Training at AI Frontiers 2018 - LaiOffer Self-Driving-Car-Lecture 3: Any-Angl...
 
Percy Liang at AI Frontiers : Pushing the Limits of Machine Learning
Percy Liang at AI Frontiers : Pushing the Limits of Machine LearningPercy Liang at AI Frontiers : Pushing the Limits of Machine Learning
Percy Liang at AI Frontiers : Pushing the Limits of Machine Learning
 
Mark Moore at AI Frontiers : Uber Elevate
Mark Moore at AI Frontiers : Uber ElevateMark Moore at AI Frontiers : Uber Elevate
Mark Moore at AI Frontiers : Uber Elevate
 
Mario Munich at AI Frontiers : Consumer robotics: embedding affordable AI in ...
Mario Munich at AI Frontiers : Consumer robotics: embedding affordable AI in ...Mario Munich at AI Frontiers : Consumer robotics: embedding affordable AI in ...
Mario Munich at AI Frontiers : Consumer robotics: embedding affordable AI in ...
 
Arnaud Thiercelin at AI Frontiers : AI in the Sky
Arnaud Thiercelin at AI Frontiers : AI in the SkyArnaud Thiercelin at AI Frontiers : AI in the Sky
Arnaud Thiercelin at AI Frontiers : AI in the Sky
 
Anima Anandkumar at AI Frontiers : Modern ML : Deep, distributed, Multi-dimen...
Anima Anandkumar at AI Frontiers : Modern ML : Deep, distributed, Multi-dimen...Anima Anandkumar at AI Frontiers : Modern ML : Deep, distributed, Multi-dimen...
Anima Anandkumar at AI Frontiers : Modern ML : Deep, distributed, Multi-dimen...
 
Sumit Gupta at AI Frontiers : AI for Enterprise
Sumit Gupta at AI Frontiers : AI for EnterpriseSumit Gupta at AI Frontiers : AI for Enterprise
Sumit Gupta at AI Frontiers : AI for Enterprise
 
Yuandong Tian at AI Frontiers : Planning in Reinforcement Learning
Yuandong Tian at AI Frontiers : Planning in Reinforcement LearningYuandong Tian at AI Frontiers : Planning in Reinforcement Learning
Yuandong Tian at AI Frontiers : Planning in Reinforcement Learning
 
Alex Ermolaev at AI Frontiers : Major Applications of AI in Healthcare
Alex Ermolaev at AI Frontiers : Major Applications of AI in HealthcareAlex Ermolaev at AI Frontiers : Major Applications of AI in Healthcare
Alex Ermolaev at AI Frontiers : Major Applications of AI in Healthcare
 
Long Lin at AI Frontiers : AI in Gaming
Long Lin at AI Frontiers : AI in GamingLong Lin at AI Frontiers : AI in Gaming
Long Lin at AI Frontiers : AI in Gaming
 
Melissa Goldman at AI Frontiers : AI & Finance
Melissa Goldman at AI Frontiers : AI & FinanceMelissa Goldman at AI Frontiers : AI & Finance
Melissa Goldman at AI Frontiers : AI & Finance
 
Li Deng at AI Frontiers : From Modeling Speech/Language to Modeling Financial...
Li Deng at AI Frontiers : From Modeling Speech/Language to Modeling Financial...Li Deng at AI Frontiers : From Modeling Speech/Language to Modeling Financial...
Li Deng at AI Frontiers : From Modeling Speech/Language to Modeling Financial...
 
Ashok Srivastava at AI Frontiers : Using AI to Solve Complex Economic Problems
Ashok Srivastava at AI Frontiers : Using AI to Solve Complex Economic ProblemsAshok Srivastava at AI Frontiers : Using AI to Solve Complex Economic Problems
Ashok Srivastava at AI Frontiers : Using AI to Solve Complex Economic Problems
 
Rohit Tripathi at AI Frontiers : Using intelligent connectivity and AI to tra...
Rohit Tripathi at AI Frontiers : Using intelligent connectivity and AI to tra...Rohit Tripathi at AI Frontiers : Using intelligent connectivity and AI to tra...
Rohit Tripathi at AI Frontiers : Using intelligent connectivity and AI to tra...
 
Kai-Fu Lee at AI Frontiers : The Era of Artificial Intelligence
Kai-Fu Lee at AI Frontiers : The Era of Artificial IntelligenceKai-Fu Lee at AI Frontiers : The Era of Artificial Intelligence
Kai-Fu Lee at AI Frontiers : The Era of Artificial Intelligence
 

Recently uploaded

Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...CzechDreamin
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyUXDXConf
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe中 央社
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfFIDO Alliance
 
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties ReimaginedEasier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties Reimaginedpanagenda
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfFIDO Alliance
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024Lorenzo Miniero
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...FIDO Alliance
 
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfFIDO Alliance
 
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptxBT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptxNeo4j
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024Stephen Perrenod
 
Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsStefano
 
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FIDO Alliance
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentationyogeshlabana357357
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101vincent683379
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfUK Journal
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfFIDO Alliance
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireExakis Nelite
 

Recently uploaded (20)

Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties ReimaginedEasier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
 
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
 
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptxBT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
 
Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 

Wei Xu at AI Frontiers : Language Learning in an Interactive and Embodied Setting

  • 1. Horizon Robotics Language Learning in an Interactive and Embodied Setting 11/2018 Wei Xu 1 Horizon Robotics
  • 2. Horizon RoboticsA Developmental Approach to Machine Intelligence 1. It might be easier than solving all the tasks a human adult can do 2. Learn skills and knowledges unspecified at design time 3. Gradually proceed from easy tasks to difficult tasks 2 “Instead of trying to produce a program to simulate the adult mind, why not rather try to produce one which simulates the child's? If this were then subjected to an appropriate course of education one would obtain the adult brain.” - Alan Turing (1950) Language learning in an interactive and embodied setting
  • 3. Horizon RoboticsWhy Embodied?  Learn from the experiences coming from the machine’s interactions with its environment  Learn commonsense through the observation and interaction with the environment  Meaning emerges by “grounding” language in modalities in our environment 3Language learning in an interactive and embodied setting Human driving: < 1000 miles Self-driving: >10 million miles
  • 4. Horizon RoboticsWhy Interactive?  A useful robot needs to be able to understand and communicate effectively  It is easier for human to teach machines directly using language than writing code  Humans are great teachers  Learn the effects of speaking by observing feedbacks from conversational partner  Learn human value through the interaction 4Language learning in an interactive and embodied setting
  • 5. Horizon RoboticsAnswering Questions and Following Commands 1. Is it possible to learn to follow commands using end-to-end reinforcement learning without any pretraining for vision or language? 2. Whether learning question answering can help learning command 3. Can the machine understand words under new context not seen in training? 5 Haonan Yu, Haichao. Zhang, Wei Xu “Interactive Grounded Language Acquisition and Generalization in a 2D World” ICLR 2018
  • 6. Horizon RoboticsProblem Setup 6Answering questions and following commands east and avocado never appears together in training Watermelon only appears in answers during training
  • 7. Horizon RoboticsModel architecture 7Answering questions and following commands answer action value
  • 8. Horizon RoboticsExperiments 8Answering questions and following commands No QA training
  • 9. Horizon RoboticsGeneralization Ability 9 We can generalize to word combinations never seen in training We can generalize to questions containing words never seen in training Answering questions and following commands Held out X(%): %X of word/combinations are held out from training
  • 10. Horizon Robotics Challenges:  Partially observed  Much longer delay of reward  More visual variations “Navigate to the dog!”Navigation in a 3D Environment 10
  • 11. Horizon RoboticsGuided Feature Transformation Haonan Yu, Xiaochen Lian, Haichao Zhang, Wei. Xu “Guided Feature Transformation (GFT): A Neural Language Grounding Module for Embodied Agents” CoRL 2018 11Navigation in 3D environment action value
  • 13. Horizon RoboticsDemo the object besides candle is your target . please move to the object that is front of the basketball . can you reach the object right of toilet ?go to the object to the right of bike please .reach the location between car and trampoline please.please navigate to the grid between gift and tower .please navigate to the grid between bucket and chair .please move to the object that is front of basketball . 13Navigation in 3D environment
  • 14. Horizon RoboticsLearning to Speak and Remember 1. How to learn to speak by talking with other people? 2. What information should be remembered? 3. How to utilize knowledge in memory? 14 Haichao Zhang, Haonan Yu, Wei Xu “Interactive Language Acquisition with One-Shot Visual Concept Learning through a Conversation Game” ACL 2018
  • 15. Horizon RoboticsProblem Setup Rewards are given for each learner response based on its appropriateness 15Learning to speak and remember
  • 16. Horizon RoboticsMemory Augmented Imitation + Behavior Shaping Through RL Interpreter Speaker Vision Memory What is this? It is a bird. Reward 16Learning to speak and remember
  • 17. Horizon RoboticsModel Detail 17 Trained end-to-end using gradient descent over Imitation Cost + Reinforce Cost Learning to speak and remember
  • 18. Horizon RoboticsExample Dialogs 18Learning to speak and remember T: Virtual teacher L: Learner (machine) T: i see grape L: watermelon grape watermelon T: tell what you see L: see see see see see T: there is grape L: grape grape watermelon T: i can observe coconut L: fox watermelon watermelon ------------------------------------------------- _________________________________ ------------------------------------------------- ------------------------------------------------- _________________________________ Before learning After learning
  • 19. Horizon RoboticsSummary  What we have now:  Learning to understand and use simple language, memorize useful information, and execute simple commands from the interactions with a virtual teacher in virtual environments  What we will do in the future:  Simple → complex  Virtual → real 19
  • 20. Horizon RoboticsAI Research at Horizon Robotics  About the company  A leading technology powerhouse of edge AI platform  Provide algorithms, processors and hardware jointly optimized for high-performance, low- power and low-cost edge AI capabilities  CES 2019 Innovation Reward  General AI Lab @ Silicon Valley  Research towards the company’s long term vision for artificial general intelligence  Build machines that can learn skills and knowledges unspecified at design time  Applied AI Lab @ Silicon Valley  Applied research focusing on near term needs  Developing novel AI technologies that are critical to our current products Job: bit.ly/general-ai-lab bit.ly/applied-ai-lab 20

Editor's Notes

  1. Good afternoon everyone. I am Wei Xu from Horizon Robotics. Today I am going to talk about our recent work on language learning in an interactive and embodied setting
  2. In 1950, in the same article where the famous Turing test was proposed, Turing also proposed a solution. “Instead of trying to produce a program to simulate the adult mind, why not rather try to produce one which simulates the child's? If this were then subjected to an appropriate course of education one would obtain the adult brain”. There are several advantages of this approach. First, there are so many things that a human adult can do, it will be too expensive and difficult to individually solve each one of them. Second, emphasizing that all the skills and knowledge of the machine are acquired through its own learning can make sure that the machine will be able to learn new skills and new knowledge unspecified at design time. Third, learning in a developmental way lets the machine gradually proceed from easier tasks to more difficult tasks, which can make the learning easier. This is like curriculum learning which is found to be effective in many difficult learning problems.
  3. For embodied learning, the learning experiences are from machine’s physical interactions with its environment. By actually doing things and observing the effects, the machine can learn a lots of common sense knowledge about the environment. These kinds of knowledges are typically very hard to be captured by rules or a static dataset. Self-driving car is a great example. Waymo recently announced that the total mileage of their cars is exceeding 10 million miles. Yet they are still not fully ready for deployment. On the other hand, we all know from our experience that a human can learn to drive very well with a few hundred miles practice. A key difference between the self-driving car and human is that human has a lot of commonsense knowledge about the world. For example, even without learning to drive, a human driver knows what situation is unsafe, what obstacles should be avoided, and so on. But for self-driving, all of these commonsense knowledge has to be either coded by rules or obtained from huge amount of driving data. Embodied learning is also very help for understanding language. In order for the machine to understand and use language, it needs to connect word sequences with the actual objects and events in the environment. ……..
  4. Why should the machine learn in an interactive way? There are several reasons. First, a useful robot needs to interact with human, so it should be able to understand and communicate effectively with human. Second, it is easier for human to teach machines directly using language than writing code. And human are great teachers because they are good at adjusting the teaching based on the state of the learner. And in order to be able to use language, the machine needs to learn the effects of speaking by observing feedbacks from its conversational partner. Finally, through the interaction with human, the machine can learn the human value, which is very important to make sure it will do things consistent with human value. So I’ve talked about our motivation of learning language in an interactive and embodied setting. In the rest of the talk, I will talk about our recent work along this direction.
  5. The first one is about learning to answer questions and follow commands. This work was published in this year’s ICLR conference. There are two problems we want to study in this paper.
  6. Here is the problem setup. We developed a 2D simulator. For each session, we generate a random map, question and instructions. The answer is provided as direct supervision. The agent is given reward based on whether it successfully executed the instruction. At test time, the agent will be given commands with words or word combinations never seen in training commands or questions.
  7. This is the high level structure of our model. I won’t go into the detail of the model. What I want to say here we design the structure focusing on its generalization ability.
  8. This is a short video demo showing how the agent navigates following the commands. The current command is “please move to the object that is front of basket ball”. The agent needs to approach the toilet paper from the direction where it is in front of the basket ball. After it finishes a task, a new map and command will be generated.
  9. So far our agent is able to understand some language. In this work, we want the agent to learn to use language through conversation.
  10. Here the problem setup. In initially, the agent has zero language ability, cannot understand nor use it, just like a new born baby.
  11. This is a high level structure of our model. First, it needs to have memory module because it needs to remember information coming from teacher utterances and images. The vision module generates the visual representation. The interpreter module is for understanding teacher utterance and decide whether to store things into memory. The speaker module is responsible for generating responses based understanding of teacher utterances and information retrieved from memory. The whole system is trained by predicting teacher word sequence and the rewards indicating the appropriateness of the response.
  12. I will skip the detail of the model. Just note that it’s trained end-to-end using gradient descent over Imitation Cost + Reinforce Cost
  13. Here I show some dialog examples. This is a dialog before learning. The agent just generates some garbage responses, just like a newborn baby. Then dialogs after the learning. Here I want to mention is that the machine never see these types of object during training. From these dialogs we can see that the machine learned several things. It can confirm the statements from the teacher. It can actively seeking information by asking questions. It can remember relevant information provided by the teacher so later it can use it for answering questions. And it can also answer teacher’s questions if it knows the answer. It somehow learned to uses shape as major cue to differentiate objects. I want to emphasize that, unlike most chatbots, where the behavior of the bot is pretty much designed by human, here non of these behaviors are programmed. The machine learned all these behaviors through its interaction with the teacher, in a similar way as a baby learn from their parents.
  14. In this final slide, I am going to say a little bit about Horizon Robotics. It’s a leading technology powerhouse of edge AI platform. Its current focus is providing algorithms, processors and hardware jointly optimized for high-performance, low-power and low-cost edge AI capabilities. And I want to share a good news with you that we just received the CES 2019 Innovation Reward Vehicle Intelligence and Self-Driving Technology We have two AI Labs in Silicon Valley, one is general AI Lab. It’s doing the kind of research I just talked about, building machines that can learn skills and knowledges unspecified at design time We also have applied AI Lab. It’s doing applied research focusing on near term needs of the company, developing novel AI technologies that are critical to our current products We are actively hiring. If you are interested, please visit either of these two websites.