Presented at The 15th International Conference on Ubiquitous Robots June 28th, 2018

1. Jungsik Hwang1,2 and Jun Tani2
1Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea
2Okinawa Institute of Science and Technology (OIST), Japan
The 15th International Conference on Ubiquitous Robots
June 28th, 2018
A Dynamic Neural Network Approach
to Generating Robot’s Novel Actions
: A Simulation Experiment

2. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
How to Teach Actions
“Tutoring” “Learned Actions”
Learning
From Demonstration

3. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
Research Questions
“Tutoring” “Learned Actions”
Learning
From Demonstration
How can a robot generate novel
actions from learning basic actions?
“Novel Actions”

4. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
Research Questions
“Tutoring” “Learned Actions”
Learning
From Demonstration
How can a robot generate novel
actions from learning basic actions?
“Dynamic Neural Network Approach”
• Source of Novelty = Non-linear Memory Dynamics
“Novel Actions”

5. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
• Multiple Timescales RNN with Parametric Biases
Dynamic Neural Network Approach
Action
Generation
Action
Encoding
PF
PM
PS
PB
Basic Actions

6. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
• Maps robot’s actions into the low-dimensional space
Action Encoding Module
Action
Generation
Action
Encoding
PF
PM
PS
PB
Basic Actions
Self-organized
during
Training

7. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
• Learns actions in a multiple timescales structure
Action Generation Module
Action
Generation
Action
Encoding
PF
PM
PS
PB
Slowly
Changing
Neural
Activity
Fast
Changing
Neural
Activity
Basic Actions

8. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
• With given PB values in Action Encoding Modules
Generation of Action
Action
Generation
Action
Encoding
PF
PM
PS
PB
Basic Actions
New Actions

9. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
Results – Generation of Novel Action

10. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
• Appropriateness
– Actions that “can be used”
• e.g., Maximum angular velocity < Threshold
• Novelty
– Actions “different from learned action”
• Distance(Learned Action, Generated Action)
• Diversity
– Actions “different each other”
• Distance(Generated Action, Generated Action)
Measuring the Level of Creativity

11. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
• Appropriateness
– Actions that “can be used”
• e.g., Maximum angular velocity < Threshold
• Novelty
– Actions “different from learned action”
• Distance(Learned Action, Generated Action)
• Diversity
– Actions “different each other”
• Distance(Generated Action, Generated Action)
Measuring the Level of Creativity

12. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
• Appropriateness
– Actions that “can be used”
• e.g., Maximum angular velocity < Threshold
• Novelty
– Actions “different from learned action”
• Distance(Learned Action, Generated Action)
• Diversity
– Actions “different each other”
• Distance(Generated Action, Generated Action)
Measuring the Level of Creativity

13. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
• Appropriateness
– Actions that “can be used”
• e.g., Maximum angular velocity < Threshold
• Novelty
– Actions “different from learned action”
• Distance(Learned Action, Generated Action)
• Diversity
– Actions “different each other”
• Distance(Generated Action, Generated Action)
Measuring the Level of Creativity
The model’s level of
creativity depends on
the learning method.

14. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
Internal Structure in the Action Encoding Module
Action
Generation
Action
Encoding
PF
PM
PS
PB
• 200 x 200 points in 2D PB Space
• Each point encodes one action
• e.g., (0.3, 0.2) → Left Jab
• e.g., (0.1, 0.5) → Right Straight

15. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
Internal Structure in the Action Encoding Module
Action
Generation
Action
Encoding
PF
PM
PS
PB
• 200 x 200 points in 2D PB Space
• Each point encodes one action
• e.g., (0.3, 0.2) → Left Jab
• e.g., (0.1, 0.5) → Right Straight
• Self-organized during training

16. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
Internal Structure in the Action Encoding Module
Actions learned during training
Actions that cannot be used
Novel
Actions
Combination of
Right Straight & Left Uppercut

17. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
Internal Structure in the Action Encoding Module
Actions learned during training
Actions that cannot be used
Novel
Actions
Combination of
Right Straight & Left Uppercut

18. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
Internal Structure in the Action Encoding Module
Actions learned during training
Actions that cannot be used
Novel
Actions
Combination of
Right Straight & Left Uppercut

19. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
Internal Structure in the Action Encoding Module
Current PB Value
(in Action Encoding Module)
Robot’s Action
(Position values of 8 Joints)
“Rugged” PB Space
: Small changes in the PB Values ➔ Abrupt changes in robot’s action

20. A Dynamic Neural Network Approach to Generating Robot’s Novel Actions : A Simulation Experiment
[ThP1O]
• Generating Creative Robot Actions
– From the Dynamic Neural Network Perspective
• Neural Network Model (MTRNN-PB)
– Reproduces learned actions
– Generates novel actions
• Through modulating & combining those learned actions
– Self-organizes non-linear memory dynamics
• Source of novel actions
Summary

21. A photo from 2017 OIST Science Festival
“Creative Robot Dance Generation”
By Jungsik Hwang, Nadine Wirkuttis
and Jun Tani
Acknowledgement. This work was supported by the Industr
ial Strategic Technology Development Program (10044009) f
unded by the Ministry of Knowledge Economy in Korea and
Okinawa Institute of Science and Technology Graduate Univ
ersity in Japan.
A Dynamic Neural Network Approach
to Generating Robot’s Novel Actions
: A Simulation Experiment
By Jungsik Hwang and Jun Tani
[ThP10]