2020.01.20 RoboCup@Home Education (Introduction) [EN]

AI + Service Robot = AI Education Platform
RoboCup@Home EDUCATION
AI-Focused Robotics Education by Home Service Robot DIY
2020-01-20 | Jeffrey Too Chuan TAN

Jeffrey Too Chuan TAN（陈图川）
[ Education Background ]
2007 - 2010 The University of Tokyo (Japan), Department of Precision Engineering, Doctor of Engineering
2004 - 2007 Universiti Tenaga Nasional (Malaysia), Master of Mechanical Engineering
1999 - 2003 Universiti Tenaga Nasional (Malaysia), Bachelor of Mechanical Engineering (Hons.)
[ Working Experience ]
2017 - Present Associate Professor, Nankai University (China)《天津市青年千人计划》
2017 - Present Research Fellow, Tamagawa University (Japan)
2014 - 2017 Project Assistant Professor, Institute of Industrial Science, The University of Tokyo (Japan)
2015 - 2017 Adjunct Lecturer, Tokyo City University (Japan)
2013 - 2014 Project Researcher, Institute of Industrial Science, The University of Tokyo (Japan)
2011 - 2013 Project Researcher, National Institute of Informatics (Japan)
2010 - 2011 Project Researcher, Graduate School of Engineering, The University of Tokyo (Japan)
2004 - 2007 Tutor, Universiti Tenaga Nasional (Malaysia)
[ Professional Services ]
2016 - Present Committee (Service and Junior), World Robot Summit
2016 - 2019 Organizing Committee, RoboCup Federation (@Home)
2015 - Present Committee, RoboCup@Home Education
2014 - Present Organizing Committee, RoboCup Japan (@Home)
2
Profile

RoboCup@Home
RoboCup@Home aims to foster the development of service and assistive robot technology to make possible
future personal domestic applications. The competitions comprise of a set of benchmark tests to evaluate the
robots’ capabilities in realistic home environment settings and scenarios, with the research focuses on: human-
robot interaction and cooperation, navigation in dynamic environments, computer vision and object recognition
under natural light conditions, object manipulation, adaptive behaviors and learning, ambient intelligence, and
system integration.
3

Outline
1. Prologue: Team KameRider
2. RoboCup@Home EDUCATION Initiative
a. Education Challenge
b. Open Source Educational Robot Platforms
c. OpenCourseWare
d. Outreach Programs
3. Service Robotics R&D

1. PROLOGUE: TEAM KAMERIDER
http://openbotics.org/kamerider/

2013 The Beginning of Team KameRider
2013.05.03-06 RoboCup Japan Open 2013 Tokyo, Japan
• [UT] Jeffrey
• [Award] JSAI Award [SIGVerse for RoboCup @Home Simulation]
• [Award] RoboCup @Home Simulation [2nd Place]
2013.06.24-07.01 RoboCup 2013 Eindhoven, Netherlands
(International)
• [Symposium] Poster: “Open Web Based Development Platform for
RoboCup @Home Simulation”
• [Symposium] Oral: “Development of RoboCup@Home Simulation
towards Long-term Large Scale HRI”

2014 Entering RoboCup Japan Open
The Japanese Society for Artificial Intelligence Award
2014.03-06 Internship of Mr. Tey @ SIT, Japan
• [Internship] Mr. Tey (UTM) assisted Jeffrey's team in the
development of a basic robot platform for RoboCup
@Home
2014.05.03-06 RoboCup Japan Open 2014 Fukuoka, Japan
• [UT] Jeffrey, [NKU] 6 members, [UTM] Tey Wei Kang
• [Award] JSAI Award [Standard Platform for RoboCup
@Home]

2014 Open Source Educational
Robot Platform for @Home
2014.06-09 Internship of Mr. Seow @ UT, Japan
• [Internship] Mr. Seow (UTM) develops the basic robot
platform for RoboCup @Home based on the RCF support
2014.12.06 Intelligent Home Robotics Challenge 2014, Tokyo
• [UT] Jeffrey, [UTM] Lim Kian Sheng, Mohamad Hafizuddin
bin Majek, Muhammad Faiz bin Muhammad Rozi
• [Award] Mobile Robot Category 3rd Place
• [Award] Overall 3rd Place

2015 First Education Challenge
2015.05.03-06 RoboCup Japan Open 2015 Fukui, Japan
• [UT] Jeffrey, [NKU] 3 members, [UTM] Muhammad
Najib Abdullah, Nicole Tham Lei May
• [Award] RoboCup @Home SPL (Beta) [1st Place]
• [Award] RoboCup @Home Simulation [3rd Place]

2015 Entering International RoboCup
2015.07.17-23 RoboCup 2015 Hefei, China
(International)
• [UT] Jeffrey, [NKU] 7 members, [UTM]
Yeong Che Fai, Seow Yip Loon, Nicole Tham
Lei May
• Overall ranked 7th out of 17 qualified teams
• Top 9 teams to enter Stage 2

2016 Collaborative Team UT-NKU-UTM-SIT
2016.03.24-27 RoboCup Japan Open 2016 Aichi,
Japan
• [Award] RoboCup @Home Education [2nd Place]
• [Award] RoboCup @Home Simulation [1st Place]
2016.06.30-07.04 RoboCup 2015 Leipzig, Germany
(International)
• Overall ranked 7th out of 23 qualified teams
11

2017 Collaborative Team NKU-UTM-SIT
RoboCup Japan Open 2017 Nagoya
• [Award] RoboCup @Home Education [1st Place]
RoboCup 2017 Nagoya (International)
• [Award] RoboCup @Home SSPL [Overall ranked 4th
out of 7 qualified teams]
RoboCup Asia-Pacific 2017 Bangkok
• [Award] RoboCup @Home [1st Place]
• [Award] RoboCup @Home Education [1st Place]
12

Competition Achievements
2019 RoboCup Japan Open 2019: @Home OPL [3rd Place]
2019 RoboCup Japan Open 2019: @Home Education [3rd Place]
2019 RoboCup 2019 Sydney, Australia (International): @Home SSPL [3rd Place]
2019 RoboCup Asia-Pacific Tianjin Invitational Tournament 2019: @Home OPL [3rd Place]
2019 RoboCup Asia-Pacific Tianjin Invitational Tournament 2019: @Home OPL Technical Challenge [2nd Place]
2019 RoboCup Asia-Pacific Tianjin Invitational Tournament 2019: @Home SSPL [1st Place]
2019 RoboCup Asia-Pacific Tianjin Invitational Tournament 2019: @Home SSPL Technical Challenge [2nd Place]
2019 RoboCup China Open 2019: @Home SSPL [3rd Place]
2019 RoboCup China Open 2019: @Home SSPL Technical Challenge [1st Place]
2019 RoboCup China Open 2019: @Home [3rd Place]
2019 RoboCup China Open 2019: @Home Technical Challenge [1st Place]
2018 China Robot Skills Competition 2018: SoftBank Robotics Cup [1st Place]
2018 RoboCup Japan Open 2018 Ogaki: @Home Education [2nd Place]
2018 RoboCup China Open 2018: @Home Technical Challenge [1st Place]
2017 RoboCup Asia-Pacific 2017 Bangkok: @Home [1st Place]
2017 RoboCup Asia-Pacific 2017 Bangkok: @Home Education [1st Place]
2017 RoboCup 2017 Nagoya, Japan (International): @Home SSPL [Overall ranked 4th out of 7 qualified teams]
2017 RoboCup Japan Open 2017 Nagoya: @Home Education [1st Place]
2017 RoboCup Japan Open 2017 Nagoya: @Home Simulation [2nd Place]
2016 RoboCup 2016 Leipzig, Germany (International): @Home [Overall ranked 7th out of 23 qualified teams]
2016 RoboCup Japan Open 2016 Aichi: @Home Education [2nd Place]
2016 RoboCup Japan Open 2016 Aichi: @Home Simulation [1st Place]
2015 RoboCup 2015 Hefei, China (International): @Home [Overall ranked 7th out of 17 qualified teams]
2015 RoboCup Japan Open 2015 Fukui: @Home SPL [1st Place]
2015 RoboCup Japan Open 2015 Fukui: @Home Simulation [3rd Place]
2014 Intelligent Home Robotics Challenge 2014 [Overall 3rd Place]
2014 Intelligent Home Robotics Challenge 2014 [Mobile Robot Category 3rd Place]
2014 Japanese Society for Artificial Intelligence (JSAI) Award [Standard Platform for RoboCup @Home]
2014 RoboCup Japan Open 2014 Fukuoka: @Home Simulation [2nd Place]
2013 Japanese Society for Artificial Intelligence (JSAI) Award [SIGVerse for RoboCup @Home Simulation]
2013 RoboCup Japan Open 2013 Tokyo: @Home Simulation [2nd Place]
14

2. ROBOCUP@HOME EDUCATION
INITIATIVE
http://www.robocupathomeedu.org/
15

AI-Focused Robotics Education by
Home Service Robot DIY
The “Bridging Problem”
School-level Robotics Education vs University-level Robotics Research
• Bottom-up vs Top-down
• Conceptual Problems vs Real World Problems
The Blooming of AI, Cloud and Big Data
• Learning Platform and Ecosystem
16

RoboCup@Home EDUCATION
RoboCup@Home EDUCATION is an educational initiative in
RoboCup@Home that promotes educational efforts to boost
RoboCup@Home participation and artificial intelligence (AI)-
focused service robot development.
Under this initiative, currently there are 4 efforts in operation:
1. RoboCup@Home Education Challenge
2. Open Source Educational Robot Platforms for
RoboCup@Home
3. OpenCourseWare for the learning of AI-focused service
robotics
4. Outreach Programs (local workshops, international
academic exchanges, etc.)
http://www.robocupathomeedu.org/
https://www.facebook.com/robocupathomeedu/ 17

RC@Home
Education
RC@Home
Education
RC@Home
Education
Challenge
RC@Home
Education
Challenge
Open
Robot
Platform
Open
Robot
Platform
OutreachOutreach
RoboCup
Community
RoboCup
Community
iHR Challenge
Exchange Program
Workshop
RoboCup Japan Open
Promote
New Entry
Evolution by New
Contribution
Open
Courseware
Evaluate the
learning
Open Design &
Source Codes
Open Design &
Source Codes
Core Robot Platform
with Modular Add-ons
Robots in RoboCup
Japan Open 2016
Invited Lecture
Universities
& Research
Community
Universities
& Research
Community
RoboCup
Committee
RoboCup
Committee
Join RoboCup
Outreach to Public
Improve Service
Robot Development

Regional
Collaborators
[ Japan ]
• Organizer: RoboCup Japan Committee
• Coordinator:
– Hiroyuki OKADA (Tamagawa University)
– Yoshinobu HAGIWARA (Ritsumeikan University)
– Jeffrey Too Chuan TAN (Nankai University, China)
• Supporter:
– Yasuhiro MASUTANI (Osaka Electro-Communication University)
– Kosei DEMURA (Kanazawa Institute of Technology)
– Yuki INOUE (Osaka Institute of Technology)
– Kenichi OHARA (Meijo University)
[ Italy ]
• Organizer: Italian RoboCup Regional Committee
• Coordinator:
– Luca Iocchi (Sapienza University of Rome)
– Paola Ferrarelli (Sapienza University of Rome)
[ USA ]
• Collaborator: Amy EGUCHI (Bloomfield College), M. Q. Azhar (BMCC.CUNY)
[ Canada ]
• Collaborator: Sara Iatauro (English Montreal School Board)
[ China]
• Collaborator: 李实 (中科院自动化所)
[ Thailand ]
• Collaborator: Kanjanapan SUKVICHAI (Kasetsart University)
[ Malaysia ]
• Collaborator: Zool Hilmi Ismail (Universiti Teknologi Malaysia), Kwan Ban Hoe, Danny Ng Wee Kiat
(University Tunku Abdul Rahman), Wan Zuha Wan Hasan, Hafiz Rashidi Harun (Universiti Putra
Malaysia)
[ Iran ]
• Collaborator: Reza Javanmard (University of Science and Technology of Mazandaran) 19

2-A EDUCATION CHALLENGE
http://www.robocupathomeedu.org/challenges
20

RoboCup@Home
Education Challenge
• RoboCup@Home (Main)
– Since 2006
• RoboCup@Home Education Challenge
– RoboCup Japan Open 2015, Fukui (SPL Beta), Japan
– RoboCup Japan Open 2016, Aichi, Japan
– RoboCup Japan Open 2017, Nagoya, Japan
– RoboCupJunior Italian Open 2017, Montesilvano, Italy
– RoboCup Asia-Pacific 2017 Bangkok, Thailand
– RoboCup Japan Open 2018, Ogaki, Japan
– European RoboCupJunior Championship (EURCJ) 2018, Montesilvano, Italy
– RoboCup 2018 Montreal, Canada
– RoboCup China Open 2019, Shaoxing, China
– European RoboCup@Home Education Challenge 2019, Trieste, Italy
– RoboCup 2019 Sydney, Australia
– RoboCup Japan Open 2019 Nagaoka, Japan (August)
– RoboCup Junior Australia Open 2019 Melbourne, Australia (October)
– RoboCup Asia-Pacific 2019 Moscow, Russia (November)
• Upcoming events
– RoboCup@Home Education Challenge India 2020, India (January)
– RoboCup Japan Open 2020 Aichi, Japan (March)
– Mexican Tournament of Robotics 2020, Mexico (March)
– RoboCup China Open 2020, Shaoxing, China (April)
– RoboCup Asia-Pacific Tianjin 2020, China (April)
– RoboCupJunior Austrian Open 2020, Austria (April)
– European RoboCup@Home Education Challenge 2020, Portugal (May)
– RoboCup 2020 Bordeaux, France (June) [+ new Pepper Challenge]
– World Robot Summit 2020 Aichi, Japan (October) 21

RoboCup@Home Education Challenge
RoboCup Japan Open 2015, 2016, 2017, 2018
RoboCup Japan Open 2015, Fukui (SPL Beta)
RoboCup Japan Open 2016, Aichi

RoboCup@Home Education Challenge 2017
RoboCup Asia-Pacific (RCAP) 2017 Thailand
23
https://blogs.mathworks.com/racing-lounge/2018/01/17/robocupathome-education-workshop/

European RoboCupJunior Championship
(EURCJ) 2018, Montesilvano, Italy
24

European RoboCupJunior Championship
(EURCJ) 2018, Montesilvano, Italy
25

Workshop June 16 (Sat) ~ 18 (Mon), 2018
• 6/16
– AM Workshop 1 Hardware and Software
Setup
– PM Workshop 2 Speech, Navigation
• 6/17
– AM Workshop 3 Vision
– PM Workshop 4 Arm, System Integration
• 6/18
– AM Field Testing
– PM Robot Inspection and Presentation
Competition June 19 (Tue) ~ 21 (Thu), 2018
• 6/19
– AM Team Setup
– PM Task 1 Speech and Person Recognition
• 6/20
– AM Task 2 Help-me-carry
– PM Task 3 Restaurant
• 6/21
– AM Finals (Demo and Presentation)
***AM 09:00~12:00; PM 13:00~16:00 26

RoboCup 2018 Montreal, Canada
27

RoboCup 2018 Montreal, Canada
2811 teams, over 40 participants, 9 different countries

@Home Education Challenge
RoboCupJunior China Open 2019 Shaoxing
29

@Home Education Challenge
RoboCupJunior China Open 2019 Shaoxing
30

RoboCup@Home Education Outreach Initiative to Australia
in Promotion of RoboCup 2019
Workshop July 2 (Tue) ~ 4 (Thu), 2019
• 7/2
– AM Workshop 1 Hardware and Software
Setup
– PM Workshop 2 Speech, Navigation
• 7/3
– AM Workshop 3 Vision
– PM Workshop 4 Arm, System Integration
• 7/4
– AM Field Testing
– PM Robot Inspection and Presentation
Competition July 5 (Fri) ~ 7 (Sun), 2019
• 7/5
– AM Team Setup
– PM Task 1 Speech and Person Recognition
• 7/6
– AM Task 2 Help-me-carry
– PM Task 3 Restaurant
• 7/7
– AM Finals (Demo and Presentation)
***AM 09:00~12:00; PM 13:00~16:00 31

32
15 teams, over 70 participants, 7 different countries

2-B EDUCATIONAL OPEN ROBOT PLATFORMS
2-C OPENCOURSEWARE
http://www.robocupathomeedu.org/robots
33

MARRTINO ROBOT
An open source, open hardware robotic platform
A mobile robot to learn and discover
34https://www.marrtino.org/

Educational Open Robot Platform for
RoboCup@Home (Service Robot)

Approach
• Open source platform
for service robot
– Startup base, cost
effective and community
support
• Current design:
– Basic robot platform
– Modular add-ons

Specifications
• Mobile Base
– TurtleBot2 (Kobuki)
• Perception Systems
– Kinect for Xbox 360
• Robot Arm
– TurtleBot Arm
– Elevated Platform
• User Interface
– Digital I/O
– Android interface
– Iconic robot facial expression system
• Software framework
– Navigation
– Manipulation
– Voice Interaction
– People/object recognition

Various Implementations
• Different hardware configurations of Team
KameRider for Japan Open 2015 and RoboCup
2015 Hefei
38

Open Source Solution
Open robot platform for service robotics
• OpenCourseWare:
– http://www.robocupathomeedu.org/learn
– http://robotforall.org/opencourseware/
• Support wiki: http://robotforall.org/wiki/
• Source codes: https://github.com/robocupathomeedu/
• Demo videos: https://www.youtube.com/user/kameriderteam
• Development examples:
– https://github.com/robocupathomeedu/rc-home-edu-learn-ros
rc-home-edu-learn-ros
– rchomeedu_speech
– rchomeedu_vision
– rchomeedu_navigation
– rchomeedu_arm
– rchomeedu_apps
• rchomeedu_follower
• rchomeedu_partybot 39

2-D OUTREACH PROGRAMS
http://www.robocupathomeedu.org/outreach
40

Local
Workshops
[ Japan ]
• Family & Robotics Workshops
– 2014.08.31 Introduction to ROS and TurtleBot2
– 2014.09.04 RoboCup@Home Challenge with TurtleBot2
– 2014.09.28 “Grab a bottle” with TurtleBot2
– 2014.11.16 “Follow me” with TurtleBot2
• 2016.01.23 RC@HomeEDU Workshop (Kanto)
• 2016.02.20 RC@HomeEDU Workshop (Kansai)
• 2016.12 3rd RC@HomeEDU Workshop (Kansai)
• 2017.02 4th RC@HomeEDU Workshop (Kanto)
[ Malaysia ]
• 2017.02 RC@HomeEDU Workshop
[ Italy ]
• 2017.03 RC@HomeEDU Workshop @ RomeCup
[ Iran ]
• 2017.04 RC@HomeEDU Workshop @ RoboCup Iran
Open 2017
[ China ]
• 2017.08 RC@HomeEDU Seminar @ ROS技术及应用
培训，中国机器人大赛2017
[ USA ]
• 2017.09 RC@HomeEDU Exhibition @ Maker Faire,
New York
[ Thailand ]
• 2017.12 RC@HomeEDU Workshop @ RoboCup Asia
Pacific 2017
42

Hands-on Workshops (Japan, Malaysia, Iran)
43

Workshop Development in Malaysia
UTAR, UPM, UNIKL
45

Workshop development for high
school students in China
46

University undergraduate course
<Robotics Software Engineering>
47

International Academic Meetings
48

International Academic
Exchange Programs
• 2017.01.09-18 SAKURA Science Program @ Japan
– Host: Tamagawa University (Japan)
– Visitor: 10 students and 1 staff from Kasetsart University
(Thailand)
• 2016.12-2017.03 RoboCup Internship @ Japan
– Host: The University of Tokyo (Japan)
– Intern: 1 student from Univerisiti Teknologi Malaysia
(Malaysia)
• 2016.02.26-03.06 SAKURA Science Program @ Japan
– Visitor: 10 students and 1 staff from Nankai University (China)
• 2016.02.03-19 SAKURA Science Program @ Japan
– Host: Shibaura Institute of Technology (Japan)
– Visitor: 10 students and 2 staff from Universiti Teknologi
Malaysia (Malaysia)
• 2014.12.06 Intelligent Home Robotics Challenge 2014
@ Japan
– Venue: Tokyo
– Participated the challenge and workshop by 3 students from
Univerisiti Teknologi Malaysia (Malaysia)
• 2014.06-09 RoboCup Internship @ Japan
(Malaysia)
• 2014.03-06 Robotics Internship @ Japan
– Host: Shibaura Institute of Technology (Japan)
(Malaysia)

Student Development PhD Scholarship at
Australian National University
Internship in Japan Internship in ItalyInternship in Italy

Next Step
• Worldwide Initiative
– RoboCup@Home
Education Community
(Challenge, Workshop)
– USA, Europe (Italy),
Thailand, China, Iran,
Malaysia, Singapore, etc.
51
• Collaboration with RoboCup Junior
• Collaboration with Industrial Partners
– MathWorks, NVIDIA, ROBOTIS
• Open Courseware and Open Robot (Hardware/Software)
Development

Bridging Robotics Education between High School and
University: An Outreach Development in Southeast Asia
Jeffrey Too Chuan Tan1, Kanjanapan Sukvichai2, Zool Hilmi Ismail3, Ban Hoe Kwan4,
Danny Wee Kiat Ng4, Hafiz Rashidi Harun5, Amy Eguchi6 and Luca Iocchi7
MOTIVATION – There is a big gap of missing advanced skill
sets between high school and university level of robotics
education due to the differences in bottom-up and top-
down learning approaches.
SOLUTION – We aim to initiate a bridging education layer
that abstracts advanced university level robotics
development into a learning platform suitable for high
school students. The students learn by building practical
robots and competing their robots with peers.
PROJECT – We are developing a set of hardware and
software solutions as the learning platform (Fig. 1), and
organizing a series of educational activities in the form of
workshop and competition (Fig. 2). The objective of this
work is to outreach and evaluate this effort in developing
countries in Southeast Asia.
Regional Collaborators
1. Nankai University, China
2. Kasetsart University, Thailand
3. Universiti Teknology Malaysia, Malaysia
4. Universiti Tunku Abdul Rahman, Malaysia
5. Universiti Putra Malaysia, Malaysia
6. Bloomfield College, USA
7. Sapienza University of Rome, Italy
Fig. 1 Affordable robot platforms TurtleBot2 and MARRtino
Fig. 2 Outreach programs including workshop and competition
activities in China, Japan, USA and Italy (clockwise from top left)

World Robot Summit – Junior Category
School Robot Challenge Workshop & Trial 2017
53

World Robot Summit – Junior Category
School Robot Challenge Workshop & Trial 2018
54

3. SERVICE ROBOTICS R&D
http://www.openbotics.org/
55

Home Service Robot DIY
a. Robot LEG
– Autonomous Navigation
b. Robot EYES
– Visual Perception
c. Robot ARM
– Object Manipulation
d. Robot MOUTH
– Human-Robot Interaction
e. Robot BRAIN
– AI, Machine Learning,
Cloud Computing, Big Data
56

Robot EYES – Visual Perception
• Robot Vision
– Perception by RGB-D Sensor
57
[https://msdn.microsoft.com/en-us/library/jj131033.aspx]

• Image Processing by OpenCV
• Deep Learning Object Detection by YOLO
58

• Person recognition result in RoboCup 2016
59

Improving Deep Learning Based Object Detection by
CycleGAN Method Under Inconsistent Illumination Conditions
60
Three illumination conditions of
the real environment
CycleGAN is used to realize the mutual
transformation of scenes
Dark environment before
brightness enhancement
Dark environment after
Object detection after
The top view of the
visual task scene and
the robot vision with
supplementary light
Object detection
confidence level
improvement
[F. Wang, J. T. C. Tan, “Improving Deep Learning Based Object Detection of Mobile Robot Vision by HSI Preprocessing Method and
CycleGAN Method Under Inconsistent Illumination Conditions in Real Environment,” in Proc. of the 2019 IEEE/ASME AIM, October 2019]

Robot LEG – Autonomous Navigation
61
• Indoor Autonomous Navigation
– Adaptive Monte Carlo Localization (AMCL)
– Simultaneous Localization and Mapping (SLAM)
– Static and Dynamic Obstacle Avoidance

Robot ARM – Object Manipulation
62
http://wiki.ros.org/turtlebot_block_manipulation
Object Manipulation

Multi-Object Grasp Planning in High Distribution Density
using Inverse Reachability Map and Base Repositioning
63
Experiment environment and object distribution IRM of different type of objects
System components and operation flow
Experiment results
[Y. Xi, J. T. C. Tan, F. Wang, H. Song, “Multi-Object Grasp Planning in High Distribution Density of Service Robot
Using Inverse Reachability Map and Base Repositioning,” in Proc. of the 2019 IEEE ARSO, November 2019]

Robot MOUTH – Human-Robot Interaction
• Speech Synthesis
(Text-to-Speech)
– Festival, ROS
sound_play
• Speech Recognition
(offline)
– CMUSphinx, ROS
Pocketsphinx
• Speech Recognition
(online)
– XunFei, Web
Speech API
• Facial Expression by
Emoticon
64
[H. Song, J. T. C. Tan, Y. Xing, G. Hou, “Communication Efficiency and User Experience Analysis of Visual and Audio
Feedback Cues in Human and Service Robot Voice Interaction Cycle,” in Proc. of the 2019 WRC SARA, August 2019]

Robot BRAIN – AI, Machine Learning, Cloud Computing, Big Data
65
Client Systems
Robot Learning
Knowledge
Transfer
Cloud System
• Processing Servers
• Databases
?

• Crowdsourcing of virtual HRI for collaborative strategy learning
66[J. T. C. Tan, Y. Hagiwara, T. Inamura, “Robot Learning Framework via Crowdsourcing of Human-Robot Interaction
for Collaborative Strategy Learning,” in Proc. of the 24th IEEE RO-MAN (Interactive Session), IS04, 2015]

State parameters:
• Self
• Action
• Object(Target)
• Location
𝑆𝑒𝑙𝑓_𝐴𝑐𝑡𝑖𝑜𝑛𝑖 = 𝑓 𝑆𝑒𝑙𝑓_𝐴𝑐𝑡𝑖𝑜𝑛𝑖−1, 𝑃𝑎𝑟𝑡𝑛𝑒𝑟_𝐴𝑐𝑡𝑖𝑜𝑛𝑖, 𝑊𝑜𝑟𝑘_𝐶𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛𝑖
𝐴𝑔𝑒𝑛𝑡_𝐴𝑐𝑡𝑖𝑜𝑛(𝑂𝑏𝑗𝑒𝑐𝑡, 𝐿𝑜𝑐𝑎𝑡𝑖𝑜𝑛)
𝑊𝑜𝑟𝑘_𝐶𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 = 𝑂𝑏𝑗𝑒𝑐𝑡1(𝐿𝑜𝑐𝑎𝑡𝑖𝑜𝑛), … , 𝑂𝑏𝑗𝑒𝑐𝑡 𝑛(𝐿𝑜𝑐𝑎𝑡𝑖𝑜𝑛)
“Minimum information” to describe the current state
Collaborative Intelligence
67
• Partner
• Action
• Object(Target)
• Location
• Work
• Action(Static)
• Object1-n
• Location1-n
• Condition1-n(Omitted)

Extraction of Embodied Collaborative
Behaviors from Cyber-Physical HRI with
Immersive User Interfaces
• Contents
– (See) Visual Observation
• Movement of HMD to
determine observed target
– (Say) Verbal Communication
• Spoken speech
– (Do) Action
• Agent’s body movement to
determine traveled path
• Timing
– Contents’ occurrence timings
w.r.t. collaboration operation

Crowdsourcing of HRI and Robot
“Collaborativeness” Evaluation
69

70

71
Handyman (GPSR) Interactive Clean Up
Human Navigation

x
Personal
Shopping
Assistant
Tele-
Shopping
Personalized
Services
AI + Service Robots > Commercial Services
72
Convenient and Efficient
Customer-Centered and
Store-Enhanced
New Shopping Experience

From Industrial Automation to
Commercial Services
Industrial Automation Commercial Services
73
EffectivenessEffectiveness EfficiencyEfficiency AdaptiveAdaptive HelpfulHelpful
ProductivityProductivity ServiceService
SpeedSpeed AccuracyAccuracy ThoughtfulnessThoughtfulness WarmthWarmth
Intelligence

Personalized Services
• Multi-lingual
• Face Recognition
• Personal Database
• Data for Store
74

Recommender Systems — User-Based
and Item-Based Collaborative Filtering
75
Nearest Neighbor algorithm
Similarity Affection

Personal Shopping Assistant
• Information
Assistance
• Physical
Assistance
76

Tele-Shopping
• Immersive Shopping Experience
77

Take-Home Messages
1. Prologue: Team KameRider
 “It works!”
2. RoboCup@Home EDUCATION Initiative
a. Education Challenge
 “Let’s organize together at your region!”
b. Educational Open Robot Platforms
 “Give everyone a robot!”
c. OpenCourseWare
 “Everyone can learn AI & robotics!”
d. Outreach Programs
 “Bring us to your community!”
3. Service Robotics R&D
 “Everyone can learn AI and Robotics!”

Thank you!
www.RoboCupatHomeEDU.org
Supported by:
iHR研究専門委員会

2020.01.20 RoboCup@Home Education (Introduction) [EN]

Recommended

Recommended

More Related Content

Similar to 2020.01.20 RoboCup@Home Education (Introduction) [EN]

Similar to 2020.01.20 RoboCup@Home Education (Introduction) [EN] (20)

Recently uploaded

Recently uploaded (20)

2020.01.20 RoboCup@Home Education (Introduction) [EN]