Jeffrey Tan is an expert in service robotics education. He founded the RoboCup@Home Education initiative to promote the development of service robots through robotics competitions and an open-source education platform. Some key aspects of the initiative include the RoboCup@Home Education Challenge competition, an open-source robotics platform for education, and various outreach activities like training camps and academic exchanges. His team has participated in numerous RoboCup competitions internationally since 2013, winning several awards.

1. 人工智能+服务机器人=AI教育平台
RoboCup@Home EDUCATION
AI-Focused Robotics Education by Home Service Robot DIY
2019-12-05 | Jeffrey 陈图川

2. Jeffrey Too Chuan TAN（陈图川）
【教育背景】
2007 - 2010 The University of Tokyo (日本), Department of Precision Engineering, Doctor of Engineering
2004 - 2007 Universiti Tenaga Nasional (马来西亚), Master of Mechanical Engineering
1999 - 2003 Universiti Tenaga Nasional (马来西亚), Bachelor of Mechanical Engineering (Hons.)
【工作经验】
2017 – 现今 副教授，南开大学（中国）《天津市青年千人计划》
2017 – 现今 特别研究员，玉川大学（日本）
2014 - 2017 Project Assistant Professor, Institute of Industrial Science, The University of Tokyo (日本)
2015 - 2017 Adjunct Lecturer, Tokyo City University (日本)
2013 - 2014 Project Researcher, Institute of Industrial Science, The University of Tokyo (日本)
2011 - 2013 Project Researcher, National Institute of Informatics (日本)
2010 - 2011 Project Researcher, Graduate School of Engineering, The University of Tokyo (日本)
2004 - 2007 Tutor, Universiti Tenaga Nasional (马来西亚)
【国际学术组织兼职】
2016 - 现今 Committee (Service and Junior), World Robot Summit
2016 - 现今 Organizing Committee, RoboCup Federation (@Home)
2015 - 现今 Committee, RoboCup@Home Education
2014 - 现今 Organizing Committee, RoboCup Japan (@Home)
2
简历

3. RoboCup@Home
RoboCup@Home旨在促进服务和助手机器人技术的发展，以使其未来的民间应用成为可能。比赛包含一组
衡量标准，用以评估机器人在真实家庭环境中和其他场景中的能力。比赛的关键技术有：人机交互
(human-robot interaction and cooperation)，动态环境中的导航，计算机视觉和自然光照条件下物体的识
别 ， 操 纵 物 体 ， 适 应 性 行 为 和 自 适 应 学 习 (adaptive behaviors and learning) ， 环 境 智 能 (ambient
intelligence)和系统集成(system integration)。
3

4. 目录
1. 起源：Team KameRider
2. RoboCup@Home EDUCATION Initiative
a. Education Challenge 教育竞赛
b. 开源机器人教育平台
c. 推广活动
3. 服务机器人科研开发

5. 1. 起源：TEAM KAMERIDER
http://openbotics.org/kamerider/

6. 2013 Team KameRider成立
2013.05.03-06 RoboCup Japan Open 2013 Tokyo, Japan
• [UT] Jeffrey
• [Award] JSAI Award [SIGVerse for RoboCup @Home Simulation]
• [Award] RoboCup @Home Simulation [2nd Place]
2013.06.24-07.01 RoboCup 2013 Eindhoven, Netherlands
(International)
• [Symposium] Poster: “Open Web Based Development Platform for
RoboCup @Home Simulation”
• [Symposium] Oral: “Development of RoboCup@Home Simulation
towards Long-term Large Scale HRI”

7. 2014 RoboCup日本公开赛初赛
荣获日本人工智能学会奖
2014.03-06 Internship of Mr. Tey @ SIT, Japan
• [Internship] Mr. Tey (UTM) assisted Jeffrey's team in the
development of a basic robot platform for RoboCup
@Home
2014.05.03-06 RoboCup Japan Open 2014 Fukuoka, Japan
• [UT] Jeffrey, [NKU] 6 members, [UTM] Tey Wei Kang
• [Award] JSAI Award [Standard Platform for RoboCup
@Home]
• [Award] RoboCup @Home Simulation [2nd Place]

8. 2014 创发@Home开源机器人教育平台
2014.06-09 Internship of Mr. Seow @ UT, Japan
• [Internship] Mr. Seow (UTM) develops the basic robot
platform for RoboCup @Home based on the RCF support
2014.12.06 Intelligent Home Robotics Challenge 2014, Tokyo
• [UT] Jeffrey, [UTM] Lim Kian Sheng, Mohamad Hafizuddin
bin Majek, Muhammad Faiz bin Muhammad Rozi
• [Award] Mobile Robot Category 3rd Place
• [Award] Overall 3rd Place

9. 2015 创办首届Education Challenge
2015.05.03-06 RoboCup Japan Open 2015 Fukui, Japan
• [UT] Jeffrey, [NKU] 3 members, [UTM] Muhammad
Najib Abdullah, Nicole Tham Lei May
• [Award] RoboCup @Home SPL (Beta) [1st Place]
• [Award] RoboCup @Home Simulation [3rd Place]

10. 2015 以开源机器人教育平台参加RoboCup国际赛
2015.07.17-23 RoboCup 2015 Hefei, China
(International)
• [UT] Jeffrey, [NKU] 7 members, [UTM]
Yeong Che Fai, Seow Yip Loon, Nicole Tham
Lei May
• Overall ranked 7th out of 17 qualified teams
• Top 9 teams to enter Stage 2

11. 2016 国际合作队伍 UT-NKU-UTM-SIT
2016.03.24-27 RoboCup Japan Open 2016 Aichi,
Japan
• [Award] RoboCup @Home Education [2nd Place]
• [Award] RoboCup @Home Simulation [1st Place]
2016.06.30-07.04 RoboCup 2015 Leipzig, Germany
(International)
• Overall ranked 7th out of 23 qualified teams
11

12. 2017 国际合作队伍 NKU-UTM-SIT
RoboCup Japan Open 2017 Nagoya
• [Award] RoboCup @Home Education [1st Place]
• [Award] RoboCup @Home Simulation [2nd Place]
RoboCup 2017 Nagoya (International)
• [Award] RoboCup @Home SSPL [Overall ranked 4th
out of 7 qualified teams]
RoboCup Asia-Pacific 2017 Bangkok
• [Award] RoboCup @Home [1st Place]
• [Award] RoboCup @Home Education [1st Place]
12

13. 团队历年参赛成绩
13

14. 2019 RoboCup机器人世界杯日本公开赛，@Home OPL [季军]
2019 RoboCup机器人世界杯日本公开赛，@Home Education [季军]
2019 RoboCup 2019 Sydney, Australia (International), RoboCup @Home Education [3rd Place]
2019 RoboCup机器人世界杯亚太天津邀请赛，@Home OPL [季军]
2019 RoboCup机器人世界杯亚太天津邀请赛，@Home OPL技术挑战赛 [亚军]
2019 RoboCup机器人世界杯亚太天津邀请赛，@Home SSPL [冠军]
2019 RoboCup机器人世界杯亚太天津邀请赛，@Home SSPL技术挑战赛 [亚军]
2019 RoboCup机器人世界杯中国赛，家庭组SSPL [三等奖]
2019 RoboCup机器人世界杯中国赛，家庭组SSPL技术挑战赛 [冠军]
2019 RoboCup机器人世界杯中国赛，家庭组 [三等奖]
2019 RoboCup机器人世界杯中国赛，家庭组技术挑战赛 [冠军]
2018 中国机器人技能大赛“软银机器人杯” [冠军]
2018 RoboCup Japan Open 2018 Ogaki, RoboCup @Home Education [2nd Place]
2018 RoboCup机器人世界杯中国赛, RoboCup家庭组技术挑战赛 [冠军]
2017 RoboCup Asia-Pacific 2017 Bangkok, RoboCup @Home [1st Place]
2017 RoboCup Asia-Pacific 2017 Bangkok, RoboCup @Home [1st Place]
2017 RoboCup Asia-Pacific 2017 Bangkok, RoboCup @Home Education [1st Place]
2017 RoboCup 2017 Nagoya, Japan (International), RoboCup @Home SSPL [Overall ranked 4th out of 7 qualified teams]
2017 RoboCup Japan Open 2017 Nagoya, RoboCup @Home Education [1st Place]
2017 RoboCup Japan Open 2017 Nagoya, RoboCup @Home Simulation [2nd Place]
2016 RoboCup 2016 Leipzig, Germany (International), RoboCup @Home League [Overall ranked 7th out of 23 qualified teams]
2016 RoboCup Japan Open 2016 Aichi, RoboCup @Home Education [2nd Place]
2016 RoboCup Japan Open 2016 Aichi, RoboCup @Home Simulation [1st Place]
2015 RoboCup 2015 Hefei, China (International), RoboCup @Home League [Overall ranked 7th out of 17 qualified teams]
2015 RoboCup Japan Open 2015 Fukui, RoboCup @Home SPL [1st Place]
2015 RoboCup Japan Open 2015 Fukui, RoboCup @Home Simulation [3rd Place]
2014 Intelligent Home Robotics Challenge 2014 [Overall 3rd Place]
2014 Intelligent Home Robotics Challenge 2014 [Mobile Robot Category 3rd Place]
2014 Japanese Society for Artificial Intelligence (JSAI) Award [Standard Platform for RoboCup @Home]
2014 RoboCup Japan Open 2014 Fukuoka, RoboCup @Home Simulation [2nd Place]
2013 Japanese Society for Artificial Intelligence (JSAI) Award [SIGVerse for RoboCup @Home Simulation]
2013 RoboCup Japan Open 2013 Tokyo, RoboCup @Home Simulation [2nd Place] 14
团队历年参赛成绩

15. 2. ROBOCUP@HOME EDUCATION
INITIATIVE
http://www.robocupathomeedu.org/
15

16. AI-Focused Robotics Education by
Home Service Robot DIY
The “Bridging Problem”
School-level Robotics Education vs University-level Robotics Research
• Bottom-up vs Top-down
• Conceptual Problems vs Real World Problems
The Blooming of AI, Cloud and Big Data
• Learning Platform and Ecosystem
16

17. AI机器人教育生态环境建设
RoboCup@Home Education
RoboCup@Home Education促进RoboCup@Home的教
育工作，以推动RoboCup@Home的参与和家庭服务机
器人的开发。目前的活动：
1. 创办RoboCup@Home Education Challenge
2. 推动家庭服务机器人的开源教育平台
3. 推广活动（国内培训营，国际学术交流项目等）
http://www.robocupathomeedu.org/
https://www.facebook.com/robocupathomeedu/
17

18. RC@Home
Education
RC@Home
Education
RC@Home
Education
Challenge
RC@Home
Education
Challenge
Open
Robot
Platform
Open
Robot
Platform
OutreachOutreach
RoboCup
Community
RoboCup
Community
iHR Challenge
Exchange Program
Workshop
RoboCup Japan Open
Promote
New Entry
Evolution by New
Contribution
Open
Courseware
Evaluate the
learning
Open Design &
Source Codes
Open Design &
Source Codes
Core Robot Platform
with Modular Add-ons
Robots in RoboCup
Japan Open 2016
Invited Lecture
Universities
& Research
Community
Universities
& Research
Community
RoboCup
Committee
RoboCup
Committee
Join RoboCup
Outreach to Public
Improve Service
Robot Development

19. 合作成员
[ 日本 ]
• Organizer: RoboCup Japan Committee
• Coordinator:
– Hiroyuki OKADA (Tamagawa University)
– Yoshinobu HAGIWARA (Ritsumeikan University)
– Jeffrey Too Chuan TAN (Nankai University, China)
• Supporter:
– Yasuhiro MASUTANI (Osaka Electro-Communication University)
– Kosei DEMURA (Kanazawa Institute of Technology)
– Yuki INOUE (Osaka Institute of Technology)
– Kenichi OHARA (Meijo University)
[ 意大利 ]
• Organizer: Italian RoboCup Regional Committee
• Coordinator:
– Luca Iocchi (Sapienza University of Rome)
– Paola Ferrarelli (Sapienza University of Rome)
[ 美国 ]
• Collaborator: Amy EGUCHI (Bloomfield College), M. Q. Azhar (BMCC.CUNY)
[ 加拿大 ]
• Collaborator: Sara Iatauro (English Montreal School Board)
[ 中国 ]
• Collaborator: 李实 (中科院自动化所)
[ 泰国 ]
• Collaborator: Kanjanapan SUKVICHAI (Kasetsart University)
[ 马来西亚 ]
• Collaborator: Zool Hilmi Ismail (Universiti Teknologi Malaysia), Kwan Ban Hoe (University Tunku
Abdul Rahman), Danny Ng Wee Kiat (University Tunku Abdul Rahman), Hafiz Rashidi Harun
(Universiti Putra Malaysia)
[ 伊朗 ]
• Collaborator: Reza Javanmard (University of Science and Technology of Mazandaran) 19

20. AI机器人教育生态环境建设
ROBOCUP@HOME EDUCATION CHALLENGE
机器人世界杯 家庭组 教育竞赛
http://www.robocupathomeedu.org/challenges
20

21. RoboCup@Home Education Challenge
竞赛活动
• RoboCup@Home主竞赛
– Since 2006
• RoboCup@Home Education Challenge
– RoboCup Japan Open 2015, Fukui (SPL Beta), 日本
– RoboCup Japan Open 2016, Aichi, 日本
– RoboCup Japan Open 2017, Nagoya, 日本
– RoboCupJunior Italian Open 2017, Montesilvano, 意大利
– RoboCup Asia-Pacific 2017 Bangkok, 泰国
– RoboCup Japan Open 2018, Ogaki, 日本
– European RoboCupJunior Championship (EURCJ) 2018, Montesilvano,意大利
– RoboCup 2018 Montreal, 加拿大
– RoboCup China Open 2019, Shaoxing, 中国
– European RoboCup@Home Education Challenge 2019, Trieste, 意大利
– RoboCup 2019 Sydney, 澳大利亚
– RoboCup Japan Open 2019 Nagaoka, 日本 (August)
– RoboCup Junior Australia Open 2019 Melbourne, 澳大利亚 (October)
– RoboCup Asia-Pacific 2019 Moscow, 俄罗斯 (November)
• Upcoming events
– RoboCup@Home Education Challenge India 2020, 印度 (January)
– RoboCup Japan Open 2020 Aichi, 日本 (March)
– Mexican Tournament of Robotics 2020, 墨西科 (March)
– RoboCup China Open 2020, Shaoxing, 中国 (April)
– RoboCup Asia-Pacific Tianjin 2020, 中国 (April)
– RoboCupJunior Austrian Open 2020, 奥地利 (April)
– European RoboCup@Home Education Challenge 2020, Portugal, 葡萄牙 (May)
– RoboCup 2020 Bordeaux, 法国 (June) [+ new Pepper Challenge]
– World Robot Summit 2020 Aichi, 日本 (October) 21

22. RoboCup@Home Education Challenge
RoboCup Japan Open 2015, 2016, 2017, 2018
RoboCup Japan Open 2015, Fukui (SPL Beta)
RoboCup Japan Open 2016, Aichi

23. RoboCup@Home Education Challenge 2017
RoboCup Asia-Pacific (RCAP) 2017 Thailand
23
https://blogs.mathworks.com/racing-lounge/2018/01/17/robocupathome-education-workshop/

24. RoboCup@Home Education Challenge
European RoboCupJunior Championship
(EURCJ) 2018, Montesilvano, Italy
24

25. RoboCup@Home Education Challenge
European RoboCupJunior Championship
(EURCJ) 2018, Montesilvano, Italy
25

26. Workshop June 16 (Sat) ~ 18 (Mon), 2018
• 6/16
– AM Workshop 1 Hardware and Software
Setup
– PM Workshop 2 Speech, Navigation
• 6/17
– AM Workshop 3 Vision
– PM Workshop 4 Arm, System Integration
• 6/18
– AM Field Testing
– PM Robot Inspection and Presentation
Competition June 19 (Tue) ~ 21 (Thu), 2018
• 6/19
– AM Team Setup
– PM Task 1 Speech and Person Recognition
• 6/20
– AM Task 2 Help-me-carry
– PM Task 3 Restaurant
• 6/21
– AM Finals (Demo and Presentation)
***AM 09:00~12:00; PM 13:00~16:00 26

27. RoboCup@Home Education Challenge 2018
RoboCup 2018 Montreal, Canada
27

28. RoboCup@Home Education Challenge 2018
RoboCup 2018 Montreal, Canada
2811 teams, over 40 participants, 9 different countries

29. @Home Education Challenge
RoboCupJunior China Open 2019 Shaoxing
29

30. @Home Education Challenge
RoboCupJunior China Open 2019 Shaoxing
30

31. RoboCup@Home Education Outreach Initiative to Australia
in Promotion of RoboCup 2019
RoboCup@Home Education Challenge 2019
AI-Focused Robotics Education by Home Service Robot DIY
Workshop July 2 (Tue) ~ 4 (Thu), 2019
• 7/2
– AM Workshop 1 Hardware and Software
Setup
– PM Workshop 2 Speech, Navigation
• 7/3
– AM Workshop 3 Vision
– PM Workshop 4 Arm, System Integration
• 7/4
– AM Field Testing
– PM Robot Inspection and Presentation
Competition July 5 (Fri) ~ 7 (Sun), 2019
• 7/5
– AM Team Setup
– PM Task 1 Speech and Person Recognition
• 7/6
– AM Task 2 Help-me-carry
– PM Task 3 Restaurant
• 7/7
– AM Finals (Demo and Presentation)
***AM 09:00~12:00; PM 13:00~16:00 31

32. RoboCup@Home Education Challenge 2019
AI-Focused Robotics Education by Home Service Robot DIY
32
15 teams, over 70 participants, 7 different countries

33. AI机器人教育生态环境建设
推动家庭服务机器人的开源教育平台
http://www.robocupathomeedu.org/challenges
33

34. 家庭服务机器人的开源教育平台

35. 基本概念
• 开源教育平台
– 简易入门级平台，
成本廉宜，广泛
的开源社区支持
• 现今设计
– 基本机器人平台
– 模块化加载项

36. 系统细节
• 移动平台
– TurtleBot2 (Kobuki)
• 机器人视觉
– Kinect for Xbox 360
• 机械手臂
– TurtleBot Arm
– 升降平台
• 人机交互
– 数码I/O
– 安卓界面
– 机器人脸部表情系统
• 软件框架
– 导航
– 抓取物件
– 语音识别
– 人物/物件识别

37. 多元的机器人组合
• RoboCup Japan Open 2015和RoboCup 2015合
肥赛中，KameRider队伍机器人不同的硬件
组合
37

38. 开源的解决方案
开源家庭服务机器人教育平台
• 开放课程：http://robotforall.org/opencourseware/
• 开发教程：http://robotforall.org/wiki/
• 开源代码：https://github.com/robocupathomeedu/
• 在线视频：http://i.youku.com/robocupathomeedu
• 开发例子：https://github.com/robocupathomeedu/rc-home-edu-learn-ros
rc-home-edu-learn-ros
– rchomeedu_speech
– rchomeedu_vision
– rchomeedu_navigation
– rchomeedu_arm
– rchomeedu_apps
• rchomeedu_follower
• rchomeedu_partybot
38

39. AI机器人教育生态环境建设
推广活动（培训营，学术交流项目等）
http://www.robocupathomeedu.org/challenges
39

40. 科普讲座
40

41. 培训营
[ Japan ]
• Family & Robotics Workshops
– 2014.08.31 Introduction to ROS and TurtleBot2
– 2014.09.04 RoboCup@Home Challenge with TurtleBot2
– 2014.09.28 “Grab a bottle” with TurtleBot2
– 2014.11.16 “Follow me” with TurtleBot2
• 2016.01.23 RC@HomeEDU Workshop (Kanto)
• 2016.02.20 RC@HomeEDU Workshop (Kansai)
• 2016.12 3rd RC@HomeEDU Workshop (Kansai)
• 2017.02 4th RC@HomeEDU Workshop (Kanto)
[ Malaysia ]
• 2017.02 RC@HomeEDU Workshop
[ Italy ]
• 2017.03 RC@HomeEDU Workshop @ RomeCup
[ Iran ]
• 2017.04 RC@HomeEDU Workshop @ RoboCup Iran
Open 2017
[ China ]
• 2017.08 RC@HomeEDU Seminar @ ROS技术及应用
培训，中国机器人大赛2017
[ USA ]
• 2017.09 RC@HomeEDU Exhibition @ Maker Faire,
New York
[ Thailand ]
• 2017.12 RC@HomeEDU Workshop @ RoboCup Asia
Pacific 2017
41

42. 各国的培训会(日本、马来西亚、伊朗)
42

43. 43

44. 各国的培训会(马来西亚)
44

45. 各国的培训会(澳门)
45

46. 大学课程
《机器人软件工程学》
46

47. 国际学术会议
47

48. 国际学术交流• 2017.01.09-18 SAKURA Science Program @ Japan
– Host: Tamagawa University (Japan)
– Visitor: 10 students and 1 staff from Kasetsart University
(Thailand)
• 2016.12-2017.03 RoboCup Internship @ Japan
– Host: The University of Tokyo (Japan)
– Intern: 1 student from Univerisiti Teknologi Malaysia
(Malaysia)
• 2016.02.26-03.06 SAKURA Science Program @ Japan
– Host: The University of Tokyo (Japan)
– Visitor: 10 students and 1 staff from Nankai University (China)
• 2016.02.03-19 SAKURA Science Program @ Japan
– Host: Shibaura Institute of Technology (Japan)
– Visitor: 10 students and 2 staff from Universiti Teknologi
Malaysia (Malaysia)
• 2014.12.06 Intelligent Home Robotics Challenge 2014
@ Japan
– Venue: Tokyo
– Participated the challenge and workshop by 3 students from
Univerisiti Teknologi Malaysia (Malaysia)
• 2014.06-09 RoboCup Internship @ Japan
– Host: The University of Tokyo (Japan)
– Intern: 1 student from Univerisiti Teknologi Malaysia
(Malaysia)
• 2014.03-06 Robotics Internship @ Japan
– Host: Shibaura Institute of Technology (Japan)
– Intern: 1 student from Univerisiti Teknologi Malaysia
(Malaysia)

49. 学生后期学习的发展 PhD Scholarship at
Australian National University
Internship in Japan Internship in ItalyInternship in Italy

50. Next Step
• Worldwide Initiative
– RoboCup@Home
Education Community
(Challenge, Workshop)
– USA, Europe (Italy),
Thailand, China, Iran,
Malaysia, Singapore, etc.
50
• Collaboration with RoboCup Junior
• Collaboration with Industrial Partners
– MathWorks, NVIDIA, ROBOTIS
• Open Courseware and Open Robot (Hardware/Software)
Development

51. Bridging Robotics Education between High School and
University: An Outreach Development in Southeast Asia
Jeffrey Too Chuan Tan1, Kanjanapan Sukvichai2, Zool Hilmi Ismail3, Ban Hoe Kwan4,
Danny Wee Kiat Ng4, Hafiz Rashidi Harun5, Amy Eguchi6 and Luca Iocchi7
MOTIVATION – There is a big gap of missing advanced skill
sets between high school and university level of robotics
education due to the differences in bottom-up and top-
down learning approaches.
SOLUTION – We aim to initiate a bridging education layer
that abstracts advanced university level robotics
development into a learning platform suitable for high
school students. The students learn by building practical
robots and competing their robots with peers.
PROJECT – We are developing a set of hardware and
software solutions as the learning platform (Fig. 1), and
organizing a series of educational activities in the form of
workshop and competition (Fig. 2). The objective of this
work is to outreach and evaluate this effort in developing
countries in Southeast Asia.
Regional Collaborators
1. Nankai University, China
2. Kasetsart University, Thailand
3. Universiti Teknology Malaysia, Malaysia
4. Universiti Tunku Abdul Rahman, Malaysia
5. Universiti Putra Malaysia, Malaysia
6. Bloomfield College, USA
7. Sapienza University of Rome, Italy
Fig. 1 Affordable robot platforms TurtleBot2 and MARRtino
Fig. 2 Outreach programs including workshop and competition
activities in China, Japan, USA and Italy (clockwise from top left)

52. 3. 服务机器人科研开发
http://www.openbotics.org/
52

53. 家庭服务机器人DIY，你我都做得到
a. 机器人的腿
– 移动平台，室内导航
b. 机器人的眼
– 机器人视觉，物件识别
c. 机器人的手
– 机械手臂，抓取物件
d. 机器人的嘴
– 语音识别，人机交互
e. 机器人的脑
– 人工智能，机械学习，
云计算与大数据
53

54. 机器人的眼: 机器人视觉，物件识别
• 机器人视觉
– 使用RGB-D传感器环境识别
54
[https://msdn.microsoft.com/en-us/library/jj131033.aspx]

55. 机器人的眼: 机器人视觉，物件识别
• OpenCV图像处理
• YOLO深度学习物体检测
55

56. 机器人的眼: 机器人视觉，物件识别
• 在RoboCup 2016中实施人物识别任务的结果
56

57. Improving Deep Learning Based Object Detection by
CycleGAN Method Under Inconsistent Illumination Conditions
57
Three illumination conditions of
the real environment
CycleGAN is used to realize the mutual
transformation of scenes
Dark environment before
brightness enhancement
Dark environment after
brightness enhancement
Object detection after
brightness enhancement
The top view of the
visual task scene and
the robot vision with
supplementary light
Object detection
confidence level
improvement
[F. Wang, J. T. C. Tan, “Improving Deep Learning Based Object Detection of Mobile Robot Vision by HSI Preprocessing Method and
CycleGAN Method Under Inconsistent Illumination Conditions in Real Environment,” in Proc. of the 2019 IEEE/ASME AIM, October 2019]

58. 机器人的腿: 移动平台，室内导航
58
• 室内自主导航
– Adaptive Monte Carlo Localization (AMCL)
– Simultaneous Localization and Mapping (SLAM)
– 静态和动态避障

59. 机器人的手: 机械手臂，抓取物件
59
http://wiki.ros.org/turtlebot_block_manipulation
Object Manipulation

60. Multi-Object Grasp Planning in High Distribution Density
using Inverse Reachability Map and Base Repositioning
60
Experiment environment and object distribution IRM of different type of objects
System components and operation flow
Experiment results
[Y. Xi, J. T. C. Tan, F. Wang, H. Song, “Multi-Object Grasp Planning in High Distribution Density of Service Robot
Using Inverse Reachability Map and Base Repositioning,” in Proc. of the 2019 IEEE ARSO, November 2019]

61. 机器人的嘴: 语音识别，人机交互
• 语音合成 (Text-
to-Speech)
– Festival, ROS
sound_play
• 语音识别
(offline)
– CMUSphinx,
ROS
Pocketsphinx
• 语音识别 (online)
– XunFei, Web
Speech API
• Facial Expression
by Emoticon
61
[H. Song, J. T. C. Tan, Y. Xing, G. Hou, “Communication Efficiency and User Experience Analysis of Visual and Audio
Feedback Cues in Human and Service Robot Voice Interaction Cycle,” in Proc. of the 2019 WRC SARA, August 2019]

62. 机器人的脑: 人工智能，机械学习，云计算与大数据
62
Client Systems
Robot Learning
Knowledge
Transfer
Cloud System
• Processing Servers
• Databases
?

63. 机器人的脑: 人工智能，机械学习，云计算与大数据
• 云端众包大量虚拟人机交互实验，用于协作策略学习
63[J. T. C. Tan, Y. Hagiwara, T. Inamura, “Robot Learning Framework via Crowdsourcing of Human-Robot Interaction
for Collaborative Strategy Learning,” in Proc. of the 24th IEEE RO-MAN (Interactive Session), IS04, 2015]

64. State parameters:
• Self
• Action
• Object(Target)
• Location
𝑆𝑒𝑙𝑓_𝐴𝑐𝑡𝑖𝑜𝑛𝑖 = 𝑓 𝑆𝑒𝑙𝑓_𝐴𝑐𝑡𝑖𝑜𝑛𝑖−1, 𝑃𝑎𝑟𝑡𝑛𝑒𝑟_𝐴𝑐𝑡𝑖𝑜𝑛𝑖, 𝑊𝑜𝑟𝑘_𝐶𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛𝑖
𝐴𝑔𝑒𝑛𝑡_𝐴𝑐𝑡𝑖𝑜𝑛(𝑂𝑏𝑗𝑒𝑐𝑡, 𝐿𝑜𝑐𝑎𝑡𝑖𝑜𝑛)
𝑊𝑜𝑟𝑘_𝐶𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 = 𝑂𝑏𝑗𝑒𝑐𝑡1(𝐿𝑜𝑐𝑎𝑡𝑖𝑜𝑛), … , 𝑂𝑏𝑗𝑒𝑐𝑡 𝑛(𝐿𝑜𝑐𝑎𝑡𝑖𝑜𝑛)
“Minimum information” to describe the current state
Collaborative Intelligence
64
• Partner
• Action
• Object(Target)
• Location
• Work
• Action(Static)
• Object1-n
• Location1-n
• Condition1-n(Omitted)

65. Extraction of Embodied Collaborative
Behaviors from Cyber-Physical HRI with
Immersive User Interfaces
• Contents
– (See) Visual Observation
• Movement of HMD to
determine observed target
– (Say) Verbal Communication
• Spoken speech
– (Do) Action
• Agent’s body movement to
determine traveled path
• Timing
– Contents’ occurrence timings
w.r.t. collaboration operation

66. Crowdsourcing of HRI and Robot
“Collaborativeness” Evaluation
66

67. 机器人的脑: 人工智能，机械学习，云计算与大数据
• 云端众包大量虚拟人机交互实验，用于协作策略学习
67

68. 机器人的脑: 人工智能，机械学习，云计算与大数据
• 云端众包大量虚拟人机交互实验，用于协作策略学习
68
Handyman (GPSR) Interactive Clean Up
Human Navigation

69. Take-Home Messages
1. 起源：Team KameRider
 “能行！”
2. RoboCup@Home EDUCATION Initiative
a. Education Challenge 教育竞赛
 “我们大家一起组织！”
b. 开源机器人教育平台
 “每个人都可以开发服务机器人！”
c. 推广活动
 “带我们到您的社区！”
3. 服务机器人科研开发
 “每个人都可以学习AI和机器人！”

70. 谢谢！
www.RoboCupatHomeEDU.org
Supported by:
iHR研究専門委員会