Leading EDGE COMPUTING 11
The Next Generation of
Smart Manufacturing-
Using Robot Operating System 2 & DDS
Bill Wang, Ph.D
Project Manager
Advanced Robotic Platform Group
2018.11.10
Leading EDGE COMPUTING 22
PrismTech Acquired in 2016
Leading EDGE COMPUTING 33
Definition of Industrial/Service Robots
*AGV in manufacturing is here
Field robot
Inspection/maintenance/construction/logistics/medical/rescue/security Cleaning/mowing/toy/hobby/
multimedia/education/elderly/
person aid
Leading EDGE COMPUTING 44
Service Robotics: Strong Growth and Technology Burst
Main drivers: logistic systems
Burst
Leading EDGE COMPUTING 55
Open Source Software systems hugely popular; e.g. >2/3 of all service
robot suppliers use Robot Operating System ROS (and other OSS)
by IFA (International Federation of Robotics) 2017
Leading EDGE COMPUTING 66
ROS 2 Alpha version(Ardent Apalone) release on 2017.12
ROS 10 Years of History ~
Zhang et al., Sci. Robot. 2, eaar1868 (2017) 25 October 2017
• ROS is widely used from mobile robots,
industrial robots, probing robots, human–
computer interaction robots, to
autonomous car and group robots
• Integrated navigation, simulation,
trajectory planning, collision avoidance,
environmental awareness
ROS
2017.12
Leading EDGE COMPUTING 77
Revolution of AGV
Multiple AGVsAGV in the past
To Be:
VS
As Is:
Leading EDGE COMPUTING 88
Release in June 2018
ROS 2 release Bouncy Bolson!
•Ubuntu 18.04 (Bionic)
• Debian packages for amd64 as well as arm64
•Ubuntu 16.04 (Xenial)
• no Debian packages but building from source is
supported
•Mac OS X 10.12 (Sierra)
•Windows 10 with Visual Studio 2017
•Binary packages / fat archives support three RMW
implementations by default (without the need to build from
source):
• eProsima's FastRTPS (default)
• RTI's Connext
• ADLINK's OpenSplice
https://github.com/ros2/ros2/wiki/Release-Bouncy-Bolson
Leading EDGE COMPUTING 99
De-centralized International Open
Architecture Becomes Popular!
OPC-UA PUB/SUB over TSN
ROS
Node 2
ROS
Node 3
ROS
Node 4
ROS
Node 1
Distributed (UDP)
ROS 2 from ROS 1
Factory Automation Robotics Autonomous Car
MISSION CRITICAL
BUSINESS CRITICAL
Leading EDGE COMPUTING 1212
Fault Propagation in Centralized System
Leading EDGE COMPUTING 1313
Fault Propagation in Centralized System
Leading EDGE COMPUTING 1414
Fault Propagation in Centralized System
Leading EDGE COMPUTING 1515
Fault Propagation in Centralized System
Leading EDGE COMPUTING 1616
Fault Propagation in Centralized System
Leading EDGE COMPUTING 1717
DDS Architecture
Physical View
DDS MIDDLEWARE
Stable Interfaces
Scalable Architecture
Infrastructure ensures “the right data at the right place, at
the right time”
Autonomous Software
Components
Leading EDGE COMPUTING 1818
• The world's largest robots open source platform
• 3M, ABB, BMW, Ford, BOSCH, Boeing,
Siemens, MIT, Stanford
• 3000+ developers
• 14 million download (2017)
• 180,000+ commits
• 140+ kinds of robots
• Reusable modular components
• Support all kinds of H/S components
• OpenCV, TensorFlow, etc.
• CUDA, CPU, ARM, X86
• Lidar, Infrared sensors, depth sensor, camera
ROS Industrial (ROS-I)
Leading EDGE COMPUTING 1919
5G Factories of the Future
Leading EDGE COMPUTING 2020
5G Factories of the Future
• Mobile robot
• Production line connectivity
• Human-Robot Collaboration
• Wireless private network
SCM
CRM
PLM
ERP
APS MES
• Multiple AGVs with collision avoidance
ROS2 DDS 5G
charging area
UAV
AMR
Pallet
Customers
Customized Order
• Real-time wireless private network
u-sensors
u-meter
u-actuator
u-actuator
ROS 2
@Zenoh
• ROS2 over
u-controller/transceiver
Leading EDGE COMPUTING 2121
Related Open Source Resource of ADLINK
- YAKS
- FogOS
- Cyclone DDS (https://github.com/eclipse/cyclonedds)
- RMW_Cyclone DDS
- Zenoh
- RMW_Zhe
- Docker for DDS
- NeuronBot
- DDSBot
- OmniBot
- RT-Kernel
- ROS 2.0 version of EAI ydlidar pkg
- adlink_ros2_tools (performance)
https://github.com/atolab
Leading EDGE COMPUTING 2222
• ROS 1 Standalone
• ROS 2 coordination among machines, adopt
DDS (pub-sub)
ROS
Master
ROS
Node 1
ROS
Node 2
ROS
Node 3
ROS
Node 4
Robot 1
Server-client (TCP)
Registration
Robot 2
Message
ROS 1 (2010)
ROS 2 (2014)
ROS
Node 2
ROS
Node 3
ROS
Node 4
ROS
Node 1
Distributed (UDP)
• Real-time
• Powerful QoS
• Standard tech.
• Dynamic discovery)
ROS2Merit
ROS 2 removes the
Master Node
Using Distributed/De-centralized Way
Leading EDGE COMPUTING 2323
ROS & ROS 2 could Bridge
ROS 1 (Kinetic) Environment
(Nodes, packages)
Linux O.S. (Ubuntu)
Hardware
GPIO, RS-232/RS-485
Intel® RealSense™
Depth Camera
Linux driver Linux driver
Opensplice DDS
API (C/C++/python)
ROS 2 Environment
(Nodes, packages)
ROS 1 &2
Bridge
API (C/C++/python)
Realtime kernel
Patch (Xenomai)
Intel Movidius VPU (USB3.0)
Leading EDGE COMPUTING 2424
Time-Deterministic Communication
Application ….
Motion
planning
Localization
and mapping
Computer
Vision
3D data
processing
Robot control
ROS client library (& drivers)
Vortex OpenSplice
Commercial version
Vortex OpenSplice
Community version
ROS abstract middleware layer (RMW)
Motion
planning
Localization
and mapping
Computer
Vision
3D data
processing
Robot control
ROS client library (& drivers)
DDS Vortex
XRCE
Huawei
TSN (Ethernet)
IEEE 802.1
• Global time
• Schedule
Time Deterministic
Network
(Guaranteed Latency)
5G (Wireless)
Leading EDGE COMPUTING 2525
Architecture I: Cloud of Robots X AI
ADLINK
ROS2
controller
Smart
Camera
5G EPC+MEC
EtherCAT
passed failed
pick & place
Defect
check
1. Dynamic
trajectory
inference
Motion control
deploy
deploy pretrained model
new result
new result
2. Advanced
analytics
Leading EDGE COMPUTING 2626
ROS 2
Architecture II: Swarm of Robots
Leading EDGE COMPUTING 2727
Give ROS users a useful tool- ADLINK Neuron
the world’s first industrial-grade ROS Starter Kit
• High flexible hardware specification
➢ Intel CPU i3/i5/i7/Celeron
➢ RAM ~32G DDR4
➢ SSD: 32G ~ (mSATA)
• Plenty of I/O interface (ALL-IN-ONE)
➢ PCIe x16 gen3 * 1
➢ PCIe x1 gen2 * 1
➢ RS232 * 3, RS485* 1, GPIO * 10, USB 3/2 * 4 (both)
➢ GbE port * 2, miniPCIe * 2, Input: 12 or 5 Volt
• Mini-ITX form factor is a compatible design
• Supported DC battery (12V, 24V DC)
Leading EDGE COMPUTING 2828
3 Robot Reference Architecture
ROS 2 Starter Kit - Neuron
DDSBot NeuronBot OmniBot
Promotion of peer-to-peer swarm
architecture of ROS 2
Promotion ROS 2 w/ AI Promotion in Education
Leading EDGE COMPUTING 2929
NVIDIA® Jetson™ TX2 Edge Inference Platform
Give ROS and AI users a useful tool - ADLINK M200-JT2
• Key Features
➢Equipped with NVIDIA® Jetson™ TX2 System on Module
➢Dual Core Denver + Quad Core A57
➢8GB LPDDR4, 32GB eMMC
➢Rich I/O & expansion
➢2x HDMI (lockable), 3x USB 3.0, 1x OTG, 1x GbE, 1x RS-232, 12Vdc input
➢1x mPCIe slot with USIM, 1x m.2 SATA slot, 1x SD card slot
https://www.adlinktech.com/Products/Deep_Learning_Accelerator_Plat
form_and_Server/Inference_Platform/M200-JT2?lang=en
To accelerate deep learning workloads for
Object Detection, Recognition and Classification !
Leading EDGE COMPUTING 3030
• ADLINK® M200-JT2 Edge Inference Platform
➢ NVIDIA® Jetson™ TX2
• AI & Visioning
➢ Depth Camera
➢ Cuda + CudNN
• Robotic Operating System
➢ ROS 1/ROS 2
➢ DDS
• Lidar and Navigation
➢ SLAM
➢ Navigation
• A Fast Moving Base
➢ TurtleBot
Combine ROS and AI with TX2
ADLINK TegraBot
Depth Camera
Lidar
TurtleBot
ADLINK M200-JT2 (TX 2)
Leading EDGE COMPUTING 3131
ADLINK TegraBot Function Architecture
Robot Navigation
Localization
Lidar
Navigation
Map Info Sys
AI tracker
Target Filter
Depth Camera
TF Classifier
Target Controller
Base Driver
State machine
IO communication
ROS
Visualization
State monitoring
Remote Robot
Control
SSH
Management
Launch
TegraBot Neuron
Wireless
Combine ROS and AI with TX2 for people tracking
Leading EDGE COMPUTING 3232
AI training method and data
ADLINK TegraBot - Autonomous tracking by cloth
• CNN: Tensor flow
➢Single Shot MultiBox Detector
➢MobileNets
✓Convolutional Neural Networks for Mobile Vision
✓Depthwise separable convolution
• Cloth classifier
➢Training set: ~300 pic
➢Classes: Tshirt, Shirt, Suit, and Polo Shirt
➢Train step: 200,000 on a GPU
Leading EDGE COMPUTING 3333
ADLINK TegraBot - ROS Node Graph
Depth Camera
People Tracker
TensorFlow
Classifier Base Driver
Robot Navigation
Leading EDGE COMPUTING 3434
The robot would recognize and track the person who wear shirt
ADLINK TegraBot with ROS and AI
Tracking People Recognize the cloth with 2 people Recognize the cloth with 4 people
Depth Camera
RGB Image
Inferred Image
Depth Image
TensorFlow Classifier
SSD
MobilNet
People Tracker
person ID
Depth acquisition
tracking filter
Tracking Controller
Target filter
Command Gen
Dynamic filter
Depth Resolver
Open NI
Point Cloud Library
Base Driver
State machine IO comm
Leading EDGE COMPUTING 3535
Power by ADLINK, ROS-I, ABB
ADLINK use ROS to control ABB robot arm
https://www.youtube.com/watch?v=q-ansSW0Cec
Leading EDGE COMPUTING 3636
https://neuron.adlinktech.com
ROS 2 Starter Kit - Neuron
Leading EDGE COMPUTING 3737
ADLINK’s experience in ROS/DDS training course
Hsin-Ming Youth Hub, Taoyuan City Government, Taiwan Automotive component manufacturing company, Japan
Leading EDGE COMPUTING 3838
We Want You!
凌華科技 - 機器人平台團隊現在有在徵一位應用工程師(FAE)
• 主要的工作內容如下:
1. Pre-sale technical support(present, training).
2. Pose-sale technical support(co-work, Q&A).
3. Support the engineering part of some specific projects.
4. Co-work with sales team.
5. POC/demonstration of ADLINK DDS(data distribution service)
• 需要的相關技能如下:
1. MS or higher in technical field(Computer science, engineering).
2. Strong understanding of network stack and OS.
3. Understanding of messaging system
4. Familiar with C/C++/Python.
5. Embedded system experience.
6. Team work player.
7. Enjoy problem solving.
8. Good English.
9. Ability to travel about 30% of time.
• 我們團隊是非常年輕的團隊,主管才不到35歲,部門平均年齡不到30歲,
整個思維非常創新啊,部門會有不定期聚餐以及各項活動。
Please contact, Dr. Bill Wang:
bill.wang@adlinktech.com
Leading EDGE COMPUTING 3939

【1110ROS社群開講】ROS 2與DDS應用於工業領域_王健豪

  • 1.
    Leading EDGE COMPUTING11 The Next Generation of Smart Manufacturing- Using Robot Operating System 2 & DDS Bill Wang, Ph.D Project Manager Advanced Robotic Platform Group 2018.11.10
  • 2.
    Leading EDGE COMPUTING22 PrismTech Acquired in 2016
  • 3.
    Leading EDGE COMPUTING33 Definition of Industrial/Service Robots *AGV in manufacturing is here Field robot Inspection/maintenance/construction/logistics/medical/rescue/security Cleaning/mowing/toy/hobby/ multimedia/education/elderly/ person aid
  • 4.
    Leading EDGE COMPUTING44 Service Robotics: Strong Growth and Technology Burst Main drivers: logistic systems Burst
  • 5.
    Leading EDGE COMPUTING55 Open Source Software systems hugely popular; e.g. >2/3 of all service robot suppliers use Robot Operating System ROS (and other OSS) by IFA (International Federation of Robotics) 2017
  • 6.
    Leading EDGE COMPUTING66 ROS 2 Alpha version(Ardent Apalone) release on 2017.12 ROS 10 Years of History ~ Zhang et al., Sci. Robot. 2, eaar1868 (2017) 25 October 2017 • ROS is widely used from mobile robots, industrial robots, probing robots, human– computer interaction robots, to autonomous car and group robots • Integrated navigation, simulation, trajectory planning, collision avoidance, environmental awareness ROS 2017.12
  • 7.
    Leading EDGE COMPUTING77 Revolution of AGV Multiple AGVsAGV in the past To Be: VS As Is:
  • 8.
    Leading EDGE COMPUTING88 Release in June 2018 ROS 2 release Bouncy Bolson! •Ubuntu 18.04 (Bionic) • Debian packages for amd64 as well as arm64 •Ubuntu 16.04 (Xenial) • no Debian packages but building from source is supported •Mac OS X 10.12 (Sierra) •Windows 10 with Visual Studio 2017 •Binary packages / fat archives support three RMW implementations by default (without the need to build from source): • eProsima's FastRTPS (default) • RTI's Connext • ADLINK's OpenSplice https://github.com/ros2/ros2/wiki/Release-Bouncy-Bolson
  • 9.
    Leading EDGE COMPUTING99 De-centralized International Open Architecture Becomes Popular! OPC-UA PUB/SUB over TSN ROS Node 2 ROS Node 3 ROS Node 4 ROS Node 1 Distributed (UDP) ROS 2 from ROS 1 Factory Automation Robotics Autonomous Car
  • 10.
  • 11.
  • 12.
    Leading EDGE COMPUTING1212 Fault Propagation in Centralized System
  • 13.
    Leading EDGE COMPUTING1313 Fault Propagation in Centralized System
  • 14.
    Leading EDGE COMPUTING1414 Fault Propagation in Centralized System
  • 15.
    Leading EDGE COMPUTING1515 Fault Propagation in Centralized System
  • 16.
    Leading EDGE COMPUTING1616 Fault Propagation in Centralized System
  • 17.
    Leading EDGE COMPUTING1717 DDS Architecture Physical View DDS MIDDLEWARE Stable Interfaces Scalable Architecture Infrastructure ensures “the right data at the right place, at the right time” Autonomous Software Components
  • 18.
    Leading EDGE COMPUTING1818 • The world's largest robots open source platform • 3M, ABB, BMW, Ford, BOSCH, Boeing, Siemens, MIT, Stanford • 3000+ developers • 14 million download (2017) • 180,000+ commits • 140+ kinds of robots • Reusable modular components • Support all kinds of H/S components • OpenCV, TensorFlow, etc. • CUDA, CPU, ARM, X86 • Lidar, Infrared sensors, depth sensor, camera ROS Industrial (ROS-I)
  • 19.
    Leading EDGE COMPUTING1919 5G Factories of the Future
  • 20.
    Leading EDGE COMPUTING2020 5G Factories of the Future • Mobile robot • Production line connectivity • Human-Robot Collaboration • Wireless private network SCM CRM PLM ERP APS MES • Multiple AGVs with collision avoidance ROS2 DDS 5G charging area UAV AMR Pallet Customers Customized Order • Real-time wireless private network u-sensors u-meter u-actuator u-actuator ROS 2 @Zenoh • ROS2 over u-controller/transceiver
  • 21.
    Leading EDGE COMPUTING2121 Related Open Source Resource of ADLINK - YAKS - FogOS - Cyclone DDS (https://github.com/eclipse/cyclonedds) - RMW_Cyclone DDS - Zenoh - RMW_Zhe - Docker for DDS - NeuronBot - DDSBot - OmniBot - RT-Kernel - ROS 2.0 version of EAI ydlidar pkg - adlink_ros2_tools (performance) https://github.com/atolab
  • 22.
    Leading EDGE COMPUTING2222 • ROS 1 Standalone • ROS 2 coordination among machines, adopt DDS (pub-sub) ROS Master ROS Node 1 ROS Node 2 ROS Node 3 ROS Node 4 Robot 1 Server-client (TCP) Registration Robot 2 Message ROS 1 (2010) ROS 2 (2014) ROS Node 2 ROS Node 3 ROS Node 4 ROS Node 1 Distributed (UDP) • Real-time • Powerful QoS • Standard tech. • Dynamic discovery) ROS2Merit ROS 2 removes the Master Node Using Distributed/De-centralized Way
  • 23.
    Leading EDGE COMPUTING2323 ROS & ROS 2 could Bridge ROS 1 (Kinetic) Environment (Nodes, packages) Linux O.S. (Ubuntu) Hardware GPIO, RS-232/RS-485 Intel® RealSense™ Depth Camera Linux driver Linux driver Opensplice DDS API (C/C++/python) ROS 2 Environment (Nodes, packages) ROS 1 &2 Bridge API (C/C++/python) Realtime kernel Patch (Xenomai) Intel Movidius VPU (USB3.0)
  • 24.
    Leading EDGE COMPUTING2424 Time-Deterministic Communication Application …. Motion planning Localization and mapping Computer Vision 3D data processing Robot control ROS client library (& drivers) Vortex OpenSplice Commercial version Vortex OpenSplice Community version ROS abstract middleware layer (RMW) Motion planning Localization and mapping Computer Vision 3D data processing Robot control ROS client library (& drivers) DDS Vortex XRCE Huawei TSN (Ethernet) IEEE 802.1 • Global time • Schedule Time Deterministic Network (Guaranteed Latency) 5G (Wireless)
  • 25.
    Leading EDGE COMPUTING2525 Architecture I: Cloud of Robots X AI ADLINK ROS2 controller Smart Camera 5G EPC+MEC EtherCAT passed failed pick & place Defect check 1. Dynamic trajectory inference Motion control deploy deploy pretrained model new result new result 2. Advanced analytics
  • 26.
    Leading EDGE COMPUTING2626 ROS 2 Architecture II: Swarm of Robots
  • 27.
    Leading EDGE COMPUTING2727 Give ROS users a useful tool- ADLINK Neuron the world’s first industrial-grade ROS Starter Kit • High flexible hardware specification ➢ Intel CPU i3/i5/i7/Celeron ➢ RAM ~32G DDR4 ➢ SSD: 32G ~ (mSATA) • Plenty of I/O interface (ALL-IN-ONE) ➢ PCIe x16 gen3 * 1 ➢ PCIe x1 gen2 * 1 ➢ RS232 * 3, RS485* 1, GPIO * 10, USB 3/2 * 4 (both) ➢ GbE port * 2, miniPCIe * 2, Input: 12 or 5 Volt • Mini-ITX form factor is a compatible design • Supported DC battery (12V, 24V DC)
  • 28.
    Leading EDGE COMPUTING2828 3 Robot Reference Architecture ROS 2 Starter Kit - Neuron DDSBot NeuronBot OmniBot Promotion of peer-to-peer swarm architecture of ROS 2 Promotion ROS 2 w/ AI Promotion in Education
  • 29.
    Leading EDGE COMPUTING2929 NVIDIA® Jetson™ TX2 Edge Inference Platform Give ROS and AI users a useful tool - ADLINK M200-JT2 • Key Features ➢Equipped with NVIDIA® Jetson™ TX2 System on Module ➢Dual Core Denver + Quad Core A57 ➢8GB LPDDR4, 32GB eMMC ➢Rich I/O & expansion ➢2x HDMI (lockable), 3x USB 3.0, 1x OTG, 1x GbE, 1x RS-232, 12Vdc input ➢1x mPCIe slot with USIM, 1x m.2 SATA slot, 1x SD card slot https://www.adlinktech.com/Products/Deep_Learning_Accelerator_Plat form_and_Server/Inference_Platform/M200-JT2?lang=en To accelerate deep learning workloads for Object Detection, Recognition and Classification !
  • 30.
    Leading EDGE COMPUTING3030 • ADLINK® M200-JT2 Edge Inference Platform ➢ NVIDIA® Jetson™ TX2 • AI & Visioning ➢ Depth Camera ➢ Cuda + CudNN • Robotic Operating System ➢ ROS 1/ROS 2 ➢ DDS • Lidar and Navigation ➢ SLAM ➢ Navigation • A Fast Moving Base ➢ TurtleBot Combine ROS and AI with TX2 ADLINK TegraBot Depth Camera Lidar TurtleBot ADLINK M200-JT2 (TX 2)
  • 31.
    Leading EDGE COMPUTING3131 ADLINK TegraBot Function Architecture Robot Navigation Localization Lidar Navigation Map Info Sys AI tracker Target Filter Depth Camera TF Classifier Target Controller Base Driver State machine IO communication ROS Visualization State monitoring Remote Robot Control SSH Management Launch TegraBot Neuron Wireless Combine ROS and AI with TX2 for people tracking
  • 32.
    Leading EDGE COMPUTING3232 AI training method and data ADLINK TegraBot - Autonomous tracking by cloth • CNN: Tensor flow ➢Single Shot MultiBox Detector ➢MobileNets ✓Convolutional Neural Networks for Mobile Vision ✓Depthwise separable convolution • Cloth classifier ➢Training set: ~300 pic ➢Classes: Tshirt, Shirt, Suit, and Polo Shirt ➢Train step: 200,000 on a GPU
  • 33.
    Leading EDGE COMPUTING3333 ADLINK TegraBot - ROS Node Graph Depth Camera People Tracker TensorFlow Classifier Base Driver Robot Navigation
  • 34.
    Leading EDGE COMPUTING3434 The robot would recognize and track the person who wear shirt ADLINK TegraBot with ROS and AI Tracking People Recognize the cloth with 2 people Recognize the cloth with 4 people Depth Camera RGB Image Inferred Image Depth Image TensorFlow Classifier SSD MobilNet People Tracker person ID Depth acquisition tracking filter Tracking Controller Target filter Command Gen Dynamic filter Depth Resolver Open NI Point Cloud Library Base Driver State machine IO comm
  • 35.
    Leading EDGE COMPUTING3535 Power by ADLINK, ROS-I, ABB ADLINK use ROS to control ABB robot arm https://www.youtube.com/watch?v=q-ansSW0Cec
  • 36.
    Leading EDGE COMPUTING3636 https://neuron.adlinktech.com ROS 2 Starter Kit - Neuron
  • 37.
    Leading EDGE COMPUTING3737 ADLINK’s experience in ROS/DDS training course Hsin-Ming Youth Hub, Taoyuan City Government, Taiwan Automotive component manufacturing company, Japan
  • 38.
    Leading EDGE COMPUTING3838 We Want You! 凌華科技 - 機器人平台團隊現在有在徵一位應用工程師(FAE) • 主要的工作內容如下: 1. Pre-sale technical support(present, training). 2. Pose-sale technical support(co-work, Q&A). 3. Support the engineering part of some specific projects. 4. Co-work with sales team. 5. POC/demonstration of ADLINK DDS(data distribution service) • 需要的相關技能如下: 1. MS or higher in technical field(Computer science, engineering). 2. Strong understanding of network stack and OS. 3. Understanding of messaging system 4. Familiar with C/C++/Python. 5. Embedded system experience. 6. Team work player. 7. Enjoy problem solving. 8. Good English. 9. Ability to travel about 30% of time. • 我們團隊是非常年輕的團隊,主管才不到35歲,部門平均年齡不到30歲, 整個思維非常創新啊,部門會有不定期聚餐以及各項活動。 Please contact, Dr. Bill Wang: bill.wang@adlinktech.com
  • 39.