This document discusses real-time cloud robotics and its applications in smart cities. It introduces the C2RO cloud robotics platform, which uses stream processing to connect low-cost robotic devices virtually and process data. The document addresses the challenge of latency in cloud robotics and proposes a hybrid cloud/edge computing model to distribute processing. It provides examples of using C2RO for object recognition and SLAM. The document also describes a people counting project in collaboration with Philips and MIT using C2RO to process data from smart street lights.

1. Real-Time Cloud Robotics
in Practical Smart City Applications
Nazli Khan Beigi, Bahar Partov, Soodeh Farokhi
C2RO: Collaborative Cloud Robotics
Montreal, Canada

2. Paper Contribution
• Cloud robotics as a means of
resource optimization in smart city applications.
• C2RO platform contributing in
• Parallel computing,
• Cloud and edge computing,
• Machine learning,
• Computer vision.

3. What is Cloud Robotics?
• The major trend in today’s robotics since its
emergence in 2010.
• Network-connected robots offloading
• Intensive and complex computation tasks
• Data sharing available in a centralized location.
• Advancements in the cloud computing and big data:
• To push the barriers in artificial intelligence applications.

4. C2RO Platform
• C2RO cloud robotics platform uses real-time stream processing technology
• Virtually connects, stores, and processes
• The energy-efficient and low-cost mobile robotic devices or sensors.

5. Cloud Robotics Challenge: Latency
End-to-end latency = processing time + network response time
Robotics applications classifications:
• Hard-real time applications, e.g., collision avoidance
• Soft real-time applications, e.g., object recognition
• Delay tolerant applications, e.g., mapping

6. Latency, Processing Time
In a collision avoidance application, for instance, the PT includes:
• Video capturing
• Video compression/ decompression
• Object recognition
• Algorithm Implementation

7. Latency, Network Response Time
• Latency in lower layers of communication, i.e., from devices to the edge of the network
• Latency in higher levels of the communication network, i.e., from edge to the cloud:
• Professional data stream network (DSN)
• To reduce the routing latency
• To provide redundant paths
• A partnership with PubNub, globally scaled DSN
• With 14 data centers and 99.999% service level agreements (SLAs).

8. Ultimate Solution to Latency:
Hybrid Cloud Robotics
• The real-time processing is dynamically
distributed among resources
• on-board,
• on the edge,
• on the cloud.
• Edge computing, i.e., fog, to fill the latency
and network incompetency gap in real-time
applications.

9. C2RO Hybrid Cloud Robotics
Computation Model Examples
• LIDAR sensor | car processing unit | cloud;
• Vacuum cleaner or toy | personal computer or home network router | cloud;
• Surveillance camera | significant robot | cloud.

10. Experimental Setup:
• Real time Object Recognition
• C-SLAM: Localization and Autonomous Mapping

11. Application of C2RO Platform in Smart Cities
• Smart cities have scattered massive number of
sensors
• Complicated infrastructure to connect and process
• Cloud robotics: an efficient computing means in
intensive data processing applications.
• A high-profile collaborative urban “People
Counting” project done in collaboration with
Philips and MIT.
• Street lights are transformed the into
multifunctional digital urban platforms.
• Cloud connected smart thermal sensors are added

12. People Counting System Configuration
• Raspberry Pi 3 SBC,
• Radiometric-capable LWIR camera, i.e. FLIR Lepton.
• Four VMs
• The platform and dashboard at AmazonEC2.
VMs in the Cloud
CPU Model Intel Xeon E5
CPU Frequency/ Mhz 2400
Cores 4/8
Thread per core 1
Memory/MB 8192
OS Ubuntu 16.04 (Linux 4.4)

13. People Counting Snapshot

14. People Counting Video

15. Conclusion
• Cloud robotics has been significantly pushed the barriers of robotics.
• It is not still a proper model for real-time, latency sensitive smart city
applications.
• We propose our hybrid cloud robotics model
• We present a scalable platform, named C2RO platform, for parallel
computing
• The application of this platform for a high-profile urban project was reported

16. References
1. B. Tang et al., “Incorporating Intelligence in Fog Computing for Big Data Analysis in Smart Cities,” in IEEE Trans. on Industrial Informatics, March 2017
2. B. Kehoe, et al., “A Survey of Research on Cloud Robotics and Automation,” in IEEE Trans. On Automation & Eng., VOL. 12, NO. 2, April 2015
3. J. Wan, et al., “Cloud Robotics: Current Status and Open Issues,” in Special Section in IEEE Access: The Plethora of Research in Internet of Things (IoT),
2016
4. N. Gangid, B. Sharma, “Cloud Computing and Robotics for Disaster Management,” in IEEE Inter. Conference on Intelligent Systems, Modelling and
Simulation, 2016
5. J. Salmeron-Garcia, et al., “A Tradeoff Analysis of a Cloud-Based Robot Navigation Assistant Using Stereo Image Processing”, in IEEE Trans. On Automation
Science and Engineering, Vol.12, No.2, April 2015
6. L. Riazuelo et al., “RoboEarth Semantic Mapping: A Cloud Enabled Knowledge-Based Approach,” in IEEE Trans. On Automation Science and Eng., VOL. 12,
NO. 2, April 2015
7. K. Sugiura, K. Zettsu, “Rospeex: A Cloud Robotics Platform for Human-Robot Spoken Dialogues,” in IROS, Sept. 2015, pp. 6155-6060.
8. J. Mahler et al., “Dex-Net 1.0: A Cloud-Based Network of 3D Objects for Robust Grasp Planning Using a Multi-Armed Bandit Model with Correlated
Rewards,” in ICRA, May 2016
9. S. Dey, A. Mukherjee, “Robotic SLAM - a Review from Fog Computing and Mobile Edge Computing Perspective,” in IEEE International Conference on
Mobile and Ubiquitous Systems: Computing Networking and Services, 2016
10. M. Satyanarayanan, “Edge Computing for Situational Awareness,” in IEEE Local and Metropolitan Area Networks (LANMAN), 2017
11. K. Bilal, A. Erbad, “Edge Computing for Interactive Media and Video Streaming,” in IEEE International Conference on Fog and Mobile Edge Computing
(FMEC), 2017.
12. S. Kamburugamuve, L. Christiansen, G. Fox, “A framework for real-time processing of sensor data in the cloud”, in Journal of Sensors, 2015
13. H. He, et al., “Cloud based Real-time Multi-Robot Collision Avoidance for Swarm Robotics”, in International Journal of Grid and Distributed Computing, Vol.
9, No.6, 2016, pp. 339- 358
14. S. Farokhi et al., “Performance Boost with Hybrid Cloud Robotics,” in International Conference on Intelligent Robots and Systems, Sept. 2017
15. A. Anjomshoaa, et al., “Quantifying the Anthropogenic Heat in Urban Areas Using Thermal Images,” in CSCI, 2016

17. Contact Us
• Soodeh Farokhi, Founder and CTO, C2RO Robotics, soodeh.farokhi@c2ro.com
• Nazli Khan Beigi, Communication Specialist, C2RO Robotics, nazli.khanbeigi@c2ro.com
• www.c2ro.com

18. Additional Slides
Use cases of C2RO Hybrid Cloud Robotics

19. Use cases of C2RO Hybrid Cloud Robotics
Object Recognition (OR):
• The robotic platform is equipped with a single-board computer (SBC), i.e.
Raspberry Pi 3, and an RGBD sensor.
• In case that OR is performed only on the SBC, the performance of the
process is ~0.07fps, and the user sees new images with detected objects
every ~13 seconds.
• While using our proposed model, the entire process improves to ~50fps,
which translates into a real-time visualization of the detection process.

20. Use cases of C2RO Hybrid Cloud Robotics
Simultaneous Localization and Mapping (SLAM)
• The robotic platform is equipped with a single-board computer (SBC), i.e.
Raspberry Pi 3, and an RGBD sensor.
• If the map is built locally on the SBC, the SLAM algorithm update rate is
~5Hz;
• While by using our hybrid cloud robotics model, the map update rate was
enhanced to 30Hz.

21. Additional Slides
Parallel Cloud Computation

22. Cloud Computation Topologies
1. Carries out all computation in one processor. Though this reduces the
transmission of multi-robot state information between processors, but due
to huge computations, the processor would be very busy and this leads
into slow computation.
2. Assigns each computational component to one processor. This would
require transfer of robots’ state information between processors, in
addition to an imbalance of processing between delay-sensitive and non-
delay-sensitive components. So, the overall delay of this topology was
rather high too.
3. Parallel processing topology that unlike other parallel algorithms, would
focus on entity or agent level parallelization and would study mainly
computation resource scaling based on the computation load.

23. Parallel Cloud Computation
The topology is to prevent the delays caused by non-delay sensitive functions
by combining the processes of the similar essence in one processor.
Also, although each processor could run multiple instances in parallel, we
made sure that the proper data types are sent to the processor instance that
catches the right robot information.
Hence, by proposing an innovative load balancing technique, we provide an
efficient, low latency parallel computation algorithm

24. Parallel Cloud Computation
The same algorithm is implemented on SBC, a desktop computer server, a desktop
computer GPU, and C2RO cloud, including 4, 4, 4, and 20 processing cores, respectively.
Hence, by proposing an innovative load balancing technique, we provide an efficient,
low latency parallel computation algorithm.