The document provides an overview of Robot Operating System (ROS) and demonstrates the Kinect XBOX 360 sensor using ROS. It describes the different levels of ROS including packages, stacks, messages and services. It also explains node communication and concepts like coordinate frames, actions, and plugins. Finally, it gives technical details of the Kinect sensor and demonstrates its capabilities for skeletal tracking and gesture recognition using OpenNI and ROS.

1. Piyush Chand (Master Student) NEC Europe Laboratories Ltd. Date: 3/02/2011 Robot Operating Systems (ROS) Overview & Kinect XBOX 360 Demonstration

2. Introduction ROS(Robot Operating System) ROS Classification Service Invocation in ROS Technical Overview: ROS Technical Overview: Kinect XBOX 360 OpenNI Framework OpenNI: Abstract Layered view Kinect: Process of Tracking Kinect Demonstration

5. ROS Levels ROS File system Level Packages: Packages are the main unit for organizing software in ROS. Manifests: Manifests (manifest.xml) provide metadata about a package, including its license information and dependencies, as well as language-specific information such as compiler flags. Stacks: Stacks are collections of packages that provide aggregate functionality, such as a "navigation stack." Stack manifests: Stack manifests (stack.xml) provide data about a stack, including its license information and its dependencies on other stacks. Message: Message descriptions, define the data structures for messages sent in ROS. Example: my_package/msg/MyMessageType.msg Service: Service descriptions, define the request and response data structures for services in ROS. Example: my_package/srv/MyServiceType.srv

6. ROS Computation Graph Level Nodes: Nodes are combined together into a graph and communicate with one another using streaming topics, RPC services, and the Parameter Server. ROS Master: The ROS Master provides naming and registration services to the rest of the nodes in the ROS system. Parameter Server: A parameter server is a shared, multi-variate dictionary that is accessible via network APIs. The Parameter Server is implemented using XMLRPC and runs inside of the ROS Master, which means that its API is accessible via normal XMLRPC libraries. Nodes communication: Nodes communicate with each other by passing messages. A message is a simply a data structure, comprising typed fields Topics: Topics are named buses over which nodes exchange messages based on publish / subscribe policy. Service: Request / reply is done via a Service, which is defined by a pair of messages. Bags: Bags are a format for saving and playing back ROS message data.

7. Service Invocation

8. Higher-Level Concepts Coordinate Frames/Transforms: The tf package provides a distributed, ROS-based framework for calculating the positions of multiple coordinate frames over time. Actions/Tasks: The actionlib package defines a common, topic-based interface for pre-emptible tasks in ROS. Message Ontology: The common_msgs stack provide a base message ontology for robotic systems. These includes messages for actions (actionlib_msgs), diagnostics (diagnostic_msgs), geometric primitives (geometry_msgs), robot navigation (nav_msgs), and common sensors (sensor_msgs), such as laser range finders, cameras, point clouds. Plugins : pluginlib provides a library for dynamically loading libraries in C++ code. Filters: The filters package provides a C++ library for processing data using a sequence of filters. Robot Model : The URDF(Unified Robot Description Format)package defines an XML format for representing a robot model and provides a C++ parser.

9. The ActionClient and ActionServer communicate via a "ROS Action Protocol" , which is built on top of ROS messages. The client and server then provide a simple API for users to request goals (on the client side) or to execute goals (on the server side) via function calls and callbacks. Looks like the JAVA RMI(Remote Method Invocation).

10. MIT ROS Piano Example

11. ROS based Navigation System The ROS navigation process provides the following functionality: Path planning Obstacle avoidance Automatic map making

12. Technical Overview: Kinect XBOX 360 Sensor 3D sensor RGB camera IR camera Audio device (a microphone or an array of microphones) Field of View Horizontal field of view: 57 degrees Vertical field of view: 43 degrees Physical tilt range: ± 27 degrees Depth sensor range: 1.2m – 3.5m Data Streams 320×240 16-bit depth@ 30 frames/sec 640×480 32-bit color@ 30 frames/sec 16-bit audio @ 16 kHz Skeletal Tracking System Tracks up to 6 people, including 2 active players Tracks 20 joints per active player Ability to map active players to LIVE Avatars

13. OpenNI Framework The OpenNI framework, provides an application programming interface (API) for writing applications utilizing natural interaction. This API covers communication with both low level devices (e.g. vision and audio sensors), as well as high-level middleware solutions (e.g. for visual tracking using computer vision).

14. Kinect: Abstract Layered View

15. Process of Tracking Production Nodes: Process of creating the data required for Natural Interaction. Example: User Generator, Depth Generator User Generator Node: Production node that produces body data. Depth Generator Node: A production node that is implemented by a sensor, which takes raw sensory data from a depth sensor (for example, a stream of X frames per second) and outputs a depth map. Production Chains: The sequence of nodes (user generator => depth generator), is reliant on each other in order to produce the required body data, and is called a Production Chain.

16. User Generator Node Depth Generator Node Chain Production Node

17. Florence Project Example Node functionality Wany robot

18. References: Benjamin Cohen: University of pennsylvania, Starting with ROS, 4/09/2010 Florence Multi Purpose Mobile Robot for Ambient Assisted Living ICT-2009-248730 ROS Introduction: http://www.ros.org/wiki/ROS/Introduction OpenNi User Guide: downloaded from http://www.openni.org/documentation Kinect Specifications : http://www.xbox360forum.com/xbox-forum/threads/110218-Kinect-technical-specifications-revealed

19. Kinect Demonstration: Skeleton Tracker Cursor Control Wave Gesture, Hand Tracking GL view, OpenGL (Open Graphics Library)

20. Thank You very much.........!!!! 