The document provides an introduction and overview of point cloud data and the Point Cloud Library (PCL). It discusses point clouds, how they are acquired from sensors and represented as data structures. It covers common file formats for storing point clouds like ROS messages and PCD files. The document introduces PCL as a C++ library for processing point clouds, describing its modular structure and capabilities for features, surface reconstruction, filtering, I/O and segmentation.
The International Journal of Engineering and Sciencetheijes
This document summarizes a research paper on an elastic management strategy for cloud storage. The paper proposes an elastic cloud storage framework based on HDFS that introduces virtual resource management. It proposes an elastic management strategy for allocating and scheduling virtual resources. Simulation results showed that the cloud storage approach can effectively improve the efficiency of using virtual resources. Keywords included HDFS, elastic cloud storage, virtual resource management, resource allocation and scheduling, and feedback control theory.
¿Sabías que con kinect puedes crear un sencillo robot en apenas 15 minutos? Aprende, de la mano de Jose Vicente Sogorb, la manera de hacerlo con unos sencillos pasos. Al tiempo conoceremos el impacto que han tenido estos nuevos elementos, como kinect o ROS, en la reactivación de un campo como la robótica móvil, que comienza a vivir una época de auge que promete nuevos e interesantes avances.
Ponente: José Vicente Sogorb es jefe del grupo de Automatismos en el departamento de I+D+i de Acciona Infraestructuras y miembro fundador de la Asociación de Robótica y Domótica de España (A.R.D.E.)
The document discusses robotics software and competitions. It introduces ROS as a robot operating system, Gazebo as a robot simulator, and CloudSim for managing robot simulations in the cloud. It describes DARPA's Urban Challenge for autonomous vehicles and Robotics Challenge for developing rescue robots. Examples show how to write Python code using ROS for publishing, subscribing and services.
06 history of cv computer vision - toweards a simple bruth-force utilityzukun
The document provides a history of computer vision from early developments in the 1980s to recent advances. It discusses early challenges like limited hardware and software. While remarkable progress has been made in areas like 3D reconstruction and object detection, challenges remain like developing high-level semantic understanding and overcoming segmentation difficulties. The document also examines research motives and whether computer vision problems are truly engineering problems or require more fundamental advances.
Cvpr2010 open source vision software, intro and training part viii point clou...zukun
The document discusses point cloud processing and analysis techniques including acquisition, feature estimation, surface classification, object recognition, and registration. It introduces the motivation to classify 3D scenes using either discriminative feature descriptors with machine learning or geometric reasoning techniques. It then outlines the topics to be covered including acquiring point clouds from sensors, estimating features like surface normals and curvature, computing descriptors like FPFH (Fast Point Feature Histograms), classifying surfaces, recognizing objects, and registering point clouds.
Mylyn helps address information overload and context loss when multi-tasking. It integrates tasks into the IDE workflow and uses a degree-of-interest model to monitor user interaction and provide a task-focused UI with features like view filtering, element decoration, automatic folding and content assist ranking. This creates a single view of all tasks that are centrally managed within the IDE.
Open Source Computer Vision (OpenCV) is a BSD-licensed open source library for computer vision and image processing. The document outlines OpenCV's capabilities including image enhancement, object classification and tracking, and face detection and recognition. It provides examples of using OpenCV in C++ and Python to load and display images, detect faces, and enhance images. The document concludes that OpenCV is a cross-platform library with over 2,000 algorithms for computer vision and image processing tasks.
The International Journal of Engineering and Sciencetheijes
This document summarizes a research paper on an elastic management strategy for cloud storage. The paper proposes an elastic cloud storage framework based on HDFS that introduces virtual resource management. It proposes an elastic management strategy for allocating and scheduling virtual resources. Simulation results showed that the cloud storage approach can effectively improve the efficiency of using virtual resources. Keywords included HDFS, elastic cloud storage, virtual resource management, resource allocation and scheduling, and feedback control theory.
¿Sabías que con kinect puedes crear un sencillo robot en apenas 15 minutos? Aprende, de la mano de Jose Vicente Sogorb, la manera de hacerlo con unos sencillos pasos. Al tiempo conoceremos el impacto que han tenido estos nuevos elementos, como kinect o ROS, en la reactivación de un campo como la robótica móvil, que comienza a vivir una época de auge que promete nuevos e interesantes avances.
Ponente: José Vicente Sogorb es jefe del grupo de Automatismos en el departamento de I+D+i de Acciona Infraestructuras y miembro fundador de la Asociación de Robótica y Domótica de España (A.R.D.E.)
The document discusses robotics software and competitions. It introduces ROS as a robot operating system, Gazebo as a robot simulator, and CloudSim for managing robot simulations in the cloud. It describes DARPA's Urban Challenge for autonomous vehicles and Robotics Challenge for developing rescue robots. Examples show how to write Python code using ROS for publishing, subscribing and services.
06 history of cv computer vision - toweards a simple bruth-force utilityzukun
The document provides a history of computer vision from early developments in the 1980s to recent advances. It discusses early challenges like limited hardware and software. While remarkable progress has been made in areas like 3D reconstruction and object detection, challenges remain like developing high-level semantic understanding and overcoming segmentation difficulties. The document also examines research motives and whether computer vision problems are truly engineering problems or require more fundamental advances.
Cvpr2010 open source vision software, intro and training part viii point clou...zukun
The document discusses point cloud processing and analysis techniques including acquisition, feature estimation, surface classification, object recognition, and registration. It introduces the motivation to classify 3D scenes using either discriminative feature descriptors with machine learning or geometric reasoning techniques. It then outlines the topics to be covered including acquiring point clouds from sensors, estimating features like surface normals and curvature, computing descriptors like FPFH (Fast Point Feature Histograms), classifying surfaces, recognizing objects, and registering point clouds.
Mylyn helps address information overload and context loss when multi-tasking. It integrates tasks into the IDE workflow and uses a degree-of-interest model to monitor user interaction and provide a task-focused UI with features like view filtering, element decoration, automatic folding and content assist ranking. This creates a single view of all tasks that are centrally managed within the IDE.
Open Source Computer Vision (OpenCV) is a BSD-licensed open source library for computer vision and image processing. The document outlines OpenCV's capabilities including image enhancement, object classification and tracking, and face detection and recognition. It provides examples of using OpenCV in C++ and Python to load and display images, detect faces, and enhance images. The document concludes that OpenCV is a cross-platform library with over 2,000 algorithms for computer vision and image processing tasks.
This document provides an overview of OpenCV, an open source computer vision and machine learning software library. It discusses OpenCV's core functionality for representing images as matrices and directly accessing pixel data. It also covers topics like camera calibration, feature point extraction and matching, and estimating camera pose through techniques like structure from motion and planar homography. Hints are provided for Android developers on required permissions and for planar homography estimation using additional constraints rather than OpenCV's general homography function.
This document provides information about the Computer Vision Laboratory 2012 course at the Institute of Visual Computing. The course focuses on computer vision on mobile devices and will involve 180 hours of project work per person. Students will work in groups of 1-2 people on topics like 3D reconstruction from silhouettes or stereo images on mobile devices. Key dates are provided for submitting a work plan, mid-term presentation, and final report. Contact information is given for the lecturers and teaching assistant.
This document summarizes a presentation on natural image statistics given by Siwei Lyu at the 2009 CIFAR NCAP Summer School. The presentation covered several key topics:
1) It discussed the motivation for studying natural image statistics, which is to understand representations in the visual system and develop computer vision applications like denoising.
2) It reviewed common statistical properties found in natural images like 1/f power spectra and non-Gaussian distributions.
3) Maximum entropy and Bayesian models were presented as approaches to model these statistics, with Gaussian and independent component analysis discussed as specific examples.
4) Efficient coding principles from information theory were introduced as a framework for understanding neural representations that aim to decorrelate and
Camera calibration involves determining the internal camera parameters like focal length, image center, distortion, and scaling factors that affect the imaging process. These parameters are important for applications like 3D reconstruction and robotics that require understanding the relationship between 3D world points and their 2D projections in an image. The document describes estimating internal parameters by taking images of a calibration target with known geometry and solving the equations that relate the 3D target points to their 2D image locations. Homogeneous coordinates and projection matrices are used to represent the calibration transformations mathematically.
Brunelli 2008: template matching techniques in computer visionzukun
The document discusses template matching techniques in computer vision. It begins with an overview that defines template matching and discusses some common computer vision tasks it can be used for, like object detection. It then covers topics like detection as hypothesis testing, training and testing techniques, and provides a bibliography.
The HARVEST Programme evaluates feature detectors and descriptors through indirect and direct benchmarks. Indirect benchmarks measure repeatability and matching scores on the affine covariant testbed to evaluate how features persist across transformations. Direct benchmarks evaluate features on image retrieval tasks using the Oxford 5k dataset to measure real-world performance. VLBenchmarks provides software for easily running these benchmarks and reproducing published results. It allows comparing features and selecting the best for a given application.
This document summarizes VLFeat, an open source computer vision library. It provides concise summaries of VLFeat's features, including SIFT, MSER, and other covariant detectors. It also compares VLFeat's performance to other libraries like OpenCV. The document highlights how VLFeat achieves state-of-the-art results in tasks like feature detection, description and matching while maintaining a simple MATLAB interface.
This document summarizes and compares local image descriptors. It begins with an introduction to modern descriptors like SIFT, SURF and DAISY. It then discusses efficient descriptors such as binary descriptors like BRIEF, ORB and BRISK which use comparisons of intensity value pairs. The document concludes with an overview section.
This document discusses various feature detectors used in computer vision. It begins by describing classic detectors such as the Harris detector and Hessian detector that search scale space to find distinguished locations. It then discusses detecting features at multiple scales using the Laplacian of Gaussian and determinant of Hessian. The document also covers affine covariant detectors such as maximally stable extremal regions and affine shape adaptation. It discusses approaches for speeding up detection using approximations like those in SURF and learning to emulate detectors. Finally, it outlines new developments in feature detection.
The document discusses modern feature detection techniques. It provides an introduction and agenda for a talk on advances in feature detectors and descriptors, including improvements since a 2005 paper. It also discusses software suites and benchmarks for feature detection. Several application domains are described, such as wide baseline matching, panoramic image stitching, 3D reconstruction, image search, location recognition, and object tracking.
System 1 and System 2 were basic early systems for image matching that used color and texture matching. Descriptor-based approaches like SIFT provided more invariance but not perfect invariance. Patch descriptors like SIFT were improved by making them more invariant to lighting changes like color and illumination shifts. The best performance came from combining descriptors with color invariance. Representing images as histograms of visual word occurrences captured patterns in local image patches and allowed measuring similarity between images. Large vocabularies of visual words provided more discriminative power but were costly to compute and store.
This document summarizes a research paper on internet video search. It discusses several key challenges: [1] the large variation in how the same thing can appear in images/videos due to lighting, viewpoint etc., [2] defining what defines different objects, and [3] the huge number of different things that exist. It also notes gaps in narrative understanding, shared concepts between humans and machines, and addressing diverse query contexts. The document advocates developing powerful yet simple visual features that capture uniqueness with invariance to irrelevant changes.
The document discusses computer vision techniques for object detection and localization. It describes methods like selective search that group image regions hierarchically to propose object locations. Large datasets like ImageNet and LabelMe that provide training examples are also discussed. Performance on object detection benchmarks like PASCAL VOC is shown to improve significantly over time. Evaluation standards for concept detection like those used in TRECVID are presented. The document concludes that results are impressively improving each year but that the number of detectable concepts remains limited. It also discusses making feature extraction more efficient using techniques like SURF that take advantage of integral images.
This document provides an outline and overview of Yoshua Bengio's 2012 tutorial on representation learning. The key points covered include:
1) The tutorial will cover motivations for representation learning, algorithms such as probabilistic models and auto-encoders, and analysis and practical issues.
2) Representation learning aims to automatically learn good representations of data rather than relying on handcrafted features. Learning representations can help address challenges like exploiting unlabeled data and the curse of dimensionality.
3) Deep learning algorithms attempt to learn multiple levels of increasingly complex representations, with the goal of developing more abstract, disentangled representations that generalize beyond local patterns in the data.
Advances in discrete energy minimisation for computer visionzukun
This document discusses string algorithms and data structures. It introduces the Knuth-Morris-Pratt algorithm for finding patterns in strings in O(n+m) time where n is the length of the text and m is the length of the pattern. It also discusses common string data structures like tries, suffix trees, and suffix arrays. Suffix trees and suffix arrays store all suffixes of a string and support efficient pattern matching and other string operations in linear time or O(m+logn) time where m is the pattern length and n is the text length.
This document provides a tutorial on how to use Gephi software to analyze and visualize network graphs. It outlines the basic steps of importing a sample graph file, applying layout algorithms to organize the nodes, calculating metrics, detecting communities, filtering the graph, and exporting/saving the results. The tutorial demonstrates features of Gephi including node ranking, partitioning, and interactive visualization of the graph.
EM algorithm and its application in probabilistic latent semantic analysiszukun
The document discusses the EM algorithm and its application in Probabilistic Latent Semantic Analysis (pLSA). It begins by introducing the parameter estimation problem and comparing frequentist and Bayesian approaches. It then describes the EM algorithm, which iteratively computes lower bounds to the log-likelihood function. Finally, it applies the EM algorithm to pLSA by modeling documents and words as arising from a mixture of latent topics.
This document describes an efficient framework for part-based object recognition using pictorial structures. The framework represents objects as graphs of parts with spatial relationships. It finds the optimal configuration of parts through global minimization using distance transforms, allowing fast computation despite modeling complex spatial relationships between parts. This enables soft detection to handle partial occlusion without early decisions about part locations.
Iccv2011 learning spatiotemporal graphs of human activities zukun
The document presents a new approach for learning spatiotemporal graphs of human activities from weakly supervised video data. The approach uses 2D+t tubes as mid-level features to represent activities as segmentation graphs, with nodes describing tubes and edges describing various relations. A probabilistic graph mixture model is used to model activities, and learning estimates the model parameters and permutation matrices using a structural EM algorithm. The learned models allow recognizing and segmenting activities in new videos through robust least squares inference. Evaluation on benchmark datasets demonstrates the ability to learn characteristic parts of activities and recognize them under weak supervision.
Icml2012 learning hierarchies of invariant featureszukun
This document discusses learning hierarchies of invariant features using convolutional neural networks. It describes how convolutional networks build hierarchical representations through multiple stacked layers that each apply normalization, filtering, non-linearity, and pooling operations to learn increasingly complex features. This architecture is inspired by the hierarchical organization of the mammalian visual cortex. The document outlines applications of convolutional networks in areas like computer vision, speech recognition, and natural language processing where they have achieved state-of-the-art performance by learning hierarchical representations from data.
This document provides an overview of OpenCV, an open source computer vision and machine learning software library. It discusses OpenCV's core functionality for representing images as matrices and directly accessing pixel data. It also covers topics like camera calibration, feature point extraction and matching, and estimating camera pose through techniques like structure from motion and planar homography. Hints are provided for Android developers on required permissions and for planar homography estimation using additional constraints rather than OpenCV's general homography function.
This document provides information about the Computer Vision Laboratory 2012 course at the Institute of Visual Computing. The course focuses on computer vision on mobile devices and will involve 180 hours of project work per person. Students will work in groups of 1-2 people on topics like 3D reconstruction from silhouettes or stereo images on mobile devices. Key dates are provided for submitting a work plan, mid-term presentation, and final report. Contact information is given for the lecturers and teaching assistant.
This document summarizes a presentation on natural image statistics given by Siwei Lyu at the 2009 CIFAR NCAP Summer School. The presentation covered several key topics:
1) It discussed the motivation for studying natural image statistics, which is to understand representations in the visual system and develop computer vision applications like denoising.
2) It reviewed common statistical properties found in natural images like 1/f power spectra and non-Gaussian distributions.
3) Maximum entropy and Bayesian models were presented as approaches to model these statistics, with Gaussian and independent component analysis discussed as specific examples.
4) Efficient coding principles from information theory were introduced as a framework for understanding neural representations that aim to decorrelate and
Camera calibration involves determining the internal camera parameters like focal length, image center, distortion, and scaling factors that affect the imaging process. These parameters are important for applications like 3D reconstruction and robotics that require understanding the relationship between 3D world points and their 2D projections in an image. The document describes estimating internal parameters by taking images of a calibration target with known geometry and solving the equations that relate the 3D target points to their 2D image locations. Homogeneous coordinates and projection matrices are used to represent the calibration transformations mathematically.
Brunelli 2008: template matching techniques in computer visionzukun
The document discusses template matching techniques in computer vision. It begins with an overview that defines template matching and discusses some common computer vision tasks it can be used for, like object detection. It then covers topics like detection as hypothesis testing, training and testing techniques, and provides a bibliography.
The HARVEST Programme evaluates feature detectors and descriptors through indirect and direct benchmarks. Indirect benchmarks measure repeatability and matching scores on the affine covariant testbed to evaluate how features persist across transformations. Direct benchmarks evaluate features on image retrieval tasks using the Oxford 5k dataset to measure real-world performance. VLBenchmarks provides software for easily running these benchmarks and reproducing published results. It allows comparing features and selecting the best for a given application.
This document summarizes VLFeat, an open source computer vision library. It provides concise summaries of VLFeat's features, including SIFT, MSER, and other covariant detectors. It also compares VLFeat's performance to other libraries like OpenCV. The document highlights how VLFeat achieves state-of-the-art results in tasks like feature detection, description and matching while maintaining a simple MATLAB interface.
This document summarizes and compares local image descriptors. It begins with an introduction to modern descriptors like SIFT, SURF and DAISY. It then discusses efficient descriptors such as binary descriptors like BRIEF, ORB and BRISK which use comparisons of intensity value pairs. The document concludes with an overview section.
This document discusses various feature detectors used in computer vision. It begins by describing classic detectors such as the Harris detector and Hessian detector that search scale space to find distinguished locations. It then discusses detecting features at multiple scales using the Laplacian of Gaussian and determinant of Hessian. The document also covers affine covariant detectors such as maximally stable extremal regions and affine shape adaptation. It discusses approaches for speeding up detection using approximations like those in SURF and learning to emulate detectors. Finally, it outlines new developments in feature detection.
The document discusses modern feature detection techniques. It provides an introduction and agenda for a talk on advances in feature detectors and descriptors, including improvements since a 2005 paper. It also discusses software suites and benchmarks for feature detection. Several application domains are described, such as wide baseline matching, panoramic image stitching, 3D reconstruction, image search, location recognition, and object tracking.
System 1 and System 2 were basic early systems for image matching that used color and texture matching. Descriptor-based approaches like SIFT provided more invariance but not perfect invariance. Patch descriptors like SIFT were improved by making them more invariant to lighting changes like color and illumination shifts. The best performance came from combining descriptors with color invariance. Representing images as histograms of visual word occurrences captured patterns in local image patches and allowed measuring similarity between images. Large vocabularies of visual words provided more discriminative power but were costly to compute and store.
This document summarizes a research paper on internet video search. It discusses several key challenges: [1] the large variation in how the same thing can appear in images/videos due to lighting, viewpoint etc., [2] defining what defines different objects, and [3] the huge number of different things that exist. It also notes gaps in narrative understanding, shared concepts between humans and machines, and addressing diverse query contexts. The document advocates developing powerful yet simple visual features that capture uniqueness with invariance to irrelevant changes.
The document discusses computer vision techniques for object detection and localization. It describes methods like selective search that group image regions hierarchically to propose object locations. Large datasets like ImageNet and LabelMe that provide training examples are also discussed. Performance on object detection benchmarks like PASCAL VOC is shown to improve significantly over time. Evaluation standards for concept detection like those used in TRECVID are presented. The document concludes that results are impressively improving each year but that the number of detectable concepts remains limited. It also discusses making feature extraction more efficient using techniques like SURF that take advantage of integral images.
This document provides an outline and overview of Yoshua Bengio's 2012 tutorial on representation learning. The key points covered include:
1) The tutorial will cover motivations for representation learning, algorithms such as probabilistic models and auto-encoders, and analysis and practical issues.
2) Representation learning aims to automatically learn good representations of data rather than relying on handcrafted features. Learning representations can help address challenges like exploiting unlabeled data and the curse of dimensionality.
3) Deep learning algorithms attempt to learn multiple levels of increasingly complex representations, with the goal of developing more abstract, disentangled representations that generalize beyond local patterns in the data.
Advances in discrete energy minimisation for computer visionzukun
This document discusses string algorithms and data structures. It introduces the Knuth-Morris-Pratt algorithm for finding patterns in strings in O(n+m) time where n is the length of the text and m is the length of the pattern. It also discusses common string data structures like tries, suffix trees, and suffix arrays. Suffix trees and suffix arrays store all suffixes of a string and support efficient pattern matching and other string operations in linear time or O(m+logn) time where m is the pattern length and n is the text length.
This document provides a tutorial on how to use Gephi software to analyze and visualize network graphs. It outlines the basic steps of importing a sample graph file, applying layout algorithms to organize the nodes, calculating metrics, detecting communities, filtering the graph, and exporting/saving the results. The tutorial demonstrates features of Gephi including node ranking, partitioning, and interactive visualization of the graph.
EM algorithm and its application in probabilistic latent semantic analysiszukun
The document discusses the EM algorithm and its application in Probabilistic Latent Semantic Analysis (pLSA). It begins by introducing the parameter estimation problem and comparing frequentist and Bayesian approaches. It then describes the EM algorithm, which iteratively computes lower bounds to the log-likelihood function. Finally, it applies the EM algorithm to pLSA by modeling documents and words as arising from a mixture of latent topics.
This document describes an efficient framework for part-based object recognition using pictorial structures. The framework represents objects as graphs of parts with spatial relationships. It finds the optimal configuration of parts through global minimization using distance transforms, allowing fast computation despite modeling complex spatial relationships between parts. This enables soft detection to handle partial occlusion without early decisions about part locations.
Iccv2011 learning spatiotemporal graphs of human activities zukun
The document presents a new approach for learning spatiotemporal graphs of human activities from weakly supervised video data. The approach uses 2D+t tubes as mid-level features to represent activities as segmentation graphs, with nodes describing tubes and edges describing various relations. A probabilistic graph mixture model is used to model activities, and learning estimates the model parameters and permutation matrices using a structural EM algorithm. The learned models allow recognizing and segmenting activities in new videos through robust least squares inference. Evaluation on benchmark datasets demonstrates the ability to learn characteristic parts of activities and recognize them under weak supervision.
Icml2012 learning hierarchies of invariant featureszukun
This document discusses learning hierarchies of invariant features using convolutional neural networks. It describes how convolutional networks build hierarchical representations through multiple stacked layers that each apply normalization, filtering, non-linearity, and pooling operations to learn increasingly complex features. This architecture is inspired by the hierarchical organization of the mammalian visual cortex. The document outlines applications of convolutional networks in areas like computer vision, speech recognition, and natural language processing where they have achieved state-of-the-art performance by learning hierarchical representations from data.
Icml2012 learning hierarchies of invariant features
Cvpr2010 open source vision software, intro and training part vii point cloud procesing in ros. point cloud library. - rusu - unknown - 2010
1. Introduction Motivation Acquisition Data representation Storage PCL
PointCloud(2) procesing in ROS.
Point Cloud Library.
Radu Bogdan RUSU
June 14, 2010
2. [Introduction] Motivation Acquisition Data representation Storage PCL
Outline
1. Introduction
2. Motivation
3. Acquisition
4. Data representation
5. Storage
6. PCL
Radu Bogdan RUSU Point Cloud Library
3. [Introduction] Motivation Acquisition Data representation Storage PCL
Introduction (1/6)
What is ROS?
ROS is a comprehensive meta operating system for robotics
organized via nodes, packages, stacks, repositories
collection of building blocks
community driven effort - all free/open source (!)
stop reinventing the wheel, bring the best ouf of robotics
replicating research results
Radu Bogdan RUSU Point Cloud Library
4. [Introduction] Motivation Acquisition Data representation Storage PCL
Introduction (2/6)
What are Point Clouds?
*
Point Cloud = a ”cloud“ (i.e., collection) of nD points
(usually n = 3)
p i = {xi , yi , zi } −→ P = {p 1 , p 2 , . . . , p i , . . . , p n }
used to represent 3D information about the world
Radu Bogdan RUSU Point Cloud Library
5. [Introduction] Motivation Acquisition Data representation Storage PCL
Introduction (3/6)
What are Point Clouds?
besides XYZ data, each point p can hold additional
information
examples include: RGB colors, intensity values, distances,
segmentation results, etc
Radu Bogdan RUSU Point Cloud Library
6. [Introduction] Motivation Acquisition Data representation Storage PCL
Introduction (4-6/6)
What are Point Clouds?
Radu Bogdan RUSU Point Cloud Library
7. [Introduction] Motivation Acquisition Data representation Storage PCL
Introduction (4-6/6)
What are Point Clouds?
Radu Bogdan RUSU Point Cloud Library
8. [Introduction] Motivation Acquisition Data representation Storage PCL
Introduction (4-6/6)
What are Point Clouds?
Radu Bogdan RUSU Point Cloud Library
9. Introduction [Motivation] Acquisition Data representation Storage PCL
Outline
1. Introduction
2. Motivation
3. Acquisition
4. Data representation
5. Storage
6. PCL
Radu Bogdan RUSU Point Cloud Library
10. Introduction [Motivation] Acquisition Data representation Storage PCL
Motivation (1/4)
Why are Point Clouds important?
Point Clouds are important for a lot of reasons (!). Besides
representing geometry, they can complement and supersede
images when data has a high dimensionality.
Radu Bogdan RUSU Point Cloud Library
11. Introduction [Motivation] Acquisition Data representation Storage PCL
Motivation (2/4)
Why are Point Clouds important?
Concrete example 1: get the cup from the drawer.
⇐⇒
Handle Drawer
*
Radu Bogdan RUSU Point Cloud Library
12. Introduction [Motivation] Acquisition Data representation Storage PCL
Motivation (3/4)
Why are Point Clouds important?
Concrete example 2: find the door and its handle, and open it.
Door
⇐⇒ Handle
*
Radu Bogdan RUSU Point Cloud Library
13. Introduction [Motivation] Acquisition Data representation Storage PCL
Motivation (4/4)
Why are Point Clouds important?
Concrete example 3: safe motion planning/manipulation.
⇒
*
Radu Bogdan RUSU Point Cloud Library
14. Introduction Motivation [Acquisition] Data representation Storage PCL
Outline
1. Introduction
2. Motivation
3. Acquisition
4. Data representation
5. Storage
6. PCL
Radu Bogdan RUSU Point Cloud Library
15. Introduction Motivation [Acquisition] Data representation Storage PCL
Acquisition (1/3)
How are Point Clouds acquired? Where do they come from?
There are many different sensors that can generate 3D
information. Examples:
laser/lidar sensors (2D/3D)
stereo cameras
time-of-flight (TOF) cameras
etc...
Radu Bogdan RUSU
* Point Cloud Library
16. Introduction Motivation [Acquisition] Data representation Storage PCL
Acquisition (2/3)
How are Point Clouds acquired? Where do they come from?
The PR2 sensor head:
*
two pairs of stereo cameras (narrow + wide)
tilting laser sensor
Radu Bogdan RUSU Point Cloud Library
17. Introduction Motivation [Acquisition] Data representation Storage PCL
Acquisition (3/3)
How are Point Clouds acquired? Where do they come from?
Simulation (!):
raytracing + stereo imagery fed into the same algorithmic
modules that are used to process real data
Radu Bogdan RUSU Point Cloud Library
18. Introduction Motivation Acquisition [Data representation] Storage PCL
Outline
1. Introduction
2. Motivation
3. Acquisition
4. Data representation
5. Storage
6. PCL
Radu Bogdan RUSU Point Cloud Library
19. Introduction Motivation Acquisition [Data representation] Storage PCL
Data representation (1/7)
Representing Point Clouds
As previously presented:
a point p is represented as an n-tuple, e.g.,
p i = {xi , yi , zi , ri , gi , bi , disti , · · · }
a Point Cloud P is represented as a collection of points p i ,
e.g., P = {p 1 , p 2 , · · · , p i , · · · , p n }
Radu Bogdan RUSU Point Cloud Library
20. Introduction Motivation Acquisition [Data representation] Storage PCL
Data representation (2/7)
Point Cloud Data structures
In terms of data structures:
an XYZ point can be represented as:
float32 x
float32 y
float32 z
a n-dimensional point can be represented as:
float32[] point
which is nothing else but a:
std::vector<float32> point
in C++
potential problem: everything is represented as floats (!)
Radu Bogdan RUSU Point Cloud Library
21. Introduction Motivation Acquisition [Data representation] Storage PCL
Data representation (3/7)
Point Cloud Data structures
In terms of data structures:
therefore a point cloud P is:
Point[] points
or:
std::vector<Point> points
in C++, where Point is the structure/data type representing
a single point p
Radu Bogdan RUSU Point Cloud Library
22. Introduction Motivation Acquisition [Data representation] Storage PCL
Data representation (4/7)
Point Cloud Data structures
Because Point Clouds are big:
operations on them are typically slower (more data, more
computations)
they are expensive to store, especially if all data is
represented as floats/doubles
Solutions:
Radu Bogdan RUSU Point Cloud Library
23. Introduction Motivation Acquisition [Data representation] Storage PCL
Data representation (4/7)
Point Cloud Data structures
Because Point Clouds are big:
operations on them are typically slower (more data, more
computations)
they are expensive to store, especially if all data is
represented as floats/doubles
Solutions:
store each dimension data in different (the most
appropriate) formats, e.g., rgb - 24bits, instead of 3 × 4
(sizeof float)
group data together, and try to keep it aligned (e.g., 16bit
for SSE) to speed up computations
Radu Bogdan RUSU Point Cloud Library
24. Introduction Motivation Acquisition [Data representation] Storage PCL
Data representation (5/7)
ROS representations for Point Cloud Data
The ROS PointCloud(2) data format
(sensor_msgs/PointCloud2.msg):
# This message holds a collection of nD points , as a binary blob .
Header header
# 2 D structure of the point cloud . If the cloud is unordered ,
# height is 1 and width is the length of the point cloud .
uint32 height
uint32 width
# Describes the channels and their layout in the binary data blob
PointField [] fields
bool is_bigendian # Is this data bigendian ?
uint32 point_step # Length of a point in bytes
uint32 row_step # Length of a row in bytes
uint8 [] data # Actual point data , size is ( row_step * height )
bool is_dense # True if there are no invalid points
Radu Bogdan RUSU Point Cloud Library
25. Introduction Motivation Acquisition [Data representation] Storage PCL
Data representation (6/7)
ROS representations for Point Cloud Data
where PointField (sensor_msgs/PointField.msg) is:
# This message holds the description of one point entry in the # PointCloud2 message format .
uint8 INT8 = 1
uint8 UINT8 = 2
uint8 INT16 = 3
uint8 UINT16 = 4
uint8 INT32 = 5
uint8 UINT32 = 6
uint8 FLOAT32 = 7
uint8 FLOAT64 = 8
string name # Name of field
uint32 offset # Offset from start of point struct
uint8 datatype # Datatype enumeration see above
uint32 count # How many elements in field
PointField examples:
"x", , 7, 1
"y", 4, 7, 1
"z", 8, 7, 1
" rgba " , 12 , 6, 1
" normal_x " , 16 , 8, 1
" normal_y " , 2 , 8, 1
" normal_z " , 24 , 8, 1
" fpfh " , 32 , 7 , 33
Radu Bogdan RUSU Point Cloud Library
26. Introduction Motivation Acquisition [Data representation] Storage PCL
Data representation (7/7)
ROS representations for Point Cloud Data
binary blobs are hard to work with
we provide a custom converter, Publisher/Subscriber,
transport tools, filters, etc, similar to images
templated types: PointCloud2 −→ PointCloud<PointT>
examples of PointT:
struct PointXYZ
{
float x ;
float y ;
float z ;
}
struct Normal
{
float normal [3];
float curvature ;
}
Radu Bogdan RUSU Point Cloud Library
27. Introduction Motivation Acquisition Data representation [Storage] PCL
Outline
1. Introduction
2. Motivation
3. Acquisition
4. Data representation
5. Storage
6. PCL
Radu Bogdan RUSU Point Cloud Library
28. Introduction Motivation Acquisition Data representation [Storage] PCL
Point Cloud Data storage (1/2)
ROS input/output
PointCloud2.msg and PointField.msg are ROS messages
they can be published on the network, saved/loaded
to/from BAG files (ROS message logs)
usage example:
$ rostopic find sensor_msgs/PointCloud2 | xargs rosrecord -F foo
[ INFO ] [1271297447.6564145 2]: Recording to foo . bag .
^C
[ INFO ] [127129745 .7235 4983]: Closing foo . bag .
$ rosplay -c foo.bag
bag : foo . bag
version : 1.2
start_time : 127129744797428 542
end_time : 1271297449983577462
length : 2 929692
topics :
- name : / narrow_stereo_textured / points2
count : 3
datatype : sensor_msgs / PointCloud2
md5sum : 1158 d486dd51d683ce2f1be655c3c181
Radu Bogdan RUSU Point Cloud Library
29. Introduction Motivation Acquisition Data representation [Storage] PCL
Point Cloud Data storage (2/2)
PCD (Point Cloud Data) file format
In addition, point clouds can be stored to disk as files, into the
PCD format.
# Point Cloud Data ( PCD ) file format v .5
FIELDS x y z rgba
SIZE 4 4 4 4
TYPE F F F U
WIDTH 3 72
HEIGHT 1
POINTS 3 72
DATA binary
...
DATA can be either ascii or binary. If ascii, then
...
DATA ascii
. 54216 .11349 . 4 749
- . 17447 .11425 . 41273
- . 1 661 .11338 . 4 916
. 26422 .11499 . 32623
...
Radu Bogdan RUSU Point Cloud Library
30. Introduction Motivation Acquisition Data representation Storage [PCL]
Outline
1. Introduction
2. Motivation
3. Acquisition
4. Data representation
5. Storage
6. PCL
Radu Bogdan RUSU Point Cloud Library
31. Introduction Motivation Acquisition Data representation Storage [PCL]
Point Cloud Library (1/11)
http://www.ros.org/wiki/pcl
Radu Bogdan RUSU Point Cloud Library
32. Introduction Motivation Acquisition Data representation Storage [PCL]
Point Cloud Library (2/11)
What is PCL (Point Cloud Library)?
PCL is:
fully templated modern C++ library for 3D point cloud
processing
uses SSE optimizations (Eigen backend) for fast
computations on modern CPUs
uses OpenMP and Intel TBB for parallelization
passes data between modules (e.g., algorithms) using
Boost shared pointers
OpenCV’s 3D little sister
PCL deprecates older ROS packages such as
point_cloud_mapping and replaces
sensor_msgs/PointCloud.msg with the modern
sensor_msgs/PointCloud2.msg format (!)
Radu Bogdan RUSU Point Cloud Library
33. Introduction Motivation Acquisition Data representation Storage [PCL]
Point Cloud Library (3/11)
PCL (Point Cloud Library) structure
collection of smaller, modular C++ libraries:
libpcl_features: many 3D features (e.g., normals and curvatures,
boundary points, moment invariants, principal curvatures, Point Feature
Histograms (PFH), Fast PFH, ...)
libpcl_surface: surface reconstruction techniques (e.g.,
meshing, convex hulls, Moving Least Squares, ...)
libpcl_filters: point cloud data filters (e.g., downsampling, outlier
removal, indices extraction, projections, ...)
libpcl_io: I/O operations (e.g., writing to/reading from PCD (Point
Cloud Data) and BAG files)
libpcl_segmentation: segmentation operations (e.g.,cluster
extraction, Sample Consensus model fitting, polygonal prism extraction, ...)
libpcl_registration: point cloud registration methods (e.g.,
Iterative Closest Point (ICP), non linear optimizations, ...)
unit tests, examples, tutorials (some are work in progress)
C++ classes are templated building blocks (nodelets!)
Radu Bogdan RUSU Point Cloud Library
34. Introduction Motivation Acquisition Data representation Storage [PCL]
Point Cloud Library (4/11)
PPG: Perception Processing Graphs
Philosophy: write once, parameterize everywhere
PPG: Perception Processing Graphs
Radu Bogdan RUSU Point Cloud Library
35. Introduction Motivation Acquisition Data representation Storage [PCL]
Point Cloud Library (5/11)
PPG: Perception Processing Graphs
Why PPG?
Algorithmically:
door detection = table detection = wall detection = ...
the only thing that changes is: parameters (constraints)!
Radu Bogdan RUSU Point Cloud Library
36. Introduction Motivation Acquisition Data representation Storage [PCL]
Point Cloud Library (6/11)
More on architecture
Inheritance simplifies development and testing:
pcl :: Feature < PointT > feat ;
feat = pcl :: Normal < PointT > ( input );
feat = pcl :: FPFH < PointT > ( input );
feat = pcl :: BoundaryPoint < PointT > ( input );
...
feat . compute (& output );
...
Radu Bogdan RUSU Point Cloud Library
37. Introduction Motivation Acquisition Data representation Storage [PCL]
Point Cloud Library (7/11)
PCL 0.1.x statistics
Misc, stats:
over 100 classes
over 25k lines of code
external dependencies (for now) on eigen, cminpack, ANN,
FLANN, TBB
internal dependencies (excluding the obvious) on
dynamic_reconfigure, message_filters
tf_pcl package for TF integration
Radu Bogdan RUSU Point Cloud Library
38. Introduction Motivation Acquisition Data representation Storage [PCL]
Point Cloud Library (8/11)
Nodelets
write once, parameterize everywhere =⇒ modular code
ideally, each algorithm is a “building block” that consumes
input(s) and produces some output(s)
in ROS, this is what we call a node. inter-process data
passing however is inefficient. ideally we need shared
memory.
Solution:
nodelets = “nodes in nodes” = single-process, multi-threading
Radu Bogdan RUSU Point Cloud Library
39. Introduction Motivation Acquisition Data representation Storage [PCL]
Point Cloud Library (8/11)
Nodelets
write once, parameterize everywhere =⇒ modular code
ideally, each algorithm is a “building block” that consumes
input(s) and produces some output(s)
in ROS, this is what we call a node. inter-process data
passing however is inefficient. ideally we need shared
memory.
Solution:
nodelets = “nodes in nodes” = single-process, multi-threading
same ROS API as nodes (subscribe, advertise, publish)
dynamically (un)loadable
optimizations for zero-copy Boost shared_ptr passing
PCL nodelets use dynamic_reconfigure for on-the-fly
parameter setting
Radu Bogdan RUSU Point Cloud Library
40. Introduction Motivation Acquisition Data representation Storage [PCL]
Point Cloud Library (9/11)
Nodelets
Performance hints:
29 msg/s (inter process) vs 4664 mgs/s (intra process)
fast (un)loading for algorithms (one service request)
natively multithreaded
(boost::thread::hardware_concurrency ())
threading scheduler and granularity controllable in
OpenMP/TBB
Goals:
GUI for managing nodelets => 3D “Simulink” (!)
get to a point where fast prototyping is easy
Radu Bogdan RUSU Point Cloud Library
41. Introduction Motivation Acquisition Data representation Storage [PCL]
Point Cloud Library (10/11)
Downsample and filtering example with nodelets
< launch >
< node pkg = " nodelet " type = " standalone_nodelet "
name = " pcl_manager " output = " screen " / >
< node pkg = " nodelet " type = " nodelet " name = " foo "
args = " voxel_grid VoxelGrid pcl_manager " >
< remap from " / voxel_grid / input "
to = " / narrow_stereo_textured / points " / >
< rosparam >
# -[ Mandatory parameters
leaf_size: [ . 15 , . 15 , . 15]
# -[ O p t i o n a l p a r a m e t e r s
# field containing distance values ( for filtering )
filter_field_name: " z "
# filtering points outside of < .8 ,5. >
filter_limit_min: .8
filter_limit_max: 5.
use_indices: false # false by default
</ rosparam >
</ node >
...
</ launch >
Radu Bogdan RUSU Point Cloud Library
42. Introduction Motivation Acquisition Data representation Storage [PCL]
Point Cloud Library (11/11)
Normal estimation example with nodelets
< launch >
< node pkg = " nodelet " type = " nodelet " name = " foo "
args = " normal_estimation NormalEstimation pcl_manager " >
< remap from = " / normal_estimation / input "
to = " / voxel_grid / output " / >
< remap from = " / normal_estimation / surface "
to = " / narrow_stereo_textured / points " / >
< rosparam >
# -[ M a n d a t o r y p a r a m e t e r s
# Set either ’ k_search ’ or ’ radius_search ’
k_search:
radius_search: .1
# Set the spatial locator . Possible values are:
# ( ANN ) , 1 ( FLANN ) , 2 ( organized )
spatial_locator:
</ rosparam >
</ node >
...
</ launch >
Radu Bogdan RUSU Point Cloud Library
43. Introduction Motivation Acquisition Data representation Storage [PCL]
PCL - Table Object Detector
How to extract a table plane and the objects lying on it
Radu Bogdan RUSU Point Cloud Library
44. Introduction Motivation Acquisition Data representation Storage [PCL]
PCL - Web tutorials (1/3)
Radu Bogdan RUSU Point Cloud Library
45. Introduction Motivation Acquisition Data representation Storage [PCL]
PCL - Web tutorials (2/3)
Radu Bogdan RUSU Point Cloud Library
46. Introduction Motivation Acquisition Data representation Storage [PCL]
PCL - Web tutorials (3/3)
Demo
http://www.ros.org/wiki/pcl/Tutorials
Radu Bogdan RUSU Point Cloud Library
47. Introduction Motivation Acquisition Data representation Storage [PCL]
Questions?
Thanks!
http://www.ros.org/wiki/pcl
Radu Bogdan RUSU Point Cloud Library