This thesis proposes a system for capturing 3D models of large objects using autonomous quadcopters. A major component of such a system is accurately localizing the position and orientation, or pose, of the quadcopter in order to execute precise flight patterns. This thesis focuses on the design and implementation of a localization algorithm that uses a particle filter to combine internal sensor measurements and augmented reality tag detection in order to estimate the pose of an AR.Drone quadcopter. This system is shown to perform significantly better than integrated velocity measurements alone.
This document describes a simulation of swarm learning techniques using drones. The simulation models drones searching for a lost object. The drones have sensors to detect the lost object and characteristics like speed and sensor accuracy. The drones learn through an evolutionary process where the fittest drones with the best characteristics for finding the lost object breed to pass on their genes. Over multiple generations, the time taken for the swarm to find the lost object decreases as the drones' skills improve through evolution. The simulation outputs various data and its goal is to explore how swarm intelligence can be applied to search and rescue scenarios.
This document presents a reactive collision avoidance system for an autonomous sailboat using stereo vision. It describes selecting stereo cameras for obstacle detection and developing algorithms in MATLAB to detect obstacles in images and calculate their range using stereo vision techniques. The algorithms were optimized, integrated and implemented on an embedded Linux system (BeagleBone Black) in C/C++. The system was tested in a reservoir with different obstacle configurations and verified to reliably detect obstacles and avoid collisions.
Enhancing video game experience through a vibrotactile floorNicola Gallo
When it comes to Virtual Reality, the whole idea is to initiate the feeling of Presence, the perception of actually being within a virtual world. When discrepancies happen between what your brain expects and what it actually feels, this feeling can be broken. This generates a sense of disappointment along with a sensation of being disassociated from the virtual environment. In order to deceive your mind and give it the illusion that your body is somewhere different than what your eyes are seeing, all five human senses should perceive the digital environment to be physically real. While tricking the sense of smell and taste is not so common in the video game world, the sense of touch has been attracting the attention of a growing number of companies all around the world. However, when the pedestrian movement is involved, it is not so clear how to provide a compelling haptic feedback as you would expect to receive it directly under your feet.
This project aims to solve this problem by taking advantage of the potentialities offered by a vibrotactile floor. Two VR experiences have been developed: Infinite Runner, in which the floor was employed for the generation of particular haptic effects as a response to specific game events, and Wakaduck, in which it was tried to use the haptic feedback not only to enhance the user experience, but also and above all to provide some haptic cues whose understanding is essential to correctly play the game.
Innovative Payloads for Small Unmanned Aerial System-Based PersonAustin Jensen
This document describes Austin Jensen's doctoral dissertation titled "Innovative Payloads for Small Unmanned Aerial System-Based Personal Remote Sensing and Applications". The dissertation focuses on using small unmanned aerial systems (UAS) as remote sensing platforms to collect aerial imagery and track radio-tagged fish. It presents novel methods for calibrating imagery from commercial cameras on UAS and estimating the location of radio-tagged fish using multiple UAS. The calibration methods allow imagery to be orthorectified and used to deliver actionable information to end users. Simulations evaluate the fish tracking methods and predict their real-world performance. The dissertation contributes to advancing UAS-based remote sensing for applications such as precision agriculture, vegetation mapping,
Virtual Environments as Driving Schools for Deep Learning Vision-Based Sensor...Artur Filipowicz
At the turn of the 20th century, inventors and industrialists alike strived to enable every person to own and drive a car. Overtime, automobile ownership grew to meet that vision. One hundred years later, automobile manufacturers and technology companies are working on self-driving cars which would be neither owned nor driven by individuals. The benefits of replacing cars with fully autonomous vehicles are enormous. While it is difficult to put a value on lives saved, injuries avoided, pollution reduced, and commute time repurposed, economic savings from this technology are estimated to be on the order of trillions of dollars. The main roadblock in achieving the vision for this century is developing technology which would enable autonomous vehicles to perceive and understand the environment as well as, if not better than, human divers. Perception is a roadblock because presently no algorithm is capable of reaching human levels of cognition.
This thesis explores the interaction between virtual reality simulation and Deep Learning which may develop computer vision that rivals human vision. The specific problem considered is detection and localization of a stop object, the stop sign, based on an image. A video game, Grand Theft Auto 5, is used to collect over half a million images and corresponding ground truth labels with and without stop signs in various lighting and weather conditions. A deep convolutional neural network trained on this data and fine tuned on real world data achieves accuracy in stop sign detection of over 95% within 20 meters of the stop sign and has a false positive rate of 4% on test data from the real world. Additionally, the physical constraints on this problem are analysed, a framework for the use of simulators is developed, and domain adaptation and multi-task learning are explored.
This document is a master's thesis that aims to detect sensor failures and malfunctions in cameras. It presents methods to assess image quality and detect blurred, overexposed, underexposed, and obstructed images that could indicate sensor issues. The thesis covers implementing classifiers like support vector machines to detect blur and exploring techniques like thresholding and histograms to detect exposure range problems. It also examines detecting issues like blooming effects and power failures or obstructions that could impact the camera sensor. The goal is to continuously monitor image quality to identify sensor malfunctions in real-time.
This is my master degree dissertation project. Which has inspired the "Alagba" project.
It is the proposal for the use of a drone which detects turtle nests using the camera and deep learning.
This document provides an instruction manual for the Meade ETX-90AT, ETX-105AT, and ETX-125AT Astro Telescopes. It describes the key features of the telescopes including the optical tube, viewfinder, setting circles, locks, and computer control panel. It provides instructions on assembly, alignment, focusing, and basic operation using the arrow keys to manually move the telescope or let Autostar automatically track objects. The document also outlines Autostar's menu system and provides information on additional features and accessories.
This document describes a simulation of swarm learning techniques using drones. The simulation models drones searching for a lost object. The drones have sensors to detect the lost object and characteristics like speed and sensor accuracy. The drones learn through an evolutionary process where the fittest drones with the best characteristics for finding the lost object breed to pass on their genes. Over multiple generations, the time taken for the swarm to find the lost object decreases as the drones' skills improve through evolution. The simulation outputs various data and its goal is to explore how swarm intelligence can be applied to search and rescue scenarios.
This document presents a reactive collision avoidance system for an autonomous sailboat using stereo vision. It describes selecting stereo cameras for obstacle detection and developing algorithms in MATLAB to detect obstacles in images and calculate their range using stereo vision techniques. The algorithms were optimized, integrated and implemented on an embedded Linux system (BeagleBone Black) in C/C++. The system was tested in a reservoir with different obstacle configurations and verified to reliably detect obstacles and avoid collisions.
Enhancing video game experience through a vibrotactile floorNicola Gallo
When it comes to Virtual Reality, the whole idea is to initiate the feeling of Presence, the perception of actually being within a virtual world. When discrepancies happen between what your brain expects and what it actually feels, this feeling can be broken. This generates a sense of disappointment along with a sensation of being disassociated from the virtual environment. In order to deceive your mind and give it the illusion that your body is somewhere different than what your eyes are seeing, all five human senses should perceive the digital environment to be physically real. While tricking the sense of smell and taste is not so common in the video game world, the sense of touch has been attracting the attention of a growing number of companies all around the world. However, when the pedestrian movement is involved, it is not so clear how to provide a compelling haptic feedback as you would expect to receive it directly under your feet.
This project aims to solve this problem by taking advantage of the potentialities offered by a vibrotactile floor. Two VR experiences have been developed: Infinite Runner, in which the floor was employed for the generation of particular haptic effects as a response to specific game events, and Wakaduck, in which it was tried to use the haptic feedback not only to enhance the user experience, but also and above all to provide some haptic cues whose understanding is essential to correctly play the game.
Innovative Payloads for Small Unmanned Aerial System-Based PersonAustin Jensen
This document describes Austin Jensen's doctoral dissertation titled "Innovative Payloads for Small Unmanned Aerial System-Based Personal Remote Sensing and Applications". The dissertation focuses on using small unmanned aerial systems (UAS) as remote sensing platforms to collect aerial imagery and track radio-tagged fish. It presents novel methods for calibrating imagery from commercial cameras on UAS and estimating the location of radio-tagged fish using multiple UAS. The calibration methods allow imagery to be orthorectified and used to deliver actionable information to end users. Simulations evaluate the fish tracking methods and predict their real-world performance. The dissertation contributes to advancing UAS-based remote sensing for applications such as precision agriculture, vegetation mapping,
Virtual Environments as Driving Schools for Deep Learning Vision-Based Sensor...Artur Filipowicz
At the turn of the 20th century, inventors and industrialists alike strived to enable every person to own and drive a car. Overtime, automobile ownership grew to meet that vision. One hundred years later, automobile manufacturers and technology companies are working on self-driving cars which would be neither owned nor driven by individuals. The benefits of replacing cars with fully autonomous vehicles are enormous. While it is difficult to put a value on lives saved, injuries avoided, pollution reduced, and commute time repurposed, economic savings from this technology are estimated to be on the order of trillions of dollars. The main roadblock in achieving the vision for this century is developing technology which would enable autonomous vehicles to perceive and understand the environment as well as, if not better than, human divers. Perception is a roadblock because presently no algorithm is capable of reaching human levels of cognition.
This thesis explores the interaction between virtual reality simulation and Deep Learning which may develop computer vision that rivals human vision. The specific problem considered is detection and localization of a stop object, the stop sign, based on an image. A video game, Grand Theft Auto 5, is used to collect over half a million images and corresponding ground truth labels with and without stop signs in various lighting and weather conditions. A deep convolutional neural network trained on this data and fine tuned on real world data achieves accuracy in stop sign detection of over 95% within 20 meters of the stop sign and has a false positive rate of 4% on test data from the real world. Additionally, the physical constraints on this problem are analysed, a framework for the use of simulators is developed, and domain adaptation and multi-task learning are explored.
This document is a master's thesis that aims to detect sensor failures and malfunctions in cameras. It presents methods to assess image quality and detect blurred, overexposed, underexposed, and obstructed images that could indicate sensor issues. The thesis covers implementing classifiers like support vector machines to detect blur and exploring techniques like thresholding and histograms to detect exposure range problems. It also examines detecting issues like blooming effects and power failures or obstructions that could impact the camera sensor. The goal is to continuously monitor image quality to identify sensor malfunctions in real-time.
This is my master degree dissertation project. Which has inspired the "Alagba" project.
It is the proposal for the use of a drone which detects turtle nests using the camera and deep learning.
This document provides an instruction manual for the Meade ETX-90AT, ETX-105AT, and ETX-125AT Astro Telescopes. It describes the key features of the telescopes including the optical tube, viewfinder, setting circles, locks, and computer control panel. It provides instructions on assembly, alignment, focusing, and basic operation using the arrow keys to manually move the telescope or let Autostar automatically track objects. The document also outlines Autostar's menu system and provides information on additional features and accessories.
This thesis presents methods to extend the operational lifetime of inflatable soft robots when leaks occur in either the structural chambers or actuation chambers. For structural leaks, a control algorithm is developed to detect the leak and then lower the target pressures as needed to slow the leak rate while maintaining a specified level of accuracy. Testing on an inflatable robot arm showed the algorithm could decrease long-term error by 50% compared to no control and prolong operation. For actuator leaks, lowering the target pressure for the leaking joint is shown to trade off leak rate and performance. Tests on inflatable robots demonstrated the potential for up to a 300% increase in operation time with minimal increased error. The work aims to preserve limited compressed air resources and extend operation
This document summarizes an investigation by BSc. Dirk Ekelschot into Gortler vortices in hypersonic flow using quantitative infrared thermography (QIRT) and tomographic particle image velocimetry (Tomo-PIV) at Delft University of Technology. The study used a double compression ramp model in the Hypersonic Test Facility Delft to generate Gortler vortices. QIRT was used to analyze the growth of vortices at different Reynolds numbers. Tomo-PIV was employed to validate its use in the facility by calibrating the system and estimating accuracy and reconstruction quality. Results from both techniques provided insight into the vortex topology and growth.
MSc_Thesis_Wake_Dynamics_Study_of_an_H-type_Vertical_Axis_Wind_TurbineChenguang He
This thesis investigates the wake dynamics of an H-type vertical axis wind turbine using particle image velocimetry (PIV). Two-component PIV is used to study vorticity shedding and horizontal wake expansion at the turbine mid-span plane. Stereoscopic PIV is performed on 7 cross-stream vertical planes to analyze tip vortex dynamics and the evolution of 3D wake structures. The experimental results show asymmetrical vorticity decay in the horizontal plane, with faster decay on the leeward side. Tip vortices are stronger than shed vortices. Near the turbine axis, tip vortices move inboard behind the rotor before moving outboard towards the windward side further downstream. Vertical
This thesis presents work on a novel Compass Star Tracker (CST) instrument that can determine its local position on Earth by imaging stars. The document covers:
1) The theoretical development of the local position determination mathematics using star sensor attitude information.
2) Design and fabrication of a proof-of-concept CST, including the imaging sensor, optics, inclinometer, cooling system, and data acquisition hardware.
3) Experimental testing of the CST to validate the position determination concept, including analysis of theoretical and practical error sources.
The CST has potential applications for autonomous position sensing on the Moon and Mars. The work serves to eliminate constraints on the previous CST design and provides a
Real-time and high-speed vibrissae monitoring with dynamic vision sensors and...Aryan Esfandiari
This thesis shows that spiked-based neuromorphic sensors, such as dynamic vision sensors, open new possibilities for neuroscience instrumentation. Under specific circumstances, this can be an outstanding competitor to conventional technologies, making up for their disadvantages and rectifying current challenges during physical experiments. Such technology is more convenient while maintaining a desired level of accuracy and reliability.
This thesis discusses and considers algorithms through which multiple approaches to noise reduction are represented. Moreover, an Artificial Neural Network, as a classificatory, and the Kalman filter, as an estimator, are compared to determine the best options for a whisker movement monitoring and tracking algorithm. In this case, a modular weighted prediction-correction algorithm is generated with a noticeable level of accuracy and the ability to retain its reliability in noisy and unsuitable
experimental conditions.
This thesis discusses several conventional embedded system platforms for further implementations. All Programmable System-On-Chip is concluded as the most optimal solution for further research in this field. The platform shows noticeable capabilities for more complex dynamic vision sensors expected in the future, along with comprehensive algorithms. This thesis contributes necessary implementations for embedded system platforms with state-of-the-art technologies, such as System-On-Chip components including field-programmable gate array and microcontrollers. These can be used for further research and developments on asynchronous spiked-based neuromorphic sensors.
This document presents a project that examines using frequency response as an objective measurement for assessing the performance of cochlear implant microphones in uncontrolled environments. Tests were conducted using an Advanced Bionics Nada cochlear implant in different environments, with variations in microphone position, distance from speaker, ambient noise levels, and speaker used. The results showed high repeatability of the tests across environments, but variability increased when the microphone position changed or noise levels were high. Frequency response can still provide an overview of a cochlear implant microphone's performance in a user's home.
This document defines the mission statement for a proposed interstellar probe called "La Pinta" to study the Alpha Centauri system. It provides an overview of the state of knowledge on interstellar travel based on historical spacecraft concepts. The document outlines the mission objectives, operational concept, systems requirements, and risks. The proposed mission would utilize an ion engine or solar sail propulsion system to travel to Alpha Centauri over several decades, carrying instruments to characterize any exoplanets and the interstellar medium. Defining the mission requirements is the first step to further develop the concept and assess feasibility for a future interstellar exploration mission.
This document presents a dissertation on designing and implementing a 3D hand tracking interface using the Nintendo Wii Remote. It begins with an introduction describing the background and objectives of the project. It then provides a literature review on the Wii Remote's capabilities and technical specifications. The document outlines several conceptual models for hand tracking with increasing degrees of freedom. It presents the implementation of a 6 degree of freedom hand tracking system using two IR light sources and the Wii Remote's infrared camera. Testing and results demonstrate that the system allows for fast and accurate 3D tracking of hands at low cost, with potential applications for molecular visualization and other CAD programs.
Design and control of a compact hydraulic manipulator for quadruped robotsItfanrBeta
This document is the thesis of Bilal Ur Rehman for a Doctor of Philosophy degree from the University of Genova, Italy. The thesis describes the design and development of a novel, compact hydraulic manipulator called HyArm for use on quadruped robots. Through simulation and analysis, key specifications for the manipulator were determined, such as range of motion, torque, speed and optimal mounting position. Commercially available actuators were selected based on the simulation results. The mechanical design of HyArm is then discussed, which has 6 degrees of freedom, is compact at 0.745m, lightweight at 12.5kg and can carry a 10kg payload. Low-level and high-level controllers were also developed for Hy
This document is a project report submitted by four students to fulfill the requirements for a Bachelor of Technology degree in Information Technology. The report discusses steganography, which is hiding secret information within other information. Specifically, the report focuses on digital image steganography, where secret messages are hidden within digital images. The report provides an introduction to steganography, a literature review on related topics like cryptography, an analysis of requirements, descriptions of how image steganography works and algorithms used, system design diagrams, implementation details, applications of the system, and directions for future work.
This document provides an overview of the Metasploit Framework and how it can be used by pentesters. Metasploit is an open-source platform for developing, testing, and using exploit code. It includes an extensive database of public exploits and automated tools to help pentesters with various phases of an assessment. The document discusses key Metasploit interfaces, how to run exploits and payloads like Meterpreter, post-exploitation techniques, creating custom modules, pivoting through compromised systems, and other features. Special attention is given to automation and how Metasploit can help carry out comprehensive penetration tests efficiently.
The document discusses techniques for earthing systems and soil resistivity testing. It covers:
1. The principles and guidelines for soil resistivity testing, including test methods, traverse locations, and spacing ranges.
2. How to design lightning earth systems, including types of earth electrodes, common earthing systems, and calculating earth resistance.
3. The importance of measuring impulse impedance to assess the transient performance of an earthing system, and how to interpret impulse impedance measurements.
This document describes a dissertation on multimodal recognition of emotions. It introduces the topic of automatically recognizing human emotions from audio-visual data. The dissertation aims to develop models and techniques for detecting emotions from facial expressions in images and video, as well as speech. It outlines the goals of recognizing basic emotions like happiness, sadness, fear and anger using multiple modalities like facial expressions, body gestures and speech. The document provides an overview of the problem and outlines the various chapters that will cover topics like active appearance models for face analysis, facial expression recognition in images and video, emotion recognition from speech, and multimodal data fusion techniques.
This document describes Volker Beckmann's habilitation thesis presented at the University of Paris 7 in 2010. It provides a summary of his career experience, research interests, and publications focusing on active galactic nuclei observed with X-ray and gamma-ray telescopes. The document includes his curriculum vitae, teaching experience, student supervision, and a list of over 100 publications and conference presentations.
The document discusses the development of patch clamp techniques for measuring ion channel activity in single cells, including the conventional glass micropipette method and emerging microfabrication approaches using biochips. It describes how soft lithography and etching techniques can be used to create micropores in silicon chips as an alternative to glass micropipettes for patch clamp recordings, aiming to provide higher throughput and less labor-intensive measurements compared to conventional methods. The document also reviews several commercial systems that have implemented high-throughput patch clamping using microfabricated biochips.
This thesis implements continuous POMDP algorithms on autonomous robots to handle uncertainty in partially observable environments. It uses the ARKAQ learning algorithm to model a real world environment and achieve predefined targets with Sony AIBO robots in Webots simulation. Two problems are studied: ball approaching and scoring a goal. The ARKAQ algorithm uses a Kalman filter for state estimation, ART2 neural networks for state segmentation, and Q-learning. Experiments show the robots successfully reach the targets and score goals more effectively than random actions. The optimality of the results is discussed along with parameters that affect optimality.
This document describes a final year project to develop an inflight data acquisition system for a smaller aircraft. It includes a declaration page signed by the student, Muhamed Nur Syamim Bin Idris, and an approval page signed by the project supervisor, Dr. Ari Legowo. The system will collect parameters like position, acceleration, and attitude data and transmit it in real-time to a ground station.
The document is a design report for the Cygnus satellite by Aman Sharma, John Gehrke, Brandon Keeber, Eduardo Asuaje, Jacob Korinko, and Vaibhav Menon. It provides details on the preliminary design of the satellite, including mission analysis, payload design, subsystem designs for structure, thermal, attitude control, telemetry/tracking/command, propulsion, power, and risk/cost analysis. The design aims to meet requirements for a communications satellite in low Earth orbit with a 5 year mission lifetime.
This document is a research paper written by Craig Ferguson at the University of Cape Town that presents a high performance traffic sign detection technique for use in low power systems or high speed vehicles. The paper introduces the problem of traffic sign detection in vehicles and outlines the objectives and structure of the research. It then reviews existing literature on topics like preprocessing, detection, classification, training and testing. The paper goes on to describe the proposed method, which uses RGB thresholding for segmentation and tracks signs across frames to allow for a voting scheme. It presents results showing the method performs detection at 13ms per frame and achieves 83% detection efficiency, significantly outperforming a cascade classifier detector. The technique is constrained to midday lighting but provides a proof
This document is the bachelor's thesis of Cristóbal Cuevas García from June 2018. The thesis proposes a preliminary collision avoidance system for unmanned aerial vehicles using ultrasonic range finders and an Arduino microcontroller board. The system involves assembling a quadcopter from scratch and integrating additional hardware and software for collision avoidance. Ground and flight tests were conducted to evaluate the effectiveness of the collision avoidance system in detecting obstacles and maneuvering the quadcopter to avoid collisions. While the system was able to detect obstacles and trigger avoidance maneuvers, improving stability after avoidance maneuvers was identified as an area for future work.
This thesis developed a system to enable a quadcopter to autonomously navigate to predefined target locations while avoiding obstacles. The system uses a monocular camera for localization, mapping and object detection. It implements Simultaneous Localization and Mapping (SLAM) for localization and builds a map. Oriented FAST and Rotated BRIEF (ORB) features are used to detect and classify obstacles. Reinforcement learning is used to navigate around obstacles and reach targets, by learning from rewards and punishments. The system was implemented and tested on a Parrot AR.Drone, navigating autonomously indoors to validate the approach.
The document is a thesis submitted by Sunidhi Azad to Concordia University in partial fulfillment of the requirements for a Master of Computer Science degree. It studies group foraging by robotic swarms, which involves collectively searching for an object of interest and bringing it back to a central location. The thesis proposes five novel algorithms for the exploration phase of foraging and analyzes their competitive ratios. It also provides two approaches for the transportation phase and analyzes a simple case theoretically. The performance of the algorithms is evaluated through simulations.
This thesis presents methods to extend the operational lifetime of inflatable soft robots when leaks occur in either the structural chambers or actuation chambers. For structural leaks, a control algorithm is developed to detect the leak and then lower the target pressures as needed to slow the leak rate while maintaining a specified level of accuracy. Testing on an inflatable robot arm showed the algorithm could decrease long-term error by 50% compared to no control and prolong operation. For actuator leaks, lowering the target pressure for the leaking joint is shown to trade off leak rate and performance. Tests on inflatable robots demonstrated the potential for up to a 300% increase in operation time with minimal increased error. The work aims to preserve limited compressed air resources and extend operation
This document summarizes an investigation by BSc. Dirk Ekelschot into Gortler vortices in hypersonic flow using quantitative infrared thermography (QIRT) and tomographic particle image velocimetry (Tomo-PIV) at Delft University of Technology. The study used a double compression ramp model in the Hypersonic Test Facility Delft to generate Gortler vortices. QIRT was used to analyze the growth of vortices at different Reynolds numbers. Tomo-PIV was employed to validate its use in the facility by calibrating the system and estimating accuracy and reconstruction quality. Results from both techniques provided insight into the vortex topology and growth.
MSc_Thesis_Wake_Dynamics_Study_of_an_H-type_Vertical_Axis_Wind_TurbineChenguang He
This thesis investigates the wake dynamics of an H-type vertical axis wind turbine using particle image velocimetry (PIV). Two-component PIV is used to study vorticity shedding and horizontal wake expansion at the turbine mid-span plane. Stereoscopic PIV is performed on 7 cross-stream vertical planes to analyze tip vortex dynamics and the evolution of 3D wake structures. The experimental results show asymmetrical vorticity decay in the horizontal plane, with faster decay on the leeward side. Tip vortices are stronger than shed vortices. Near the turbine axis, tip vortices move inboard behind the rotor before moving outboard towards the windward side further downstream. Vertical
This thesis presents work on a novel Compass Star Tracker (CST) instrument that can determine its local position on Earth by imaging stars. The document covers:
1) The theoretical development of the local position determination mathematics using star sensor attitude information.
2) Design and fabrication of a proof-of-concept CST, including the imaging sensor, optics, inclinometer, cooling system, and data acquisition hardware.
3) Experimental testing of the CST to validate the position determination concept, including analysis of theoretical and practical error sources.
The CST has potential applications for autonomous position sensing on the Moon and Mars. The work serves to eliminate constraints on the previous CST design and provides a
Real-time and high-speed vibrissae monitoring with dynamic vision sensors and...Aryan Esfandiari
This thesis shows that spiked-based neuromorphic sensors, such as dynamic vision sensors, open new possibilities for neuroscience instrumentation. Under specific circumstances, this can be an outstanding competitor to conventional technologies, making up for their disadvantages and rectifying current challenges during physical experiments. Such technology is more convenient while maintaining a desired level of accuracy and reliability.
This thesis discusses and considers algorithms through which multiple approaches to noise reduction are represented. Moreover, an Artificial Neural Network, as a classificatory, and the Kalman filter, as an estimator, are compared to determine the best options for a whisker movement monitoring and tracking algorithm. In this case, a modular weighted prediction-correction algorithm is generated with a noticeable level of accuracy and the ability to retain its reliability in noisy and unsuitable
experimental conditions.
This thesis discusses several conventional embedded system platforms for further implementations. All Programmable System-On-Chip is concluded as the most optimal solution for further research in this field. The platform shows noticeable capabilities for more complex dynamic vision sensors expected in the future, along with comprehensive algorithms. This thesis contributes necessary implementations for embedded system platforms with state-of-the-art technologies, such as System-On-Chip components including field-programmable gate array and microcontrollers. These can be used for further research and developments on asynchronous spiked-based neuromorphic sensors.
This document presents a project that examines using frequency response as an objective measurement for assessing the performance of cochlear implant microphones in uncontrolled environments. Tests were conducted using an Advanced Bionics Nada cochlear implant in different environments, with variations in microphone position, distance from speaker, ambient noise levels, and speaker used. The results showed high repeatability of the tests across environments, but variability increased when the microphone position changed or noise levels were high. Frequency response can still provide an overview of a cochlear implant microphone's performance in a user's home.
This document defines the mission statement for a proposed interstellar probe called "La Pinta" to study the Alpha Centauri system. It provides an overview of the state of knowledge on interstellar travel based on historical spacecraft concepts. The document outlines the mission objectives, operational concept, systems requirements, and risks. The proposed mission would utilize an ion engine or solar sail propulsion system to travel to Alpha Centauri over several decades, carrying instruments to characterize any exoplanets and the interstellar medium. Defining the mission requirements is the first step to further develop the concept and assess feasibility for a future interstellar exploration mission.
This document presents a dissertation on designing and implementing a 3D hand tracking interface using the Nintendo Wii Remote. It begins with an introduction describing the background and objectives of the project. It then provides a literature review on the Wii Remote's capabilities and technical specifications. The document outlines several conceptual models for hand tracking with increasing degrees of freedom. It presents the implementation of a 6 degree of freedom hand tracking system using two IR light sources and the Wii Remote's infrared camera. Testing and results demonstrate that the system allows for fast and accurate 3D tracking of hands at low cost, with potential applications for molecular visualization and other CAD programs.
Design and control of a compact hydraulic manipulator for quadruped robotsItfanrBeta
This document is the thesis of Bilal Ur Rehman for a Doctor of Philosophy degree from the University of Genova, Italy. The thesis describes the design and development of a novel, compact hydraulic manipulator called HyArm for use on quadruped robots. Through simulation and analysis, key specifications for the manipulator were determined, such as range of motion, torque, speed and optimal mounting position. Commercially available actuators were selected based on the simulation results. The mechanical design of HyArm is then discussed, which has 6 degrees of freedom, is compact at 0.745m, lightweight at 12.5kg and can carry a 10kg payload. Low-level and high-level controllers were also developed for Hy
This document is a project report submitted by four students to fulfill the requirements for a Bachelor of Technology degree in Information Technology. The report discusses steganography, which is hiding secret information within other information. Specifically, the report focuses on digital image steganography, where secret messages are hidden within digital images. The report provides an introduction to steganography, a literature review on related topics like cryptography, an analysis of requirements, descriptions of how image steganography works and algorithms used, system design diagrams, implementation details, applications of the system, and directions for future work.
This document provides an overview of the Metasploit Framework and how it can be used by pentesters. Metasploit is an open-source platform for developing, testing, and using exploit code. It includes an extensive database of public exploits and automated tools to help pentesters with various phases of an assessment. The document discusses key Metasploit interfaces, how to run exploits and payloads like Meterpreter, post-exploitation techniques, creating custom modules, pivoting through compromised systems, and other features. Special attention is given to automation and how Metasploit can help carry out comprehensive penetration tests efficiently.
The document discusses techniques for earthing systems and soil resistivity testing. It covers:
1. The principles and guidelines for soil resistivity testing, including test methods, traverse locations, and spacing ranges.
2. How to design lightning earth systems, including types of earth electrodes, common earthing systems, and calculating earth resistance.
3. The importance of measuring impulse impedance to assess the transient performance of an earthing system, and how to interpret impulse impedance measurements.
This document describes a dissertation on multimodal recognition of emotions. It introduces the topic of automatically recognizing human emotions from audio-visual data. The dissertation aims to develop models and techniques for detecting emotions from facial expressions in images and video, as well as speech. It outlines the goals of recognizing basic emotions like happiness, sadness, fear and anger using multiple modalities like facial expressions, body gestures and speech. The document provides an overview of the problem and outlines the various chapters that will cover topics like active appearance models for face analysis, facial expression recognition in images and video, emotion recognition from speech, and multimodal data fusion techniques.
This document describes Volker Beckmann's habilitation thesis presented at the University of Paris 7 in 2010. It provides a summary of his career experience, research interests, and publications focusing on active galactic nuclei observed with X-ray and gamma-ray telescopes. The document includes his curriculum vitae, teaching experience, student supervision, and a list of over 100 publications and conference presentations.
The document discusses the development of patch clamp techniques for measuring ion channel activity in single cells, including the conventional glass micropipette method and emerging microfabrication approaches using biochips. It describes how soft lithography and etching techniques can be used to create micropores in silicon chips as an alternative to glass micropipettes for patch clamp recordings, aiming to provide higher throughput and less labor-intensive measurements compared to conventional methods. The document also reviews several commercial systems that have implemented high-throughput patch clamping using microfabricated biochips.
This thesis implements continuous POMDP algorithms on autonomous robots to handle uncertainty in partially observable environments. It uses the ARKAQ learning algorithm to model a real world environment and achieve predefined targets with Sony AIBO robots in Webots simulation. Two problems are studied: ball approaching and scoring a goal. The ARKAQ algorithm uses a Kalman filter for state estimation, ART2 neural networks for state segmentation, and Q-learning. Experiments show the robots successfully reach the targets and score goals more effectively than random actions. The optimality of the results is discussed along with parameters that affect optimality.
This document describes a final year project to develop an inflight data acquisition system for a smaller aircraft. It includes a declaration page signed by the student, Muhamed Nur Syamim Bin Idris, and an approval page signed by the project supervisor, Dr. Ari Legowo. The system will collect parameters like position, acceleration, and attitude data and transmit it in real-time to a ground station.
The document is a design report for the Cygnus satellite by Aman Sharma, John Gehrke, Brandon Keeber, Eduardo Asuaje, Jacob Korinko, and Vaibhav Menon. It provides details on the preliminary design of the satellite, including mission analysis, payload design, subsystem designs for structure, thermal, attitude control, telemetry/tracking/command, propulsion, power, and risk/cost analysis. The design aims to meet requirements for a communications satellite in low Earth orbit with a 5 year mission lifetime.
This document is a research paper written by Craig Ferguson at the University of Cape Town that presents a high performance traffic sign detection technique for use in low power systems or high speed vehicles. The paper introduces the problem of traffic sign detection in vehicles and outlines the objectives and structure of the research. It then reviews existing literature on topics like preprocessing, detection, classification, training and testing. The paper goes on to describe the proposed method, which uses RGB thresholding for segmentation and tracks signs across frames to allow for a voting scheme. It presents results showing the method performs detection at 13ms per frame and achieves 83% detection efficiency, significantly outperforming a cascade classifier detector. The technique is constrained to midday lighting but provides a proof
This document is the bachelor's thesis of Cristóbal Cuevas García from June 2018. The thesis proposes a preliminary collision avoidance system for unmanned aerial vehicles using ultrasonic range finders and an Arduino microcontroller board. The system involves assembling a quadcopter from scratch and integrating additional hardware and software for collision avoidance. Ground and flight tests were conducted to evaluate the effectiveness of the collision avoidance system in detecting obstacles and maneuvering the quadcopter to avoid collisions. While the system was able to detect obstacles and trigger avoidance maneuvers, improving stability after avoidance maneuvers was identified as an area for future work.
This thesis developed a system to enable a quadcopter to autonomously navigate to predefined target locations while avoiding obstacles. The system uses a monocular camera for localization, mapping and object detection. It implements Simultaneous Localization and Mapping (SLAM) for localization and builds a map. Oriented FAST and Rotated BRIEF (ORB) features are used to detect and classify obstacles. Reinforcement learning is used to navigate around obstacles and reach targets, by learning from rewards and punishments. The system was implemented and tested on a Parrot AR.Drone, navigating autonomously indoors to validate the approach.
The document is a thesis submitted by Sunidhi Azad to Concordia University in partial fulfillment of the requirements for a Master of Computer Science degree. It studies group foraging by robotic swarms, which involves collectively searching for an object of interest and bringing it back to a central location. The thesis proposes five novel algorithms for the exploration phase of foraging and analyzes their competitive ratios. It also provides two approaches for the transportation phase and analyzes a simple case theoretically. The performance of the algorithms is evaluated through simulations.
This document summarizes a degree thesis project focused on developing a hexacopter drone with a gripping module to compete in the 2018 CPS-VO challenge. The goals of the project were to 1) create a computer simulation of the hexacopter and gripping module using Gazebo and ROS, and 2) design and build a physical hexacopter integrated with a gripping module based on the simulation results. The thesis describes the hardware and software used to model and simulate the drone, as well as the process for designing, 3D printing, and testing the physical gripping module attachment. The completed hexacopter with gripping module was used to compete in the 2018 challenge.
The document presents a complete Android-based framework for automatically identifying a user's transportation mode using GPS trajectories and accelerometer measurements from a smartphone. The framework includes an architecture, design, implementation, user interface, and algorithms for transportation mode identification. It applies segmentation, simplification, and machine learning classification techniques to collected GPS and accelerometer data to identify modes like walking, running, and in-vehicle transportation. The system was evaluated on real and simulated data, achieving an overall accuracy of around 85% for identifying transportation modes, outperforming the Google Activity Recognition API.
This document is a bachelor's thesis submitted by Dimitar Dimitrov to the Technical University of Munich. The thesis analyzes velocity fields in cosmological simulations using the ParticleEngine visualization tool. It presents upgrades made to the ParticleEngine to address the needs of astrophysicists studying galaxy formation simulations. The upgrades include implementing multiple particle sources, a transfer function editor, fourth-component recalculation, displaying physical units, and exporting particle data. Visualization results are shown demonstrating the multi-probe configuration and direct volume rendering with fourth-component recalculation.
This thesis describes software tools for simulating and automatically generating swarm algorithms. It presents SWEEP, a swarm simulation and experimentation platform, which is used to test dispersion, task assignment, and chemical cloud detection swarm algorithms. The thesis also introduces an evolutionary algorithm to automatically generate swarm behaviors. It evolves solutions for dispersion, object manipulation tasks, and evaluates the results.
This master's thesis addresses the problem of loop closure detection in visual SLAM. Two approaches using binary features are presented: 1) A probabilistic approach called FAB-MAP is adjusted to use binary features for reliable loop closure detection. 2) A novel approach measures similarity between VLAD representations of image frames combined with pre-filtering to improve detection rate and speed. Both approaches are implemented and evaluated in the ORB-SLAM2 framework on the KITTI dataset, showing they achieve similar trajectory accuracy as ORB-SLAM2 while saving considerable time in loop detection compared to the original method.
This document discusses automatic Android malware analysis. It begins with introductions to Android application fundamentals like application components and intents. It then discusses the APK file format and Dex file format. It covers static analysis using the Androguard tool to extract information from APKs. It also covers dynamic analysis using the CuckooDroid tool and discusses fixes needed for it to work with newer Android versions. It explores techniques for emulator evasion/detection and discusses using Frida for instrumentation. The conclusion discusses areas for future work like integrating Frida with CuckooDroid and improving emulator performance and Android support.
This document summarizes a student project on predicting malicious activity using real-time video surveillance. The project applies techniques like super-resolution, face and object recognition using HOG features, and neural networks to enhance video quality, identify objects and faces, and semantically describe scenes to detect unusual activity. Algorithms were implemented in MATLAB and results were stored in a MongoDB database. Key techniques included super-resolution, PCA-based face recognition, HOG-based object detection, and neural networks like CNNs and RNNs for image captioning. The project aims to help detect criminal activity and track convicted individuals in public spaces.
This thesis proposes and evaluates a compressive sensing (CS)-based indoor positioning and tracking system using received signal strength (RSS) from wireless local area network access points. The system is designed and implemented on mobile devices with limited resources.
In the offline phase, RSS fingerprints are collected and clustered using affinity propagation. In the online phase, coarse localization is done by matching RSS measurements to precomputed clusters, and fine localization refines the position using CS recovery on the sparse location signal.
An indoor tracking system is also presented, which integrates the CS-based positioning with a Kalman filter for sequential location estimates. Experimental results on two testbeds show the system achieves better accuracy than other fingerprinting methods, suitable for implementation
This thesis presents an approach for non-rigid multi-modal object tracking using Gaussian mixture models (GMM). The target is represented by a GMM with each ellipsoid corresponding to a different fragment of the target. A region growing algorithm is used to automatically adapt the fragment set and extract accurate boundaries. Tracking performance is improved by incorporating joint Lucas-Kanade feature tracking to handle large motions. Experimental results demonstrate the effectiveness of the approach on challenging sequences.
This document is a master's thesis submitted by Milan Tepić to the University of Stuttgart exploring host-based intrusion detection to enhance cybersecurity in real-time automotive systems. The thesis was supervised by Dr.-Ing. Mohamed Abdelaal and examined by Prof. Dr. Kurt Rothermel. It explores using timing elements of control unit functions to detect anomalies and intrusions. The goal is to develop a host-based intrusion detection system called AutoSec that can detect anomalies while keeping false alarms close to zero, in compliance with the AUTOSAR automotive software standard.
Im-ception - An exploration into facial PAD through the use of fine tuning de...Cooper Wakefield
This document is a thesis submitted by Cooper Wakefield to the University of Queensland for the degree of Bachelor of Engineering. The thesis proposes developing a presentation attack detection (PAD) system through fine tuning a deep convolutional neural network. It aims to leverage pre-trained networks and fine tune the upper layers to differentiate between real and fake facial images with a high degree of accuracy. The thesis outlines the problem of presentation attacks on facial recognition systems, reviews prior approaches to PAD, and describes the proposed solution of using transfer learning on a CNN to classify images as real or fake.
This document summarizes the system requirements for Project RIDES, which is being developed by Team Omni at Embry-Riddle Aeronautical University. It details the revision history of the document, provides an overview of the key subsystems and their requirements, and describes use cases and sequence diagrams for core functions like starting a ride, stopping a ride, and updating vehicle locations. The document is intended to specify the intellectual property and technical requirements for the autonomous vehicle project.
This doctoral thesis examines methods for estimating the authenticity of videos by analyzing their visual quality and structure. It aims to determine the proportion of information an edited video retains from its original parent video. The thesis first evaluates existing no-reference algorithms for visual quality assessment. It then explores techniques for shot segmentation and comparison. It also develops models for calculating a video's authenticity degree based on factors like visual quality, shot importance, and evidence of global modifications. The goal is to objectively estimate a video's authenticity when only the video itself is available for analysis, without relying on external metadata.
The document discusses precise autonomous orbit control for spacecraft in low Earth orbit. It presents two main approaches: 1) reconsidering state-of-the-art orbit control methods from an autonomous perspective, and 2) formally defining the autonomous orbit control problem as a virtual formation with one spacecraft (reference) affected only by Earth's gravity. The author develops analytical and numerical control algorithms, simulates their performance, and describes implementing an autonomous orbit control system for the PRISMA mission which demonstrated the capability in flight.
Similar to Particle Filter Localization for Unmanned Aerial Vehicles Using Augmented Reality Tags (20)
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
20 Comprehensive Checklist of Designing and Developing a WebsitePixlogix Infotech
Dive into the world of Website Designing and Developing with Pixlogix! Looking to create a stunning online presence? Look no further! Our comprehensive checklist covers everything you need to know to craft a website that stands out. From user-friendly design to seamless functionality, we've got you covered. Don't miss out on this invaluable resource! Check out our checklist now at Pixlogix and start your journey towards a captivating online presence today.
Building RAG with self-deployed Milvus vector database and Snowpark Container...Zilliz
This talk will give hands-on advice on building RAG applications with an open-source Milvus database deployed as a docker container. We will also introduce the integration of Milvus with Snowpark Container Services.
Particle Filter Localization for Unmanned Aerial Vehicles Using Augmented Reality Tags
1. Particle Filter Localization for
Unmanned Aerial Vehicles Using
Augmented Reality Tags
Edward Francis Kelley V
Submitted to the
Department of Computer Science
in partial fulfillment of the requirements for
the degree of Bachelor of Arts
Princeton University
Advisor:
Professor Szymon Rusinkiewicz
May 2013
2. This thesis represents my own work in accordance with University regulations.
Edward Francis Kelley V
ii
3. Abstract
This thesis proposes a system for capturing 3D models of large objects using autonomous quadcopters. A major component of such a system is accurately localizing
the position and orientation, or pose, of the quadcopter in order to execute precise
flight patterns. This thesis focuses on the design and implementation of a localization algorithm that uses a particle filter to combine internal sensor measurements
and augmented reality tag detection in order to estimate the pose of an AR.Drone
quadcopter. This system is shown to perform significantly better than integrated
velocity measurements alone.
iii
4. Acknowledgements
Completing this thesis has been one of the most challenging, yet fulfilling experiences
I have had in my time here at Princeton. I could never have completed this alone,
and I am indebted to a long list of mentors, friends, and family members.
First and foremost, I would like to express my gratitude to my advisor, Szymon
Rusinkiewicz, whose support and advice has been invaluable throughout this entire
process. Professor Rusinkiewicz went above and beyond what could be expected of
an undergraduate thesis advisor and I appreciate how much I have learned from him,
ever since my days of struggling through “Death Graphics.” I would also like to thank
Professor Robert Stengel for his advice and support, as well as Professor Christopher
Clark for introducing me to robotics and continuing to be a valuable source of advice
during this project.
Furthermore, this project was funded by the School of Engineering and Applied
Science, as well as the Morgan McKinzie 93 Senior Thesis Prize Fund. I am grateful
to attend a school that makes projects such as this possible.
A special thanks goes to my thesis partner, Sarah Tang. Her friendship and cheery
disposition made those long hours of watching generally-uncooperative quadcopters
a fun and memorable experience.
I would also like to thank everyone who made my time at Princeton incredible.
In particular: my friends in the Princeton Tower Club for keeping my spirits up and
providing wonderful distractions from my computer screen; my quasi-roommates and
dinner companions, Nick Adkins and Alice Fuller, for bringing me so much happiness
and laughter; John Subosits for allowing me to pretend that I am an engineer; Rodrigo
Menezes for being a great partner in crime, both in the US and abroad; the Menezes
and Adkins families for each claiming me as one of their own; Jonathan Yergler, Coach
Zoltan Dudas, and the rest of the Princeton Fencing Team for all of the wonderful
memories, both on and off the strip.
iv
5. Finally, I would like to thank my parents. It is only with their love and support
that I have made it to this point. I am unable to fully convey my appreciation for
everything they have done.
v
12. Chapter 1
Introduction
In recent years, research using small-scale Unmanned Aerial Vehicles (UAVs) has increased rapidly. Multi-rotor aircraft, such as quadcopters (also known as quadrotors),
have proven to be powerful platforms for applications in a variety of fields, from aerial
photography to search and rescue [18, 17]. Once prohibitively expensive for many applications, multi-rotor aircraft have decreased substantially in price, ranging from a
few hundred dollars to several thousand dollars. Additionally, on-board control systems have greatly added to the stability and ease of control of these aircraft, with
many quadcopters using pre-programmed routines for difficult procedures such as
takeoff and landing.
Although quadcopters have seen interest from the military and hobbyists for quite
some time, these recent developments in price and stability have resulted in a large
increase of research using quadcopters. In 2010, a French electronics company, Parrot,
released the AR.Drone, a quadcopter intended for consumers. Unlike most other
quadcopters, which are sold in unassembled kits targeted to experienced hobbyists
or researchers, the AR.Drone was designed to be ready-to-fly right out of the box
by an inexperienced pilot. Although affordable and easy to use, these quadcopters
are far from just toys. Equipped with an array sensors and multiple cameras, the
1
13. AR.Drone and other consumer-grade vehicles have been used by research groups to
explore autonomous flight with a low barrier of entry, both in cost and complexity.
With the democratization of this technology, quadcopters can be used to be solved
problems where cost and complexity for such a solution were once prohibitive.
1.1
Motivation
For applications ranging from developing video games to preserving archaeological
artifacts, capturing 3D models of real-world objects has become an increasingly important task.
While there are currently several different methods for capturing these models,
each of these methods has associated limitations and drawbacks.
1.2
1.2.1
Current Acquisition Methods
Manual Model Creation
The most common method of model acquisition is using a trained modeler to manually
produce a 3D model using reference imagery and measurements. Models are tediously
crafted in CAD software in a process similar to sculpting. While this technique is
used extensively in video games and films, it is typically too time consuming and
expensive for many applications such as archeology.
1.2.2
Laser Scanners
Laser rangefinder technology is the “gold standard” of 3D model acquisition in terms
of precision. Modern scanners are able to produce models with sub-millimeter precision, which make them a great choice for detailed digitization of statues. Combined
with high-resolution photograph texture-mapping, very few techniques can match the
2
14. Figure 1.1: Laser scanner rig used by the Digital Michelangelo Project [23].
precision and quality of these scans. The Digital Michelangelo Project showed the
power and precision of laser scanners by scanning several different statues, including
Michelangelo’s David, to 1/4mm accuracy [23].
However, laser scanners do have several drawbacks. The equipment is extremely
expensive, bulky, and fragile. The Michelangelo Project had to transport over 4 tons
of equipment to Italy in order to produce their scans. Additionally, laser scans involve
immense setup and can take many hours. The model of David took over a thousand
man-hours to scan and even more than that in post processing [23].
1.2.3
Microsoft Kinect
Several groups have researched using the Microsoft Kinect for model acquisition. As
the Kinect produces an RGB image with depth information, imagery gathered from
multiple angles can be used to create a 3D model. While this has been found to
be useful in scanning small to medium size objects, including people, the Kinect
has several limitations. First of all, the depth range is rather short, on the order
of a couple meters. Additionally, because the depth is partially determined using
3
15. Figure 1.2: 3D model of a statue generated by Agisoft Photoscan [1].
an infrared pattern, the Kinect does not work in locations with a large amount of
background infrared light, such as outdoors.
1.2.4
Multi-View Stereo
Multi-view stereo uses a collection of 2D images to reconstruct a 3D object. By
viewing a single object from hundreds of different camera positions, it is possible to
infer depth information and generate a 3D model. Although this technique originally
required precisely known camera coordinates, recent algorithms can produce a 3D
model from an unordered collection of images with unknown camera positions, assuming that there is sufficient coverage. Existing software packages such as Bundler
and Agisoft Photoscan can produce high-quality 3D reconstructions using these unordered image collections [6, 1].
The ability to use a collection of images without precise camera position information means that these 3D objects can be modeled substantially faster than with a
laser scanner. For a smaller object, it is a relatively simple process to take pictures of
the object from many different angles. However, for a larger object, such as a statue
or building, the task of gathering imagery becomes substantially more difficult.
4
16. 1.3
Problem Definition
We look to create a system for capturing imagery of large objects for use in multi-view
stereo software. There are several desirable attributes of such a system:
1. Low Cost
The system should be low cost, making it accessible to as many users as possible.
2. Easy to Use
The system should be able to be deployed by users with minimal training and
setup. Additionally, the hardware should be off-the-shelf and easily attainable.
3. Complete Coverage
The system must be able to capture images from a wide variety of positions,
completely covering every part of the target object.
4. High Quality Imagery
The system must produce sufficiently high resolution, non-distorted images for
use in multi-view stereo software.
1.4
Proposed Solution
We propose using low-cost autonomous quadcopters to gather the imagery needed
for use in multi-view stereo software. By flying a quadcopter with a mounted camera
around the target object, imagery can be gathered quickly from many viewing angles.
Using quadcopters has many advantages:
1. Quadcopters can capture images from positions unreachable by ground-based
cameras.
5
17. Figure 1.3: Example of desired camera positions for use in multi-view stereo [18].
2. By methodically flying around the target object at different altitudes, complete
coverage of the target object can be achieved.
3. The imagery can be captured very quickly, on the order of a few minutes.
4. Quadcopters are small, portable, and easily deployable.
1.5
Solution Components
In order to implement such a solution, there are two main components which must
be created. First, a localization algorithm is needed to produce position and orientation estimates of the quadcopter in the global frame. Then, a controller is needed
to produce the control inputs for the quadcopter to move along a desired path. This
thesis focuses on the design, implementation, and testing of the localization component, while the thesis of Sarah Tang in the Mechanical and Aerospace Engineering
Department addresses the controller component of this system [30].
6
18. Chapter 2
Related Work
The past decade has seen a huge increase in the use of quadcopters for a variety of
applications. With the improvement of stabilization software, quadcopters have seen
a rise in popularity as a stable, cheap, and highly maneuverable aerial platform.
2.1
3D Model Acquisition Using Quadcopters
Although a relatively new field, several research groups have studied the use of quadcopters in 3D model construction. Irschara et al. created a system to generate 3D
models using images taken from UAVs. While a quadcopter was used for gathering
imagery, the quadcopter was manually controlled and the main focus of their work was
multi-view stereo model creation [18]. Steffen et al. studied surface reconstruction
using aerial photography captured by UAVs [29].
2.2
GPS-denied Navigation of Quadcopters
There has been substantial work done on autonomous navigation of quadcopters
without the use of GPS. As GPS signal cannot typically penetrate walls or ceilings,
7
19. Figure 2.1: 3D Model of a building By Irschara et al. [18]
(a) Flying environment
(b) Camera view with detected features
Figure 2.2: AR.Drone localization using monocular SLAM [13].
any autonomous flights taking place indoors must use methods other than GPS for
localization.
Most relevant to this project, Engel et al. have published multiple papers on
the camera-based navigation and localization of the AR.Drone [14, 13]. While they
were able to achieve very accurate navigation, their work relies on the drone facing
a mostly planar surface during the entire flight. As constructing models requires
capturing imagery from a variety of positions and headings, this constraint is not
acceptable for the proposed system.
Bills et al. presented a method for navigating in an unknown interior space with
the AR.Drone. By performing edge detection on the forward and downward facing
cameras, their algorithm is able to determine the type of environment the quad8
20. (a) Vanishing point detection
(b) Stair detection
Figure 2.3: Localization using perspective cues [10].
copter is in, either hallway or staircase, and issues the appropriate commands to
autonomously navigate the space [10]. While this system is a great step in exploring
interior space, it does not provide a solution to our problem. Unlike the work by Bills
et al., our system must operate in essentially open space, with few visual cues, such
as vanishing points, available for reference.
9
21. Chapter 3
System Design
3.1
Quadcopter Characteristics
Quadcopters and other multi-rotor aircraft are mechanically much simpler than their
conventional rotorcraft counterparts. In order to change roll, pitch, and yaw, a traditional helicopter utilizes mechanical linkages to change the pitch of the main rotor
blades. Quadcopters, on the other hand, use four fixed-pitch rotors which can be independently controlled to achieve a full range of motion. Because of their mechanical
simplicity, quadcopters are relatively simple to maintain and repair. Additionally,
quadcopters are very highly maneuverable due to the ability to rapidly change the
thrust of any individual rotor. Also, as there are four rotors contributing to the vertical thrust, each individual rotor possesses less kinetic energy than a single main rotor
on a traditional helicopter, making them safer to be used in indoor spaces and around
humans.
3.1.1
Basics of Quadcopter Flight
Each rotor produces a downwards thrust and torque opposite to the direction of
rotation. In order to counter this torque, rotors along a single arm rotate in one
10
22. (a) Hover
(b) Yaw
(c) Pitch
(d) Roll
Figure 3.1: Quadcopter flight control.
direction, with the other rotors rotating in the other direction, as seen in Figure 3.1.
If all of the rotors are turning at the same rate, then the torque forces will cancel out and the quadcopter will hover without rotating (Figure 3.1a). In order to
change yaw, two of the rotors spinning in the same direction increase their thrust,
and therefore their torque, and the other two rotors slow (Figure 3.1b). This induces
a change in yaw without affecting the overall thrust. In order to change pitch or roll,
two rotors on the same side increase thrust, with the other rotors decreasing thrust
(Figures 3.1c and 3.1d) [12].
11
23. 3.2
Parrot AR.Drone 2.0
(a) Indoor Hull
(b) Outdoor Hull
Figure 3.2: Parrot AR.Drone 2.0 [8].
Our system will use the AR.Drone 2.0, the second generation of the consumergrade quadcopter released by Parrot in 2010. The AR.Drone is a stabilized aerial
platform that can be controlled by a user-friendly interface on a variety of mobile
devices such as the Apple iPhone or iPad. The quadcopter is equipped with cameras
and can be used for recording videos and playing augmented reality games.
Figure 3.3: AR.Drone FreeFlight application for the Apple iPhone [8].
12
24. Forward Camera
HD, 720p
92◦ diagonal viewing area
Bottom Camera
QVGA, 320x240
64◦ diagonal viewing area
Computational Resources
1 GHz ARM Cortex-A8 CPU
800 MHz Video Digital Signal Processor
256 MB (1 Gbit) DDR2 RAM
Networking
802.11n WiFi
Sensors
3-axis gyroscope (2000 degree/second)
3-axis accelerometer (+/- 50 mg precision)
3-axis magnetometer (6 degree precision)
Pressure sensor (+/- 10 Pa precision)
Ultrasound altitude sensor
Table 3.1: AR.Drone 2.0 technical specifications [11].
3.2.1
Features
Considering its target audience of consumers, the AR.Drone is actually a very powerful research platform. The quadcopter is ready-to-fly out of the box. Unlike most
quadcopters which are sold as kits, there is no assembly or technology knowledge
needed to get started. Additionally, with the provided SDK, it is relatively easy to
get off the ground and start controlling the quadcopter programmatically. Finally,
at only $300, the AR.Drone is much easier to fit into most research budgets than kit
quadcopters which can cost thousands of dollars.
As this quadcopter is designed to be used by novice pilots, the AR.Drone comes
with a protective expanded polypropylene shell which prevents the blades from coming
in direct contact with anything. Additionally, if any of the motors senses collision,
all of the rotors immediately shut off. This makes the AR.Drone an especially safe
13
25. platform which is extremely unlikely to cause damage to either the target object or
people.
The AR.Drone has two cameras, one forward-facing HD camera, and one lower
resolution high frame-rate camera facing downwards. The AR.Drone processes the
visual imagery on board to produce a velocity estimate. Depending on the ground
material and lighting quality, the AR.Drone uses either muti-resolution optical flow
or FAST corner detection with least-squares minimization to estimate movement
between frames. The AR.Drone also uses the gyroscope and accelerometer on the
navigation board to produce a velocity estimate and fuses this estimate with the
vision-based velocity to create a relatively robust combined velocity estimation [11].
Figure 3.4: Fusion of accelerometer and vision-based velocity measurements [11].
For altitude estimation, the AR.Drone uses a combination of an ultrasonic range
sensor and pressure sensor. At heights under 6 meters, the AR.Drone relies solely on
the ultrasonic sensor. Above those heights, where the ultrasonic sensor is not in its
operational range, the AR.Drone estimates altitude based on the difference between
the current pressure and the pressure measured on takeoff.
14
26. Figure 3.5: On-board AR.Drone control architecture [11].
The on-board processor handles low-level stabilization and wind compensation, allowing the quadcopter to hold position when not receiving control inputs. Commands
to the AR.Drone are sent as desired pitch and roll angles for translational movements,
angular rate for yaw adjustment, and velocity for altitude adjustments. These high
level commands are then translated by the on-board controller into rotor speed adjustments. More difficult actions, such as takeoff and landing, are completely handled
by the on-board control. When the takeoff command is issued, the AR.Drone quickly
takes off to a default height and hovers before accepting any movement commands.
3.2.2
Limitations
While the AR.Drone is a great platform for many research projects, it does have
limitations when compared to hobbyist or professional-grade quadcopters.
The hardware design allows for very little customization. While most professionalgrade quadcopters have ports for adding additional sensors, there is no straightfor15
27. ward way to add any extra electronics to the AR.Drone. Even if it were possible to
customize, the AR.Drone is designed to only lift its own weight, with most hobbyists
claiming to add a maximum of 100 grams payload before the flight characteristics are
significantly affected [2]. Professional quadcopters of a similar size are typically able
to fly with payloads between 400 and 600 grams [7].
Another limitation of the AR.Drone is the flight time. The maximum flight time
of the AR.Drone is only around 15 minutes, with the additional weight of the indoor
hull bringing this down to 10-12 minutes. Similar sized quadcopters, such as the
Mikrocopter, typically achieve around 30 minutes of flight time, depending on weight
and battery size [7].
Additionally, the AR.Drone has no built in GPS system, meaning that the onboard sensors provide only relative measurements. This leads to errors due to drift
and makes flying autonomously in a precise pattern an extremely challenging task.
A downside of the extra protection offered by the AR.Drone hull is that it is
much larger than the hull of the Mikrocopter or similar quadcopters. This results
in a larger surface area that can be affected by the wind, making outdoor flights
particularly difficult even with the on-board stabilization.
3.3
3.3.1
System Architecture
Robot Operating System
The Robot Operating System (ROS) is used to organize the interaction between programs and libraries. Although not an “operating system” in the traditional sense,
ROS is an open source communication layer used in a wide variety of robotics applications. Supported by Willow Garage, ROS has a large amount of documentation
and packages which can handle many common tasks in robotics. Many of these packages are hardware-independent, meaning that they can be quickly implemented on an
16
28. Figure 3.6: System architecture diagram.
array of different robotics systems. ROS also provides a standard message protocol,
allowing packages to work together in a language-agnostic manner [27].
3.3.2
ardrone autonomy
ardrone autonomy is an open-source ROS wrapper for the Parrot AR.Drone SDK
developed in the Autonomy Lab at Simon Fraser University [3]. This package handles
the interface of navdata messages, video feeds, and control commands between ROS
and the AR.Drone. This allows the use of many existing ROS packages in localizing
and controlling the quadcopter.
17
29. 3.3.3
ARToolKit
ARToolKit is an open-source software library designed to be used for creating augmented reality applications. Developed by Dr. Hirokazu Kato and maintained by
the HIT lab at the University of Washington, ARToolKit uses computer vision algorithms to identify visual tags, such as the one in Figure 3.7, and calculate the relative
position and orientation.
For augmented reality applications, this can be used to superimpose 3D graphics
onto a video feed in real time based on the tag position and orientation. In this system,
the tags will be used to generate global positioning estimates for the quadcopter by
combining estimated tag transformations with known tag locations.
Specifically, ARToolKit will be implemented using a slightly modified version of
the ar pose library, a ROS wrapper for ARToolKit developed by Ivan Dryanovski et
al. at the CCNY robotics lab [4].
Figure 3.7: Augmented reality tag with ID 42.
18
30. 3.3.4
Localization
The purpose of the localization module is to produce an estimated pose of the
quadcopter. The localization module receives the navdata and video feed from the
AR.Drone via ardrone autonomy. The localization module then sends the video to
ar pose in order get any augmented reality tag transformations. By using measurements from the navdata messages and detected tags from ar pose, the localization
module produces a global pose estimation of the quadcopter.
3.3.5
Controller
The purpose of the controller is to produce the control inputs which move the quadcopter from its current pose to a desired pose. The controller receives the estimated
pose from the localization module and sends the flight commands to the AR.Drone
via ardrone autonomy.
3.3.6
3D Reconstruction Software
After the flight is complete, 3D reconstruction software is used to turn the collection of images into a 3D model. There are multiple off-the-shelf libraries, including
open-source Clustering Views for Muti-View Stereo (CVMS) and commercial Agisoft
Photoscan [16, 1].
19
31. Chapter 4
Localization
4.1
Problem Description
For a robot to perform precise maneuvers, it must first have an understanding of
its position and orientation, or pose. This is the problem of localization. By using
a variety of sensor measurements, the localization algorithm must produce a single
estimate of the quadcopter’s pose for use in the controller.
4.2
Considerations of the AR.Drone
As this project uses the AR.Drone 2.0, the localization algorithm is built around the
capabilities and limitations of this hardware. Considering the low load-capacity of
the AR.Drone and the fact that this project aims to use off-the-shelf hardware, the
localization algorithm may only use the sensors included in the AR.Drone.
Therefore, the localization must produce an estimate by using some combination
of the forward camera, downward camera, accelerometer, gyroscope, magnetometer,
ultrasound altimeter, and pressure altimeter.
20
32. 4.3
Localization Methods
Localization for mobile robots fall into three main categories: Kalman, Grid-Based
Markov, and Monte Carlo.
4.3.1
Extended Kalman Filter
The extended Kalman Filter (EKF) is used extensively in mobile robot localization.
The EKF is a nonlinear version of the Discrete Kalman Filter first introduced by
Rudolf Emil Kalman in 1960 [19]. The EKF linearizes about the current mean and
covariance in method similar to a Taylor series. The EKF contains two stages, a
prediction step and a correction step. In the prediction step, the new state and
error covariance are projected using proprioceptive sensors. Then, in the correction
step, the estimate and error covariance are updated in response to exteroceptive
sensor measurements [33]. Kalman filters have the disadvantage of only being able to
approximate normal distributions.
4.3.2
Grid-Based Markov Localization
Grid-based localization uses a “fine grained” grid approximation of the belief space,
the space that covers all of the potential position and orientations of the robot [15].
For each time step, the probability of a robot being in any one of the grid cells is
calculated, first by using odometry and then by exteroceptive sensors such as range
finders. While relatively straightforward to implement, this process has many drawbacks. First of all, picking the size of the grid cells can be difficult. If the cells are
too large, then the estimate will not be precise. However, if the cells are too small,
the algorithm will be slow and very memory-intensive. Additionally, grid-based localization performs poorly in higher-dimensional spaces, as the number of cells grows
exponentially with the number of dimensions.
21
33. 4.3.3
Particle Filter
A particle filter is a type of Monte Carlo simulation with sequential importance sampling [9]. Essentially, a particle filter keeps track of a large number of particles, with
each particle representing a possible pose estimation, in order to represent the probability distribution of the belief space. The particle filter typically moves these particles
using proprioceptive sensor measurements convolved with Gaussian noise [15]. Then,
the particles are weighted with respect to exteroceptive sensor measurements. The
particles are then randomly resampled based on these weight values, producing a
corrected distribution of particles.
There are many advantages to using a particle filter. Due to the way the process creates an approximation by a set of weighted samples, without any explicit
assumptions of the approximation’s form, it can be used in applications where the
assumption of Gaussian noise doesn’t necessarily apply [9].
4.4
Particle Filter with Augmented Reality Tags
Algorithm 1 Particle Filter with Augmented Reality Tag Correction
1: for all t do
2:
if buffer full() then
3:
predict(∆t, vx , vy , altd, θ)
4:
end if
5:
if recieved tag() then
6:
correct(P)
Transformation matrix from camera to marker
7:
end if
8:
xest ←get estimate()
9: end for
The particle filter has been chosen for this project due to its flexibility, ease of
implementation, and performance. In typical implementations of particle filters for
mobile robots, the prediction step uses proprioceptive sensors, such as accelerometers,
22
34. gyroscopes, and rotary encoders. Then, this estimate is typically corrected by using
exteroceptive sensors, such as infrared, laser, or ultrasound.
However, due to the lack of horizontal range sensors, this particle filter uses a
different division between the prediction and correction steps. The prediction step
uses the stock configuration of sensors in the AR.Drone, specifically the fused velocity,
gyroscope, and ultrasound altimeter measurements. Then, the correction step uses
an estimated global position, as determined by augmented reality tags, to resample
the particles.
4.4.1
Buffering Navdata
The localization module receives navdata at 50Hz. Depending on the number of
particles and computational resources available to the localization algorithm, this
can be a higher rate than the particle filter can run the propagation step. Reducing
the rate at which propagation is run allows the particle filter to use more particles,
providing better coverage of the pose estimate space and increasing the likelihood of
convergence.
Additionally, while a more rapidly updated pose estimate would be preferable,
the accuracy of the measurement is not such that it is especially useful to update at
a rate of 50Hz. For example, the maximum velocity that the quadcopter should ever
achieve in this system is around 1m/s. In .02 seconds, the quadcopter will have only
moved 20mm, or 2cm. Considering that the desired accuracy of the localization is
on the order of tens of centimeters, updating the estimated pose at a reduced rate is
acceptable.
As the navdata is recieved, the navdata measurements, such as velocity and yaw,
are added to a buffer of size n. Every n measurements, the prediction step is called
with the simple moving average of the previous n values and the sum of the ∆t values
since the last call to propagate. This results in an update rate of
23
50
Hz.
n
35. Although the buffer size is currently a hard-coded value, this could be dynamically
changed based on the amount of delay between receiving navdata measurements and
processing them in the prediction step. This would result in the highest propagate
rate possible given a fixed number of particles. On the other hand, the buffer size
could remain fixed with the particle filter adjusting the total number of particles,
allowing for the best possible coverage at a guaranteed update rate.
4.4.2
Initialization
The particle filter is initialized by creating a set of N particles. Each of these particles
represents a potential pose in the belief space. In particular, each particle at a given
time step t is of the form:
xt
yt
xt =
z
t
θt
Where xt , yt , zt are the position, in mm, and θt is the heading, in degrees, of the
particle in the global coordinate space. As the low level stabilization and control of
the quadcopter is handled by the on-board processor, it is not necessary to include
roll and pitch in the pose estimate of the quadcopter since these are not needed for
the high level control. The entire set of particles of size N at time step t can be
described as:
Xt = [xt [0], xt [1], ..., xt [N ]]T
Additionally, for each time step, there is a set of associated weights
Wt = [wt [0], wt [1], ..., wt [N ]]T
24
36. Figure 4.1: AR.Drone coordinate system [21].
Normalized, such that
N
wt [i] = 1
i=0
Coordinate Frame Conventions
The coordinate frame follows the standard set by the ardrone autonomy package. As
shown in Figure 4.1, the coordinate frame is right-handed, with positive x as forward,
positive y as left, and positive z as up. In terms of rotation, a counter clockwise
rotation about an axis is positive. The heading value ranges from -180 degrees to 180
degrees, with 0 centered along the x-axis. When the particle filter is initialized, the
global coordinate frame is set equal to the first instance of the local coordinate frame.
4.4.3
Prediction Step
The first component of the particle filter is the prediction step. In this step, the
position of every particle is updated by using the sensor measurements contained in
the navdata messages. Specifically, the prediction step uses the elapsed time, ∆t,
25
37. Algorithm 2 Prediction Step
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
function Predict(∆t, vx , vy , θ, zultra )
∆θ ← θ − θt−1
for i = 0 → N do
∆θnoise ← randn(∆θ, σθ )
θt [i] ← θt−1 [i] + ∆θnoisy
(vx,global , vy,global ) ← transform(vx , vy , θt [i])
vx,global,noisy ← randn(vx,global , σv )
vy,global,noisy ← randn(vy,global , σv )
xt [i] ← ∆t ∗ vx, global, noise + xt−1 [i]
yt [i] ← ∆t ∗ vy, global, noise + yt−1 [i]
zt [i] ← randn(zultra , σultra )
end for
end function
fused velocity measurements, vx and vy , yaw reading, θ, and ultrasound altimeter
value, zultra .
The prediction step then subtracts the value of θt−1 , the value of the overall θ estimate from the previous step, from θ to produce ∆θ. Then, for every particle i, a new
pose xt [i] is generated using the previous pose, xt−1 [i] and the values ∆t, vx , vy , ∆θ,
and zultra .
Adding Noise to Sensor Measurements
In order to update the pose estimation for a given particle, noise is first added to
the sensor measurements in so as to model sensor inaccuracies. By adding noise to
the particles, the distribution of particles should expand to include all of the possible
belief states.
For each sensor value, a new “noisy” value is generated by sampling from a Gaussian distribution with a mean of the sensor reading and a standard deviation, σ, which
models the accuracy of the sensor.
26
38. Converting Local Velocity to Global Velocity
In order to update the pose of each particle in the global frame, the values vx,noisy
and vy,noisy , must be transformed from the local coordinate frame of the quadcopter
to the global coordinate frame. First, on Line 5, the new heading of the particle is
determined by adding the value ∆θnoisy to the estimated heading of the last time step,
θt−1 [i].
Then, on Line 6, the global velocity is found by
vx,global,noisy cos(θt [i]) −sin(θt [i]) vx,noisy
=
vy,global,noisy
sin(θt [i]) cos(θt [i])
vy,noisy
Position Update
Once the velocity is in the global coordinate frame, Euler integration is used on Line 9
to determine the new position of the particle.
4.4.4
Correction Step
Determining Global Position from Augmented Reality Tag
The general concept in the correction step is to use a known marker position and a
calculated camera-to-marker transformation in order to determine the global position
of the quadcopter. Using this calculated global position, the particles, weighted by
their similarity to the calculated global position, are resampled to correct for the drift
accumulated by using local measurements.
The first step is to determine the pose of the quadcopter in the global coordinate
frame using augmented reality tags. These tags, each with unique binary patterns,
are placed in known positions and orientations (e.g. axis aligned in the corners of a
5m square). These positions are published as ROS static transforms, allowing them
27
39. Algorithm 3 Correction Step
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:
37:
38:
39:
40:
41:
function Correct(marker id, P)
P is transformation from camera to marker
M ← lookupTransform(world, marker id)
B ← lookupTransform(base, camera)
T ← MPT BT
(xcorrect , ycorrect , zcorrect ) ← getTranslation(T)
θcorrect ← getHeading(T)
(xcorrect ← [xcorrect , ycorrect , zcorrect , θcorrect ]T )
wsum ← 0
for i = 0 → N do
Weighting Particles
2 + (y [i] − y
2 + (z [i] − z
2
dist ← (xt [i] − xcorrect )
t
correct )
t
correct )
θdiff ← |θt [i] − θcorrect |
wdist ← pdf(dist, 0, σx,artag )
wθ ← pdf (θdif f , 0, σθ,artag )
wt [i] ← wdist + wθ
wsum ← wsum + wt [i]
end for
for i = 0 → N do
wt [i] ← wt [i]/wsum
end for
Normalizing Weights
walkerInitialize(Xt , Wt )
for i = 0 → N do
randVal ← rand(0, 1)
if randVal < resampleThreshold then
xnew = xcorrect
else
xnew = walkerSelect()
end if
Xtemp ← xnew
end for
Xt ← Xtemp
Random Resampling
Weighted Sampling of Particles
Update Particles
end function
28
40. to be retrieved by a lookupTransform() command. When a tag is detected by the
ar pose library, a marker is published containing the marker id of the tag and the
transformation matrix from the camera to the tag, P. Then, the transformation
matrices from the world frame to the tag, M, and from quadcopter base to camera, B, are retrieved using lookupTransform(). Through matrix multiplication, the
transformation from world to quadcopter, T, can be obtained as follows:
T = MPT BT
On lines 6 and 7, the ROS transformation library is used to derive the estimated
position and orientation of the quadcopter, xcorrect , from the transformation matrix
T.
Weighting Particles
Each particle is assigned a weight by comparing the particle pose, xt [i], to the pose
estimate generated using the augmented reality tag, xcorrect . First, the Euclidean distance between the two positions and the difference between θ estimates are calculated
on lines 13 and 14.
Then, on line 15, position and orientation weights are determined by inputing
dist and diff into zero-mean probability density functions with standard deviations
derived from testing.
pdf (x, µ, σ) is defined as:
1 x−µ 2
1
√ e− 2 ( σ )
σ 2π
Where
29
41. x
Input value
µ
Mean of normal distribution
σ
Standard deviation of normal distribution
Random Resampling
Every time the correction step is called, a small percentage of particles are replaced
using, xcorrect , the pose as estimated by using the augmented reality tags. As the
quadcopter will often fly for several meters without picking up a tag, the particles
will become very spread out to account for sensor drift. Without replacement, there
is a chance that the quadcopter would not be able to recover from a large amount of
drift as no particle would be close enough to the actual position to accurately correct.
By resampling, the particles will more rapidly cluster when the quadcopter is over a
tag. There is a trade-off of doing this, however. On rare occasions, the ar pose library
mistakes the marker id of the tag, resulting in an incorrect transformation. If the
resample rate is too high, then a large number of particles will essentially “teleport” to
the wrong position, resulting in a large jump of estimated pose. If this resample rate is
small enough, then it will still accomplish the goal of quickly clustering particles when
above tags, but will be robust enough to recover from incorrect tag transformations.
Weighted Sampling of Particles
The remaining particles are chosen using a weighted sampling. Essentially, particles
that are closest to xcorrect will be chosen with a higher probability, allowing other
particles to die out. This is often called “survial of the fittest” sampling. The simple
method of performing weighted sampling would be to keep an array of cumulative
weights. This way, a random number generated between 0 and 1 could be used to
search through the array and pick particles with the correct distribution. This would
result in a lookup time of O(log N ). However, using Walker’s alias method, a small
30
42. amount of preprocessing can reduce these lookups to O(1) [32]. As there are many
particles and the correction step can be called at a very high frequency, it is important
to reduce the lookup time as much as possible.
4.4.5
Pose Estimate from Particles
At the end of call to the localization algorithm, a new pose estimate is generated using
the particle distribution. This is done by taking a linear combination of all of the
particles. This method works reasonably well for this application as the particles tend
to stay in a single cluster. In applications where multiple belief clusters are likely to
exist, such as in a particle filter with an unknown start position, this solution would
not be sufficient and a technique such as mean shift clustering should be used.
31
43. Chapter 5
Results and Analysis
5.1
Sensor Testing
In order to correctly apply Gaussian noise to the sensor measurements during the
prediction step, we need to model the performance of the sensors. Additionally, we
must determine the accuracy of the augmented reality tag detection for use in the
weighting function of the correction step.
5.1.1
Gyroscope
The gyroscope was tested by placing the AR.Drone on a flat surface and recording
the measured yaw angle. These readings were then centered about the mean and are
shown in Figure 5.1.
5.1.2
Ultrasound Altimeter
The best way to test the ultrasound altimeter would be to attach the AR.Drone
to a stationary rig and take measurements at several different heights within the
operating range. Unfortunately, the AR.Drone SDK does not provide direct access to
the ultrasound sensor and returns a value of zero if the quadcopter is not currently
32
44. Figure 5.1: Stationary gyroscope test.
flying. Therefore, the ultrasound altimeter was tested by hovering the quadcopter at
a constant height in a controlled, indoor environment. While some of the noise in
the sensor measurement is due to the quadcopter actually changing altitude, these
measurements (Figures 5.2a and 5.2b) still give us a general idea of the accuracy of
the sensor.
(a) Hovering at approximately 700mm
(b) Hovering at approximately 2000mm
Figure 5.2: Hovering ultrasound test.
33
45. 5.1.3
Augmented Reality Tag Detection
Detection Range
The quadcopter was mounted horizontally to test different aspects of the AR tag
detection (Figure 5.3). In determining the size of tag to use, there were two main
considerations. The smaller the tags, the earlier they could be detected in the takeoff
sequence, which is when visual localization performs poorly and it is especially important to use a tag for correction. On the other hand, if the tags were too small,
then the altitude at which it could be detected would be too limited. We chose a tag
size of 135cm, which we found to have a detection range of approximately 30cm to
4m (Figure 5.4). In order to increase the operational range of the AR tag correction,
we could potentially use a combination of tags of different sizes.
Figure 5.3: AR tag testing setup.
Tag Position Variance vs. Distance
In order to determine the variance of the probability density function used in the
correction step, we tested the precision of the AR tag estimation algorithm at several
different distances. By measuring the position of a stationary tag, we can see that
the variance of measurements increases with the distance (Figure 5.5). Interestingly,
34
46. (a) Minimum distance for tag detection: (b) Maximum distance for tag detection:
30cm
4m
Figure 5.4: Tag detection range.
the variance along the z, or vertical axis, is substantially lower than that in the x or
y axes.
Figure 5.5: Tag estimated position standard deviation.
Tag Detection Space
The tag detection space, which is the 3D volume above each tag in which it can be
detected, is determined both by the aspect ratio and resolution of the camera and the
size of the AR tag. The tag is first fully in the frame at 30cm. At an altitude of 3.5m,
the area in which the tag can be detected is approximately a 2.8m by 1.8m rectangle.
A result of this is that it is easier to “hit” a tag at a higher altitude. Therefore, it is
probably best to immediately fly to a higher altitude after taking off, increasing the
chance that the quadcopter will see the first tag.
35
47. Figure 5.6: Tag detection space.
Tag Detection Rate
We found that stationary tags were detected with a very high likelihood throughout the entire operational range. At every distance tested, the tags were correctly
identified in over 98% of frames.
Sources of Tag Error
We found several sources of error which could prevent the detection of AR tags. First,
the movement of the quadcopter sometimes causes the camera image to be blurred
as seen in Figure 5.7. Additionally, in environments with bright overhead light, such
as in direct sunlight, the quadcopter would cast a shadow over the tag, preventing it
from being detected. Finally, we found that the auto-exposure of the on-board camera
would often cause tags to overexpose if the background were dark, as demonstrated
in Figure 5.8.
36
48. Figure 5.7: Tag motion blur.
Figure 5.8: Effect of auto-exposure on tag contrast.
5.2
Localization
In order to test the localization algorithm, the quadcopter was manually flown in a
rectangular pattern measuring 3.5m by 2.5m. AR tags were placed at each of the
corners of this pattern. Figures 5.9 through 5.13 show a comparison of localization
using only visual odometry and localization using the particle filter with augmented
tag correction.
The visual odometry measurement suffers from substantial drift, estimating that
the quadcopter landed over 2m away from its actual landing location. On the other
hand, the particle filter localization is able to correct for drift and accurately reflects
the path of the flight.
37
54. Chapter 6
Conclusion
This thesis proposed a method for 3D model generation using autonomous quadcopters and multi-view stereo. Such a system would be portable, cheap, and easily
deployable in a range of applications such as archeology and video game development.
Specifically, this thesis presented a localization method for low-cost quadcopters using a particle filter with augmented reality tags. This system was shown to perform
substantially better than using local sensor measurements alone.
6.1
Future Work
The next step in creating this system for model generation would be to fully integrate
this localization algorithm with a controller and path planner. While much effort was
put into integrating this work and the controller produced by Sarah Tang, various
hardware issues prevented fully autonomous flight from being achieved.
Once basic waypoint tracking is implemented, the system could autonomously
generate the path it flies around the object so as too improve coverage and fly around
irregularly shaped objects. Eventually, such a system should be packaged in a way
such that it can easily be used with the AR.Drone with very little setup, bringing
43
55. the ability to quickly develop 3D models of large objects to researchers across many
fields.
44
56. Bibliography
[1] Agisoft photoscan. http://www.agisoft.ru/products/photoscan.
[2] Ar.drone forum: Weight capacity. http://forum.parrot.com/ardrone/en/
viewtopic.php?id=4545.
[3] ardrone autonomy. https://github.com/AutonomyLab/ardrone_autonomy.
[4] ar pose. http://www.ros.org/wiki/ar_pose.
[5] Artoolkit. http://www.hitl.washington.edu/artoolkit/.
[6] Bundler: Structure from motion (sfm) for unordered image collections. http:
//phototour.cs.washington.edu/bundler/.
[7] Mk-quadrokopter. http://www.mikrokopter.de/ucwiki/en/MK-Quadro.
[8] Parrot Press Photos. http://ardrone2.parrot.com/photos/photo-album/.
[9] Hamza Alkhatib, Ingo Neumann, Hans Neuner, and Hansj¨rg Kutterer. Como
parison of sequential monte carlo filtering with kalman filtering for nonlinear
state estimation. In Proceedings of the 1th International Conference on Machine
Control & Guidance, June, pages 24–26, 2008.
[10] C. Bills, J. Chen, and A. Saxena. Autonomous mav flight in indoor environments
using single image perspective cues. In Robotics and Automation (ICRA), 2011
IEEE International Conference on, pages 5776–5783, 2011.
[11] Pierre-Jean Bristeau, Franois Callou, David Vissire, and Nicolas Petit. The
navigation and control technology inside the ar.drone micro uav, 2011.
[12] Nick Dijkshoorn. Simultaneous localization and mapping with the ar.drone, 2012.
[13] J. Engel, J. Sturm, and D. Cremers. Accurate figure flying with a quadrocopter
using onboard visual and inertial sensing. IMU, 320:240.
[14] J. Engel, J. Sturm, and D. Cremers. Camera-based navigation of a low-cost
quadrocopter. In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pages 2815 –2821, oct. 2012.
45
57. [15] Dieter Fox, Sebastian Thrun, Wolfram Burgard, and Frank Dellaert. Particle
filters for mobile robot localization, 2001.
[16] Yasutaka Furukawa. Clustering views for multi-view stereo (cmvs). http://
www.di.ens.fr/cmvs/.
[17] S. Gupte, P.I.T. Mohandas, and J.M. Conrad. A survey of quadrotor unmanned
aerial vehicles. In Southeastcon, 2012 Proceedings of IEEE, pages 1–6, 2012.
[18] A. Irschara, V. Kaufmann, M. Klopschitz, H. Bischof, and F. Leberl. Towards
fully automatic photogrammetric reconstruction using digital images taken from
uavs. In Proceedings of the ISPRS TC VII Symposium100 Years ISPRS, 2010.
[19] Rudolph Emil Kalman et al. A new approach to linear filtering and prediction
problems. Journal of basic Engineering, 82(1):35–45, 1960.
[20] Hirokazu Kato. Artoolkit: library for vision-based augmented reality. IEICE,
PRMU, pages 79–86, 2002.
[21] Tom´ˇ Krajn´ Vojtˇch Von´sek, Daniel Fiˇer, and Jan Faigl. Ar-drone as
as
ık,
e
a
s
a platform for robotic research and education. In Research and Education in
Robotics-EUROBOT 2011, pages 172–186. Springer, 2011.
[22] K.Y.K. Leung, C.M. Clark, and J.P. Huissoon. Localization in urban environments by matching ground level video images with an aerial image. In Robotics
and Automation, 2008. ICRA 2008. IEEE International Conference on, pages
551 –556, may 2008.
[23] Marc Levoy, Kari Pulli, Brian Curless, Szymon Rusinkiewicz, David Koller,
Lucas Pereira, Matt Ginzton, Sean Anderson, James Davis, Jeremy Ginsberg,
Jonathan Shade, and Duane Fulk. The digital michelangelo project: 3d scanning of large statues. In Proceedings of the 27th annual conference on Computer
graphics and interactive techniques, SIGGRAPH ’00, pages 131–144, New York,
NY, USA, 2000. ACM Press/Addison-Wesley Publishing Co.
[24] Joao Pedro Baptista Mendes. Assisted teleoperation of quadcopters using obstacle avoidance. 2012.
[25] Rudy Negenborn. Robot localization and kalman filters. PhD thesis, Citeseer,
2003.
[26] Paul Pounds, Robert Mahony, and Peter Corke. Modelling and control of a
quad-rotor robot.
[27] Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy
Leibs, Rob Wheeler, and Andrew Y Ng. Ros: an open-source robot operating
system. In ICRA workshop on open source software, volume 3, 2009.
46
58. [28] Szymon Rusinkiewicz, Olaf Hall-Holt, and Marc Levoy. Real-time 3d model
acquisition, 2002.
[29] R. Steffen and W. F¨rstner. On visual real time mapping for unmanned aerial
o
vehicles. In 21st Congress of the International Society for Photogrammetry and
Remote Sensing (ISPRS), pages 57–62, 2008.
[30] Sarah Tang. Vision-based control for autonomous quadrotor uavs, 2013.
[31] Jing Tong, Jin Zhou, Ligang Liu, Zhigeng Pan, and Hao Yan. Scanning 3d
full human bodies using kinects. Visualization and Computer Graphics, IEEE
Transactions on, 18(4):643–650, 2012.
[32] Alastair J. Walker. An efficient method for generating discrete random variables
with general distributions. ACM Trans. Math. Softw., 3(3):253–256, September
1977.
[33] Greg Welch and Gary Bishop. An introduction to the kalman filter, 1995.
[34] Teddy Yap, Mingyang Li, Anastasios I. Mourikis, and Christian R. Shelton. A
particle filter for monocular vision-aided odometry.
47