Σχεδιασµός και υλοποίηση αρθρωτού συστήµατος υπολογισµού τρισδιάστατης θέσης κινούµενου αντικειµένου µε χρήση ετερογενών αισθητήρων και ϕίλτρων

1.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Σχεδιασμός και υλοποίηση αρθρωτού συστήματος υπολογισμού τρισδιάστατης θέσης κινούμενου αντικειμένου με χρήση ετερογενών αισθητήρων και φίλτρων Διπλωματική Εργασία Εκπόνηση: Θεόδωρος Παπαφωτίου AEM: 9708 Επίβλεψη: Καθηγητής Ανδρέας Συμεωνίδης Ερευνητής Εμμανουήλ Τσαρδούλιας

2.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Presentation Structure 2 Introduction Methodology Conclusions Experiments

3.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Sensors calibration on robots/devices by using external local. systems Improve user experience with more accurate human tracking Problem Description 3 Why Indoor Localization? 5 1 3 2 4 Navigation Robot & human navigation in indoor spaces Tours Guided tours in enclosed spaces (i.e. museums) LBS Location-based services in airports, malls etc Calibration VR

4.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Thesis Purpose 4 01 02 03 04 Develop a complete, modular and user-friendly indoor localization proposal, regardless the installation location Integrate as many as possible indoor localization methods and review them both individually and in combinations Develop custom software solutions for methods implementations, fusion techniques and optimization Develop custom design solutions for all systems included in the project, that can be 3D-printed

5.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Indoor Localization Methods 5 ● Particle Filter ● Kalman Filter ● Landmark Detection ● Optical Flow ● Dead Reckoning ● Signal TX/RX

6.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Software Architecture Inputs provided by the systems’ equipment Perception of the input data to estimate the moving-device’s 3D-position & Filtering Fusion of the estimated data from each method based on the selected methods from the user Output to the user via broker and configuration modifications to the systems’ equipment 6

7.

8.

9.

10.

11.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Localization Methods Depth Frame Deprojecting Fingerprinting Dead reckoning N-angulation Optical Flow Calculation 7 LED pattern detection Features Extraction & Tracking Perspective Projection Model 9-DOF IMU data acquisition UWB-nodes cluster RSSI calculation Camera Server UWB Server IMU Odometry Visual Odometry

12.

13.

14.

15.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Fusion Methods Lab Coordinates Outputs of N Localization Methods [ Xi Yi Zi vxi vyi vzi ], i = 1, 2 … N Kalman FIlter k-Means Weighted Average Median Selected Fusion Method by user Fused Estimation Output [ Xf Yf Zf vxf vyf vzf ] If N = 1, Fusion process is skipped We can implement a “weighted” kalman filter by adjusting its matrices correspondingly 8

16.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Moving Localization Device • LED pattern • UWB tag • ToF Ranging Sensor • 9-DoF IMU • PiCamera 3 • LCD screen • Power Sensor • RaspberryPi 4B • Powerbank X Y 9

17.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Moving Device - IMU Odometry X Y Calibrate IMU Sensors (accelerometer/gyroscope/ magnetometer) Acquire sensors data (9x1 vector) Insert data vector into a Kalman Filter Retrieve Position, Velocity & Acceleration of the IMU from Kalman Estimation output 10 Relative Position (!)

18.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Moving Device - Visual Odometry Capture two sequential frames [Ft], [Ft+1] and undistort them Detect features on Ft Track those features to Ft+1 Compute the Essential Matrix Estimate Rotation matrices & Translation vectors Extract X, Y, Z World Coordinates 11 Relative Position (!)

19.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Calculate relative camera positions assuming their height known by placing the device on pattern points External System - Camera Server Camera Server 1 Camera Server 2 Pan-Tilt Arm RaspberryPi 4B computational unit D455 Intel RealSense Color/Depth Camera Rotate camera on Pan and Tilt to keep the tracked device constantly inside the “active” area of the frame 12

20.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Calculate relative camera positions assuming their height known by placing the device on pattern points External System - Camera Server Camera Server 1 Camera Server 2 Pan-Tilt Arm RaspberryPi 4B computational unit D455 Intel RealSense Color/Depth Camera Rotate camera on Pan and Tilt to keep the tracked device constantly inside the “active” area of the frame 12

21.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 External System - Camera Server Capture a frame and undistort it Separate the areas with HSV values in a range, specified by the tracked pattern color Detect the contours in the masked areas and select the contour with the mostly matching pattern shape Change the frame colorspace from RGB to HSV 13

22.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Fingerprinting Perspective Projection Model Depth Frame Point Deprojection External System - Camera Server Multi-output Regression with Support Vector Regression (SVR) as the base model 14

23.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Fingerprinting Perspective Projection Model Depth Frame Point Deprojection External System - Camera Server Multi-output Regression with Support Vector Regression (SVR) as the base model 14 Rotation (!)

24.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 External System UWB-nodes 25 Tag Anchor X Initiator Publish to Broker S1 S2 S3 S4 3D Position Estimation Signal transmitted from moving device UWB tag N-angulation applied using the Received Signal Strength (RSS) from 4 anchors at each timestep 16

25.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Graphical User Interface Framework for app dev. Database for data handling 17

26.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Experiments Environment Laboratory of ECE AUTH Research Student Team “VROOM” ● 2x Camera Servers placed diagonally at the area ● 6x UWB Anchors covering the area of interest ● 1x Computer for hosting the user app and the database 18

27.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Experiments Environment Points for moving experiments Points for static experiments 19

28.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 All experiments Moving experiments (2D) Static experiments Total Displacement experiments ● Individual Methods ○ UWB system ○ Cameras system (Color) ○ Cameras system (Depth) ○ IMU Odometry ○ Visual Odometry ● Combinations ○ UWB & IMU Odometry ○ All Local methods ○ All Global methods ○ All methods With & Without Smoothing Filter Only With Smoothing Filter ● 2D movement (Ζ = ct) ○ UWB system & Cameras system (Color) ○ UWB system & Cameras system (Depth) ● 3D movement (Z = var) ○ UWB system & Cameras system (Color) ○ UWB system & Cameras system (Depth) ● All methods enabled - 2D movement ● All methods enabled - 3D movement 20

29.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Results - Moving Experiments Without output filter With output filter Kalman Filter & Rolling median 21

30.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Results - Moving Experiments Absolute Only Methods All Methods UWB System & IMU Odometry 22

31.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Conclusions Absolute Localization (AL) is more accurate compared to Reference Localization (RL) RL is highly affected by sharp alterations of the moving direction Camera Server localization methods provide the best standalone localization results The combination of all AL presents the best performance through all experiments The best low-cost combinations include the UWB method standalone or its combination with the IMU Odometry method The Visual Odometry poses the greatest errors due to the lack of enough visually trackable features of the environment 23

32.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Results - Static Experiments Z = ct Z = var UWB & CS-Color 24 UWB & CS-Depth 13cm 22cm 22cm 19cm

33.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Results - Static Experiments Z = ct Z = var UWB & CS-Color 24 UWB & CS-Depth

34.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Future implications Use of BLE transmitters for additional low- cost tracking Simplification of systems installation at a laboratory by managing all the labs through a common broker, database and application that will be hosted remotely Custom equipment (PCB, connectors) for the moving- device to significantly reduce its size Various patterns on the moving device to be detected via the Camera Server on various device orientation-angles for optimal & robust tracking Fine-tuning of localization methods and fusion techniques by extensive testing using robots, drones and humans in various indoor environments Integrate Neural Networks in the Pattern Detection, Noise Filtering and Fusion processes Integration of additional localization methods to increase the range of this project’s applications 25

35.

36.

37.

38.

39.
Design and implementationof an integrated and modular system for 3D position calculation of a moving object using heterogeneous sensors and filters 01/04/2024 Thank you! Questions? 26

Σχεδιασµός και υλοποίηση αρθρωτού συστήµατος υπολογισµού τρισδιάστατης θέσης κινούµενου αντικειµένου µε χρήση ετερογενών αισθητήρων και ϕίλτρων

More Related Content

Similar to Σχεδιασµός και υλοποίηση αρθρωτού συστήµατος υπολογισµού τρισδιάστατης θέσης κινούµενου αντικειµένου µε χρήση ετερογενών αισθητήρων και ϕίλτρων

More from ISSEL

Recently uploaded

Σχεδιασµός και υλοποίηση αρθρωτού συστήµατος υπολογισµού τρισδιάστατης θέσης κινούµενου αντικειµένου µε χρήση ετερογενών αισθητήρων και ϕίλτρων