Download to read offline
Uploaded bySURF Events
Biking on the edge - Jerome Mies - SRD23
Jerome Mies presented on using edge computing and AWS IoT Greengrass to monitor street conditions in Amsterdam via bicycle inspections. The project aims to cover the entire city with LiDAR and video AI to provide an accurate overview of deficiencies. Edge devices like NVIDIA Jetson Nano mounted on bicycles can perform real-time image analysis and inference to detect issues like light posts, garbage, and road damage. AWS Greengrass enables managing remote edge devices and orchestrating event-driven workflows. Models are trained with Amazon SageMaker and optimized for battery-powered edge applications. The cloud allows scalability to multiple edge devices for real-time data processing and updates to improve public safety.
More Related Content
Similar to Biking on the edge - Jerome Mies - SRD23
Biking on the edge - Jerome Mies - SRD23
- 1. Jerome Mies (Lecturer/Researcher at Hogeschool van Amsterdam) 23 May, 2023 Biking on the edge Monitoring the conditions of streets in Amsterdam with AWS IoT Greengrass
- 2.
- 3. Overview • What arewe working on? • What is edge computing? • How do we use edge computing? • How do we use AWS? • How can we integrate AWS? • Example models
- 4. Together improving thewellbeing in public spaces Hogeschool van Amsterdam Centre of expertise for applied artificial intelligence Smart Asset Management Lab SURF SURF is the collaborative organisation for IT in Dutch education and research Municipalities Velotech Solutions Sustainable object detections in the public space
- 5. Conditions of theassets in the city impact our safety and wellbeing Light posts Traffic lights Garbage Traffic signs Road markings Road surface Trees 1
- 6. Our ambition isto cover the complete city maintenance with LiDAR & video AI Video image recognition LiDAR mapping Light posts Traffic lights Garbage Traffic signs Road markings Road surface Trees 1
- 7. Provide an accurate& complete overview of deficiencies in the city through eco-friendly inspections 3 Detailed routes per area Image analysis using algorithms Registration in eform-app Bicycles can go where cars can’t Validation of the inspections Integration with maintenance subcontractor systems Key features of the inspections Inspections (periodically) of designated areas by bicycle/vehicle Number of identified objects, defects & exact GPS locations Reporting & dashboard that is linked with the existing workflows Purpose Improve social & traffic safety Minimize liability risks Preventing peak workload for maintenance parties in winter
- 8. Route planning Schedule riders Data gathering & mobile mapping Before& after inspections Data analysis Reporting municipality Reporting maintenance contractor Green and fast methods to get insights in the public space by using bikes & technology AI 2 Our approach:
- 9. Overview • What arewe working on? • What is edge computing? • How do we use edge computing? • How do we use AWS? • How can we integrate AWS? • Example models
- 10. Edge computing isreal-time computing at the end devices
- 11. What are edgedevices?
- 12. A cloud computingprovider can help integrate applications with edge devices • Cloud providers • Amazon AWS • Microsoft Azure • Edge Impulse • IoT (Edge) devices • Jetson Nano • Raspberry Pi • Arduino Nano
- 13. Edge computing versusCloud computing Advantages of edge computing • Real-time computing • Limited sending of information • Privacy
- 14. Overview • What arewe working on? • What is edge computing? • How do we use edge computing? • How do we use AWS? • How can we integrate AWS? • Example models
- 15. Light post detectionat the edge 4 Inspections on a bicycle Real-time inference on images to detect defects Bicycle-mounted edge devices operate autonomously.
- 16. From light postdetection to tilted street lights The angle of a light post detected in an image is determined in multiple steps. The angle of the light post is compared to the horizon. 5 Light posts in a training set are labelled using the Roboflow application and models are trained.
- 17. Modular edge device • NVIDIAJetson Nano or NVIDIA Xavier NX • 32 GB SD card • 4G and WiFi connectivity • Battery pack • Camera • GPS sensor and gyroscope
- 18.
- 19. Overview • What arewe working on? • What is edge computing? • How do we use edge computing? • How do we use AWS? • How can we integrate AWS? • Example models
- 20. • Deploy softwareto all remote devices at once from a central cloud location • Software versioning • CI/CD, Infrastructure as Code • Modular Greengrass components make it possible to design flexible architectures adapting to multiple use cases • Docker support • Dealing with unstable network connections reliably Managing remote devices with AWS Greengrass
- 21. • Every Greengrasscomponent acts as a microservice in a data pipeline, e.g. taking pictures with a camera, doing the AI inference, removing sensitive data, uploading to the cloud, etc. • Orchestrate event-driven workflows at the device level, with components talking over inter-process communication • Trigger or schedule workflows remotely over MQTT from AWS IoT Core • Address individual edge devices using a dedicated MQTT topic structure Greengrass components and event-driven workflow
- 22. • Using publiclyavailable models • Dedicated models to detect objects of interest, e.g. models from municipalities for garbage detection • Pre-trained models, such as YOLOv5 trained on a COCO dataset, to detect faces for further removal • Training custom models with transfer learning • Training data is collected from the bicycles and labelled manually. • Train models with Amazon SageMaker Studio • Track training experiments and model performance with MLflow Models and training
- 23. • Model optimizationis crucial for battery-powered edge applications. • Limited to GPU capacity of the device (NVIDIA Jetson Nano vs NVIDIA Xavier NX) • Faster models (and more powerful devices) will allow us to move from image processing to video processing to LIDAR processing at the edge. Model optimization and limitations of the edge
- 24. • With remotedevice management, scaling towards multiple edge devices is easy. • Remote connectivity enables real-time updates and event-driven data processing. • Modular design allows us to accommodate multiple use cases. • Let’s make cities a safer place through cloud and edge technology and AI. • We have developed a blueprint for smart edge-computing applications for use in public sector and research. Cloud is the key to scalability at the edge
- 25. We can useAWS to integrate with our dashboard for simpler use Integration AWS with a dashboard
- 26. Overview • What arewe working on? • What is edge computing? • How do we use edge computing? • How do we use AWS? • How can we integrate AWS? • Example models
- 27. Examples of usecases for VeloTech • Road deterioration • Broken street lights • Tilted lanterns • Street signs pollution • Lidar scans
- 28. Overview • What arewe working on? • What is edge computing? • How do we use edge computing? • How do we use AWS? • How can we integrate AWS? • Example models
- 29. Road deterioration example -text in Dutch
- 30.
- 31.
- 32.
- 33. Lidar scans –some ideas Tilted street lights Falling trees Collapsing quays (‘kades’) and bridges Mapping the city
- 34.