Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Engineering the software of robotic systems - 3 - Underwater robots [ICSE 2017 - technical briefing]

952 views

Published on

Slides of the technical briefing we held at the ICSE 2017 conference.

The accompanying paper is available here: http://www.ivanomalavolta.com/files/papers/ICSE_2017_TB.pdf
Involved researchers: Federico Ciccozzi (1), Davide Di Ruscio (2), Ivano Malavolta (3), Patrizio Pelliccione (4), Jana Tumova (5)

1) Mälardalen University (Sweden)
2) University of L’Aquila (Italy)
3) Vrije Universiteit Amsterdam (The Netherlands)
4) Chalmers University of Technology | GU (Sweden)
5) KTH Royal Institute of Technology (Sweden)

Published in: Software
  • Be the first to comment

  • Be the first to like this

Engineering the software of robotic systems - 3 - Underwater robots [ICSE 2017 - technical briefing]

  1. 1. UNIVERSITY OF L’AQUILA Federico Ciccozzi1, Davide Di Ruscio2, Ivano Malavolta3, Patrizio Pelliccione4, Jana Tumova5 1Mälardalen University (Sweden) 2University of L’Aquila (Italy) 3Vrije Universiteit Amsterdam (The Netherlands) 4Chalmers University of Technology | GU (Sweden) 5KTH Royal Institute of Technology (Sweden) Engineering the software of robotic systems UAV VASA
  2. 2. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova What is VASA?
  3. 3. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Underwater robot with following characteristics: • 5 thrusters, omni-directionality • other actuators: torpedoes, two arms (grab/release, push/rotate) • complex control routines due to very variable underwater conditions • equipped with stereo-vision system (two separate cameras taking in individual image streams which are merged into one stereo-image) What is VASA?
  4. 4. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova • Decisions based on input from stereo vision system • Mission-oriented and mission-critical • Software roughly divided in three parts: • Control software – for thrusters and other actuators • Vision software – input image streaming, image recognition, image storing (big data processing) • Application software – needed for the robot to exploit image streaming/recognition and control software to perform missions Peculiarities
  5. 5. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Control software • + Well-established best practices – we know how to optimize the software for a thruster, we know how to manage the interplay between thrusters operating on different axes • - Usually engineered through platform-specific coding, low reusability, low adaptability to replacement of heterogeneous parts Engineering Software for vision-based UAVs
  6. 6. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Vision software • + Well-established common libraries (e.g. OpenGL) with algorithms for object detection, colour detection, image recognition, filtering, etc • - Underwater environment is extremely variable, due to weather conditions, streams, depth, turbidity o this makes it hard to reuse SW from aerial and terrestrial robots o this software must be engineered so to allow an extremely wide spectrum of different environmental situations in order to be able to recognize, e.g., colours Engineering Software for vision-based UAVs
  7. 7. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Application software • Up to now not given as much as attention as to the previous two • Big parts of the missions are hard-coded (e.g., known distances) o + Less calls to time-consuming vision software o - Low adaptability • Mission-specific (and often platform-specific) o + High optimization o - Low reusability o - Low adaptability Engineering Software for vision-based UAVs
  8. 8. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova • International AUSVI RoboSub Competition, yearly, held in San Diego, open to students only (http://www.robonation.org/competition/robosub) • VASA team, Mälardalen University • Mission-based tournament (like a World-cup for UAVs) Application scenario
  9. 9. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Typical RoboSub missions
  10. 10. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova • No communication with the robot allowed after put in water • Outdoor pool with turbid water • Unknown environment (size, colours, weather conditions) • Missions know in advance but several details unknown (order, colours) Challenges
  11. 11. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova • No communication with the robot allowed after put in water • Outdoor pool with turbid water • Unknown environment (size, colours, weather conditions) • Missions know in advance but several details unknown (order, colours) Heavily relying on vision Challenges
  12. 12. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Example on varying visual conditions underwater
  13. 13. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Example on varying visual conditions underwater
  14. 14. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Example on varying visual conditions underwater
  15. 15. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Control software • Issue: replacement of actuators managed by platform-specific control code • Solution: reverse-engineering control code to executable (UML) models from which different target platforms can be hit Engineering Software for RoboSub
  16. 16. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Vision software • Issue: too low adaptability to weather conditions (too high response time when adding a larger number of ”known” colour nuances) • Solution: created a unique board with CPU, GPU and FPGA able to cooperate in order to allow for much faster object and colour recognition and heavier processing of larger amounts of data • The side-effect was that vision SW had to be re-engineered almost completely • How to effectively make heterogeneous processors to interact is still an open SW engineering issue Engineering Software for RoboSub
  17. 17. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Application software (1) • Issue: some parts of the missions were hard-coded (e.g., known distances between obstacles and/or objects) • Solution: we removed all mission-specific hard-coding and made all missions independent from each other independency led to much higher adaptability to changed order of missions and higher fault- tolerance (the robot does not get stuck in one mission when it can not carry it out) Engineering Software for RoboSub
  18. 18. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Application software (2) • Issue: engineered ad-hoc for specific missions. Changes to one mission could affect other missions • Solution: a DSML by which we could model and simulate missions. From the same model, different platforms (even different types of robots) can be targeted (recall FlyAQ?) Engineering Software for RoboSub
  19. 19. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova • Tool for simulating variability of the underwater environment • Collaborative missions with UAVs and other types of autonomous vehicles • Improvement of performance of vision software through effective exploitation of FPGA capabilities • Improvement of image filtering through more efficient minimum-volume enclosing ellipsoids algorithms Current work

×