Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Engineering the software of robotic systems - 2 - Drones [ICSE 2017 - technical briefing]

888 views

Published on

Slides of the technical briefing we held at the ICSE 2017 conference.

The accompanying paper is available here: http://www.ivanomalavolta.com/files/papers/ICSE_2017_TB.pdf
Involved researchers: Federico Ciccozzi (1), Davide Di Ruscio (2), Ivano Malavolta (3), Patrizio Pelliccione (4), Jana Tumova (5)

1) Mälardalen University (Sweden)
2) University of L’Aquila (Italy)
3) Vrije Universiteit Amsterdam (The Netherlands)
4) Chalmers University of Technology | GU (Sweden)
5) KTH Royal Institute of Technology (Sweden)

Published in: Software
  • Be the first to comment

Engineering the software of robotic systems - 2 - Drones [ICSE 2017 - technical briefing]

  1. 1. Federico Ciccozzi1, Davide Di Ruscio2, Ivano Malavolta3, Patrizio Pelliccione4, Jana Tumova5 1Malardalen University (Sweden) 2University of L’Aquila (Italy) 3Vrije Universiteit Amsterdam (The Netherlands) 4Chalmers University of Technology | GU (Sweden) 5KTH Royal Institute of Technology (Sweden) Engineering the software of robotic systems - FLYAQ UNIVERSITY OF L’AQUILA
  2. 2. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Special kind of helicopter with: • high stability • omni-directional • smaller fixed-pitch rotors à safer than classical helicopters • simple to design and construct • relatively inexpensive image from http://goo.gl/FJFS5l What is a drone?
  3. 3. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Application scenario Mission to monitor the CO2 level in a geographical area by monitoring it along a grid composed of cells of size 12x12 meters Contextual entities obstacles emergency areas no fly zones home NO FLY ZONE
  4. 4. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Mission modeling Mission Context Map MML BL Behavior specification Detailed flight plan synthesis Robot type models Mission Execution Engine this layer is extensible RL
  5. 5. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Mask complexity • usable by non-technical experts • domain-specific concepts • separation of concerns Independence w.r.t. the types of robots • behaviour generation capability Reuse • context models • robot models Robots must be autonomous Principles Mission Context Map MML BL Behaviorspecification Detailed flight plan synthesis Robot type models Mission Execution EngineRL
  6. 6. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Modeling languages Mission Context Map MML BL Behaviorspecification Detailed flight plan synthesis Robot type models Mission Execution EngineRL
  7. 7. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Operator in-the-field stakeholder specifying the mission Robot engineer – models a specific kind of robot – develops the controller that instructs the robot on how to perform BL basic operations Platform extender – extends the MML metamodel with new kinds of tasks – develops a synthesizer for transforming each new task to its corresponding BL operations MML RL + software controller MML + synthesizer Involved stakeholders
  8. 8. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Mission layer sequence of tasks executed by a team of robots Extension point Monitoring mission language (MML)
  9. 9. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Photogrid task • identifies a virtual grid within an area • each cell of the grid has a size of n meters • each drone flies over each cell of the grid at an altitude of z meters and takes a picture of the ground Example of extension
  10. 10. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Example (1) MML model (in the tool) NF1 NF2 home RT PGT
  11. 11. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Example (2) Robot model (Parrot)
  12. 12. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Example (3) Behavioural model Drone& D1& Drone& D2& Drone& D3& Start&(ε,&ε)& Start&(ε,&ε)& Start&(ε,&ε)& TakeOff&(ε,&ε)& TakeOff&(ε,&ε)& TakeOff&(ε,&ε)& GoTo&(ε,&ε)&GoTo&(ε,&ε)& GoTo&(ε,&ε)& GoTo&(ε,&{Photo})&GoTo&(ε,&{Photo})& GoTo&(ε,&{Photo})& GoTo&(ε,{Photo,BroadCast(D3.R1.Done)})& GoTo&(ε,&ε)& Land&(ε,&ε)& Stop&(ε,&ε)& GoTo&(ε,&ε)& Land&(ε,&ε)& Stop&(ε,&ε)& 0GoTo&(ε,&{Photo,&& BroadCast&(D2.PG1.Done)})& 0 GoTo&(ε,&ε)& Land&(ε,&ε)& Stop&(ε,&ε)& GoTo(ε,&{Photo,&& BroadCast&(D1.PG1.Done)})& PG1 PG1 R1
  13. 13. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Web UI Mission Design and Behaviour Synthesis request MML & drones configurations 1 2 MML concepts & drones 3 mission design 4 submit MML models 6 BL model and analysis results 5 behaviour analysis and synthesis 7 start mission Mission Execution Engine Mission modeling workflow
  14. 14. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Mission execution Mission Execution Engine 6 send status* (continuous flow) start mission setup drone controllers * steps executed in parallel Monitoring Engine tasks execution commands* feedback* feedback* 25 1 3 4 C1 D1
  15. 15. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova
  16. 16. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova s1 0 TakeOff(p1.z) s1 Mission entering s2s'''2 Goto(c1) s''2 Goto(c2) s'2 Goto(p1)Goto(home1)Land s1 f land1 Mission leaving s'1 Goto(c2) s''1 Goto(p1) v1 v1 1 Goto(q1) / DoPhoto(…) v2 1v16 1 … NoOp Mission tasks execution Goto(q16) / DoPhoto(…) NoOp r1 NoOp Goto(c1) v17 1 Flight plan synthesis MML Each MML task is constrained by: • geometry • behavioural strategy • actions
  17. 17. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova q4= p2 q1= p1 q3 q14 q5 q6 q15 q16 q2 q13 q12 q11 q7 q8 q9 q10 p3 p4 c1 c2 p1 Synthesis method on the example
  18. 18. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, and Patrizio Pelliccione. Adopting mde for specifying and executing civilian missions of mobile multi-robot systems. IEEE Access, PP(99):1–1, 2016. Darko Bozhinoski, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Massimo Tivoli (2015). FLYAQ: Enabling Non-Expert Users to Specify and Generate Missions of Autonomous Multicopters. In 30th IEEE/ACM International Conference on Automated Software Engineering (ASE 2015). Darko Bozhinoski, Ivano Malavolta, Antonio Bucchiarone, Annapaola Marconi (2015). Sustainable Safety in Mobile Multi-Robot Systems via Collective Adaptation. In Ninth IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2015, Cambridge, Massachusetts, USA, September 21-25, 2015, Darko Bozhinoski (2015). Managing Safety and Adaptability in Mobile Multi-Robot Systems. In Proceedings of the 11th International ACM SIGSOFT Conference on Quality of Software Architectures, pp. 135–140. Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione (2014). A family of Domain-Specific Languages for specifying Civilian Missions of Multi-Robot Systems. In Proceedings of the 1st International Workshop on Model-Driven Robot Software Engineering (MORSE), pp. 13–26. Davide Di Ruscio and Ivano Malavolta and Patrizio Pelliccione (2014). The Role of Parts in the System Behaviour. In Software Engineering for Resilient Systems - 6th International Workshop, SERENE 2014, Budapest, Hungary, October 15-16, 2014. Proceedings, pp. 24–39. Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione (2013). Engineering a Platform for Mission Planning of Autonomous and Resilient Quadrotors. In Software Engineering for Resilient Systems - Fifth International Workshop, SERENE 2013, pp. 33–47. References
  19. 19. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Backup slides
  20. 20. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Tool support … ROS HTTPS and web sockets Web interface Editor for MML models M2M transformation + models validation + mission workflow manager Layer of controllers that interpret BL models at run-time HTML5, CSS3, JavaScript Java + OCL + Rosbridge Any technology supported by ROS Drone driver any C1 C2 C3 … Any protocol
  21. 21. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova point line polygon volume Geometries supported by MML
  22. 22. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova sweep(d) – full coverage of an area discretized by a grid whose cells dimension is d search(target, d) – visit performed on a d-sized grid towards the discovery of target (e.g., represented by a PNG image) track(m, m’, d) – it is like sweep(d) where m and m’are the visit restart and stop messages, respectively Actions: • taking a picture • making a video • detecting the presence of CO2 • etc. Strategies supported by MML
  23. 23. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Let’s go back to the example… NF1 NF2 home RT PGT
  24. 24. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova Context = ( (NF1,NF2), (OB) ) PGT = ( (p1,p2,p3,p4,p1), sweep(10), (doPhoto(2480 3508), i) ) Swarm ={ (d1, {PGT}), (d2, {PGT}), (d3, {RT}) } Task dependency graph ( (t0, PGT, RT, t3), ∅ ∪ { (t0, {PGT, RT}) } ∪ { ({PGT, RT}, t3) } ) t0 PGT RT t3 Synthesis method on the example
  25. 25. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova • Safety • avoidance of collisions (P1), and of traversing no-fly zones (P2) • no concurrency issues (P3) • P1&P2 • correct and complete specification • thanks to specific auxiliary functions • volume partitioning algorithms • visibility graph • coverage path planning • P3 • partitioning in independent areas • no message lost • sound mechanisms for sequencing, join, and fork • a drone cannot be involved in concurrent tasks Properties of the synthesized BL model
  26. 26. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova EXPERIMENTATION models@runtime to control the mission execution and manage adaptation Support other types of robots Mission Execution Engine navigation commands (MAVLink) telemetry (MAVLink) C1 MAV Proxy Drone Kit Arducopter physics simulator physics data Google Earth drones positions at run-time Future work
  27. 27. UNIVERSITY OF L’AQUILA Federico Ciccozzi, Davide Di Ruscio, Ivano Malavolta, Patrizio Pelliccione, Jana Tumova BL model automatic synthesis BL Mission entering Mission leaving Mission tasks execution … … dn … d2 … d1 State transition State Synchronization and communication message Flight plan synthesis MML

×