Open-World Mission Specification for Reactive Robots - ICRA 2014

1. Open-World Mission Specification for Reactive Robots Spyros Maniatopoulos Matthew Blair, Cameron Finucane, Hadas Kress-Gazit Sibley School of Mechanical and Aerospace Engineering Cornell University sm2296@cornell.edu verifiablerobotics.com
2. Controller Synthesis Robot Environment Mission Specification Workspace Hybrid Controller Reactive Synthesis Discrete Strategy
3. Meet the mailbot 3
4. Challenges 4 Letters letter_Spyros letter_Matt letter_Cameron The mailbot’s world is open with respect to letter recipients. letter_Hadas
5. “Closed” vs. “Open” Worlds 5 “Closed-World Assumption” Only elements of the world modeled a priori can be taken into account during execution. Robotic assembly by DENSO’s robot arms. Source: www.robots.com
6. “Closed” vs. “Open” Worlds 6 “Open-World” New elements and objectives arise during execution. Photo: Panagiotis Papadakis Source: spectrum.ieee.org
7. Open-World Challenges • Not all mission variables known a priori • Mission objectives change during execution • React to unexpected events during execution • Incorporate/learn new functionality on-the-fly 7
8. Related Work 8 S. C. Livingston, R. M. Murray, and J. W. Burdick, “Backtracking temporal logic synthesis for uncertain environments,” ICRA, 2012. S. C. Livingston, P. Prabhakar, A. B. Jose, and R. M. Murray, “Patching task-level robot controllers based on a local μ-calculus formula,” ICRA, 2013. M. Guo, K. H. Johansson, and D. V. Dimarogonas, “Revising motion planning under linear temporal logic specifications in partially known workspaces,” ICRA, 2013. A. I. Medina Ayala, S. B. Andersson, and C. Belta, “Temporal logic motion planning in unknown environments,” RSS, 2013. S. Sarid, B. Xu, and H. Kress-Gazit, “Guaranteeing high-level behaviors while exploring partially known maps,” RSS, 2012.
9. Discrete Abstraction 9 Region Propositions Action Propositions Sensor Propositions
10. Logic-based Mission Specification 10 Boolean Operators Temporal Operators Linear Temporal Logic Robot starts in r1 If you are sensing alarm then visit r2 Structure d English “Starting in region r1, repeatedly visit the region r2 if you are not sensing an alarm” GR(1)
11. Approach Overview 11
12. Detecting New Elements 12 Detector propositions Proposition Grounding function
13. Adding New Elements 13
14. Adding New Elements 14
16. Open-World Abstractions – Groups 16 Group Letters is letter_Spyros, letter_Matt, letter_Cameron Groups of Propositions Letters (sensors) letter_Spyros letter_Matt letter_Cameron Offices (regions) office_Spyros office_Matt office_Cameron
17. Open-World Abstractions – Quantifiers 17 Quantification over Groups of propositions – all Letters letter_Spyros letter_Matt letter_Cameron all Letters
18. Open-World Abstractions – Quantifiers 18 Quantification over Groups of propositions – any Letters letter_Spyros letter_Matt letter_Cameron any Letters
19. Open-World Abstractions – Quantifiers 19 Quantification over Groups of propositions – each Offices office_Spyros office_Matt office_Cameron Go to each Office • Go to office_Spyros • Go to office_Matt • Go to office_Cameron
20. Open-World Abstractions – Correspondence 20 Correspondence between Propositions Letters letter_Spyros letter_Matt letter_Cameron Offices office_Spyros office_Matt office_Cameron
21. Open-World Abstractions – Correspondence 21 Letters correspond to Offices If you are sensing any Letter then go to the corresponding Office Correspondence between Propositions
22. Open-World Abstractions – Example 22 What about a new proposition, say, letter3?
24. Letters letter_Spyros letter_Matt letter_Cameron Offices office_Spyros office_Matt office_Cameron Adding new propositions 24 letter_Hadas
25. Adding new propositions 25 If you are sensing newLetter then add to group Letters
27. Resynthesis 27 If you are sensing newLetter then add to group Letters and resynthesize Global Resynthesis S. Sarid, B. Xu, and H. Kress-Gazit, “Guaranteeing high-level behaviors while exploring partially known maps,” RSS, 2012
28. Summary of our Approach 28
29. Simulation in LTLMoP* – Mission 29 “Patrol, and deliver letters to their recipients’ offices.” *Linear Temporal Logic MissiOn Planning (ltlmop.github.io)
30. Simulation in LTLMoP – Execution 30
31. Implementing an open-world mailbot 31 Abhishek Sriraman, Undergraduate Research Project (Spring 2014)
32. Summary & Future Work 32 Summary • Open-world abstractions allow implicit reference to propositions • New elements are automatically incorporated in the specification • Changes reflected in robot’s controller via (global) resynthesis Future Work • Automatic removal of propositions • Efficient resynthesis • Proposition grounding
33. Open-World Mission Specification for Reactive Robots Spyros Maniatopoulos Matthew Blair, Cameron Finucane, Hadas Kress-Gazit Sibley School of Mechanical and Aerospace Engineering Cornell University sm2296@cornell.edu verifiablerobotics.com

Editor's Notes

Hi all, thank you for sticking around. I’m all that’s left between you and the banquet!
My name is … work done in collaboration with …
Today I will be talking about OWMS
But before I get to that, let me first tell you what our group does.

Our group is interested in high-level mission planning.
Given, … automatically synthesize a correct-by-construction reactive controller.
In order to continuously implement the discrete strategy, we construct a hybrid controller. Then execute in simulation or experimentally
Tell a story: “Last summer, we decided get together as a team and work on a big experimental demo.

“We decided to work on an autonomous robotic courier, a mailbot!” “It consists of an Aldebaran Nao humanoid robot mounted on a Segway-based platform.”
“Its task was to pickup letters, like the one it’s holding, and deliver them to their recipients’ offices.

The mission was specified based on letter recipients and their offices. Those variables were hardcoded a priori in the mission specification.
However, what if a new professor moves in our department building? In the current setting, the user would have to pause execution, manually re-write the mission specification to incorporate this new letter recipient, synthesize a new controller, and then resume execution. That doesn’t sound very autonomous.
Reason? World open wrt letter recipients. What do I mean by “open”? Well, let’s look at what “closed” means first.

Assembly line tasks:
the environment is predictable,
the workspace highly structured,
and the objectives are mostly fixed and repeated over and over

Search & Rescue: workspaces are partially unknown and highly unstructured, and the mission objectives change on-the-fly
Other open-world missions can be found in Autonomous space and planetary exploration, Military scenarios, and Personal robots
In such settings, the closed-world assumption no longer holds.

In general, these are the challenges an open-world presents the robot with …
So, within our high-level mission specification and planning framework, we want to account for these situations in a systematic and autonomous fashion.
This work is but a first step towards tackling these challenges.
Before I get to the approach, I want to give you a brief overview of recent work related to this paper.

Prior work within our group, before I joined. We use a result of that work in our approach. That work involves …
Scott Livingston at Caltech has been working on patching reactive controllers in order to account for local topological changes in the robot’s workspace. The approach is complementary to ours. Specifically, Scott’s work deals with the controller synthesis level, while what I’m about to present deals with the mission specification level.
Also, at KTH in Sweden, they’ve worked on updating a robot’s plan as it’s navigating a partially-known workspace.
Finally, this paper by Calin Belta’s group at BU provides an interesting perspective. There, a robot starts by exploring its unknown workspace without falsifying its mission specification. Once it has enough information, it proceeds to satisfy it.

Set of Atomic Propositions (binary variables)
Explain what each type of proposition is abstracting, in terms of the mailbot scenario.

Use atomic propositions to write LTL formulas.
We are working with the GR(1) fragment of LTL.
“A simple reactive robot task would be …”
In our group’s framework, we also use Structured English, which parses directly to the GR(1) fragment of LTL. Same example “…” Our framework also supports Natural Language, but this is not demonstrated in this presentation.

Overview of our approach.
It consists of 4 main components.
Let’s start with the first one; modeling new elements of the open world.

We define a special class of sensor propositions, detectors propositions.
For example, the mailbot would have 1 such detector, which would trigger if the robot is handed a letter addressed to a previously unknown recipient.
Function D creates a new proposition (at the discrete level) and grounds it to the new element of the world (in the continuous domain).
For example, in the mailbot scenario, a letter proposition would be grounded to a radial barcode.
Quick aside on notation: t denotes the execution time step, whereas k is only incremented when a new element is detected.

When a detector proposition becomes true, a new proposition is generated …

… and added to the set of atomic propositions
Now, we want to augment the specification language s.t. the robot’s mission can be automatically updated when a new proposition is added.

We will augment the specification language with open-world abstractions.
You can think of it as adding elements of first-order logic.

There are three types of open-world abstractions.
The first one is groups of propositions.

All Group is substituted by the logical conjunction of all propositions in the group.

Any Group is substituted by the logical disjunction of all propositions in the group.

The task has to happen for each proposition in the group.

Finally, we define correspondence between propositions.
If the propositions in two groups have a 1-to-1 correspondence, we also say that the groups themselves correspond to each other.

In a mission specification written in Structured English, this is how we would declare and use correspondence.
We first declare correspondence between groups, and then use the corresponding operator to specify a task.
And in general, this is the substitution that takes place.

Using these open-world abstractions, we can now specify missions without explicitly referring to individual propositions.
…
But this still doesn’t solve the problem of dealing with new propositions…

1) … to accomplish that, we’ll need a mechanism that allows us to incorporate new propositions in the mission specification.

This mechanism is the add_to_group operator.
Going back to the mailbot scenario, when a new letter recipient is detected, a new proposition is generated.
This operator adds this new proposition to the appropriate group.
However, this new proposition now has to correspond to something in the other group.

So the add_to_group operator not only adds a new proposition to a group, but it also generates more propositions, if necessary, and adds them to the corresponding groups.
I’m later expecting questions about proposition grounding.
In a mission specification, we would use this operator as such: …
Quick clarification: the detector proposition “newLetter” doesn’t trigger upon seeing an envelope. Rather it abstracts the sensing of a letter addressed to a previously unknown recipient!

So far we’ve seen how the newly detected elements of the world are incorporated in the mission specification.
The final step is to ensure that the changes to the mission are reflected in the robot’s controller.

To accomplish that, we leverage work done previously in our group.
Namely, global resynthesis, which involves saving the current mission state, synthesizing a new robot controller, and resuming execution.
Notice that, in the mission specification, the user has control over when resynthesis will take place, by appropriately specifying that task.

1) Quick summary …

Let’s see how a simplified version of the mailbot mission plays out in simulation.
We use LTLMoP, our group’s Python toolkit for mission specification, controller synthesis, and simulation.
Explain spec …

Explain buttons: red = false, green = true
After resynthesis, there’s a new letter proposition and a new mission objective.
So that was in simulation, and you saw how we queried the user in order to ground the propositions.

Each radial barcode corresponds to a letter recipient and address.
And the addresses are grounded to region propositions
The recipient and address fields on the letter are used instead of a user query.

Open-World Mission Specification for Reactive Robots - ICRA 2014

You are reading a preview.

Open-World Mission Specification for Reactive Robots - ICRA 2014

More Related Content

Featured

Related Books

Related Audiobooks