Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A software agent controlling 2 robot arms in co-operating concurrent tasks


Published on

TeleoR is a major extension of Nilsson’s Teleo-Reactive (TR)
rule based robotic agent programming language. Programs comprise sequences of guarded action rules grouped into parameterised procedures.
The guards are deductive queries to a set of rapidly changing percept and other dynamic facts in the agent’s Belief Store. The actions are either tuples of primitive actions for external robotic resources, to be executed in parallel, or a single call to a TeleoR procedure, which can be a recursive call. The guards form a sub-goal tree routed at the guard of the first rule. When partially instantiated by the arguments of some call, this guard is the goal of the call.
TeleoR extends TR in being typed and higher order, with extra forms of rules that allow finer control over sub-goal achieving task behaviour.
Its Belief Store inference language is a higher order logic+function rule language, QuLog. QuLog also has action rules and primitive actions for updating the Belief Store and sending messages. The action of a TeleoR rule may be a combination of the action of a TR rule and a sequence of
QuLog actions. TeleoR’s most important extension of TR is the concept of task atomic procedures, some arguments of which belong to a special but application specific resource type. This allows the high level programming of multitasking agents using multiple robotic resources. When two or more tasks
need to use overlapping resources their use is alternated between task atomic calls in each task, in such a way that there is no interference, deadlock or task starvation.
This multi-task programming is illustrated by giving the essentials of a program for an agent controlling two robotic arms in multiple block tower assembly tasks. It has been used to control both a Python interactive graphical simulation and a Baxter robot building real block towers, in each case with help or hindrance from a human. The arms move in parallel whenever it can be done without risk of clashing.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

A software agent controlling 2 robot arms in co-operating concurrent tasks

  1. 1. Background slides for invited challenge demo at Rule-ML 2017 A software agent controlling 2 robot arms in co-operating concurrent tasks Robotic agent programming in QuLog and TeleoR Keith Clark Imperial College & University of Queensland Joint work with Peter Robinson University of Queensland 1
  2. 2. QuLog - LP+FP+AR Language •  Flexibly typed, multi-threaded, higher order •  Relation and function defining rules •  Function calls and set expression arguments in relation calls •  Function rule guards can query relations •  Relation rules more declarative than Prolog, closer to predicate logic •  Relations are typed and moded •  Dynamic relations defined only by facts •  Used for the agent’s Belief Store •  Top layer of action rules •  Update dynamic relations, fork threads, do I/O, communicate with other Qulog, C, Python or Java processes •  Inter-process comms uses our Pedro pub/sub and addressed message router •  Threads execute actions calling funs and rels as needed •  Rel and fun rules cannot call actions 2
  3. 3. TeleoR •  Development of Nilsson’s T-R robotic agent language of guard ~> action rules sequenced in parameterized procedures. •  guard is a QuLog deductive query to the BS •  action is a tuple of robotic actions executed in parallel or a T-R procedure call, maybe a recursive call •  TeleoR has forms of rules and actions not in T-R –  Extra rules for more fine grained behaviour control –  Can combine robotic action with QuLog action sequence: action ++ qact1 ;...; qact1 –  This allows communication and updates of BS as parallel non-robotic actions •  Compile time guarantee: –  all rule actions are correctly typed and fully instantiated when a rule is fired •  The concept of task atomic procedures: –  Used for multi-tasking sharing one or more robotic resources –  No interference, no starvation, deadlock free –  Compiler generates code that uses BS for co-ordination 3
  4. 4. Deductive Belief Store Dynamic facts Relation and function rules ? Percepts Handler Message Handler ag@host Percept fact Interpretations of Sensor data Mid-level Action control messages Incoming Messages and replies Control Thread ? Atomic re-evaluation of rule guards after each atomic update Atomic Updates Multi-threaded Agent Architecture Only outgoing Messages 4 pedro Other Agents Pub/Sub and Addressed Message Router TeleoR + QuLog QuLog
  5. 5. One arm block tower building 5
  6. 6. Task Ontology 6 def block::= 1..9 durative pickup(block), put_on_block(block), put_on_table() percept on(block,block), on_table(block), holding(block) rel sub_tower(list(block)), tower(list(block)) sub_tower([B]) <= on_table(B) sub_tower([B1,B2,..Blocks]) <= on(B1,B2) & sub_tower([B2,..Blocks]) tower([B,..Blocks]) <= not on(_,B) & sub_tower([B,..Blocks])
  7. 7. Nilsson’s universal conditional top- level plan for block tower building 7 tel makeTower(list(block)) makeTower(Blocks){ tower(Blocks) ~> () % Goal is achieved sub_tower(Blocks) & Blocks=[B,..] ~> make_clear(B) Blocks=[B] ~> move_to_table(B) Blocks=[B1,B2,..Bs] & tower([B2,..Bs]) ~> move_to_block(B1,B2) Blocks=[B,..Bs] ~> makeTower(Bs) } Action achieves
  8. 8. Mutually recursive procedures 8 tel make_clear(block) make_clear(Block){ not on(_,Block) ~> () % Goal is achieved on(OnBlock,Block) ~> move_to_table(OnBlock) } tel move_to_table(block) move_to_table(OnBlock){ on_table(OnBlock) ~> () % Goal is achieved holding(OnBlock) ~> put_on_table() not on(_, OnBlock) & not holding(_) ~> pickup(OnBlock) not holding(_) ~> make_clear(OnBlock) holding(_) ~> put_on_table() }
  9. 9. Interleaving the building of several towers 9 Just need to add: task_start makeTower task_atomic move_to_table, move_to_block
  10. 10. Baxter concurrent tower building 10
  11. 11. TR procedures Sensor data Control actions for different robotic resources BeliefStore Dynamic Facts Percepts Handler TR Evaluatio n ThreadsAll incoming messages Message Handler Task1 Using R1,R2 Task2 Using R3 Outgoing messages Multi-tasking architecture Task3 Waiting for R1,R3 Task4 Waiting for R1,R5Fixed Facts & Rules ?
  12. 12. Elements of the two arm tower builder program 12 def table ::= table1 | shared | table2 def block ::= ... def arm ::= arm1 | arm2 def resource::= arm || table % resource a reserved type name durative pickup(arm, block, table), put_on_block(arm, block, table), put_on_table(arm, table) tel makeTower(arm, list(block), table) makeTower(Arm,Blks,Tbl){ % Tbl is where tower must be built, tower(Blks, Tbl) ~> {} sub_tower(Blks, Tbl) & Blocks=[B,..] ~> makeClear(Arm, B, Tbl) Blks=[B,.. Bs] & tower(Bs, Tbl) ~> moveAcrossToBlock(Arm, B, firstOf(Bs), Tbl) Blks=[B] ~> moveAcrossToTable(Arm, B, Tbl) Blks=[B,.. Bs] ~> makeTower(Arm, Bs, Tbl) } Entire control program 40 TeleoR rules clustered into 6 procedures.
  13. 13. Auxiliary procedures 13 tel moveAcrossToTable(arm, block, table) % Will fetch a block from wherever it is and put it onto the table % May need two calls to proc below using different arms to first % move the block to shared then to destination table task_atomic oneArmMoveToTable(arm, block, table, table) % Has 3 resource args. an arm and two tables which might be the same oneArmMoveToTable(Arm, Blk, FromTab, ToTab){ on(Blk, ToTab) ~> () clear(Blk) ~> clearOneArmMoveToTable(Arm, Blk, FromTab, ToTab) not holding(Arm,_) ~> makeClear(Arm, Blk, FromTab) holding(Arm,_) ~> put_on_table(Arm, FromTab) }
  14. 14. TeleoR semantics and implementation 14 • Formal State Transition Semantics • Optimized Reference Implementation • Runtime system that implements state transition semantics • Compile time analysis ensures rule guards are not re-evaluated on BS update if no relevant change made • Currently compiled to multi-threaded Qu-Prolog • Will soon be compiled to specialized Abstract Machine Code similar to but simpler than Warren’s Prolog AMC
  15. 15. Sources and software 15 Clark & Robinson, Robotic Agent Programming in TeleoR, ICRA 2015, IEEE Clark & Robinson, Multi-tasking Robotic Agent Programming in TeleoR, Research Report, on Clark et al, A Framework for Integrating Symbolic and Sub-symbolic Representations, IJCAI 2016, AAAI Press Programming Communicating Multi-tasking Robotic Agents: A Teleo-Reactive Rule Based Approach, Springer, Early 2018 first 5 chapters at including a formal operational semantics of Nilsson’s TR lang. Clark & Robinson, Engineering Agent Applications in QuLog, Springer, 2017/8, TeleoR and QuLog Software for Unix, Linux and OS X at Collaboration and users welcomed