The document describes a method for learning relational grammars from sequences of actions performed by a user steering a robot. It induces grammars from sequences of state-action pairs by identifying repeated n-grams and generalizing rules. Experiments applying the method to navigation tasks and gesture recognition are discussed.

1. Learning Relational Grammars from Sequences of
Actions
Blanca A. Vargas-Govea and Eduardo F. Morales
{blanca,emorales}@inaoep.mx
Robotics Lab
´
INAOE, Mexico
CIARP 09
November 18, 2009
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 1 / 23

2. Outline
Outline
Introduction
Grammar learning
Induction
Evaluation
Generalization
Experiments
Navigation
Gesture recognition
Conclusions
Future work
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 2 / 23

3. Introduction
Motivation
Robot, go to the living room Increasing demand of service robots
◮ a robot has to be programmed
according to the task to be
accomplished
◮ programming abilities to perform
tasks is difficult and usually,
users are not programmers or
robotic experts
◮ a possibility is that the user
shows the robot what to do
A mobile robot that learns to navigate and to identify gestures
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 3 / 23

4. Introduction
Motivation
Sequences are used to describe different problems in many fields
◮ The robot can learn from
Goal
dynamic obstacle sequences of actions generated
turn-right
by the user
turn-left
orient go-forward
◮ In this work the idea is to learn
leave-a-
grammars from sequences of
orient
trap
actions that can be used to
execute tasks or as classifiers
Grammars are used as control programs and as classifiers
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 4 / 23

5. Grammar learning
Grammar learning
FOSeq (First Order Sequence learning)
Input: a set of sequences of first order state-action pairs. FOSeq:
1. Induces a grammar for each sequence, identifying repeated
elements (n-grams e.g., sub-sequences of n-items, in our case
n-predicates) .
2. Evaluates the sequences with each learned grammar.
3. Selects the best evaluated grammar and a list of candidates to
improve the evaluation of the best grammar.
4. Applies a generalization process to the best grammar.
Output: A grammar that can be applied as controller or as classifier.
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 5 / 23

6. Grammar learning
Input
The user steers the robot to execute the task. Raw data from the
sensors are transformed into predicates
The input is a sequence of first order state-action pairs like the
following:
pred1(State1,action1),pred2(State2,action1),pred1(State3,action2),. . .
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 6 / 23

7. Grammar learning Induction
Induction
◮ The algorithm looks for n-grams that appear at least twice in the
sequence.
◮ The candidate n-grams are incrementally searched by their length.
◮ The n-gram with the highest frequency of each iteration is
selected, generating a new grammar rule and replacing in the
sequence, all occurrences of the n-gram with a new non-terminal
symbol.
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 7 / 23

8. Grammar learning Evaluation
Induction: example
S→a b c b c b c b a b c b d b e b c S2 → R2 d b e R1
b c (5) R1 → b c
Add: R1 → b c R2 → a R1 b
S1 → a R1 R1 R1 b a R1 b d b e R1
Removing repeated items:
S1 → a R1 b a R1 b d b e R1
R1 b (2)
a R1 (2) R2 d b e R1
a R1 b (2)
Add: R2 → a R1 b a R1 b b c
S2 → R2 R2 d b e R1
Removing repeated items: b c
S2 → R2 d b e R1
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 8 / 23

9. Grammar learning Evaluation
Induction: predicates
When the items of the sequence are first order predicates, the learned
grammar is a definite clause grammar (DCG). DCGs:
◮ are an extension of context free grammars,
◮ can have arguments, and
◮ are expressed and executed in Prolog.
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 9 / 23

10. Grammar learning Evaluation
Evaluation
◮ Every learned grammar is used to parse all the sequences in the
set of traces provided by the user.
◮ For each parsed sequence, each grammar is evaluated using the
following function:
n
cj
eval(Gi ) = (1)
cj + fj
j=1
where Gi is the grammar being evaluated, cj and fj are the number
of items that the grammar is able or unable to parse respectively.
◮ The best evaluated grammar is selected.
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 10 / 23

11. Grammar learning Generalization
Generalization
The key idea is to obtain a new grammar that improves the covering of
the best grammar.
1. Select the grammar that provides the largest number of different
instantiations of predicates.
2. Compute the lgg* between both grammars.
3. Accept the new grammar rule if it improves the original coverage,
otherwise discard it.
4. The process continues until a coverage threshold is reached or
there is no longer improvement with the generalization process.
*Least general generalization (lgg) [Plotkin,1970]
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 11 / 23

12. Grammar learning Generalization
lgg example
c1 = pred(State,action1) ← c2 = pred(State,action3) ←
cond1(State,action1), cond1(State,action1),
cond2(State,action2). cond2(State,action3).
Computing lgg(c1,c2), the clause:
pred(State,Action) ←
cond1(State,action1),
cond2(State,Action).
where the constants action2 and action3 of predicate cond2 are
replaced by the variable Action.
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 12 / 23

13. Experiments
Experiments
Learning navigation tasks Classification of dynamic gestures
Stop
Left
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 13 / 23

14. Experiments Navigation
Learning navigation tasks: goto (wander + orient)
Input: sequences of basic TRPs: wander and orient.
orient(State,turn-left),wander(State,turn-left), wander(State,turn-right),
wander(State,go-forward),orient(State,turn-left),wander(State,go-forward),.. .
...
Output:
goto(State,nil) ←
in-goal(State).
goto(State,Action) ←
orient(State,none),
wander(State,Action).
´
goto(State,Accion) ←
wander(State,go-forward),
orient(State,Action).
goto(State,Action) ←
wander(State,Action).
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 14 / 23

15. Experiments Navigation
Navigation experiments: simulation
TRP seq Tasks Acc
wander + orient 8 30 86.67 %
(goto)
wander + orient + 12 30 93.33 %
leave-trap
follow + wander 10 40 90.00 %
Goto follow + wander + 14 40 100.00 %
leave-trap
visiting 5 places* - 30 96.67 %
Visiting different places
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 15 / 23

16. Experiments Navigation
Navigation experiments: Markovito (1/2)
A service ActivMedia robot equipped with a sonar ring, a Laser SICK
LMS200 and a vision stereo system.
◮ Navigating to several places in the environment.
Each place has a previously defined name (e.g.,
kitchen, sofa).
◮ following a human under user commands,
◮ finding one of a set of different objects in a house,
and
◮ delivering messages and/or objects between
different people.
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 16 / 23

17. Experiments Navigation
Navigation experiments: Markovito (2/2)
Tasks Interv. Acc
Goto 120 20 85.00 %
Follow 20 3 90.00 %
a person
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 17 / 23

18. Experiments Gesture recognition
Learning gesture grammars
Goal: learn grammars from sequences of dynamic gestures and use
them to classify new sequences.
(a) (b) (c) (d) (e)
(f) (g) (h) (i) (j)
(a) initial/final position, (b) atenttion, (c) come, (d) left, (e) right, (f) stop, (g) turn-right, (h) turn-left,
(i) waving-hand, (j) point ´
[Aviles,2006]
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 18 / 23

19. Experiments Gesture recognition
Representation
Each sequence is a vector with sets of seven attributes describing the
executed gesture. An example of a sequence is:
(+ + −− T F F),(+ + −+ T F F),(+ + +0 T F F),(+ + ++T F F)...
Motion features Posture features
◮ ∆area ◮ form : horizontal(−),
vertical(+), tilted (0).
◮ ∆x
◮ above (the head).
◮ ∆y
◮ right (to the head).
These features take one of
three possible values: {+,−,0} ◮ torso (hand over the torso).
indicating increment, decrement
or no change.
hmov(State,right), vmov(State,up), size(State,inc), forma(State,vertical),
right(State,yes), above face(State,no),. . .
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 19 / 23

20. Experiments Gesture recognition
Overall accuracy FOSeq/HMMs
2 sequences 10 sequences
FOSeq HMMs FOSeq HMMs
95.17 % 94.85 % 97.34 % 97.56 %
Confusion matrix for 10 sequences
Prediction
1 2 3 4 5 6 7 8 9 Total
1 99.33 0.67 300
2 97.24 2.76 290
3 100.00 290
Real class
4 13.10 1.72 85.17 290
5 100.00 300
6 1.03 97.59 1.38 290
7 2.07 96.55 1.38 290
8 100.00 290
9 100.00 290
Total 301 320 295 255 306 283 280 292 298 2630
1) attention, 2) come, 3) left, 4) point, 5) right, 6) stop, 7) turn-left, 8) turn-right, 9) waving.
Best classified gestures: left, right, turn-right and waving.
Worst classified gesture: point, (13 % were classified as come).
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 20 / 23

21. Experiments Gesture recognition
Example of gesture grammars
Come Point
S → ...R8,size(dec),R6,R1,R12,R7
S → ... R3,size(dec),form(horizontal),
R1 → above head(State,no),
right(si),above head(no),over torso(no)
over torso(State,yes)
R1 → above head(State,no),
R8 → hmov(State,right),
over torso(State,yes)
vmov(State,down)
R3 → hmov(State,right),
...
vmov(State,down)
Relational grammars help to find similarities between different gestures
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 21 / 23

22. Conclusions
Conclusions
◮ We have introduced an algorithm called FOSeq, that takes
sequences of states and actions and induces a grammar able to
parse and reproduce the sequences.
◮ FOSeq learns a grammar for each sequence, followed by a
generalization process between the best evaluated grammar and
other grammars to produce a generalized grammar covering most
of the sequences.
◮ FOSeq has been applied to learn navigation tasks and to learn
grammars from gesture sequences with very competitive results.
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 22 / 23

23. Future work
Future work
◮ Learn more TRPs to solve other robotic tasks,
◮ extend the experiments with gestures to obtain a general grammar
for a gesture performed by more than one person,
◮ reproduce gestures with a manipulator.
Thank you for your attention
´
(Robotics Lab INAOE, Mexico) CIARP 09 November 18, 2009 23 / 23