The slides for my PhD proposal presentation in Nov 2013 at the Università della Svizzera Italiana (USI). The proposal can be found on my webpage: http://Juxi.net/phd/

1. Juxi Leitner
istituto dalle molle di studi sull’intelligenza artificiale
idsia / supsi / usi
From Vision to Actions
towards adaptive & autonomous robots

2. adapt |əәˈdapt|
verb [ with obj. ]
[...]
• [ no obj. ] become adjusted to new conditions:
a large organization can be slow to adapt to change.
[...]
adaptive

3. adaptive

4. autonomous
autonomous |ɔːˈtɒnəәməәs|
adjective
(of a region) having the freedom to govern itself or control its own
affairs: the federation included sixteen autonomous republics.
• having the freedom to act independently: school governors are legally
autonomous.
[...]

5. autonomous

6. robotics

7. [Metta et al., 2010]
iCub
thanks to G. Metta, IIT and U.Plymouth for their pictures

8. aimed at
manipulation
[Metta et al., 2010]

9. “CHESS is
SOLVED”

10. towards learned
object manipulation
http://robotics.idsia.ch/

11. PhD Project
parts

12. visual
perception

13. robust object
detection
[Leitner et al, 2012a/b, Harding et al., 2013]

14. current
cv approaches

15. INP
INP
INP
+
dilate
min
avg
cartesian genetic
programming
[Leitner et al, 2012a/b, Harding et al., 2013]

16. !
!
!
!
!
!
!
!
!
icImage* BlueCupDetector::runFilter() {
icImage* node0 = new GpImage(-8.2, ImgW, ImgH);
icImage* node1 = InputImages[4];
icImage* node2 = node0->avg(node1);
icImage* node4 = node2->threshold(64);
icImage* node6 = node4->erode(5);
icImage* node42 = node6->SmoothMedian(5);
icImage* node79 = node42->gauss(9);
icImage* node99 = node79->dilate(5);
}
//cleanup
...
return node99;
detection

17. [Leitner et al, 2013]
visual
hand detection

18. detecting
hands

19. locate

20. [P2,1] * F * [P1,1]' = 0
current
localisation approaches

21. transferring
spatial perception
[Leitner et al, IROS2012]

22. F
Ob
9DCu
i
CoCar
or tes
din ian
ate
s
ox
g be
din ey
un er
bo 6 p
.
.
.
trainingset

23. 9DOF
iCub
bounding box
6 per eye
fully connected
.
.
.
Cartesian
Coordinates
fully connected
.
.
.
spatial perception
neural network
[Leitner et al, IJCNN 2013]

24. -200
300
-300
500
200
-400
450
100
-500
0
-600
Y (mm)
400
350
300
Z (mm)
400
550
X (mm)
600
-100
-200
-800
-300
150
-1000
-500
200
-900
-400
250
-1100
Predicted
Expected
-600
100
0
200
400
600
Sample Index
800
1000
-700
0
200
400
600
Sample Index
800
1000
-300
-400
-500
-600
-700
results
-800
-900
-1000
-1100
-1200
-1300
0
200
400
600
Sample Index
800
1000
-1200
-1300
-200
Z (mm)
-700
ANN
[Leitner et al, IJCNN 2013]
0

25. model update
[Leitner et al, IJARS 2012, BICA 2013]

26. robot
motion

27. robot
motion
[Stollenga et al, IROS 2013]

28. MoBeE v1
[Frank et al., 2011,2012, 2013]

29. model

30. roadmap
exploration

31. generating
useful actions
[Stollenga et al., IROS 2013]

32. action
repertoire

33. action
repertoire
[Leitner et al, in prep]

34. op-space forcing
hand/arm
[Leitner et al, in prep]
CSR/CSL
CSHand
CSWorld
q’ = J(q)-¹ v

35. hand-eye
coordination

36. action perception loop

37. first
integration
[Leitner et al, in prep]

40. futureworkpost PhD
improving online, continuous learning
sensorimotor integration
active vision/learning
action outcome/prediction
human-robot interaction

41. http://dilbert.com/strips/comic/2013-10-24/
thank
you

42. after 30 months:
breadth (15 ECTS)
Teaching
publications
USI PhD
requirements

43. robotics conference papers
space robotics conference paper
robotics journal paper
IROS, EpiRob
iSAIRAS
IJARS
outreach & publications
AI/ML conference papers
journal publications
GP book chapter
IJCNN, CEC, BICA
BICA, Acta Futura
+ journal papers in preparation Frontiers, RAS, CSEd

44. outreach & publications