My presentation, given at multiple labs in Europe, about the activities of the ACRV in July 2016. (including TUBerlin, DLR Oberpfaffenhofen,...)

1. Juxi Leitner
arc centre of excellence for robotic vision
queensland university of technology
<j.leitner@qut.edu.au> http://Juxi.net
robotic vision
an australian vision for robotics
ACRV @ UC Berkeley

2. http://roboticvision.org/2
tinyurl.com/QUTRobotics
Brisbane, Australia

3. http://roboticvision.org/

4. SIFR slide

5. http://roboticvision.org/

6. http://roboticvision.org/
tinyurl.com/QUTRobotics
roboticvision.org

7. http://roboticvision.org/
10 96%eye
designs
animals
haveeyes
Nature Reviews | Neuroscience
Protostomes
Bilateria
Ecdysozoa
Lophotrochozoa
~580 Mya
~550 Mya
~530 Mya
~500 Mya
~430 Mya
Deuterostomes
Chordates*
Craniates*
Vertebrates*
Gnathostomes*
Arthropods
Annelids
Molluscs
Hemichordates
Echinoderms
Cephalochordates*
Tunicates*
Myxiniformes*
Petromyzoniformes*
Last fossil jawless fish
Stages of interest in
vertebrate eye evolution Cambrian
Mya
600 550 500 450 400 0
21 3 4 5 6
Lampreys
Jawed vertebrates
Hagfish
Lancelets
Sea squirts
Ocellus
Eye patch
Figure1|The origin of vertebrates. Theevolutionofjawedvertebratesisillustratedagainstanapproximatetime-scaleof
millionsofyearsago(Mya).ThetaxaconsideredinthisReviewareindicatedwithanasteriskandareaccompaniedby
schematicsanddiagramsofthe‘eye’region.Theearliestchordates,representedbyextantcephalochordatesandtunicates,
arethoughttohaveappearedaround550Mya.Jawlesscraniates(agnathans)werepresentintheearlyCambrian,by525Mya,
andatimeof530Myahasbeenindicatedfortheirpresumedfirstappearance.AselaboratedoninBOX 1,thereis
considerablecontroversyastowhethermyxiniformes(solelyrepresentedbyextanthagfish)divergedbeforeorafterthe
separationoflampreysfromjawedvertebrates(shownasdashedblackandgreylines).Numerouslinesofjawlessfishevolved
between500and430Myaago,althoughnonehavesurvivedtothepresentday.Thefirstjawedvertebratearosearound430
Mya,andthislineisrepresentedtodaybycartilagenousfish,bonyfishandtetrapods.Six‘stagesofinterest’invertebrateeye
evolutioncorrespondtothetimeintervalsbetweenthedivergenceofimportantsurvivingtaxa.Thisdiagramdoesnot
includetheevolutionarychangesthathaveoccurredinthelast400millionyears.Thepresentedtimelineisbasedprimarily
onevidencefromthefossilrecord;seeREFS 2,13,15,17,18,144,160–163.Theschematicsaremodified,withpermission,from
REF. 11 (1996)OxfordUniversityPress(lancelet,seasquirt,hagfishandlamprey)andREF. 164 (2004)AcademicPress
(jawedvertebrate).Theeyeimagesarereproduced,withpermission,fromthefollowingreferences:lancelet,REF. 165
BIODIDAC(1996)UniversityofCaliforniaMuseumofPaleontology;seasquirt,REF. 63 (2006)BlackwellPublishing;hagfish,
REF. 166 (2006)AustralianMuseum.LampreyandjawedvertebrateeyeimagesarecourtesyofG.WesthoffandS.P.Collin).
CIENCE VOLUME 8 | DECEMBER 2007 | 961

8. http://roboticvision.org/
Australian Centre for Robotic Vision
Robust Vision
Vision and Ac/on
Seman/c Representa/ons
Algorithms and Architecture
Visual Learning

9. Dalle Molle Institute for AI (IDSIA)
Work
Juxi
Leitner
PhD Informatics / Intelligent Systems
MSc Space Robotics & Automation
BSc Information & Software Engineering
Intelligent (Space) Robots
European Space Agency (ESA)
Erasmus Intelligent Systems
Work (Humanoid) Robot Vision
Instituto Superior Técnico (IST)
Mobility Intelligent Space Systems Laboratory
About Me
Current Robotic Vision and Actions
h9p://Juxi.net
Queensland University of Technology (QUT)

10. coordination
eye-hand
move to see
see to move
Vision and Action (VA)
http://Juxi.net/projects/VisionAndAction/

11. http://Juxi.net/projects
https://www.youtube.com/watch?v=w_qDH5tSe7g&list=PLOqMcGR_zUyH4IaYurBhB5D02TurBQOQW&index=2

12. manipulation
for improved perception

13. http://roboticvision.org/
cameraslightfield
work lead by Donald G. Dansereau
Deblurring
3D scenes yield complex blur
Generalized deconvolution for
6DOF motion

14. http://roboticvision.org/
propertiesdetectingvisually
[Dansereau, Singh, Leitner, ICRA2016]
http://Juxi.net/projects/squishy/

15. http://roboticvision.org/

16. http://roboticvision.org/
http://Juxi.net/projects/AmazonPickingChallenge/

17. http://roboticvision.org/
http://Juxi.net/projects/AmazonPickingChallenge/
systemdesign

18. http://Juxi.net/projects/AmazonPickingChallenge/
systemdesign
Object
Detection
Object
Localisation
Object
Identification
Shelf
Localisation
Move
to Bin
Grasp
Selection
Grasp
Execution

19. systemdesign
Object
Detection
Object
Localisation
Object
Identification
Shelf
Localisation
perception
Kinect2
(once)
RealSense
(every attempt)
KinectFusion
2D regions
(CNN)
point cloud
segmentation
grasp point
detection
http://Juxi.net/projects/AmazonPickingChallenge/

20. http://roboticvision.org/
score

21. http://roboticvision.org/

22. ARC Centre of Excellence for Robotic Vision roboticvision.org
fruit picking
https://www.youtube.com/watch?v=8rq4iSTsg68

23. tinyurl.com/QUTLunaRoo
LUNAROO: CREATING A HOPPING
LUNAR SCIENCE PAYLOAD

24. tinyurl.com/QUTLunaRoo STUDENT SUMMER PROJECT
ROB LEE, TIM HOJNIK
SKP-1

25. tinyurl.com/QUTLunaRoo STUDENT SUMMER PROJECT
ROB LEE, TIM HOJNIK
Robotic
Vision Testbed

26. tinyurl.com/QUTLunaRoo
Going
Higher
SKP-2SKP-1A

27. tinyurl.com/QUTLunaRoo TIM HOJNIK
Leaving Footprints

28. [Zhang et al, ACRA 2015, IROS 2016]
understanding limita/ons of deep nets,
reinforcement learning & transfer of knowledge
deep learning visual control

29. http://roboticvision.org/

30. http://roboticvision.org/
http://Juxi.net/special-issue/ijrr-deep-learning-2016/
IJRR Special Issue
R:SS WS
(what) scep/cs?
deeper understanding
tool (how to use it?)
RV vs CV
data driven models

31. for listening
thank you j.leitner@roboticvision.org
http://Juxi.net/projects
ICRA 2018
Brisbane,Australia

32. for listening
thank you
j.leitner@roboticvision.org
http://Juxi.net/projects