Robotic Vision - Vision for Robotics #IEEE #QLD #CIS #Colloquium
•
3 likes•903 views
Presenation about my current research in computer vision, machine learning and robotics at the IEEE Queensland Computational Intelligence Society Colloquium at Griffith University.
More Related Content
Similar to Robotic Vision - Vision for Robotics #IEEE #QLD #CIS #Colloquium
Similar to Robotic Vision - Vision for Robotics #IEEE #QLD #CIS #Colloquium (20)
More from Juxi Leitner
More from Juxi Leitner (20)
Recently uploaded
Recently uploaded (20)
Robotic Vision - Vision for Robotics #IEEE #QLD #CIS #Colloquium
- 1. Robotic Vision Jürgen ‘Juxi’ Leitner arc centre of excellence for robotic vision queensland university of technology a vision for robotics j.leitner@qut.edu.au - http://Juxi.net
- 3. http://roboticvision.org/3 recognising objects & stuff recognising places detec4ng mo4on move to see see to move context for seeing seeing for context seeing creates memories memory helps seeing paying a;en4on recognizing humans, their ac4vi4es and intent Seeing
- 4. http://roboticvision.org/4 10 96%eye designs animals haveeyes Nature Reviews | Neuroscience Protostomes Bilateria Ecdysozoa Lophotrochozoa ~580 Mya ~550 Mya ~530 Mya ~500 Mya ~430 Mya Deuterostomes Chordates* Craniates* Vertebrates* Gnathostomes* Arthropods Annelids Molluscs Hemichordates Echinoderms Cephalochordates* Tunicates* Myxiniformes* Petromyzoniformes* Last fossil jawless fish Stages of interest in vertebrate eye evolution Cambrian Mya 600 550 500 450 400 0 21 3 4 5 6 Lampreys Jawed vertebrates Hagfish Lancelets Sea squirts Ocellus Eye patch Figure1|The origin of vertebrates. Theevolutionofjawedvertebratesisillustratedagainstanapproximatetime-scaleof millionsofyearsago(Mya).ThetaxaconsideredinthisReviewareindicatedwithanasteriskandareaccompaniedby schematicsanddiagramsofthe‘eye’region.Theearliestchordates,representedbyextantcephalochordatesandtunicates, arethoughttohaveappearedaround550Mya.Jawlesscraniates(agnathans)werepresentintheearlyCambrian,by525Mya, andatimeof530Myahasbeenindicatedfortheirpresumedfirstappearance.AselaboratedoninBOX 1,thereis considerablecontroversyastowhethermyxiniformes(solelyrepresentedbyextanthagfish)divergedbeforeorafterthe separationoflampreysfromjawedvertebrates(shownasdashedblackandgreylines).Numerouslinesofjawlessfishevolved between500and430Myaago,althoughnonehavesurvivedtothepresentday.Thefirstjawedvertebratearosearound430 Mya,andthislineisrepresentedtodaybycartilagenousfish,bonyfishandtetrapods.Six‘stagesofinterest’invertebrateeye evolutioncorrespondtothetimeintervalsbetweenthedivergenceofimportantsurvivingtaxa.Thisdiagramdoesnot includetheevolutionarychangesthathaveoccurredinthelast400millionyears.Thepresentedtimelineisbasedprimarily onevidencefromthefossilrecord;seeREFS 2,13,15,17,18,144,160–163.Theschematicsaremodified,withpermission,from REF. 11 (1996)OxfordUniversityPress(lancelet,seasquirt,hagfishandlamprey)andREF. 164 (2004)AcademicPress (jawedvertebrate).Theeyeimagesarereproduced,withpermission,fromthefollowingreferences:lancelet,REF. 165 BIODIDAC(1996)UniversityofCaliforniaMuseumofPaleontology;seasquirt,REF. 63 (2006)BlackwellPublishing;hagfish, REF. 166 (2006)AustralianMuseum.LampreyandjawedvertebrateeyeimagesarecourtesyofG.WesthoffandS.P.Collin). CIENCE VOLUME 8 | DECEMBER 2007 | 961
- 10. http://roboticvision.org/ Dalle Molle Institute for AI (IDSIA) 10 Work Juxi Leitner PhD Informatics / Intelligent Systems MSc Space Robotics & Automation BSc Information & Software Engineering Intelligent (Space) Robots European Space Agency (ESA) Erasmus Intelligent Systems Work (Humanoid) Robot Vision Instituto Superior Técnico (IST) Mobility Intelligent Space Systems Laboratory About Me Work Robotic Vision and Actions h"p://Juxi.net Queensland University of Technology (QUT)
- 11. controlling spacecraft with ANNs [Leitner et al, iSAIRAS 2010]
- 12. position, velocity, orientation thruster [Leitner et al, iSAIRAS 2010]
- 13. trajectory robustness [Leitner et al, iSAIRAS 2010]
- 16. perception visual thanks to G. Metta and IIT for this picture
- 19. objects detecting Harding, Leitner, Schmidhuber, 2013 Leitner et al., ICDL 2012, IJARS 2012, BICA 2012, CEC 2013
- 20. embedding domain knowledge + mindilate avg OpenCV func5ons full images
- 21. + mindilateINP INP diff using building blocksOpenCV + dilateINP INP thresh+ mindilate avgINP blur+ mindilate avg normali ze mindilate avginput
- 22. icVision icImage* BlueCupFilter::runFilter() { icImage* node43 = InputImages[4]; icImage* node49 = node43->LocalAvg(15); icImage* out = node49->threshold(81.532f); return out; } framework
- 23. cartesian genetic programming + mindilate avgINP INP INP Harding, Leitner, Schmidhuber, 2013 Leitner et al., ICDL 2012, IJARS 2012, BICA 2012, CEC 2013
- 25. detection icImage GreenTeaBoxDetector::runFilter() { icImage node0 = InputImages[6]; icImage node1 = InputImages[1]; icImage node2 = node0.absdiff(node1); icImage node5 = node2.SmoothBilateral(11); icImage node12 = InputImages[0]; icImage node16 = node12.Sqrt(); icImage node33 = node16.erode(6); icImage node34 = node33.log(); icImage node36 = node34.min(node5); icImage node49 = node36.Normalize(); //cleanup ... icImage out = node49.threshold(230.7218f); return out; }
- 26. detect
- 27. detect
- 28. detection
- 29. approach cgp [Leitner et al, iSAIRAS 2012]
- 30. visual hand detection [Leitner et al, CEC 2013]
- 31. hands detecting [Leitner et al, CEC 2013]
- 33. clustering feature saliency map Autonomous Approach [Leitner et al, ICDL/EpiRob 2012]
- 34. pre segmentation
- 35. results comparing [Leitner et al, ICDL/EpiRob 2012] features CGP-‐IP
- 38. trainingset 9DOF iCub position in the frame 2/6 per eye Cartesian Coordinates
- 39. transferring spatial perception [Leitner et al, IROS 2012]
- 40. setup learning
- 41. trainingset 9DOF iCub position in the frame 2/6 per eye Cartesian Coordinates . . . ~1000
- 43. results ANN-1300 -1200 -1100 -1000 -900 -800 -700 -600 -500 -400 -300 -200 0 200 400 600 800 1000 Z(mm) Sample Index -700 -600 -500 -400 -300 -200 -100 0 100 200 300 400 0 200 400 600 800 1000 Y(mm) Sample Index Predicted Expected 100 150 200 250 300 350 400 450 500 550 600 0 200 400 600 800 1000 X(mm) Sample Index -1300 -1200 -1100 -1000 -900 -800 -700 -600 -500 -400 -300 -200 0 Z(mm) [Leitner et al, IJCNN 2013]
- 45. MoBeEv2 [Frank, Leitner et al. 2012, 2013]
- 46. [Frank, Leitner et al., ICINCO, 2012]
- 47. MoBeE framework [Frank, Leitner et al., ICINCO, 2012]
- 48. hand/arm op-space forcing CSWorld CSHand CSR/CSL [Leitner et al, in prep]
- 49. tele operation
- 50. motion robot [Stollenga, Leitner et al, IROS 2013]
- 54. model
- 57. http://roboticvision.org/57 case #1learning from supervised to robotic-assisted unsupervised learning Autonomous Learning Of Robust Visual Object Detection And Identification On A Humanoid. J. Leitner, P. Chandrashekhariah, S. Harding, M. Frank, G. Spina, A. Förster, J. Triesch, J. Schmidhuber. ICDL/EpiRob 2012.
- 58. http://roboticvision.org/58 case : poking segmentation before & after action
- 59. http://roboticvision.org/59 deep learning visual control [Zhang et al, ACRA 2015/ICRA2016]
- 60. http://roboticvision.org/60 deep learning visual control [Zhang et al, ACRA 2015/ICRA2016]
- 61. http://roboticvision.org/61 conclusions a novel way of object segmentation learning and teaching perception integration action-perception side reactive reaching/grasping improving perception with (inter-)actions learning neuro-controllers
- 63. http://roboticvision.org/ Australian Deep Learning WS 63 http://Juxi.net/workshop/deeplearning-applications-vision-robotics-2015/