Towards Spatial Perception: Learning to Locate Objects from Vision #robotDoC

Juxi Leitner
Juxi LeitnerResearch Fellow (Robotic Vision & Actions) at Brisbane.AI
Jürgen ’Juxi’ Leitner
                                             S. Harding, M. Frank, A. Förster, J. Schmidhuber

                                             istituto dalle molle di studi sull’intelligenza artificiale
                                                                    università della svizzera italiana
                                                                                      idsia / usi / supsi




                                                   towards
                                         spatial perception
                             learning to locate objects from vision

                                                                                     #robotdoc 2012


Monday, September 10, 2012
iCub



                             thanks to G. Metta, IIT and U.Plymouth for their pictures

Monday, September 10, 2012
our iCub
                              setup is for
                             manipulation
Monday, September 10, 2012
thanks to G. Metta and IIT for this picture




                                                   visual
                                              perception




Monday, September 10, 2012
three parts




Monday, September 10, 2012
detecting
            objects
                             Harding et al., GPTP 2012, Leitner et al., ICDL 2012
Monday, September 10, 2012
Image credit: NASA




                 also works for
                detecting rocks
                on Mars
                                  Leitner et al., iSAIRAS 2012
Monday, September 10, 2012
detecting
            objects
                             Harding et al., GPTP 2012, Leitner et al., ICDL 2012
Monday, September 10, 2012
icVision
        framework            Leitner et al., BICA 2012

Monday, September 10, 2012
three parts
  GPTP2012, ICDL2012(?)                    BICA 2012




Monday, September 10, 2012
locate
                             objects
Monday, September 10, 2012
current
         localisation approaches
Monday, September 10, 2012
learning
                                 approach


                             +


Monday, September 10, 2012
learning
                             setup




Monday, September 10, 2012
spatial perception
                             neural network
Monday, September 10, 2012
spatial perception
                             genetic programming
Monday, September 10, 2012
x = 17.81 − 0.01906 v1 + 0.1527 v4 + 0.1378 v7 +
                               0.01108 v10 − 0.0296 v11 − 0.1207 v12

                              y = 1.124224045 + 0.1295920897 v10 +
                                0.1156011386 v8 + 0.01695234993 v0




                                        spatial perception
                              genetic programming
Monday, September 10, 2012
relative
                             errors
Monday, September 10, 2012
transferring
                             spatial perception
Monday, September 10, 2012
future
            work

                 automate the learning (IROS 2012)
                 different spatial representations
                 “intrinsic motivation” learning
                 online, continuous learning
                 eye-hand coordination


Monday, September 10, 2012
thanks
                                                             for listening

                             juxi@idsia.ch   http://Juxi.net/projects


Monday, September 10, 2012
1 of 21

Recommended

The year 2 sleepover by
The year 2 sleepoverThe year 2 sleepover
The year 2 sleepovergemmasharp
204 views2 slides
Foe lndscape aug_2012 by
Foe lndscape aug_2012Foe lndscape aug_2012
Foe lndscape aug_2012Mike KEPPELL
648 views19 slides
Autonomous Learning of Robust Visual Object Detection & Identification on a H... by
Autonomous Learning of Robust Visual Object Detection & Identification on a H...Autonomous Learning of Robust Visual Object Detection & Identification on a H...
Autonomous Learning of Robust Visual Object Detection & Identification on a H...Juxi Leitner
837 views20 slides
IDSIA's Presentation IM-CLeVeR, WP4 by
IDSIA's Presentation IM-CLeVeR, WP4IDSIA's Presentation IM-CLeVeR, WP4
IDSIA's Presentation IM-CLeVeR, WP4Juxi Leitner
321 views11 slides
ALife in Humanoid Robots #ecal2013 by
ALife in Humanoid Robots #ecal2013ALife in Humanoid Robots #ecal2013
ALife in Humanoid Robots #ecal2013Juxi Leitner
938 views24 slides
Joining Up Content by
Joining Up ContentJoining Up Content
Joining Up ContentAlastair Dunning
509 views15 slides

More Related Content

Similar to Towards Spatial Perception: Learning to Locate Objects from Vision #robotDoC

FailChat: UX Comes First Because UX is Everything! by
FailChat: UX Comes First Because UX is Everything!FailChat: UX Comes First Because UX is Everything!
FailChat: UX Comes First Because UX is Everything!ellendunne
1.4K views42 slides
Artificial Neural Networks For Spatial Perception: Towards Visual Object Loca... by
Artificial Neural Networks For Spatial Perception: Towards Visual Object Loca...Artificial Neural Networks For Spatial Perception: Towards Visual Object Loca...
Artificial Neural Networks For Spatial Perception: Towards Visual Object Loca...Juxi Leitner
1.1K views17 slides
Learning organisations and design thinking by
Learning organisations and design thinkingLearning organisations and design thinking
Learning organisations and design thinkingemilia åström
1.2K views46 slides
Website Usability | Class 1 by
Website Usability | Class 1Website Usability | Class 1
Website Usability | Class 1studiokandm
1.4K views182 slides
Persuasive design presentationd3=r1 by
Persuasive design presentationd3=r1Persuasive design presentationd3=r1
Persuasive design presentationd3=r1Sebastian Daum
371 views37 slides
Persuasive design presentationd3=r1 by
Persuasive design presentationd3=r1Persuasive design presentationd3=r1
Persuasive design presentationd3=r1Sebastian Daum
873 views37 slides

Similar to Towards Spatial Perception: Learning to Locate Objects from Vision #robotDoC(8)

FailChat: UX Comes First Because UX is Everything! by ellendunne
FailChat: UX Comes First Because UX is Everything!FailChat: UX Comes First Because UX is Everything!
FailChat: UX Comes First Because UX is Everything!
ellendunne1.4K views
Artificial Neural Networks For Spatial Perception: Towards Visual Object Loca... by Juxi Leitner
Artificial Neural Networks For Spatial Perception: Towards Visual Object Loca...Artificial Neural Networks For Spatial Perception: Towards Visual Object Loca...
Artificial Neural Networks For Spatial Perception: Towards Visual Object Loca...
Juxi Leitner1.1K views
Learning organisations and design thinking by emilia åström
Learning organisations and design thinkingLearning organisations and design thinking
Learning organisations and design thinking
emilia åström1.2K views
Website Usability | Class 1 by studiokandm
Website Usability | Class 1Website Usability | Class 1
Website Usability | Class 1
studiokandm1.4K views
Persuasive design presentationd3=r1 by Sebastian Daum
Persuasive design presentationd3=r1Persuasive design presentationd3=r1
Persuasive design presentationd3=r1
Sebastian Daum371 views
Persuasive design presentationd3=r1 by Sebastian Daum
Persuasive design presentationd3=r1Persuasive design presentationd3=r1
Persuasive design presentationd3=r1
Sebastian Daum873 views
Mars Terrain Image Classification Using Cartesian Genetic Programming #isaira... by Juxi Leitner
Mars Terrain Image Classification Using Cartesian Genetic Programming #isaira...Mars Terrain Image Classification Using Cartesian Genetic Programming #isaira...
Mars Terrain Image Classification Using Cartesian Genetic Programming #isaira...
Juxi Leitner1.1K views
Virtual Worlds in Higher Education: More Than Simulation (A Case-Study of Sec... by Simon Kear
Virtual Worlds in Higher Education: More Than Simulation (A Case-Study of Sec...Virtual Worlds in Higher Education: More Than Simulation (A Case-Study of Sec...
Virtual Worlds in Higher Education: More Than Simulation (A Case-Study of Sec...
Simon Kear1K views

More from Juxi Leitner

The Need For Robots To Grasp the World by
The Need For Robots To Grasp the WorldThe Need For Robots To Grasp the World
The Need For Robots To Grasp the WorldJuxi Leitner
330 views30 slides
Robots that Grasp the World by
Robots that Grasp the WorldRobots that Grasp the World
Robots that Grasp the WorldJuxi Leitner
609 views41 slides
Improving Robotic Manipulation with Vision and Learning @AmazonDevCentre Berlin by
Improving Robotic Manipulation with Vision and Learning @AmazonDevCentre BerlinImproving Robotic Manipulation with Vision and Learning @AmazonDevCentre Berlin
Improving Robotic Manipulation with Vision and Learning @AmazonDevCentre BerlinJuxi Leitner
274 views29 slides
Cartman, how to win the amazon robotics challenge with robotic vision and dee... by
Cartman, how to win the amazon robotics challenge with robotic vision and dee...Cartman, how to win the amazon robotics challenge with robotic vision and dee...
Cartman, how to win the amazon robotics challenge with robotic vision and dee...Juxi Leitner
459 views51 slides
Deep Reinforcement Learning | Amazon Robotics Challenge, Image Processing Lec... by
Deep Reinforcement Learning | Amazon Robotics Challenge, Image Processing Lec...Deep Reinforcement Learning | Amazon Robotics Challenge, Image Processing Lec...
Deep Reinforcement Learning | Amazon Robotics Challenge, Image Processing Lec...Juxi Leitner
317 views65 slides
ACRV Picking Benchmark: how to benchmark pick and place robotics research by
ACRV Picking Benchmark: how to benchmark pick and place robotics researchACRV Picking Benchmark: how to benchmark pick and place robotics research
ACRV Picking Benchmark: how to benchmark pick and place robotics researchJuxi Leitner
424 views14 slides

More from Juxi Leitner(20)

The Need For Robots To Grasp the World by Juxi Leitner
The Need For Robots To Grasp the WorldThe Need For Robots To Grasp the World
The Need For Robots To Grasp the World
Juxi Leitner330 views
Robots that Grasp the World by Juxi Leitner
Robots that Grasp the WorldRobots that Grasp the World
Robots that Grasp the World
Juxi Leitner609 views
Improving Robotic Manipulation with Vision and Learning @AmazonDevCentre Berlin by Juxi Leitner
Improving Robotic Manipulation with Vision and Learning @AmazonDevCentre BerlinImproving Robotic Manipulation with Vision and Learning @AmazonDevCentre Berlin
Improving Robotic Manipulation with Vision and Learning @AmazonDevCentre Berlin
Juxi Leitner274 views
Cartman, how to win the amazon robotics challenge with robotic vision and dee... by Juxi Leitner
Cartman, how to win the amazon robotics challenge with robotic vision and dee...Cartman, how to win the amazon robotics challenge with robotic vision and dee...
Cartman, how to win the amazon robotics challenge with robotic vision and dee...
Juxi Leitner459 views
Deep Reinforcement Learning | Amazon Robotics Challenge, Image Processing Lec... by Juxi Leitner
Deep Reinforcement Learning | Amazon Robotics Challenge, Image Processing Lec...Deep Reinforcement Learning | Amazon Robotics Challenge, Image Processing Lec...
Deep Reinforcement Learning | Amazon Robotics Challenge, Image Processing Lec...
Juxi Leitner317 views
ACRV Picking Benchmark: how to benchmark pick and place robotics research by Juxi Leitner
ACRV Picking Benchmark: how to benchmark pick and place robotics researchACRV Picking Benchmark: how to benchmark pick and place robotics research
ACRV Picking Benchmark: how to benchmark pick and place robotics research
Juxi Leitner424 views
Team ACRV's experience at #AmazonPickingChallenge 2016 by Juxi Leitner
Team ACRV's experience at #AmazonPickingChallenge 2016Team ACRV's experience at #AmazonPickingChallenge 2016
Team ACRV's experience at #AmazonPickingChallenge 2016
Juxi Leitner205 views
(deep) reinforcement learning - CAB420 by Juxi Leitner
(deep) reinforcement learning - CAB420(deep) reinforcement learning - CAB420
(deep) reinforcement learning - CAB420
Juxi Leitner619 views
ACRV : Robotic Vision presentation in Lisbon at IST by Juxi Leitner
ACRV : Robotic Vision presentation in Lisbon at ISTACRV : Robotic Vision presentation in Lisbon at IST
ACRV : Robotic Vision presentation in Lisbon at IST
Juxi Leitner342 views
The Australian Centre for Robotic Vision (ACRV) by Juxi Leitner
The Australian Centre for Robotic Vision (ACRV)The Australian Centre for Robotic Vision (ACRV)
The Australian Centre for Robotic Vision (ACRV)
Juxi Leitner338 views
How to place 6th in the Amazon Picking Challenge (ENB329, QUT) by Juxi Leitner
How to place 6th in the Amazon Picking Challenge (ENB329, QUT)How to place 6th in the Amazon Picking Challenge (ENB329, QUT)
How to place 6th in the Amazon Picking Challenge (ENB329, QUT)
Juxi Leitner268 views
LunaRoo: Designing a Hopping Lunar Science Payload #space #exploration by Juxi Leitner
LunaRoo: Designing a Hopping Lunar Science Payload #space #explorationLunaRoo: Designing a Hopping Lunar Science Payload #space #exploration
LunaRoo: Designing a Hopping Lunar Science Payload #space #exploration
Juxi Leitner1.2K views
Robotic Vision - Vision for Robotics #IEEE #QLD #CIS #Colloquium by Juxi Leitner
Robotic Vision - Vision for Robotics #IEEE #QLD #CIS #ColloquiumRobotic Vision - Vision for Robotics #IEEE #QLD #CIS #Colloquium
Robotic Vision - Vision for Robotics #IEEE #QLD #CIS #Colloquium
Juxi Leitner902 views
ROS Hands-On Intro/Tutorial (Robotic Vision Summer School 2015) #RVSS #ACRV by Juxi Leitner
ROS Hands-On Intro/Tutorial (Robotic Vision Summer School 2015) #RVSS #ACRVROS Hands-On Intro/Tutorial (Robotic Vision Summer School 2015) #RVSS #ACRV
ROS Hands-On Intro/Tutorial (Robotic Vision Summer School 2015) #RVSS #ACRV
Juxi Leitner3.6K views
ACRV Research Fellow Intro/Tutorial [Vision and Action] by Juxi Leitner
ACRV Research Fellow Intro/Tutorial [Vision and Action]ACRV Research Fellow Intro/Tutorial [Vision and Action]
ACRV Research Fellow Intro/Tutorial [Vision and Action]
Juxi Leitner536 views
From Vision to Actions - Towards Adaptive & Autonomous Humanoid Robots [PhD D... by Juxi Leitner
From Vision to Actions - Towards Adaptive & Autonomous Humanoid Robots [PhD D...From Vision to Actions - Towards Adaptive & Autonomous Humanoid Robots [PhD D...
From Vision to Actions - Towards Adaptive & Autonomous Humanoid Robots [PhD D...
Juxi Leitner2K views
Reactive Reaching and Grasping on a Humanoid: Towards Closing the Action-Perc... by Juxi Leitner
Reactive Reaching and Grasping on a Humanoid: Towards Closing the Action-Perc...Reactive Reaching and Grasping on a Humanoid: Towards Closing the Action-Perc...
Reactive Reaching and Grasping on a Humanoid: Towards Closing the Action-Perc...
Juxi Leitner1.4K views
Tele-operation of a Humanoid Robot, Using Operator Bio-data by Juxi Leitner
Tele-operation of a Humanoid Robot, Using Operator Bio-dataTele-operation of a Humanoid Robot, Using Operator Bio-data
Tele-operation of a Humanoid Robot, Using Operator Bio-data
Juxi Leitner1.1K views
Improving Robot Vision Models for Object Detection Through Interaction #ijcnn... by Juxi Leitner
Improving Robot Vision Models for Object Detection Through Interaction #ijcnn...Improving Robot Vision Models for Object Detection Through Interaction #ijcnn...
Improving Robot Vision Models for Object Detection Through Interaction #ijcnn...
Juxi Leitner922 views
How does it feel to be a SpaceMaster? [Erasmus Mundus - ACE Talk] by Juxi Leitner
How does it feel to be a SpaceMaster? [Erasmus Mundus - ACE Talk]How does it feel to be a SpaceMaster? [Erasmus Mundus - ACE Talk]
How does it feel to be a SpaceMaster? [Erasmus Mundus - ACE Talk]
Juxi Leitner707 views

Recently uploaded

Case Study Copenhagen Energy and Business Central.pdf by
Case Study Copenhagen Energy and Business Central.pdfCase Study Copenhagen Energy and Business Central.pdf
Case Study Copenhagen Energy and Business Central.pdfAitana
16 views3 slides
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive by
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLiveAutomating a World-Class Technology Conference; Behind the Scenes of CiscoLive
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLiveNetwork Automation Forum
31 views35 slides
SAP Automation Using Bar Code and FIORI.pdf by
SAP Automation Using Bar Code and FIORI.pdfSAP Automation Using Bar Code and FIORI.pdf
SAP Automation Using Bar Code and FIORI.pdfVirendra Rai, PMP
23 views38 slides
Evolving the Network Automation Journey from Python to Platforms by
Evolving the Network Automation Journey from Python to PlatformsEvolving the Network Automation Journey from Python to Platforms
Evolving the Network Automation Journey from Python to PlatformsNetwork Automation Forum
13 views21 slides
Transcript: The Details of Description Techniques tips and tangents on altern... by
Transcript: The Details of Description Techniques tips and tangents on altern...Transcript: The Details of Description Techniques tips and tangents on altern...
Transcript: The Details of Description Techniques tips and tangents on altern...BookNet Canada
136 views15 slides

Recently uploaded(20)

Case Study Copenhagen Energy and Business Central.pdf by Aitana
Case Study Copenhagen Energy and Business Central.pdfCase Study Copenhagen Energy and Business Central.pdf
Case Study Copenhagen Energy and Business Central.pdf
Aitana16 views
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive by Network Automation Forum
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLiveAutomating a World-Class Technology Conference; Behind the Scenes of CiscoLive
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive
SAP Automation Using Bar Code and FIORI.pdf by Virendra Rai, PMP
SAP Automation Using Bar Code and FIORI.pdfSAP Automation Using Bar Code and FIORI.pdf
SAP Automation Using Bar Code and FIORI.pdf
Transcript: The Details of Description Techniques tips and tangents on altern... by BookNet Canada
Transcript: The Details of Description Techniques tips and tangents on altern...Transcript: The Details of Description Techniques tips and tangents on altern...
Transcript: The Details of Description Techniques tips and tangents on altern...
BookNet Canada136 views
Empathic Computing: Delivering the Potential of the Metaverse by Mark Billinghurst
Empathic Computing: Delivering  the Potential of the MetaverseEmpathic Computing: Delivering  the Potential of the Metaverse
Empathic Computing: Delivering the Potential of the Metaverse
Mark Billinghurst478 views
Igniting Next Level Productivity with AI-Infused Data Integration Workflows by Safe Software
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software263 views
Unit 1_Lecture 2_Physical Design of IoT.pdf by StephenTec
Unit 1_Lecture 2_Physical Design of IoT.pdfUnit 1_Lecture 2_Physical Design of IoT.pdf
Unit 1_Lecture 2_Physical Design of IoT.pdf
StephenTec12 views
STPI OctaNE CoE Brochure.pdf by madhurjyapb
STPI OctaNE CoE Brochure.pdfSTPI OctaNE CoE Brochure.pdf
STPI OctaNE CoE Brochure.pdf
madhurjyapb14 views
Five Things You SHOULD Know About Postman by Postman
Five Things You SHOULD Know About PostmanFive Things You SHOULD Know About Postman
Five Things You SHOULD Know About Postman
Postman33 views
PharoJS - Zürich Smalltalk Group Meetup November 2023 by Noury Bouraqadi
PharoJS - Zürich Smalltalk Group Meetup November 2023PharoJS - Zürich Smalltalk Group Meetup November 2023
PharoJS - Zürich Smalltalk Group Meetup November 2023
Noury Bouraqadi127 views
Business Analyst Series 2023 - Week 3 Session 5 by DianaGray10
Business Analyst Series 2023 -  Week 3 Session 5Business Analyst Series 2023 -  Week 3 Session 5
Business Analyst Series 2023 - Week 3 Session 5
DianaGray10248 views
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院 by IttrainingIttraining
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院

Towards Spatial Perception: Learning to Locate Objects from Vision #robotDoC

  • 1. Jürgen ’Juxi’ Leitner S. Harding, M. Frank, A. Förster, J. Schmidhuber istituto dalle molle di studi sull’intelligenza artificiale università della svizzera italiana idsia / usi / supsi towards spatial perception learning to locate objects from vision #robotdoc 2012 Monday, September 10, 2012
  • 2. iCub thanks to G. Metta, IIT and U.Plymouth for their pictures Monday, September 10, 2012
  • 3. our iCub setup is for manipulation Monday, September 10, 2012
  • 4. thanks to G. Metta and IIT for this picture visual perception Monday, September 10, 2012
  • 6. detecting objects Harding et al., GPTP 2012, Leitner et al., ICDL 2012 Monday, September 10, 2012
  • 7. Image credit: NASA also works for detecting rocks on Mars Leitner et al., iSAIRAS 2012 Monday, September 10, 2012
  • 8. detecting objects Harding et al., GPTP 2012, Leitner et al., ICDL 2012 Monday, September 10, 2012
  • 9. icVision framework Leitner et al., BICA 2012 Monday, September 10, 2012
  • 10. three parts GPTP2012, ICDL2012(?) BICA 2012 Monday, September 10, 2012
  • 11. locate objects Monday, September 10, 2012
  • 12. current localisation approaches Monday, September 10, 2012
  • 13. learning approach + Monday, September 10, 2012
  • 14. learning setup Monday, September 10, 2012
  • 15. spatial perception neural network Monday, September 10, 2012
  • 16. spatial perception genetic programming Monday, September 10, 2012
  • 17. x = 17.81 − 0.01906 v1 + 0.1527 v4 + 0.1378 v7 + 0.01108 v10 − 0.0296 v11 − 0.1207 v12 y = 1.124224045 + 0.1295920897 v10 + 0.1156011386 v8 + 0.01695234993 v0 spatial perception genetic programming Monday, September 10, 2012
  • 18. relative errors Monday, September 10, 2012
  • 19. transferring spatial perception Monday, September 10, 2012
  • 20. future work automate the learning (IROS 2012) different spatial representations “intrinsic motivation” learning online, continuous learning eye-hand coordination Monday, September 10, 2012
  • 21. thanks for listening juxi@idsia.ch http://Juxi.net/projects Monday, September 10, 2012