Towards Spatial Perception: Learning to Locate Objects from Vision #robotDoC

•

1 like•849 views

Juxi Leitner

Technology Education

iCub

thanks to G. Metta, IIT and U.Plymouth for their pictures

Monday, September 10, 2012

our iCub
setup is for
manipulation
Monday, September 10, 2012

thanks to G. Metta and IIT for this picture

visual
perception

Monday, September 10, 2012

detecting
objects
Harding et al., GPTP 2012, Leitner et al., ICDL 2012
Monday, September 10, 2012

Image credit: NASA

also works for
detecting rocks
on Mars
Leitner et al., iSAIRAS 2012
Monday, September 10, 2012

icVision
framework Leitner et al., BICA 2012

Monday, September 10, 2012

three parts
GPTP2012, ICDL2012(?) BICA 2012

Monday, September 10, 2012

locate
objects
Monday, September 10, 2012

current
localisation approaches
Monday, September 10, 2012

learning
approach

+

Monday, September 10, 2012

learning
setup

Monday, September 10, 2012

spatial perception
neural network
Monday, September 10, 2012

spatial perception
genetic programming
Monday, September 10, 2012

x = 17.81 − 0.01906 v1 + 0.1527 v4 + 0.1378 v7 +
0.01108 v10 − 0.0296 v11 − 0.1207 v12

y = 1.124224045 + 0.1295920897 v10 +
0.1156011386 v8 + 0.01695234993 v0

spatial perception
genetic programming
Monday, September 10, 2012

relative
errors
Monday, September 10, 2012

transferring
spatial perception
Monday, September 10, 2012

future
work

automate the learning (IROS 2012)
different spatial representations
“intrinsic motivation” learning
online, continuous learning
eye-hand coordination

Monday, September 10, 2012

thanks
for listening

juxi@idsia.ch http://Juxi.net/projects

Monday, September 10, 2012

Similar to Towards Spatial Perception: Learning to Locate Objects from Vision #robotDoC

FailChat: UX Comes First Because UX is Everything!

ellendunne

Artificial Neural Networks For Spatial Perception: Towards Visual Object Loca...

Juxi Leitner

Learning organisations and design thinking

emilia åström

Website Usability | Class 1

studiokandm

Persuasive design presentationd3=r1

Sebastian Daum

Persuasive design presentationd3=r1

Sebastian Daum

Mars Terrain Image Classification Using Cartesian Genetic Programming #isaira...

Juxi Leitner

Virtual Worlds in Higher Education: More Than Simulation (A Case-Study of Sec...

Simon Kear

Similar to Towards Spatial Perception: Learning to Locate Objects from Vision #robotDoC (8)

FailChat: UX Comes First Because UX is Everything!

Artificial Neural Networks For Spatial Perception: Towards Visual Object Loca...

Learning organisations and design thinking

Website Usability | Class 1

Persuasive design presentationd3=r1

Mars Terrain Image Classification Using Cartesian Genetic Programming #isaira...

Virtual Worlds in Higher Education: More Than Simulation (A Case-Study of Sec...

More from Juxi Leitner

These slides were used for a few talks in the last couple of months to excite people about the intelligent robotic systems. In particular, why I believe that it is important for robots to grasp the world, both in the sense of perceiving and understanding but also in the physical sense of actually changing the state of the world by picking objects and interacting with a wide range of items. These slides (with slight variations) were presented at QUT, Uni Sydney, Uni Cambridge, DeepMind, Uni Birmingham, Amazon Robotics, ...

The Need For Robots To Grasp the World

Juxi Leitner

Significant progress in computer vision in the past years has excited a whole field of researchers. In robotics we are now able to use these techniques to build robotic systems that can observe, understand, and interact with the world, in short, we can build robots that grasp the world. This is an overview of the efforts in the Australien Centre for Robotic Vision under the umbrella of "Robotic Manipulation" led by Dr. Juxi Leitner. Slides used for a series of presentations in Australia and Europe in Sep/Oct 2018. Feel free to reach out for opportunities to juxi@lyro.io

Robots that Grasp the World

Juxi Leitner

Improving Robotic Manipulation with Vision and Learning @AmazonDevCentre Berlin

Juxi Leitner

Cartman, how to win the amazon robotics challenge with robotic vision and dee...

Juxi Leitner

Deep Reinforcement Learning | Amazon Robotics Challenge, Image Processing Lec...

Juxi Leitner

Presented at the IROS workshop on "DEVELOPMENT OF BENCHMARKING PROTOCOLS FOR ROBOTIC MANIPULATION" http://ycbbenchmarks.org/IROS2017workshop.html The ACRV Picking Benchmark has been developed over the last year to facilitate comparison of robotic systems in pick and place settings! With the ABP we propose a physical benchmark for robotic picking: overall design, objects, configuration, and guidance on appropriate technologies to solve it. Challenges are an important way to drive progress but they occur only occasionally and the test conditions are difficult to replicate outside the challenge. This benchmark is motivated by experience in the recent Amazon Picking Challenge and contains a commonly-available shelf, 42 objects, a set of stencils and standardized task setups. A major focus through the design of this benchmark was to maximise reproducibility: a number of carefully chosen scenarios with precise instructions on how to place, orient, and align objects with the help of printable stencils are defined. To make the benchmark as accessible as possible to the research community, a white IKEA shelf is used for all picking tasks. Furthermore, we carefully curated a set of 42 objects to ensure global availability and reduced chance of import restrictions.

ACRV Picking Benchmark: how to benchmark pick and place robotics research

Juxi Leitner

Building on Repeatable Grasping Experiments Team ACRV: Lessons Learned from the Amazon Picking Challenge 2016 Juxi Leitner, ACRV, Queensland University of Technology (Team ACRV, 2016, 2017) We describe our entry into the 2016 Amazon Picking Challenge (APC) and the lessons learned from deploying a complex, robotic system outside of the lab. To help future developments decided to create a new physical benchmark challenge for robotic picking to drive scientific progress and make research into (end-to-end) picking comparable. It consists of a set of 42 common objects, a widely available shelf, and exact guidelines for object arrangement using stencils.

Team ACRV's experience at #AmazonPickingChallenge 2016

Juxi Leitner

(deep) reinforcement learning - CAB420

Juxi Leitner

ACRV : Robotic Vision presentation in Lisbon at IST

Juxi Leitner

The Australian Centre for Robotic Vision (ACRV)

Juxi Leitner

How to place 6th in the Amazon Picking Challenge (ENB329, QUT)

Juxi Leitner

Presentation slides from the talk given at the IEEE Aerospace Conference (@IEEEAeroConf) 2016 in Big Sky, Montana, USA. We describe a hopping science payload solution designed to exploit the Moon’s lower gravity to leap up to 20m above the surface. The entire solar-powered robot is compact enough to fit within a 10cm cube, whilst providing unique observation and mission capabilities by creating imagery during the hop. The LunaRoo concept is a proposed payload to fly onboard a Google Lunar XPrize entry. Its compact form is specifically designed for lunar exploration and science mission within the constraints given by PTScientists. The core features of LunaRoo are its method of locomotion – hopping like a kangaroo - and its imaging system capable of unique over-the- horizon perception. The payload will serve as a proof of concept, highlighting the benefits of alternative mobility solutions, in particular enabling observation and exploration of terrain not traversable by wheeled robots. in addition providing data for beyond line-of-sight planning and communications for surface assets, extending overall mission capabilities.

LunaRoo: Designing a Hopping Lunar Science Payload #space #exploration

Juxi Leitner

Robotic Vision - Vision for Robotics #IEEE #QLD #CIS #Colloquium

Juxi Leitner

ROS Hands-On Intro/Tutorial (Robotic Vision Summer School 2015) #RVSS #ACRV

Juxi Leitner

ACRV Research Fellow Intro/Tutorial [Vision and Action]

Juxi Leitner

From Vision to Actions - Towards Adaptive & Autonomous Humanoid Robots [PhD D...

Juxi Leitner

My presentation at the ICINCO 2014 (the 11th International Conference on Informatics in Control, Automation and Robotics) Abstract: We propose a system incorporating a tight integration between computer vision and robot control modules on a complex, high-DOF humanoid robot. Its functionality is showcased by having our iCub humanoid robot pick-up objects from a table in front of it. An important feature is that the system can avoid obstacles – other objects detected in the visual stream – while reaching for the intended target object. Our integration also allows for non-static environments, i.e. the reaching is adapted on-the-fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. Furthermore we show that this system can be used both in autonomous and tele-operation scenarios.

Reactive Reaching and Grasping on a Humanoid: Towards Closing the Action-Perc...

Juxi Leitner

We present our work on tele-operating a complex humanoid robot with the help of bio-signals collected from the operator. The frameworks (for robot vision, collision avoidance and machine learning), developed in our lab, allow for a safe interaction with the environment, when combined. This even works with noisy control signals, such as, the operator’s hand acceleration and their elec- tromyography (EMG) signals. These bio-signals are used to execute equivalent actions (such as, reaching and grasping of objects) on the 7 DOF arm.

Tele-operation of a Humanoid Robot, Using Operator Bio-data

Juxi Leitner

presentation during the WCCI 2014 in Beijing, China We propose a method for learning specific object representations that can be applied (and reused) in visual detection and identification tasks. A machine learning technique called Cartesian Genetic Programming (CGP) is used to create these models based on a series of images. Our research investi- gates how manipulation actions might allow for the development of better visual models and therefore better robot vision. This paper describes how visual object representations can be learned and improved by performing object manipulation actions, such as, poke, push and pick-up with a humanoid robot. The improvement can be measured and allows for the robot to select and perform the ‘right’ action, i.e. the action with the best possible improvement of the detector.

Improving Robot Vision Models for Object Detection Through Interaction #ijcnn...

Juxi Leitner

Last December I had the pleasure to take part in the EM-ACE workshop held at the University of Porto, Portugal. I was invited to talk about my experience studying in the "Joint European Master in Space Science and Technology" (SpaceMaster) infront of about 60 students. http://www.em-ace.eu/en/upload/public-docs/UPORTO_em-ace%20event_agenda.pdf http://www.em-a.eu/en/home/rss-feed-detail/em-ace-student-event-university-of-porto-16-december-2013-1395.html

How does it feel to be a SpaceMaster? [Erasmus Mundus - ACE Talk]

Juxi Leitner

More from Juxi Leitner (20)

The Need For Robots To Grasp the World

Robots that Grasp the World

Improving Robotic Manipulation with Vision and Learning @AmazonDevCentre Berlin

Cartman, how to win the amazon robotics challenge with robotic vision and dee...

Deep Reinforcement Learning | Amazon Robotics Challenge, Image Processing Lec...

ACRV Picking Benchmark: how to benchmark pick and place robotics research

Team ACRV's experience at #AmazonPickingChallenge 2016

(deep) reinforcement learning - CAB420

ACRV : Robotic Vision presentation in Lisbon at IST

The Australian Centre for Robotic Vision (ACRV)

How to place 6th in the Amazon Picking Challenge (ENB329, QUT)

LunaRoo: Designing a Hopping Lunar Science Payload #space #exploration

Robotic Vision - Vision for Robotics #IEEE #QLD #CIS #Colloquium

ROS Hands-On Intro/Tutorial (Robotic Vision Summer School 2015) #RVSS #ACRV

ACRV Research Fellow Intro/Tutorial [Vision and Action]

From Vision to Actions - Towards Adaptive & Autonomous Humanoid Robots [PhD D...

Reactive Reaching and Grasping on a Humanoid: Towards Closing the Action-Perc...

Tele-operation of a Humanoid Robot, Using Operator Bio-data

Improving Robot Vision Models for Object Detection Through Interaction #ijcnn...

How does it feel to be a SpaceMaster? [Erasmus Mundus - ACE Talk]

Recently uploaded

TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....

rightmanforbloodline

The presentation was made in “Web3 Fusion: Embracing AI and Beyond” is more than a conference; it's a journey into the heart of digital transformation. The conference a provided a platform where the future of technology meets practical application. This three-day hybrid event, set in the heart of innovation, served as a gateway to the latest trends and transformative discussions in AI, Blockchain, IoT, AR/VR, and their collective impact on the information space.

AI in Action: Real World Use Cases by Anitaraj

AnitaRaj43

In the ever-evolving landscape of data management, Zero-ETL is an approach that is reshaping how businesses handle and integrate their data. This webinar explores Zero-ETL, a paradigm shift from the traditional Extract, Transform, Load (ETL) process, offering a more streamlined, efficient, and real-time data integration method. We will begin with an introduction to the concept of Zero-ETL, including how it allows direct access to data in its native environment and real-time data transformation, providing up-to-date information with significantly reduced data redundancy. Next, we'll take you through several demonstrations showing how Zero-ETL can deliver real-time data and enable the free movement of data between systems. We will also discuss the various tools that support all aspects of Zero-ETL, providing attendees with an understanding of how they can adopt this innovative approach in their organizations. Lastly, the session will conclude with an interactive Q&A segment, allowing participants to gain deeper insights into how Zero-ETL can be tailored to their specific business needs and how they can get started today. Join us to discover how Zero-ETL can elevate your organization's data strategy.

The Zero-ETL Approach: Enhancing Data Agility and Insight

Safe Software

Following the popularity of “Cloud Revolution: Exploring the New Wave of Serverless Spatial Data,” we’re thrilled to announce this much-anticipated encore webinar. In this sequel, we’ll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR. Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios. Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects. Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you’re building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Safe Software

In this keynote, Asanka Abeysinghe, CTO,WSO2 will explore the shift towards platformless technology ecosystems and their importance in driving digital adaptability and innovation. We will discuss strategies for leveraging decentralized architectures and integrating diverse technologies, with a focus on building resilient, flexible, and future-ready IT infrastructures. We will also highlight WSO2's roadmap, emphasizing our commitment to supporting this transformative journey with our evolving product suite.

Platformless Horizons for Digital Adaptability

WSO2

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on the deployment of external web forms using Jotform for Bonterra Impact Management. This solution can be customized to your organization’s needs and deployed to support the common use cases below: - Intake and consent - Assessments - Surveys - Applications - Program registration Interested in deploying web form automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Jeffrey Haguewood

Intro to Passkeys and the State of Passwordless.pptx

FIDO Alliance

DBX First Quarter 2024 Investor Presentation

Dropbox

JohnPollard-hybrid-app-RailsConf2024.pptx

JohnPollard37

CNIC Information System with Pakdata Cf In Pakistan

danishmna97

Decarbonising Commercial Real Estate: The Role of Operational Performance

IES VE

Portal Kombat : extension du réseau de propagande russe

中央社

Working together SRE & Platform Engineering

Marcus Vechiato

Overview of Hyperledger Foundation

Hyperleger Tokyo Meetup

Discover the innovative features and strategic vision that keep WSO2 an industry leader. Explore the exciting 2024 roadmap of WSO2 API management, showcasing innovations, unified APIM/APK control plane, natural language API interaction, and cloud native agility. Discover how open source solutions, microservices architecture, and cloud native technologies unlock seamless API management in today's dynamic landscapes. Leave with a clear blueprint to revolutionize your API journey and achieve industry success!

WSO2's API Vision: Unifying Control, Empowering Developers

WSO2

Tracing the root cause of a performance issue requires a lot of patience, experience, and focus. It’s so hard that we sometimes attempt to guess by trying out tentative fixes, but that usually results in frustration, messy code, and a considerable waste of time and money. This talk explains how to correctly zoom in on a performance bottleneck using three levels of profiling: distributed tracing, metrics, and method profiling. After we learn to read the JVM profiler output as a flame graph, we explore a series of bottlenecks typical for backend systems, like connection/thread pool starvation, invisible aspects, blocking code, hot CPU methods, lock contention, and Virtual Thread pinning, and we learn to trace them even if they occur in library code you are not familiar with. Attend this talk and prepare for the performance issues that will eventually hit any successful system. About authorWith two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Victor Rentea

Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx

FIDO Alliance

In the dynamic field of DevOps, the quest for efficiency and productivity is endless. This talk introduces a revolutionary toolkit: Large Language Models (LLMs), including ChatGPT, Gemini, and Claude, extending far beyond traditional coding assistance. We'll explore how LLMs can automate not just code generation, but also transform day-to-day operations such as crafting compelling cover letters for TPS reports, streamlining client communications, and architecting innovative DevOps solutions. Attendees will learn effective prompting strategies and examine real-life use cases, demonstrating LLMs' potential to redefine productivity in the DevOps landscape. Join us to discover how to harness the power of LLMs for a comprehensive productivity boost across your DevOps activities.

ChatGPT and Beyond - Elevating DevOps Productivity

VictorSzoltysek

In today's digital world, trust is key to customer relationships, but keeping it is a huge challenge. Customers are well-informed and empowered, quick to change brands if their trust is broken, even if it costs them more. This puts a lot of pressure on organizations to handle trust and safety issues with great care and transparency. The challenge, however, is real. Fragmented solutions have left privacy, legal, and security teams in a perpetual cycle of catch-up, struggling to update privacy notices, manage customer data rights, and answer lengthy security questionnaires—all while trying to prove ROI to the business. It's a thankless job, filled with repetition, tedious tasks, and constant interdepartmental coordination. Combine this with fast regulatory changes and the quick evolution of AI, and it becomes overwhelming. Join this webinar to learn more about TrustArc's new innovative solution Trust Center, the only unified, no-code online hub for trust and safety information built for privacy, security, compliance, and legal teams. Trust Center streamlines your path to compliance, shortens the pre-sales cycle, and reduces both legal and regulatory risks, saving time, effort, and cost. This webinar will review: - Why companies are building unified Trust Centers for a robust privacy program. - How unified Trust Centers streamline sales cycles, ensure regulatory compliance, and reduce operational bottlenecks. - How compliance, legal, security, GRC, and privacy teams benefit from a unified Trust Center in terms of needs, pains, and outcomes. - How TrustArc Trust Center saves time and work while reducing legal, reputational, and compliance risk by effectively managing policies, notices, terms, and disclosures, and providing real-time updates on subprocessors.

TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...

TrustArc

Quantum Leap in Next-Generation Computing

WSO2

Recently uploaded (20)

TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....

AI in Action: Real World Use Cases by Anitaraj

The Zero-ETL Approach: Enhancing Data Agility and Insight

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Platformless Horizons for Digital Adaptability

Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...

Intro to Passkeys and the State of Passwordless.pptx

DBX First Quarter 2024 Investor Presentation

JohnPollard-hybrid-app-RailsConf2024.pptx

CNIC Information System with Pakdata Cf In Pakistan

Decarbonising Commercial Real Estate: The Role of Operational Performance

Portal Kombat : extension du réseau de propagande russe

Working together SRE & Platform Engineering

Overview of Hyperledger Foundation

WSO2's API Vision: Unifying Control, Empowering Developers

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx

ChatGPT and Beyond - Elevating DevOps Productivity

TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...

Quantum Leap in Next-Generation Computing

Towards Spatial Perception: Learning to Locate Objects from Vision #robotDoC

1. Jürgen ’Juxi’ Leitner S. Harding, M. Frank, A. Förster, J. Schmidhuber istituto dalle molle di studi sull’intelligenza artiﬁciale università della svizzera italiana idsia / usi / supsi towards spatial perception learning to locate objects from vision #robotdoc 2012 Monday, September 10, 2012

2. iCub thanks to G. Metta, IIT and U.Plymouth for their pictures Monday, September 10, 2012

3. our iCub setup is for manipulation Monday, September 10, 2012

4. thanks to G. Metta and IIT for this picture visual perception Monday, September 10, 2012

5. three parts Monday, September 10, 2012

6. detecting objects Harding et al., GPTP 2012, Leitner et al., ICDL 2012 Monday, September 10, 2012

7. Image credit: NASA also works for detecting rocks on Mars Leitner et al., iSAIRAS 2012 Monday, September 10, 2012

8. detecting objects Harding et al., GPTP 2012, Leitner et al., ICDL 2012 Monday, September 10, 2012

9. icVision framework Leitner et al., BICA 2012 Monday, September 10, 2012

10. three parts GPTP2012, ICDL2012(?) BICA 2012 Monday, September 10, 2012

11. locate objects Monday, September 10, 2012

12. current localisation approaches Monday, September 10, 2012

13. learning approach + Monday, September 10, 2012

14. learning setup Monday, September 10, 2012

15. spatial perception neural network Monday, September 10, 2012

16. spatial perception genetic programming Monday, September 10, 2012

17. x = 17.81 − 0.01906 v1 + 0.1527 v4 + 0.1378 v7 + 0.01108 v10 − 0.0296 v11 − 0.1207 v12 y = 1.124224045 + 0.1295920897 v10 + 0.1156011386 v8 + 0.01695234993 v0 spatial perception genetic programming Monday, September 10, 2012

18. relative errors Monday, September 10, 2012

19. transferring spatial perception Monday, September 10, 2012

20. future work automate the learning (IROS 2012) different spatial representations “intrinsic motivation” learning online, continuous learning eye-hand coordination Monday, September 10, 2012

21. thanks for listening juxi@idsia.ch http://Juxi.net/projects Monday, September 10, 2012

Towards Spatial Perception: Learning to Locate Objects from Vision #robotDoC

Recommended

Recommended

More Related Content

Similar to Towards Spatial Perception: Learning to Locate Objects from Vision #robotDoC

Similar to Towards Spatial Perception: Learning to Locate Objects from Vision #robotDoC (8)

More from Juxi Leitner

More from Juxi Leitner (20)

Recently uploaded

Recently uploaded (20)

Towards Spatial Perception: Learning to Locate Objects from Vision #robotDoC