SlideShare a Scribd company logo
Closing the gap between
neuroscience and
machine learning
Jason Toy
@jtoy
Neuroscience
• Scientific study of the nervous system
• Why? How do humans and living beings
function and learn. What are we?
• Many sub disciplines: cognitive science,
computational neuroscience, neurochemistry,
neuro anatomy, clinical neuroscience, etc
Neuroscience
• Pictures
Machine learning
• study of computer science that often uses
statistical methods to give the computers the
ability to learn
• Silo'd away
• Why? Can we make computers intelligent like
humans using current computer science
paradigms
Machine learning
Differences between computers
and human intelligence
• Hardware
• Software
• Need new algorithms?
• New hardware?
• What is missing?
Differences between computers
and human intelligence - hardware
• Programs in computer vs embodied cognitions
• Digital vs analog? Hybrid models?
• Parallel computation vs sequential?
Differences between computers
and human intelligence - software
• Single objective function vs dynamic goals
• text/images vs multimodal sensory
• ML models learn strong features vs casual
models for explaining and understanding
Reinforcement learning
• Multi arm bandits - slot machines
• Marketing and landing page optimization
• Robots
Sensorimotor system
Deep Reinforcement
learning
Deep RL Environments
sensorimotor vs RL ?
Sensenet: My contribution &
attempt at closing the gap
• Sensenet: An open source touch and
sensorimotor simulator for neuroscience and
machine learning research
• https://github.com/jtoy/sensenet
Why sensorimotor?
• Most ML algorithms are passive and static:
CNNs, classification
• Single objective function models in
classification/RL
• vision vs embodied sense
• Next gen algorithms in the sensorimotor system -
touch / motion / mapping / location / rotations
Goal of SenseNet Project
• Aid in discovery of new AI algorithms through
touch/sensorimotor simulations
• Get computer science and neuroscience
communities interested in touch and
sensorimotor research
sensorimotor system
Sensorimotor + RL?
SenseNet components
• 4 main components:
• Physics and Touch Simulator
• Simple extendable API for RL, sensorimotor, and
touch abstractions
• Standardized benchmarks and reference
• Dataset of 3D objects dataset for manipulation
Physics and Touch Simulator
• SenseNet provides different physics and touch
environments as a reference
• Hard and soft body collision detection
• Tunable and tweakable physics and touch
apparatuses
• Current touch sensor acts as edge detector
(similar to merkel cells)
Bullet Physics
Get to human like sensors
SenseNet API
• Physics in C++, everything else in python
• Researchers interact with the API in python
• Reinforcement learning API / OpenAI gym comp
• Developing API to add touch sensors and skin
• Can use standard robotics tools to add agents
(URDF/stl)
Environments / Agents
• W
Full Hand Index finger only Touch Wand
~7 agent types currently
SenseNet API & Touch
Agents
• Current API: reinforcement learning API (gym)
• In development:
• APIs to enable multiple reward functions.
• Skin API, sensorimotor API
• Embodied Agents: Hand, single finger, touch
wand, disembodied touch agents, custom agents
Random agent movement
SenseNet Dataset
SenseNet Dataset
pip install sensenet
Sensenet experiments
• Can we train a simulated hand to touch and
classify objects
• Can we train a simulated hand to touch and
classify textures
• Can we train a simulated hand to touch and
manipulate objects
Sensenet + neuroscience
Summary
• Computer .......... humans
• deep learning is awesome, but ........
• " We urge researchers in neuroscience and AI to
find a common language, allowing a free flow of
knowledge that will allow continued progress" -
Demis Hassabis, Founder of DeepMind
Jason Toy
@jtoy
jtoy@jtoy.net
Want to contribute?
Questions?
Feedback
http://github.com/jtoy/sensenet

More Related Content

What's hot

Pattern recognition
Pattern recognitionPattern recognition
Pattern recognition
Minigranth
 
Artificial inteligence
Artificial inteligenceArtificial inteligence
Artificial inteligence
Anoop Jain
 
Step Into World of Artificial Intelligence
Step Into World of Artificial IntelligenceStep Into World of Artificial Intelligence
Step Into World of Artificial Intelligence
Explore Skilled
 
Robotics
RoboticsRobotics
Robotics
aneeskokadan
 
Ai & machine learning win sple
Ai & machine learning   win spleAi & machine learning   win sple
Ai & machine learning win sple
LewisWhite16
 
Artificial2
Artificial2Artificial2
Artificial2
Noor Alam
 
The best programming language for Artificial Intelligence and Machine Learning
The best programming language for Artificial Intelligence and Machine LearningThe best programming language for Artificial Intelligence and Machine Learning
The best programming language for Artificial Intelligence and Machine Learning
🖥 Anna Danilec
 
Ai & ai with python
Ai & ai with pythonAi & ai with python
Ai & ai with python
ankitdobhal9
 
Sixthsensetechnology032 120812015706-phpapp01
Sixthsensetechnology032 120812015706-phpapp01Sixthsensetechnology032 120812015706-phpapp01
Sixthsensetechnology032 120812015706-phpapp01
Vikran Kottaisamy
 
Sixthsensetechnology
SixthsensetechnologySixthsensetechnology
Sixthsensetechnology
Vikran Kottaisamy
 
Artificial intelligence
Artificial intelligenceArtificial intelligence
Artificial intelligence
lavanya lav
 
Artificial inteligence
Artificial inteligenceArtificial inteligence
Artificial inteligence
Sharath Raj
 
Artificial general intelligence research project at Keen Software House (3/2015)
Artificial general intelligence research project at Keen Software House (3/2015)Artificial general intelligence research project at Keen Software House (3/2015)
Artificial general intelligence research project at Keen Software House (3/2015)
Marek Rosa
 
Artificial intelligence
Artificial intelligenceArtificial intelligence
Artificial intelligence
Karan Panwar
 
Machine learning
Machine learningMachine learning
Machine learning
Marlen Franto
 
Artificial intelligent
Artificial intelligentArtificial intelligent
Artificial intelligent
ALi Akram
 
Python AI tutorial
Python AI tutorialPython AI tutorial
Python AI tutorial
grinu
 
artificial intelligence ppt
artificial intelligence pptartificial intelligence ppt
artificial intelligence ppt
Nitesh Dubey
 
Artificial intelligence
Artificial intelligenceArtificial intelligence
Artificial intelligence
Learnbay Datascience
 
artificial intelligence
artificial intelligenceartificial intelligence
artificial intelligence
Bikas Sadashiv
 

What's hot (20)

Pattern recognition
Pattern recognitionPattern recognition
Pattern recognition
 
Artificial inteligence
Artificial inteligenceArtificial inteligence
Artificial inteligence
 
Step Into World of Artificial Intelligence
Step Into World of Artificial IntelligenceStep Into World of Artificial Intelligence
Step Into World of Artificial Intelligence
 
Robotics
RoboticsRobotics
Robotics
 
Ai & machine learning win sple
Ai & machine learning   win spleAi & machine learning   win sple
Ai & machine learning win sple
 
Artificial2
Artificial2Artificial2
Artificial2
 
The best programming language for Artificial Intelligence and Machine Learning
The best programming language for Artificial Intelligence and Machine LearningThe best programming language for Artificial Intelligence and Machine Learning
The best programming language for Artificial Intelligence and Machine Learning
 
Ai & ai with python
Ai & ai with pythonAi & ai with python
Ai & ai with python
 
Sixthsensetechnology032 120812015706-phpapp01
Sixthsensetechnology032 120812015706-phpapp01Sixthsensetechnology032 120812015706-phpapp01
Sixthsensetechnology032 120812015706-phpapp01
 
Sixthsensetechnology
SixthsensetechnologySixthsensetechnology
Sixthsensetechnology
 
Artificial intelligence
Artificial intelligenceArtificial intelligence
Artificial intelligence
 
Artificial inteligence
Artificial inteligenceArtificial inteligence
Artificial inteligence
 
Artificial general intelligence research project at Keen Software House (3/2015)
Artificial general intelligence research project at Keen Software House (3/2015)Artificial general intelligence research project at Keen Software House (3/2015)
Artificial general intelligence research project at Keen Software House (3/2015)
 
Artificial intelligence
Artificial intelligenceArtificial intelligence
Artificial intelligence
 
Machine learning
Machine learningMachine learning
Machine learning
 
Artificial intelligent
Artificial intelligentArtificial intelligent
Artificial intelligent
 
Python AI tutorial
Python AI tutorialPython AI tutorial
Python AI tutorial
 
artificial intelligence ppt
artificial intelligence pptartificial intelligence ppt
artificial intelligence ppt
 
Artificial intelligence
Artificial intelligenceArtificial intelligence
Artificial intelligence
 
artificial intelligence
artificial intelligenceartificial intelligence
artificial intelligence
 

Similar to Closing the gap between neuroscience and machine learning

Artificial Intelligence presentation,
Artificial Intelligence  presentation,Artificial Intelligence  presentation,
Artificial Intelligence presentation,
MuhammadAbdullah771
 
ARTIFICIAL INTELLIGENCE-New.pptx
ARTIFICIAL INTELLIGENCE-New.pptxARTIFICIAL INTELLIGENCE-New.pptx
ARTIFICIAL INTELLIGENCE-New.pptx
ParveshSachdev
 
Ontology of a temperature sensor
Ontology of a temperature sensorOntology of a temperature sensor
Ontology of a temperature sensor
haroonrashidlone
 
AI for Everyone: Master the Basics
AI for Everyone: Master the BasicsAI for Everyone: Master the Basics
AI for Everyone: Master the Basics
Stutty Srivastava
 
Unit1_AI&ML (2).pptx
Unit1_AI&ML (2).pptxUnit1_AI&ML (2).pptx
Unit1_AI&ML (2).pptx
sahilshah890338
 
MIS
MISMIS
Intro AI.pdf
Intro AI.pdfIntro AI.pdf
Intro AI.pdf
satishjadhao6
 
ARTIFICIAL INTELLIGENCE.pptx
ARTIFICIAL INTELLIGENCE.pptxARTIFICIAL INTELLIGENCE.pptx
ARTIFICIAL INTELLIGENCE.pptx
vivim10
 
arti-ficial-inte-lligence
arti-ficial-inte-lligencearti-ficial-inte-lligence
arti-ficial-inte-lligence
IconicGroup
 
Artificial intelligence- The science of intelligent programs
Artificial intelligence- The science of intelligent programsArtificial intelligence- The science of intelligent programs
Artificial intelligence- The science of intelligent programs
Derak Davis
 
Artificialintelligence
ArtificialintelligenceArtificialintelligence
Artificialintelligence
Ravi Rao
 
Presentation v3
Presentation v3Presentation v3
Presentation v3
Muhammad AL-Qurishi
 
UNIT1-AI final.pptx
UNIT1-AI final.pptxUNIT1-AI final.pptx
UNIT1-AI final.pptx
CS50Bootcamp
 
Essential concepts for machine learning
Essential concepts for machine learning Essential concepts for machine learning
Essential concepts for machine learning
pyingkodi maran
 
Introduction to AI
Introduction to AIIntroduction to AI
Introduction to AI
Sathittham Sangthong
 
Artificial_Intelligence_ppt_presentation.pptx
Artificial_Intelligence_ppt_presentation.pptxArtificial_Intelligence_ppt_presentation.pptx
Artificial_Intelligence_ppt_presentation.pptx
VenkateshBoopathi2
 
Artificial intelligence : what it is
Artificial intelligence : what it isArtificial intelligence : what it is
Artificial intelligence : what it is
www.myassignmenthelp.net
 
Ai introduction
Ai introductionAi introduction
Ai introduction
Babar Siraj
 
Artificial intelligence
Artificial intelligenceArtificial intelligence
Artificial intelligence
Minigranth
 
EELU AI lecture 1- fall 2022-2023 - Chapter 01- Introduction.ppt
EELU AI  lecture 1- fall 2022-2023 - Chapter 01- Introduction.pptEELU AI  lecture 1- fall 2022-2023 - Chapter 01- Introduction.ppt
EELU AI lecture 1- fall 2022-2023 - Chapter 01- Introduction.ppt
DaliaMagdy12
 

Similar to Closing the gap between neuroscience and machine learning (20)

Artificial Intelligence presentation,
Artificial Intelligence  presentation,Artificial Intelligence  presentation,
Artificial Intelligence presentation,
 
ARTIFICIAL INTELLIGENCE-New.pptx
ARTIFICIAL INTELLIGENCE-New.pptxARTIFICIAL INTELLIGENCE-New.pptx
ARTIFICIAL INTELLIGENCE-New.pptx
 
Ontology of a temperature sensor
Ontology of a temperature sensorOntology of a temperature sensor
Ontology of a temperature sensor
 
AI for Everyone: Master the Basics
AI for Everyone: Master the BasicsAI for Everyone: Master the Basics
AI for Everyone: Master the Basics
 
Unit1_AI&ML (2).pptx
Unit1_AI&ML (2).pptxUnit1_AI&ML (2).pptx
Unit1_AI&ML (2).pptx
 
MIS
MISMIS
MIS
 
Intro AI.pdf
Intro AI.pdfIntro AI.pdf
Intro AI.pdf
 
ARTIFICIAL INTELLIGENCE.pptx
ARTIFICIAL INTELLIGENCE.pptxARTIFICIAL INTELLIGENCE.pptx
ARTIFICIAL INTELLIGENCE.pptx
 
arti-ficial-inte-lligence
arti-ficial-inte-lligencearti-ficial-inte-lligence
arti-ficial-inte-lligence
 
Artificial intelligence- The science of intelligent programs
Artificial intelligence- The science of intelligent programsArtificial intelligence- The science of intelligent programs
Artificial intelligence- The science of intelligent programs
 
Artificialintelligence
ArtificialintelligenceArtificialintelligence
Artificialintelligence
 
Presentation v3
Presentation v3Presentation v3
Presentation v3
 
UNIT1-AI final.pptx
UNIT1-AI final.pptxUNIT1-AI final.pptx
UNIT1-AI final.pptx
 
Essential concepts for machine learning
Essential concepts for machine learning Essential concepts for machine learning
Essential concepts for machine learning
 
Introduction to AI
Introduction to AIIntroduction to AI
Introduction to AI
 
Artificial_Intelligence_ppt_presentation.pptx
Artificial_Intelligence_ppt_presentation.pptxArtificial_Intelligence_ppt_presentation.pptx
Artificial_Intelligence_ppt_presentation.pptx
 
Artificial intelligence : what it is
Artificial intelligence : what it isArtificial intelligence : what it is
Artificial intelligence : what it is
 
Ai introduction
Ai introductionAi introduction
Ai introduction
 
Artificial intelligence
Artificial intelligenceArtificial intelligence
Artificial intelligence
 
EELU AI lecture 1- fall 2022-2023 - Chapter 01- Introduction.ppt
EELU AI  lecture 1- fall 2022-2023 - Chapter 01- Introduction.pptEELU AI  lecture 1- fall 2022-2023 - Chapter 01- Introduction.ppt
EELU AI lecture 1- fall 2022-2023 - Chapter 01- Introduction.ppt
 

More from jtoy

TouchNet preview at Numenta
TouchNet preview at NumentaTouchNet preview at Numenta
TouchNet preview at Numenta
jtoy
 
Rga somatic hackday
Rga somatic hackdayRga somatic hackday
Rga somatic hackday
jtoy
 
Experiential Marketing and Deep Learning
Experiential Marketing and Deep LearningExperiential Marketing and Deep Learning
Experiential Marketing and Deep Learning
jtoy
 
Intro to myself and SFPC
Intro to myself and SFPCIntro to myself and SFPC
Intro to myself and SFPC
jtoy
 
Tensorflow ruby
Tensorflow rubyTensorflow ruby
Tensorflow ruby
jtoy
 
machine learning and art hack day
machine learning and art hack daymachine learning and art hack day
machine learning and art hack day
jtoy
 
a tour of several popular tensorflow models
a tour of several popular tensorflow modelsa tour of several popular tensorflow models
a tour of several popular tensorflow models
jtoy
 
Bm packing list
Bm packing listBm packing list
Bm packing list
jtoy
 

More from jtoy (8)

TouchNet preview at Numenta
TouchNet preview at NumentaTouchNet preview at Numenta
TouchNet preview at Numenta
 
Rga somatic hackday
Rga somatic hackdayRga somatic hackday
Rga somatic hackday
 
Experiential Marketing and Deep Learning
Experiential Marketing and Deep LearningExperiential Marketing and Deep Learning
Experiential Marketing and Deep Learning
 
Intro to myself and SFPC
Intro to myself and SFPCIntro to myself and SFPC
Intro to myself and SFPC
 
Tensorflow ruby
Tensorflow rubyTensorflow ruby
Tensorflow ruby
 
machine learning and art hack day
machine learning and art hack daymachine learning and art hack day
machine learning and art hack day
 
a tour of several popular tensorflow models
a tour of several popular tensorflow modelsa tour of several popular tensorflow models
a tour of several popular tensorflow models
 
Bm packing list
Bm packing listBm packing list
Bm packing list
 

Recently uploaded

TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Jeffrey Haguewood
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
Hiike
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
AWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptxAWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptx
HarisZaheer8
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
alexjohnson7307
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!
GDSC PJATK
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
Wouter Lemaire
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Postman
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
fredae14
 

Recently uploaded (20)

TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
AWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptxAWS Cloud Cost Optimization Presentation.pptx
AWS Cloud Cost Optimization Presentation.pptx
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
UI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentationUI5 Controls simplified - UI5con2024 presentation
UI5 Controls simplified - UI5con2024 presentation
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
 

Closing the gap between neuroscience and machine learning

  • 1. Closing the gap between neuroscience and machine learning Jason Toy @jtoy
  • 2. Neuroscience • Scientific study of the nervous system • Why? How do humans and living beings function and learn. What are we? • Many sub disciplines: cognitive science, computational neuroscience, neurochemistry, neuro anatomy, clinical neuroscience, etc
  • 4.
  • 5. Machine learning • study of computer science that often uses statistical methods to give the computers the ability to learn • Silo'd away • Why? Can we make computers intelligent like humans using current computer science paradigms
  • 7. Differences between computers and human intelligence • Hardware • Software • Need new algorithms? • New hardware? • What is missing?
  • 8. Differences between computers and human intelligence - hardware • Programs in computer vs embodied cognitions • Digital vs analog? Hybrid models? • Parallel computation vs sequential?
  • 9. Differences between computers and human intelligence - software • Single objective function vs dynamic goals • text/images vs multimodal sensory • ML models learn strong features vs casual models for explaining and understanding
  • 10. Reinforcement learning • Multi arm bandits - slot machines • Marketing and landing page optimization • Robots
  • 15. Sensenet: My contribution & attempt at closing the gap • Sensenet: An open source touch and sensorimotor simulator for neuroscience and machine learning research • https://github.com/jtoy/sensenet
  • 16. Why sensorimotor? • Most ML algorithms are passive and static: CNNs, classification • Single objective function models in classification/RL • vision vs embodied sense • Next gen algorithms in the sensorimotor system - touch / motion / mapping / location / rotations
  • 17. Goal of SenseNet Project • Aid in discovery of new AI algorithms through touch/sensorimotor simulations • Get computer science and neuroscience communities interested in touch and sensorimotor research
  • 20. SenseNet components • 4 main components: • Physics and Touch Simulator • Simple extendable API for RL, sensorimotor, and touch abstractions • Standardized benchmarks and reference • Dataset of 3D objects dataset for manipulation
  • 21. Physics and Touch Simulator • SenseNet provides different physics and touch environments as a reference • Hard and soft body collision detection • Tunable and tweakable physics and touch apparatuses • Current touch sensor acts as edge detector (similar to merkel cells)
  • 23.
  • 24. Get to human like sensors
  • 25. SenseNet API • Physics in C++, everything else in python • Researchers interact with the API in python • Reinforcement learning API / OpenAI gym comp • Developing API to add touch sensors and skin • Can use standard robotics tools to add agents (URDF/stl)
  • 26. Environments / Agents • W Full Hand Index finger only Touch Wand ~7 agent types currently
  • 27. SenseNet API & Touch Agents • Current API: reinforcement learning API (gym) • In development: • APIs to enable multiple reward functions. • Skin API, sensorimotor API • Embodied Agents: Hand, single finger, touch wand, disembodied touch agents, custom agents
  • 32. Sensenet experiments • Can we train a simulated hand to touch and classify objects • Can we train a simulated hand to touch and classify textures • Can we train a simulated hand to touch and manipulate objects
  • 34. Summary • Computer .......... humans • deep learning is awesome, but ........ • " We urge researchers in neuroscience and AI to find a common language, allowing a free flow of knowledge that will allow continued progress" - Demis Hassabis, Founder of DeepMind
  • 35. Jason Toy @jtoy jtoy@jtoy.net Want to contribute? Questions? Feedback http://github.com/jtoy/sensenet