Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Desarrollo de robots sociales con
RoboComp
Pablo Bustos
RoboLab
Universidad de Extremadura
UCM - 2016
A quick introduction to
2000-
Social and Service Robotics
● Robots that help humans in everyday
activities
Social and Service Robotics
● Robots that help humans in everyday
activities
Educational
Social and Service Robotics
● Robots that help humans in everyday
activities
Educational Rehabilitation
Social and Service Robotics
● Robots that help humans in everyday
activities
Educational Rehabilitation Advertisement
Social and Service Robotics
● Robots that help humans in everyday
activities
Educational Rehabilitation Advertisement Affe...
Social and Service Robotics
● Robots that help humans in everyday
activities
Educational Rehabilitation Advertisement Affe...
Educational robots - Robex
● 2005 - now
● college, comp. sci.
● low-cost
● versatile
● robust
● classroom
● adaptable
Educational robots - Robex
Educational robots in school - Dulce
● 2011
● 6th grade primary school
● 1 semester experiment
Educational robots in school - Dulce
Educational robots in school - Learnbot 1
● 2012
● Initiation to Python
programming for kids in
5th to 12th grades
● Build...
Educational robots in school - LearnBot 2
● Odroid only
● WiFi, battery, USs and USB camera
● under 150€ for one prototype
Educational robots in school - LearnBot 2
def code(self):
# 0 back 3 left 2 right 1 front
actions = ((150,-3), (450, 0), (...
Rehabilitation Robots
Rehabilitation robots - Ursus
● 2010
● Dynamixel servos
● Visual mark detection
Rehabilitation robots - Ursus
Rehabilitation robots - Ursus
● Technology validation experiment
● 5 patients in 7-10 age range
● mild lesions affecting t...
Social robots - Nao
Social robots - Nao
Advertisement robots - Gualzru
● a product for INDRA Spanish tech
company.
● completely designed and built our
group of re...
Advertisement robots - Gualzru
Gualzru building process
Gualzru Conversational skills
Gualzru multi-modal communication
Affective robots - Muecas
● 13 motors
● Stereo vision
● IMU
● Great for HRI
○ affective robotics
○ visual attention
○ mout...
Affective robots - Muecas
Affective robots - Muecas
Affective robots - Muecas
Eye tracking (2006)
Social robots - Loki
● too expensive
● Schunk motors and Barrett hand, hard to integrate in open source designs
Social robots - Loki
Social robots - Shelly
Social robots - Shelly
Social robots - Shelly
Social robots - Shelly
2014 Toulouse
2015 Lisboa
Social Robotics - Autonomy Lab
● 70 m²
● almost ready to live
apartment
Group of groups
robotics
planning
health vision
audio
natural language processing
A family of robots
RoboComp
Why is robotics software hard?
Drivers People Failure Algorithms
The complexity trap
Robotics frameworks
2002 2003 2006 2008 201020092004 2005 2007
RoboComp
● Distributed, component-oriented
● Formal component model
● pub/sub and RPC comm models
● Industrial grade commu...
Component model
SpecificWorker
GenericWorker
+ compute()=0
GenericMonitor
Main
thread
SpecificMonitor
thread
- SpecificWor...
Communication models
push / pull pub / subscribe
thread
publications
both can be synchronous or asynchronous
thread
reques...
Communication models (code)
pub / subscribe
while(True):
c = proxyA.getData(a,b)
d = proxyB.update(c)
do_something_useful(...
Communication models (comp view)
middleware code
requests
subscriptions
requests
publications
while(True):
c = proxyA.getD...
Component model: ComponentDSL
import "/robocomp/interfaces/IDSLs/interface1.idsl";
import "/robocomp/interfaces/IDSLs/inte...
Component model: InterfazDSL
module RoboCompDifferentialRobot {
exception HardwareFailedException { string what; };
struct...
Component model: StateMachineDSL
Name_machine
{
[ states name_state *[, name_state];]
[ initial_state name_state;]
[ end_s...
Component model: StateMachineDSL
MyMachine {
states State2, State3, State4, State5;
initial_state State1;
end_state State6...
DSL Code Generator
import "/robocomp/interfaces/IDSLs/AprilTags.idsl";
import "/robocomp/interfaces/IDSLs/AprilBasedLocali...
… generates a code directory
MyComp
myComp.cdsl bin CMakeLists.txt DoxyFileetcREADME.mdsrc
myComp main.cpp
CMakeLists.txt
...
RoboComp Tools
RoboComp’s other tools: RCManager
RoboComp other tools: 3D Simulator
Example: testing generalized inverse kinematics using LM non-linear optimization
Based ...
CORTEX
CORTEX: a proposal for a cognitive architecture
A graph-based representation of the robot and the world, shared by a set o...
RoboComp tools: DeepStateRepresentation
RoboComp tools: DeepStateRepresentation
3D view
CORTEX: a proposal for a cognitive architecture
CORTEX: a proposal for a cognitive architecture
● ¿What is the proper division of
functionalities?
CORTEX: a proposal for a cognitive architecture
● ¿What is the proper division of
functionalities?
● ¿What is the correct ...
CORTEX: a proposal for a cognitive architecture
● ¿What is the proper division of
functionalities?
● ¿What is the correct ...
… a real world problem
The BringMe(X) function
What the
hell!
Bring me the cup,
please
Bring me the
cup, please
cup is in the kitchen
Thank you!
Decision
Making
Navigation
& Manipul.
Objects Interaction
Paps i...
New project
LifeBots: Lifelong Technologies for Social Robots in Smart Homes
MINECO
Retos de la Sociedad
2016-2019
Thanks
Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro
Upcoming SlideShare
Loading in …5
×

Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro

206 views

Published on

El diseño, construcción y desarrollo de robots sociales que convivan con los humanos facilitándoles sus condiciones de vida es uno de los retos tecnológicos más importantes de la actualidad. En esta presentación describiremos el camino seguido en esta línea desde RoboLab a través de un framework de desarrollo para robótica basado en componentes llamado RoboComp, de fuentes abiertas y creado durante los últimos 10 años en colaboración con varias universidades del país. Se presentarán brevemente los diferentes tipos de robots sociales que construidos en RoboLab, su uso y evolución con el tiempo. Todos ellos se han desarrollado sobre RoboComp que proporciona un conjunto de herramientas para asistir en el ciclo de vida de los componentes software. Entre éstas, se cuenta con un generador de código que usa varios lenguajes específicos de dominio para definir la conectividad y parte de la funcionalidad de los componentes. Finalmente, se describirá el estado actual de la nueva arquitectura cognitiva CORTEX, desarrollada sobre RoboComp y que, en colaboración con varios grupos de robótica, está siendo implementada y probada en varios proyectos y nuevos robots.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro

  1. 1. Desarrollo de robots sociales con RoboComp Pablo Bustos RoboLab Universidad de Extremadura UCM - 2016
  2. 2. A quick introduction to 2000-
  3. 3. Social and Service Robotics ● Robots that help humans in everyday activities
  4. 4. Social and Service Robotics ● Robots that help humans in everyday activities Educational
  5. 5. Social and Service Robotics ● Robots that help humans in everyday activities Educational Rehabilitation
  6. 6. Social and Service Robotics ● Robots that help humans in everyday activities Educational Rehabilitation Advertisement
  7. 7. Social and Service Robotics ● Robots that help humans in everyday activities Educational Rehabilitation Advertisement Affective
  8. 8. Social and Service Robotics ● Robots that help humans in everyday activities Educational Rehabilitation Advertisement Affective Butler
  9. 9. Educational robots - Robex ● 2005 - now ● college, comp. sci. ● low-cost ● versatile ● robust ● classroom ● adaptable
  10. 10. Educational robots - Robex
  11. 11. Educational robots in school - Dulce ● 2011 ● 6th grade primary school ● 1 semester experiment
  12. 12. Educational robots in school - Dulce
  13. 13. Educational robots in school - Learnbot 1 ● 2012 ● Initiation to Python programming for kids in 5th to 12th grades ● Build with RaspberryPi and Arduino ● WiFi connection to laptop running Python
  14. 14. Educational robots in school - LearnBot 2 ● Odroid only ● WiFi, battery, USs and USB camera ● under 150€ for one prototype
  15. 15. Educational robots in school - LearnBot 2 def code(self): # 0 back 3 left 2 right 1 front actions = ((150,-3), (450, 0), (150, 3)) while(True): frame = self.getImage() rois = self.getROIS(frame) maxIndex = np.argmax(rois) self.setRobotSpeed(actions[maxIndex]) sum of black pixels in three ROIs
  16. 16. Rehabilitation Robots
  17. 17. Rehabilitation robots - Ursus ● 2010 ● Dynamixel servos ● Visual mark detection
  18. 18. Rehabilitation robots - Ursus
  19. 19. Rehabilitation robots - Ursus ● Technology validation experiment ● 5 patients in 7-10 age range ● mild lesions affecting the shoulder (brachial or cerebral paralysis)
  20. 20. Social robots - Nao
  21. 21. Social robots - Nao
  22. 22. Advertisement robots - Gualzru ● a product for INDRA Spanish tech company. ● completely designed and built our group of research teams. ● differential base, batteries, tactile panel, leds for the eyes, micro, speaker, laser, RGBD. ● runs RoboCog on top of RoboComp framework. ● dozens of hours of operation.
  23. 23. Advertisement robots - Gualzru
  24. 24. Gualzru building process
  25. 25. Gualzru Conversational skills
  26. 26. Gualzru multi-modal communication
  27. 27. Affective robots - Muecas ● 13 motors ● Stereo vision ● IMU ● Great for HRI ○ affective robotics ○ visual attention ○ mouth synchronization ○ generation of facial gestures ○ speech augmented with facial expressions
  28. 28. Affective robots - Muecas
  29. 29. Affective robots - Muecas
  30. 30. Affective robots - Muecas
  31. 31. Eye tracking (2006)
  32. 32. Social robots - Loki ● too expensive ● Schunk motors and Barrett hand, hard to integrate in open source designs
  33. 33. Social robots - Loki
  34. 34. Social robots - Shelly
  35. 35. Social robots - Shelly
  36. 36. Social robots - Shelly
  37. 37. Social robots - Shelly 2014 Toulouse 2015 Lisboa
  38. 38. Social Robotics - Autonomy Lab ● 70 m² ● almost ready to live apartment
  39. 39. Group of groups robotics planning health vision audio natural language processing
  40. 40. A family of robots
  41. 41. RoboComp
  42. 42. Why is robotics software hard? Drivers People Failure Algorithms
  43. 43. The complexity trap
  44. 44. Robotics frameworks 2002 2003 2006 2008 201020092004 2005 2007
  45. 45. RoboComp ● Distributed, component-oriented ● Formal component model ● pub/sub and RPC comm models ● Industrial grade communications middleware (Ice) with public interfaces ● Code generation using DSLs technology ● Tool set: deploy, monitor, log, simulator ● C++ and Python ● > 100 components ● Talks to ROS using DSL tech.
  46. 46. Component model SpecificWorker GenericWorker + compute()=0 GenericMonitor Main thread SpecificMonitor thread - SpecificWorker w Ice App user code automatically generated code INTERNET
  47. 47. Communication models push / pull pub / subscribe thread publications both can be synchronous or asynchronous thread requests thread thread
  48. 48. Communication models (code) pub / subscribe while(True): c = proxyA.getData(a,b) d = proxyB.update(c) do_something_useful( c, d ) push / pull while(True): a,b =check_is_something_arrived() do_something_useful( a, b ) ## Running in middleware’s thread def subscribe_method1(a): safe_copy_to_local(a) def subscribe_method2(b): safe_copy_to_local(b)
  49. 49. Communication models (comp view) middleware code requests subscriptions requests publications while(True): c = proxyA.getData(a,b) STORM.publish( topic ) a,b =check_is_something_arrived() do_something_useful( a, b ) ## Running in middleware’s thread def subscribe_method1(a): safe_copy_to_local(a) def interfaz_method(a): safe_copy_from_local(a)
  50. 50. Component model: ComponentDSL import "/robocomp/interfaces/IDSLs/interface1.idsl"; import "/robocomp/interfaces/IDSLs/interface2.idsl"; Component mycomponent { Communications { implements interface1; requires interface2; subscribesTo topicToSubscribeTo; publishes topicToPublish; }; language Cpp; gui Qt(QWidget); statemachine statemachine.smdsl; };
  51. 51. Component model: InterfazDSL module RoboCompDifferentialRobot { exception HardwareFailedException { string what; }; struct TBaseState { float x; float correctedX; float z; float correctedZ; float alpha; float correctedAlpha; float advV; float rotV; float adv; float rot; bool isMoving; float voltage; }; .idsl file interface DifferentialRobot { void getBaseState(out TBaseState state) throws HardwareFailedException; void getBasePose(out int x, out int z, out float alpha) throws HardwareFailedException; void setSpeedBase(float adv, float rot) throws HardwareFailedException; void stopBase() throws HardwareFailedException; void resetOdometer() throws HardwareFailedException; void setOdometer(TBaseState state) throws HardwareFailedException; void setOdometerPose(int x, int z, float alpha) throws HardwareFailedException; void correctOdometer(int x, int z, float alpha) throws HardwareFailedException; }; }; Ice provides binaries to generate proxies for different languages
  52. 52. Component model: StateMachineDSL Name_machine { [ states name_state *[, name_state];] [ initial_state name_state;] [ end_state name_state;] [ transition { name_state => name_state *[, name_state]; *[name_state => name_state *[, name_state];] };] }; [:parent_state [parallel] { states name_state *[, name_state]; [ initial_state name_state;] [ end_state name_state;] [ transition { name_state => name_state *[, name_state]; *[name_state => name_state *[, name_state];] }; ] };]
  53. 53. Component model: StateMachineDSL MyMachine { states State2, State3, State4, State5; initial_state State1; end_state State6; Transition { State1 => State1, State2; State2 => State3, State5, State6; State3 => State3, State4; State4 => State5; State5 => State6; }; }; :State1 parallel { states State11, State12; Transition { State11 => State11; State12 => State12; }; }; ………..
  54. 54. DSL Code Generator import "/robocomp/interfaces/IDSLs/AprilTags.idsl"; import "/robocomp/interfaces/IDSLs/AprilBasedLocalization.idsl"; import "/robocomp/interfaces/IDSLs/Laser.idsl"; import "/robocomp/interfaces/IDSLs/NewAprilPose.idsl"; Component MyComp { Communications { requires Laser; implements NewAprilPose; subscribesTo AprilTags; publishes AprilBasedLocalization; }; language Cpp; statemachine statemachine.smdsl; }; .cdsl file .idsl files communication primitives user@machine: robocompcdsl mycomponent.cdsl
  55. 55. … generates a code directory MyComp myComp.cdsl bin CMakeLists.txt DoxyFileetcREADME.mdsrc myComp main.cpp CMakeLists.txt genericmonitor.cpp specificmonitor.cpp genericworker.cpp specificworker.cpp config 1500 lines of code in C++
  56. 56. RoboComp Tools
  57. 57. RoboComp’s other tools: RCManager
  58. 58. RoboComp other tools: 3D Simulator Example: testing generalized inverse kinematics using LM non-linear optimization Based on the Open Scene Graph engine
  59. 59. CORTEX
  60. 60. CORTEX: a proposal for a cognitive architecture A graph-based representation of the robot and the world, shared by a set of reactive-deliberative agents
  61. 61. RoboComp tools: DeepStateRepresentation
  62. 62. RoboComp tools: DeepStateRepresentation 3D view
  63. 63. CORTEX: a proposal for a cognitive architecture
  64. 64. CORTEX: a proposal for a cognitive architecture ● ¿What is the proper division of functionalities?
  65. 65. CORTEX: a proposal for a cognitive architecture ● ¿What is the proper division of functionalities? ● ¿What is the correct choice of agents?
  66. 66. CORTEX: a proposal for a cognitive architecture ● ¿What is the proper division of functionalities? ● ¿What is the correct choice of agents? ● ¿Is developmental cognition the only way?
  67. 67. … a real world problem The BringMe(X) function What the hell! Bring me the cup, please
  68. 68. Bring me the cup, please cup is in the kitchen Thank you! Decision Making Navigation & Manipul. Objects Interaction Paps is in the living room You are here Reinforcement Planning / Replanning Monitoring Object detection and learning Trajectory control Path planning Annotated path Path planning ASR + NLP + TTS C O R T E X ASR + NLP + TTS Monitoring Monitoring Monitoring Monitoring Monitoring Monitoring Monitoring You are here Lifelong learning Lifelong learning Lifelong learning Lifelong learning Lifelong learning Lifelong learning Lifelong learning Lifelong learning TRANSFER & MULTI MODAL HRI INCREM. OBJECT LEARNING REACH & GRASP NATURAL LANGUAGE DIALOG SEMANTIC PATH PLANNING ANNOTATED NAVIGATION COGNITIVE SUBTRACTION 3D OBJECT FITTING POLICY LEARNING Grasping 3D registration to known models The problem
  69. 69. New project LifeBots: Lifelong Technologies for Social Robots in Smart Homes MINECO Retos de la Sociedad 2016-2019
  70. 70. Thanks

×