1.
Artiﬁcial Touch L. Ascari Artiﬁcial Touch Introduction Towards a new approach in prosthetics? The tactile system Modelling L. Ascari Validation Conclusions and Future HENESIS S.R.L. Options References Parma - February 22nd, 2012 —All the activity described in the presentation has been carried on while post-doc at Scuola Superiore Sant’Anna, Pisa (I)
2.
Outline Artiﬁcial Touch1 Introduction L. Ascari Touch in Robotics Touch in Prosthetics - Commercial SoA Introduction Approach The tactile system The pick and lift task Modelling Bioinspiration Validation Conclusions2 The tactile system and Future Options Hardware References Software3 Modelling4 Validation5 Conclusions and Future Options
3.
Outline Artiﬁcial Touch1 Introduction L. Ascari Touch in Robotics Touch in Prosthetics - Commercial SoA Introduction Approach The tactile system The pick and lift task Modelling Bioinspiration Validation Conclusions2 The tactile system and Future Options Hardware References Software3 Modelling4 Validation5 Conclusions and Future Options
4.
Outline Artiﬁcial Touch1 Introduction L. Ascari Touch in Robotics Touch in Prosthetics - Commercial SoA Introduction Approach The tactile system The pick and lift task Modelling Bioinspiration Validation Conclusions2 The tactile system and Future Options Hardware References Software3 Modelling4 Validation5 Conclusions and Future Options
5.
Outline Artiﬁcial Touch1 Introduction L. Ascari Touch in Robotics Touch in Prosthetics - Commercial SoA Introduction Approach The tactile system The pick and lift task Modelling Bioinspiration Validation Conclusions2 The tactile system and Future Options Hardware References Software3 Modelling4 Validation5 Conclusions and Future Options
6.
Outline Artiﬁcial Touch1 Introduction L. Ascari Touch in Robotics Touch in Prosthetics - Commercial SoA Introduction Approach The tactile system The pick and lift task Modelling Bioinspiration Validation Conclusions2 The tactile system and Future Options Hardware References Software3 Modelling4 Validation5 Conclusions and Future Options
7.
Outline Artiﬁcial Touch1 Introduction L. Ascari Touch in Robotics Touch in Prosthetics - Commercial SoA Introduction Touch in Robotics Approach Prosthetics SoA Approach The pick and lift task The pick and lift task Bioinspiration Bioinspiration The tactile system2 The tactile system Modelling Hardware Validation Software Conclusions and Future3 Modelling Options References4 Validation5 Conclusions and Future Options
8.
Touch in Robotics I Artiﬁcial Touch L. Ascari IntroductionRobots are now very complex and sophisticated systems. Touch in RoboticsHigher computational requirements. Prosthetics SoA Approach The pick and lift task Automation robots: very high performing and reliable Bioinspiration machines. The tactile system Outside the factory ﬂoor: limited interaction with humans, Modelling specially in terms of autonomous behavior and of friendly Validation HMIs1 , Conclusions and Future despite a huge market is expected to develop rapidly2 . Options References
9.
Touch in Robotics II Artiﬁcial Touch L. Ascari Tactile sensing can provide information about mechanical properties such as compliance, friction, and mass. Introduction Touch in Robotics Knowledge of these parameters is essential if robots are to Prosthetics SoA Approach reliably handle unknown objects in unstructured The pick and lift task Bioinspiration environments. For interaction, localization of the stimulus The tactile is essential3 . system Modelling Validation Conclusions 1 and Future J. Ayers et al. Neurotechnology for biomimetic robots. MIT Press, Options2002. References 2 WorldRobotics. World Robotics 2006. International Federation ofRobotics, Statistical Department, 2006. url:http://www.worldrobotics-online.org/. 3 R. D. Howe. “Tactile sensing and control of robotic manipulation”. In:Journal of Advanced Robotics 8 (1994), pp. 245–261.
10.
For what? Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA Approach Interaction The pick and lift task Bioinspiration The tactile system Modelling Validation Autonomy Conclusions and Future Options References Locomotion
11.
For what? Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA Approach Interaction The pick and lift task Bioinspiration The tactile system Modelling Validation Autonomy Conclusions and Future Options References Locomotion
12.
For what? Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA Approach Interaction The pick and lift task Bioinspiration The tactile system Modelling Validation Autonomy Conclusions and Future Options References Locomotion
13.
For what? Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA Approach Interaction The pick and lift task Bioinspiration The tactile system Modelling Validation Autonomy Conclusions and Future Options References Locomotion
14.
SOA in robotic skins? Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References
15.
Open Issues Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA wiring Approach The pick and lift task robustness Bioinspiration The tactile stretchability system Modelling bandwidth Validation processing Conclusions and Future Options References
16.
Open Issues Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA wiring Approach The pick and lift task robustness Bioinspiration The tactile stretchability system Modelling bandwidth Validation processing Conclusions and Future Options References
17.
Open Issues Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA wiring Approach The pick and lift task robustness Bioinspiration The tactile stretchability system Modelling bandwidth Validation processing Conclusions and Future Options References
18.
Open Issues Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA wiring Approach The pick and lift task robustness Bioinspiration The tactile stretchability system Modelling bandwidth Validation processing Conclusions and Future Options References
19.
Open Issues Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA wiring Approach The pick and lift task robustness Bioinspiration The tactile stretchability system Modelling bandwidth Validation processing Conclusions and Future Options References
20.
Touch in Prosthetics - Commercial SoA Artiﬁcial TouchMore advanced: myoelectric control L. Ascari I-Limb Ultra from Touch Bionics Introduction Ultra from BeBionics Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References
21.
Touch in Prosthetics - Commercial SoA Artiﬁcial TouchMore advanced: myoelectric control L. Ascari I-Limb Ultra from Touch Bionics Introduction Ultra from BeBionics Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options ReferencesOften refused by patients!
22.
Touch in Prosthetics - Commercial SoA Artiﬁcial TouchClassical prosthesis, cable actuated L. AscariOtto bock grippers Introduction Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References
23.
Touch in Prosthetics - Commercial SoA Artiﬁcial TouchClassical prosthesis, cable actuated L. AscariOtto bock grippers Introduction Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options ReferencesNot sensorized. Higher user acceptance. Why?
24.
Contemporary prosthetics: directions and openissues4 ArtiﬁcialDirections Touch autonomous control of low level tasks L. Ascari higher spatial resolution of the sensing system Introduction Touch in Robotics neural control (prototypes exist) Prosthetics SoA Approach feedback to the patient (preliminary results) The pick and lift task Bioinspiration The tactile systemOpen issues Modelling connection with tactile nerves Validation dexterity Conclusions and Future sensitivity Options References CONTROL (myo-electrical vs neural) feedback to the patient 4 R.G.E. Clement et al. “Bionic prosthetic hands: A review of presenttechnology and future aspirations”. In: The Surgeon 9.6 (12/2011),
25.
Contemporary prosthetics: directions and openissues4 ArtiﬁcialDirections Touch autonomous control of low level tasks L. Ascari higher spatial resolution of the sensing system Introduction Touch in Robotics neural control (prototypes exist) Prosthetics SoA Approach feedback to the patient (preliminary results) The pick and lift task Bioinspiration The tactile systemOpen issues Modelling connection with tactile nerves Validation dexterity Conclusions and Future sensitivity Options References CONTROL (myo-electrical vs neural) feedback to the patient 4 R.G.E. Clement et al. “Bionic prosthetic hands: A review of presenttechnology and future aspirations”. In: The Surgeon 9.6 (12/2011),
26.
Basic questions Artiﬁcial TouchSome fundamental questions L. Ascari What is the main issue with advanced prosthesis? Introduction Touch in Robotics Prosthetics SoA Is feedback to the user essential for this? Approach The pick and lift task Bioinspiration“Solved” Issues The tactile system low level control with many signals (here) Modelling parallel but portable processing (here) Validation Conclusions mechanics (single ﬁngers, underactuation, . . . ) and Future Options References
27.
Basic questions Artiﬁcial TouchSome fundamental questions L. Ascari What is the main issue with advanced prosthesis? Object Introduction Slippage and Grasp force control Touch in Robotics Prosthetics SoA Is feedback to the user essential for this? Approach The pick and lift task Bioinspiration“Solved” Issues The tactile system low level control with many signals (here) Modelling parallel but portable processing (here) Validation Conclusions mechanics (single ﬁngers, underactuation, . . . ) and Future Options References
28.
Basic questions Artiﬁcial TouchSome fundamental questions L. Ascari What is the main issue with advanced prosthesis? Object Introduction Slippage and Grasp force control Touch in Robotics Prosthetics SoA Is feedback to the user essential for this? No! Approach The pick and lift task Bioinspiration“Solved” Issues The tactile system low level control with many signals (here) Modelling parallel but portable processing (here) Validation Conclusions mechanics (single ﬁngers, underactuation, . . . ) and Future Options References
29.
Basic questions Artiﬁcial TouchSome fundamental questions L. Ascari What is the main issue with advanced prosthesis? Object Introduction Slippage and Grasp force control Touch in Robotics Prosthetics SoA Is feedback to the user essential for this? No! Approach The pick and lift task Bioinspiration“Solved” Issues The tactile system low level control with many signals (here) Modelling parallel but portable processing (here) Validation Conclusions mechanics (single ﬁngers, underactuation, . . . ) and Future Options References
30.
Bio-inspired approach Artiﬁcial TouchWhy and to what extent? L. Ascari Ultimate model: man Man Larger dimensions, Introduction Touch in Robotics Infinite Complexity: higher densities Prosthetics SoA sensors and processing Approach The pick and lift task Technological, Bioinspiration wiring, processing The tactile limitations system Modelling Model and Simplification Principle Validation Lower complexity validation Conclusions Innovative approach sensory systems and Future Options •Technology References •Processing •Scalability Star-nosed mole
31.
Bio-inspired approach Artiﬁcial TouchWhy and to what extent? L. Ascari Ultimate model: man Man Larger dimensions, Introduction Touch in Robotics Infinite Complexity: higher densities Prosthetics SoA sensors and processing Approach The pick and lift task Technological, Bioinspiration wiring, processing The tactile limitations system Touch sense Modelling Model and Simplification Principle Validation Lower complexity validation Conclusions Innovative approach and Future sensory systems Options •Technology References •Processing •Scalability Star-nosed mole
32.
The human hand: tactile structure Artiﬁcial Touch L. AscariHuman hand touch Structure of the skin Introduction 3 major groups of aﬀerent Touch in Robotics (tactile aﬀerents, joint Prosthetics SoA Approach mechanoreceptors, spindles) The pick and lift task Bioinspiration The glabrous skin has 17.000 The tactile system tactile units Modelling 4 main types of Validation mechanoreceptors (Ruﬃni, Conclusions and Future Pacini, Merkel, Meissner) for Options intensity, pressure, acceleration References stimuli
33.
The human hand: tactile structure Artiﬁcial Touch L. AscariHuman hand touch Structure of the skin Introduction 3 major groups of aﬀerent Touch in Robotics (tactile aﬀerents, joint Prosthetics SoA Approach mechanoreceptors, spindles) The pick and lift task Bioinspiration The glabrous skin has 17.000 The tactile system tactile units Modelling 4 main types of Validation mechanoreceptors (Ruﬃni, Conclusions and Future Pacini, Merkel, Meissner) for Options intensity, pressure, acceleration References stimulifrom Johansson and Westling (“Roles of glabrous skin receptors andsensorimotor memory in automatic control of precision grip whenlifting rougher or more slippery objects”)
34.
Sensors performance... Artiﬁcial Touch... in engineering terms L. Ascari Introduction Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References
35.
The pick and lift task ArtiﬁcialTwo aspects are crucial for a stable grasp: Touch L. Ascari the ability of the HW/SW system to avoid object slip Introduction to control in real-time the grasping force. Touch in Robotics Prosthetics SoA ApproachHuman physiology of the task The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References
36.
The pick and lift task ArtiﬁcialTwo aspects are crucial for a stable grasp: Touch L. Ascari the ability of the HW/SW system to avoid object slip Introduction to control in real-time the grasping force. Touch in Robotics Prosthetics SoA ApproachHuman physiology of the task The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References
37.
On the need for feedback Artiﬁcial Touch L. AscariEvidence Where? Introduction Johansson measured Touch in Robotics 50-60ms of reaction Prosthetics SoA Approach time The pick and lift task Bioinspiration incompatible with The tactile system propagation time to the Modelling motor cortex Validation evidence of circuit Conclusions and Future closed at subcortical Options level (olivo-cerebellar References system and thalamus).from Johansson and Westling (“Roles of glabrous skin receptors andsensorimotor memory in automatic control of precision grip whenlifting rougher or more slippery objects”)
38.
On the need for feedback Artiﬁcial Touch L. AscariEvidence Where? Introduction Johansson measured Touch in Robotics 50-60ms of reaction Prosthetics SoA Approach time The pick and lift task Bioinspiration incompatible with The tactile system propagation time to the Modelling motor cortex Validation evidence of circuit Conclusions and Future closed at subcortical Options level (olivo-cerebellar References system and thalamus).from Johansson and Westling (“Roles of glabrous skin receptors andsensorimotor memory in automatic control of precision grip whenlifting rougher or more slippery objects”)
39.
On the need for feedback Artiﬁcial Touch L. AscariEvidence Where? Introduction Johansson measured Touch in Robotics 50-60ms of reaction Prosthetics SoA Approach time The pick and lift task Bioinspiration incompatible with The tactile system propagation time to the Modelling motor cortex Validation evidence of circuit Conclusions and Future closed at subcortical Options level (olivo-cerebellar References system and thalamus).from Johansson and Westling (“Roles of glabrous skin receptors andsensorimotor memory in automatic control of precision grip whenlifting rougher or more slippery objects”)
40.
Biological vs Robotic worlds Artiﬁcial TouchDo we have these limitations (signaling speed) in robots? L. Ascari Man Introduction Biological models for the Touch in Robotics design of biomimetic robots Prosthetics SoA Approach The pick and lift task Nerves Brain Limbs Bioinspiration The tactile system Interfacing Bio and Modelling Robotics Validation Robot Conclusions and Future Options • Robots as physical platforms for validating biological models References Artificial Electric Artificial Brain wires limbs 3
41.
Biological vs Robotic worlds Artiﬁcial TouchDo we have these limitations (signaling speed) in robots? L. Ascari Man Introduction Biological models for the Touch in Robotics design of biomimetic robots Prosthetics SoA Approach The pick and lift task Nerves Brain Limbs Bioinspiration The tactile system Interfacing Bio and Modelling Robotics Validation Robot Conclusions and Future Options • Robots as physical platforms for validating biological models References Artificial Electric Artificial Brain wires limbs 3No, but other constraints exist. Ex: computational power
42.
Biological vs Robotic worlds Artiﬁcial TouchDo we have these limitations (signaling speed) in robots? L. Ascari Ultimate model: man Introduction Man Larger dimensions, Touch in Robotics Infinite Complexity: higher densities Prosthetics SoA sensors and processing Approach The pick and lift task Technological, Bioinspiration wiring, processing The tactile limitations Touch sense system Model and Modelling Simplification Principle Lower complexity validation Validation Innovative approach sensory systems Conclusions •Technology and Future Options •Processing References •Scalability Star-nosed moleNo, but other constraints exist. Ex: computational power
43.
Biological vs Robotic worlds Artiﬁcial TouchDo we have these limitations (signaling speed) in robots? L. Ascari Man Ultimate model: man Introduction Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Touch sense Modelling Validation Lower complexity Innovative approach sensory systems Conclusions and Future •Technological Options •Processing References •Scalability Star-nosed moleNo, but other constraints exist. Ex: computational power
44.
Multidisciplinarity — The animal model (touch) Artiﬁcial Touch L. AscariCondylura Cristata A nose to see / Eimer Introduction 12 mobile appendages Touch in Robotics covered with more than Prosthetics SoA Approach 25.000 tactile receptors The pick and lift task Bioinspiration (Eimer organs) The tactile system Structure of the Eimer Modelling organ: a sort of pillar with Validation 3 nervous terminations Conclusions (for constant pressures, and Future Options vibrations, ﬁne surface References details); foveated tactile vision.from Catania and Kaas (“Somatosensory Fovea in the Star-NosedMole: Behavioral Use of the Star in Relation to Innervation Patterns
45.
Multidisciplinarity — The animal model (touch) Artiﬁcial Touch L. AscariCondylura Cristata A nose to see / Eimer Introduction 12 mobile appendages Touch in Robotics covered with more than Prosthetics SoA Approach 25.000 tactile receptors The pick and lift task Bioinspiration (Eimer organs) The tactile system Structure of the Eimer Modelling organ: a sort of pillar with Validation 3 nervous terminations Conclusions (for constant pressures, and Future Options vibrations, ﬁne surface References details); foveated tactile vision.from Catania and Kaas (“Somatosensory Fovea in the Star-NosedMole: Behavioral Use of the Star in Relation to Innervation Patterns
46.
Multidisciplinarity — The animal model (touch) Artiﬁcial Touch L. AscariCondylura Cristata A nose to see / Eimer Introduction 12 mobile appendages Touch in Robotics covered with more than Prosthetics SoA Approach 25.000 tactile receptors The pick and lift task Bioinspiration (Eimer organs) The tactile system Structure of the Eimer Modelling organ: a sort of pillar with Validation 3 nervous terminations Conclusions (for constant pressures, and Future Options vibrations, ﬁne surface References details); foveated tactile vision.from Catania and Kaas (“Somatosensory Fovea in the Star-NosedMole: Behavioral Use of the Star in Relation to Innervation Patterns
47.
Multidisciplinarity — The animal model (vision) Artiﬁcial Touch L. AscariHoneybee Fixed yet good eye Introduction Non-mobile compound Touch in Robotics eyes (ommatidia); Prosthetics SoA Approach The pick and lift task 3000-4000 facets each eye Bioinspiration ( = 64x64 pixel array); The tactile system spatial resolution = 1/60 Modelling of the human eye; Validation No distance information Conclusions and Future from stereo vision; Options References Center facets larger than the peripheral sensors. yet: high performance
48.
Multidisciplinarity — The animal model (vision) Artiﬁcial Touch L. AscariHoneybee Fixed yet good eye Introduction Non-mobile compound Touch in Robotics eyes (ommatidia); Prosthetics SoA Approach The pick and lift task 3000-4000 facets each eye Bioinspiration ( = 64x64 pixel array); The tactile system spatial resolution = 1/60 Modelling of the human eye; Validation No distance information Conclusions and Future from stereo vision; Options References Center facets larger than the peripheral sensors. yet: high performance
49.
Multidisciplinarity — The animal model (vision) Artiﬁcial Touch L. AscariHoneybee Fixed yet good eye Introduction Non-mobile compound Touch in Robotics eyes (ommatidia); Prosthetics SoA Approach The pick and lift task 3000-4000 facets each eye Bioinspiration ( = 64x64 pixel array); The tactile system spatial resolution = 1/60 Modelling of the human eye; Validation No distance information Conclusions and Future from stereo vision; Options References Center facets larger than the peripheral sensors. yet: high performance
50.
Multidisciplinarity — The animal model (vision) Artiﬁcial Touch L. AscariHoneybee Fixed yet good eye Introduction Non-mobile compound Touch in Robotics eyes (ommatidia); Prosthetics SoA Approach The pick and lift task 3000-4000 facets each eye Bioinspiration ( = 64x64 pixel array); The tactile system spatial resolution = 1/60 Modelling of the human eye; Validation No distance information Conclusions and Future from stereo vision; Options References Center facets larger than the peripheral sensors. yet: high performance
51.
Multidisciplinarity — The animal model (vision) Artiﬁcial Touch L. AscariHoneybee Fixed yet good eye Introduction Non-mobile compound Touch in Robotics eyes (ommatidia); Prosthetics SoA Approach The pick and lift task 3000-4000 facets each eye Bioinspiration ( = 64x64 pixel array); The tactile system spatial resolution = 1/60 Modelling of the human eye; Validation No distance information Conclusions and Future from stereo vision; Options References Center facets larger than the peripheral sensors. yet: high performance
52.
Multidisciplinarity — The animal model (vision) Artiﬁcial Touch L. AscariHoneybee Fixed yet good eye Introduction Non-mobile compound Touch in Robotics eyes (ommatidia); Prosthetics SoA Approach The pick and lift task 3000-4000 facets each eye Bioinspiration ( = 64x64 pixel array); The tactile system spatial resolution = 1/60 Modelling of the human eye; Validation No distance information Conclusions and Future from stereo vision; Options References Center facets larger than the peripheral sensors. optical ﬂow balance yet: high performance motion detection (Flicker eﬀect)
53.
Multidisciplinarity — The computational model Artiﬁcial Touch L. AscariCellular non linear networks Parallel topological Introduction CNN is a massive parallel architecture Touch in Robotics computing paradigm deﬁned Prosthetics SoA Approach in discrete N-dimensional The pick and lift task Bioinspiration spaces. The tactile system A CNN is an N-dimensional Modelling regular array of elements Validation (cells); Conclusions and Future Cells are multiple input-single Options output analog processors, all References described by one or just some few parametric functionals.from Chua and Roska (Cellular Neural Networks and VisualComputing: Foundations and Applications)
54.
Multidisciplinarity — The computational model Artiﬁcial Touch L. AscariCellular non linear networks Parallel topological Introduction CNN is a massive parallel architecture Touch in Robotics computing paradigm deﬁned Prosthetics SoA Approach in discrete N-dimensional The pick and lift task Bioinspiration spaces. The tactile system A CNN is an N-dimensional Modelling regular array of elements Validation (cells); Conclusions and Future Cells are multiple input-single Options output analog processors, all References described by one or just some few parametric functionals.from Chua and Roska (Cellular Neural Networks and VisualComputing: Foundations and Applications)
55.
CNN characteristics I Artiﬁcial Touch L. Ascari Locality of the connections between the units: in fact the Introduction Touch in Robotics main diﬀerence between CNN and other Neural Networks Prosthetics SoA Approach paradigms is the fact that information are directly The pick and lift task exchanged just between neighbouring units. Of course this Bioinspiration The tactile characteristic allows also to obtain global parallel system processing. Modelling Validation A cell is characterized by an internal state variable, Conclusions sometimes not directly observable from outside the cell and Future Options itself; References More than one connection network can be present;
56.
CNN characteristics II Artiﬁcial Touch L. Ascari A CNN dynamical system can operate both in continuous Introduction (CT-CNN) or discrete time (DT-CNN), with analogical Touch in Robotics Prosthetics SoA signals from diﬀerent sources; Approach The pick and lift task CNN data and parameters are typically real values; Bioinspiration The tactile CNN operate typically with more than one iteration, i.e. system they are recurrent networks; It is a Universal Machine Modelling (CNN-UM); Validation Conclusions It oﬀers Stored programmability; and Future Options a Hardware implementation exists. References
57.
CNN core: the template Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation Conclusions and Future Options References
58.
Template meaning Artiﬁcial Touch L. Ascari Introduction Touch in Robotics Prosthetics SoA Approach The pick and lift task Bioinspiration The tactile system Modelling Validation State-out Conclusions and Future Options References in
59.
Features of the ACE4K (16K) chip — 3TOps Artiﬁcial Touch System Desktop PC, PC-104 industrial PC, Windows NT, 2000 L. Ascari Bus PCI, 33 MHz, 32 bit data width; Visual Microprocessor type ACE4k, 64x64 processor array Introduction Grayscale image download (64x64) 2688 frame/sec 372 !s Touch in Robotics Prosthetics SoA Grayscale image readback (64x64) 3536 frame/sec (compensated through look-up table); 283!s Approach Binary image download (64x64) 44014 frame/sec; 22.72 !s The pick and lift task Bioinspiration Binary image readback (64x64) 23937frame/sec; 41.78 !s Array operation (64x64) 9 !s + N*100ns The tactile Logical operation (64x64) 3.8 !s system DSP type Texas TMS320C6202; 250MHz, 1600 MIPS operation Modelling Memory 16MB, SDRAM 125 MHz; 2Mbyte FLASH (bootable) Validation Serial Ports 3 Other features Watch Dog, Timer Conclusions and Future Options Programmability C language, native languages References Image processing library Several image processing functions optimized for CVM Application Program Interface (API) Integrate the Aladdin systerm into different environments
60.
Recall Artiﬁcial Touch FINAL GOAL COMPUTATIONAL PLATFORM L. Ascari Introduction ROBOTIC Touch in Robotics PLATFORM Prosthetics SoA Approach The pick and lift task Tactile Bioinspiration system SW The tactile system Modelling TASK CONTROLLER Validation Tactile system HW Conclusions and Future Options References TASK, PHYSIOLOGICAL STRATEGY
61.
Outline Artiﬁcial Touch1 Introduction L. Ascari Touch in Robotics Touch in Prosthetics - Commercial SoA Introduction Approach The tactile system The pick and lift task Hardware Software Bioinspiration Modelling Validation2 The tactile system Conclusions Hardware and Future Options Software References3 Modelling4 Validation5 Conclusions and Future Options
62.
The MEMS mechanoreceptor Artiﬁcial Touch L. Ascari Introduction The tactile system Hardware Software Rpu Vc Modelling R1 R2 Validation V13 V24 Conclusions R3 R4 and Future Options References 0
63.
The array — Fabrication steps Artiﬁcial Touch L. Ascari Introduction The tactile system Hardware Software Modelling Validation Conclusions and Future Options References
64.
The whole system — HW Artiﬁcial Touch L. Ascari Introduction The tactile system Hardware Software Modelling Validation Conclusions and Future Options ReferencesFrom L Ascari et al. “A miniaturized and ﬂexible optoelectronicsensing system for tactile skin”. In: Journal of Micromechanicsand Microengineering 17.11 (11/2007), pp. 2288–2298. issn:0960-1317. doi: 10.1088/0960-1317/17/11/016. url:http://ejournals.ebsco.com/direct.asp?ArticleID=4A9A98E0B7D16F0C429C
65.
The whole system — from HW to SW Artiﬁcial Touch L. Ascari Introduction The tactile system Hardware Software Modelling Validation Conclusions and Future Options ReferencesFrom L. Ascari et al. “Bio-inspired grasp control in a robotichand with massive sensorial input”. In: Biological Cybernetics100.2 (2009), p. 109. doi: 10.1007/s00422-008-0279-0
66.
Recap Artiﬁcial Touch L. Ascari We have an array of analog multidirectional tactile signals Introduction The load cell were NOT calibrated: qualitative and only The tactile loose orthogonality system Hardware we can load and process analog tactile images on the CNN Software Modelling chip at 400 Hz Validation 54 sensors wrapped around the thumb and index ﬁngers of Conclusions a robotic underactuated hand and Future Options robotic arm controlled by DSP References
67.
Recap Artiﬁcial Touch L. Ascari We have an array of analog multidirectional tactile signals Introduction The load cell were NOT calibrated: qualitative and only The tactile loose orthogonality system Hardware we can load and process analog tactile images on the CNN Software Modelling chip at 400 Hz Validation 54 sensors wrapped around the thumb and index ﬁngers of Conclusions a robotic underactuated hand and Future Options robotic arm controlled by DSP References
68.
Recap Artiﬁcial Touch L. Ascari We have an array of analog multidirectional tactile signals Introduction The load cell were NOT calibrated: qualitative and only The tactile loose orthogonality system Hardware we can load and process analog tactile images on the CNN Software Modelling chip at 400 Hz Validation 54 sensors wrapped around the thumb and index ﬁngers of Conclusions a robotic underactuated hand and Future Options robotic arm controlled by DSP References
69.
Recap Artiﬁcial Touch L. Ascari We have an array of analog multidirectional tactile signals Introduction The load cell were NOT calibrated: qualitative and only The tactile loose orthogonality system Hardware we can load and process analog tactile images on the CNN Software Modelling chip at 400 Hz Validation 54 sensors wrapped around the thumb and index ﬁngers of Conclusions a robotic underactuated hand and Future Options robotic arm controlled by DSP References
70.
Recap Artiﬁcial Touch L. Ascari We have an array of analog multidirectional tactile signals Introduction The load cell were NOT calibrated: qualitative and only The tactile loose orthogonality system Hardware we can load and process analog tactile images on the CNN Software Modelling chip at 400 Hz Validation 54 sensors wrapped around the thumb and index ﬁngers of Conclusions a robotic underactuated hand and Future Options robotic arm controlled by DSP References
71.
Recap Artiﬁcial Touch L. Ascari We have an array of analog multidirectional tactile signals Introduction The load cell were NOT calibrated: qualitative and only The tactile loose orthogonality system Hardware we can load and process analog tactile images on the CNN Software Modelling chip at 400 Hz Validation 54 sensors wrapped around the thumb and index ﬁngers of Conclusions a robotic underactuated hand and Future Options robotic arm controlled by DSP ReferencesWhere is information? What kind of spatial and temporalpatterns? How to recognize and prevent slippage?
72.
Recap Artiﬁcial Touch L. Ascari We have an array of analog multidirectional tactile signals Introduction The load cell were NOT calibrated: qualitative and only The tactile loose orthogonality system Hardware we can load and process analog tactile images on the CNN Software Modelling chip at 400 Hz Validation 54 sensors wrapped around the thumb and index ﬁngers of Conclusions a robotic underactuated hand and Future Options robotic arm controlled by DSP ReferencesWhere is information? What kind of spatial and temporalpatterns? How to recognize and prevent slippage?We need to learn the tactile “alphabet”
73.
The task controller — FSM Artiﬁcial Touch L. Ascari Introduction The tactile system Hardware Software Modelling Validation Conclusions and Future Options References
74.
The task controller — FSM Artiﬁcial Touch L. Ascari Introduction The tactile system Hardware Software Modelling Validation Conclusions and Future Options References
75.
The task controller — Features Artiﬁcial Touch L. Ascari Introduction The tactile system Hardware Software Modelling Validation Conclusions and Future Options References
76.
Outline Artiﬁcial Touch1 Introduction L. Ascari Touch in Robotics Touch in Prosthetics - Commercial SoA Introduction Approach The tactile system The pick and lift task Modelling Bioinspiration Validation Conclusions2 The tactile system and Future Options Hardware References Software3 Modelling4 Validation5 Conclusions and Future Options
77.
The slip eﬀect in robotic grasp Artiﬁcial TouchSlip as vibrations. “Catch and snap” eﬀect on the rubber L. Ascari(60Hz stable + initial 10Hz component). Recall FAII humanmechanoreceptors. Introduction The tactile system Modelling Validation Conclusions and Future Options ReferencesHolweg et al., “Slip detection by tactile sensors: algorithms andexperimental results”
78.
Deﬁnition of Tactile Events of interest Artiﬁcial Touch L. Ascari Variations, oscillations, vibrations Introduction Time is divided in periods of The tactile duration T ∗ s system Modelling Variation change in signal Validation larger than σ in Conclusions same period and Future Options Oscillation seq. of 2 subsequent References variations of opposite sign in same T ∗ . (m,n) Vibration seq. of 2 oscillations in 2 adjacent periods σ = 2% dynamic range
79.
Deﬁnition of Tactile Events of interest Artiﬁcial Touch L. Ascari Variations, oscillations, vibrations Introduction Time is divided in periods of The tactile duration T ∗ s system Modelling Variation change in signal Validation larger than σ in Conclusions same period and Future Options Oscillation seq. of 2 subsequent References variations of opposite sign in same T ∗ . (m,n) Vibration seq. of 2 oscillations in 2 adjacent periods σ = 2% dynamic range
80.
Deﬁnition of Tactile Events of interest Artiﬁcial Touch L. Ascari Variations, oscillations, vibrations Introduction Time is divided in periods of The tactile duration T ∗ s system Modelling Variation change in signal Validation larger than σ in Conclusions same period and Future Options Oscillation seq. of 2 subsequent References variations of opposite sign in same T ∗ . (m,n) Vibration seq. of 2 oscillations in 2 adjacent periods σ = 2% dynamic range
81.
Assumptions - Naming conventions Artiﬁcial Touch L. Ascari Bandwidth = 200 Hz ⇒ st = 2.5ms Introduction Analysis period where to look for events: The tactile system T ∗ = 8 ∗ st = 20ms ≡ 50Hz ≡ period = 8 ∗ L; Modelling σ = 2%dynamicrange ≡ 5gray levels Validation (m, n) ≡ [(±m, ±n) | (±n, ±m)]. Optimal choice Conclusions and Future (empirically): (3,1) Options References Both amplitude and frequency of catch and snap back oscillations may vary (mechanical properties of skin, material, weight, roughness of object) ⇒ ﬂexible, parametrizable, robust approach.
82.
Assumptions - Naming conventions Artiﬁcial Touch L. Ascari Bandwidth = 200 Hz ⇒ st = 2.5ms Introduction Analysis period where to look for events: The tactile system T ∗ = 8 ∗ st = 20ms ≡ 50Hz ≡ period = 8 ∗ L; Modelling σ = 2%dynamicrange ≡ 5gray levels Validation (m, n) ≡ [(±m, ±n) | (±n, ±m)]. Optimal choice Conclusions and Future (empirically): (3,1) Options References Both amplitude and frequency of catch and snap back oscillations may vary (mechanical properties of skin, material, weight, roughness of object) ⇒ ﬂexible, parametrizable, robust approach.
83.
Assumptions - Naming conventions Artiﬁcial Touch L. Ascari Bandwidth = 200 Hz ⇒ st = 2.5ms Introduction Analysis period where to look for events: The tactile system T ∗ = 8 ∗ st = 20ms ≡ 50Hz ≡ period = 8 ∗ L; Modelling σ = 2%dynamicrange ≡ 5gray levels Validation (m, n) ≡ [(±m, ±n) | (±n, ±m)]. Optimal choice Conclusions and Future (empirically): (3,1) Options References Both amplitude and frequency of catch and snap back oscillations may vary (mechanical properties of skin, material, weight, roughness of object) ⇒ ﬂexible, parametrizable, robust approach.
84.
Assumptions - Naming conventions Artiﬁcial Touch L. Ascari Bandwidth = 200 Hz ⇒ st = 2.5ms Introduction Analysis period where to look for events: The tactile system T ∗ = 8 ∗ st = 20ms ≡ 50Hz ≡ period = 8 ∗ L; Modelling σ = 2%dynamicrange ≡ 5gray levels Validation (m, n) ≡ [(±m, ±n) | (±n, ±m)]. Optimal choice Conclusions and Future (empirically): (3,1) Options References Both amplitude and frequency of catch and snap back oscillations may vary (mechanical properties of skin, material, weight, roughness of object) ⇒ ﬂexible, parametrizable, robust approach.
85.
Assumptions - Naming conventions Artiﬁcial Touch L. Ascari Bandwidth = 200 Hz ⇒ st = 2.5ms Introduction Analysis period where to look for events: The tactile system T ∗ = 8 ∗ st = 20ms ≡ 50Hz ≡ period = 8 ∗ L; Modelling σ = 2%dynamicrange ≡ 5gray levels Validation (m, n) ≡ [(±m, ±n) | (±n, ±m)]. Optimal choice Conclusions and Future (empirically): (3,1) Options References Both amplitude and frequency of catch and snap back oscillations may vary (mechanical properties of skin, material, weight, roughness of object) ⇒ ﬂexible, parametrizable, robust approach.
86.
Some deﬁnitions Artiﬁcial Touch L. Ascari Introduction y (t) = a sin(2πνt + ϕ) (1) The tactile system with amplitude a, frequency Modelling ν Hz and phase Validation ϕ rad ∈ [−π, π] uniformly Conclusions and Future Options distributed. References
87.
Some deﬁnitions Artiﬁcial Touch L. Ascari Introduction y (t) = a sin(2πνt + ϕ) (1) The tactile system with amplitude a, frequency Modelling ν Hz and phase Validation ϕ rad ∈ [−π, π] uniformly Conclusions and Future Options distributed. ReferencesThis modeling is consistent with the non constancy of the meanvalue of the vibration, being the analysis limited to one period
88.
M ∗ ﬁxed: L max = 10. Therefore, Tmax = L max ∗ st = 10/400 s. 0 st t° T*/2 (L−1)⋅st T* νT ∗ ≤boundaries bring to νT ∗ < ν T ∗ = 2. In all the These 2 (7) max max following analysis we consider the boundary Equation 6 then becomes: Fig. 15 A possible conﬁguration of the input tactile signal: (+1,−3)Phase and frequency 0 st t° T*/2 (L−1)⋅st T* ∗ νTK ≤ 2 ≥ −1 (7) (8) length of the period, consisting of L consecutive frames; the K ≤ 2 6 then becomes: Equation Fig. 15 A possible conﬁguration of the input tactile signal: (+1,−3) phase ϕ ∈ [−π, π] is supposed to be uniformly distributed. K ≥ −1 values of K need then to be considered: K = Only four −1, 0, 1, 2 (for instance, K = 3 ⇒ νT ∗ > 5/2). (8) K ≤2 A.1 Phasethe period, consisting of L consecutive frames; the length of and frequency phase ϕ ∈ [−π, π] is supposed to be uniformly distributed. Only By values of K need the to be considered: t ◦ = Case B. fourinserting Eq. 3 intothencondition T ∗ /2 < K < Artiﬁcial ∗ One oscillation implies the presence of a peak (modeled as T ∗ we obtain: instance, K = 3 ⇒ νT > 5/2). −1, 0, 1, 2 (for Touch A.1 Phase and frequency a null-derivative point) in t ◦ ∈ 0, T ∗ , thus originating the Biol Cybern ∗ ∗ equation: π/2 + K By inserting < ϕ < π/2 + K π −π νT T ∗ /2 < t ◦ < Case B. π − 2π νT Eq. 3 into the condition One oscillation implies the presence of a peak (modeled as T ∗ we obtain: (9) L. Ascari 2π νt ◦=f +σ = π/2 + K inπt ◦ ∈ 0, T ∗ , thus originating (2) The product νT ∗ is worth a more detailed analysis. In the a null-derivative point) · fM + ϕ 0 the equation: π/2 +imposing this modeling lowerK π −π νT ∗ are more inter- framework of the ∗ < ϕ < π/2for the existence of at least By K π − 2π νT conditions + frequencies where K ∈ Z. Therefore: one ϕ ∈ [−π, π[ solution of Eq. 9,are more easily detected by esting than higher ones, which we obtain: (9) Introduction 2π νt ◦ + ϕf0 π 1 = π/2 + K · π (2) the ﬁlter. We limit our analysis to the range ν ∈ ]0, 80] Hz (in ∗ t◦ = · +K ·π −ϕ (3) By + K π − max 80 Hz). other words, the = > −π for latency of the ﬁlter (10) π/2 imposingνπ νTconditions Thethe existence of at least equals 2π ν 2 The tactile where K ∈ Z. Therefore: one ϕ st [−π,− 2π νT ∗ <of Eq. 9, we st = 2.5 ms: latencies π/2∈ and, π[ our implementation, obtain: L · + K π in solution π Figure 15 contains a possible situation, the case in which system 1 π higher that 25 ms would cause too slow reactions of the hand t ◦ signal increasesK · at − ϕ 1 · σ and then decreases by(3) the= · + by π least at which, + K π − π νT ∗ > −π π/2 solved, originates: 2π−3σ 2 fMσ (situation indicated by the notation (+1,−3)). ν to unexpected tactile stimuli, therefore an upper bound(10) is to L least 3 · π/2 + K π = 10. ∗ ∗ 3/2 K∗ − ﬁxed: LK /2− 2π νT < < π + Tmax = L max ∗ st = 10/400 s. 1/4 + max < νT Therefore, (11) Modelling Two casescontains a possible situation, the ∗ /2 in which Figure 15 can be identiﬁed: t ◦ ∈ 0, T case or t ◦ ∈ These boundaries bring ∗ ∗ < ν ∗ = 2. which, solved, originates: to νT [0, 2] max Tmax to Eq.In all the T /2, T , depending on the · σ and of the peak, in the the∗signal∗increases by at least 1position then decreases by at The variability range νT ∈ applied 11 to following analysis we consider the boundary Validation ﬁrst or · σthe second indicated by the notation (+1,−3)). least 3 in (situation half of the period. assure the K /2 < νTof < solution, originates: − 1/4 + existence ∗ a 3/2 + K (11) Two cases canstbe identiﬁed: tT*/2 0, T ∗ /2 or t ◦ ∈ 0 t° ◦ ∈ νT ∗ ≤ 2 (7) (L−1)⋅st T* ∗ /2, T ∗ , depending on the position of the peak, in ∗the 3/2 + K > 0 range νT ∗ ∈ [0, 2] applied to Eq. 11 to T Case A. By inserting Eq. 3 into the condition 0 < t ◦ ≤ T /2 The variability (12) Conclusions ﬁrst or in A second half of the period. obtain: −1/2 + K /2 < 2 becomes: Equation 6 then weFig. 15 thepossible conﬁguration of the input tactile signal: (+1,−3) assure the existence of a solution, originates: and Future From≥K 12, K −1 3/2 +Eq. > 0 we obtain the 6 values that K can acquire in(8) Options π/2 + K By− π νT ∗ ≤ ϕ 3 into the condition 0 < t ◦ ≤ T ∗ /2 Case A. π inserting Eq. < π/2 + K π (4) (12) length of the period, consisting of L consecutive frames; the this second K /2 < 2∈ [−1, 4]. −1/2≤ 2case: K K + we obtain: In conclusion, two sets of equations must be solved: the phase ϕ ∈ [−π, π] is supposed ∈ [−π, π[) imposes that, The range of variation of ϕ (ϕ to be uniformly distributed. References in orderK π Eq. 4 to ≤ ϕ < π/2 + K π From <four≤ T /2 of Kthe values that ∗ considered: K: = Only ◦ values and need then to be can ◦ ﬁrst for 0Eq.t12, we∗obtain the 6second for ∗ K/2 <acquire in T t < T∗ π/2 + for − π νT ∗have at least one solution, two boundary (4) conditions are satisﬁed, precisely: this second case: K ∈ [−1, K ∗= 3 ⇒ νT > 5/2). −1, 0, 2 (for instance, 4]. 1, A.1 Phase and frequency The range of variation of ϕ (ϕ ∈ [−π, π[) imposes that, In conclusion, two − π νT equations must be solved: the π/2 + K π sets of ≤ ϕ < π/2+ K π π νT ∗ < π π/2 + K π −4 to have at least one solution, two boundary Casefor 0 <By ≤ T ∗ /2 and theinto the for T ∗ /2 < ∗◦ << t∗◦: < ﬁrst A :B. νT◦∗ inserting1/2 3 second condition T t /2 (13) Case Eq. in order for Eq. t >K− T One oscillation implies the presence of a peak (modeled as (5) T ∗ we K ∈ [−1, 2] obtain: conditions are> −π π/2 + K π satisﬁed, precisely: 0, T ∗ , thus originating the π/2 + K π − π νT ∗ ≤ ϕ < π/2+ K π a null-derivative point) in t ◦ ∈ π − 2π νT ∗ < ϕ < π/2 + K π −π νT ∗ equation: originates: which, + K π − π νT ∗ < π Case A+π/2+ KK −2π νT ∗ < ϕ < π/2 + K π − π νT (13) π/2 : K ∗ ∗ νT > − 1/2 π/2 solved, π (5) (9) π/2 ◦ K ϕ > −π 2π>+ + π = π/2 + K · π K νt −3/2 ≡ K ≥ −1 (2) Case B : −1/4[−1, 2] < νT ∗ < 3/2+ K K ∈ + K /2 (14) (6) By imposing the conditions for the existence of at least K ∈ [−1, 4] which, > K ∈ Z. Therefore: ∗ solved, originates: π/2+ K π −2π νT ∗ < ϕ < π/2 + K π − π νT ∗ where K − 1/2 νT one ϕ [−π, π[ solution of Eq. 9, we obtain: ∈ ◦ > 1 π ≥ −1 Case B : −1/4 + K /2∗ νT ∗ < 3/2+ K < (14) tK = −3/2· ≡ K+ K · π − ϕ (6)(3) π/2 K π − π νT > −π + K ∈ [−1, 4] 2π ν ∗ > K − 1/2 2 (10) νT π/2 + K π − 2π νT ∗ < π 123 Figure 15 contains a possible situation, the case in which the signal increases by at least 1 · σ and then decreases by at which, solved, originates: least 3 · σ (situation indicated by the notation (+1,−3)). − 1/4 + K /2 < νT ∗ < 3/2 + K 123 (11) Two cases can be identiﬁed: t ◦ ∈ 0, T ∗ /2 or t ◦ ∈
89.
Equation 6 then becomes: t tactile signal: (+1,−3) K ≥ −1 (8) nsecutive frames; the K ≤2 Phase and frequency niformly distributed. Only four values of K need then to be considered: K = −1, 0, 1, 2 (for instance, K = 3 ⇒ νT ∗ > 5/2). Case B. By inserting Eq. 3 into the condition T ∗ /2 < t ◦ < f a peak (modeled as T ∗ we obtain: , thus originating the π/2 + K π − 2π νT ∗ < ϕ < π/2 + K π −π νT ∗ Artiﬁcial (9) Touch (2) By imposing the conditions for the existence of at least Table 2 Resume of all the possibilities for a signal of given phase ϕ Table 4 Final set of equations and frequency ν to be recognized as oscillation by the ﬁlter ∗ L. Ascari one ϕ ∈ [−π, π[ solution of Eq. 9, we obtain: νT > 0 νT ∗ > 0 −π/2 − π νT ∗ ≤ ϕ < −π/2 π/2 + K π − π νT ∗ > −π (3) K = −1 + ∪1 (10) −π/2 − π νT ∗ ≤ ϕ < −π/2 K = −1 sin(ϕ) > y π/2 + K π − 2π νT ∗ < π Introduction on, the case in which sin(2π νT ∗ ) > x νT ∗ > 0 ∗d then decreases by at which, solved, originates: K =0 + ∩1 νT > 0 π/2 − π νT ∗ ≤ ϕ < π/2 The tactile otation (+1,−3)). − 1/4 + K /2 < νT ∗ < 3/2 + K (11) π/2 − π νT ∗ ≤ ϕ < π/2∈ 0, T ∗ /2 or t ◦ ∈ νT ∗ > 1/2 K =0 system K =1 + ∪1 sin(ϕ) < −y n of the peak, in the The variability range νT ∗ ∈ [0, 2] applied to Eq. 11 to 3/2π − π νT ∗ ≤ ϕ < π sin(2π νT ∗ ) < −x . assure the existence of a solution, originates: ∗ Modelling νT ∗ > 3/2 νT > 1/2 3/2 + K > 0 K =2 + ∩1 3/2π − π νT ∗ ≤ ϕ < π dition 0 < t ◦ ≤ T ∗ /2 (12) 5/2π − π νT ∗ ≤ ϕ < π K =1 Validation −1/2 + K /2 < 2 sin(ϕ) > y 0< νT ∗ < 1/2 sin(2π νT ∗ ) > x From Eq. 12, we obtain the 6 values that K can acquire in K = −1 + ∪2 ∗ Conclusionsπ (4) −π/2 − 2π νT ∗ ≤ ϕ < −π/2 − π νT ∗ νT > 3/2 this second case: K ∈ [−1, 4]. and Future 5/2π − π νT ∗ ≤ ϕ < π−π, π[) imposes that, In conclusion, two sets of equations must be solved: the 0 < νT ∗ < 3/2olution, two boundary ﬁrst for 0 < t ◦ ≤ T ∗ /2 and the second for T ∗ /2 < t ◦ < T ∗ : K =0 + ∩2 K =2 Options sin(ϕ) < −y π/2 − 2π νT ∗ ≤ ϕ < π/2 − π νT ∗ sin(2π νT ∗ ) < −x π/2 + K π − π νT ∗ ≤ ϕ < π/2+ K π References 1/4 < νT ∗ < 5/2 Case A : νT ∗ > K − 1/2 (13) K =1 + ∪2 0 < νT ∗ < 1/2 3/2π − 2π νT ∗ ≤ ϕ < 3/2π − π νT ∗ −π/2 − 2π νT ∗ ≤ ϕ < −π/2 − (5) K ∈ [−1, 2] K = −1 3/4 < νT ∗ < 7/2 sin(ϕ) > x π/2+ K π −2π νT ∗ < ϕ < π/2 + K π − π νT ∗ K =2 + ∩2 sin(2π νT ∗ ) > y 5/2π − 2π νT ∗ ≤ ϕ < 5/2π − π νT ∗ Case B : −1/4 + K /2 < νT ∗ < 3/2+ K (14) 0 < νT ∗ < 3/2 5/4 < νT ∗ < 9/2 π/2 − 2π νT ∗ ≤ ϕ < π/2 − π νT (6) K ∈ [−1, 4] K =3 + ∪2 7/2π − 2π νT ∗ ≤ ϕ < 7/2π − π νT ∗ K =0 sin(ϕ) < −x sin(2π νT ∗ ) < −y 7/4 < νT ∗ < 11/2 K =4 + ∩2 9/2π − 2π νT ∗ ≤ ϕ < 9/2π − π νT ∗ 1/4 < νT ∗ < 5/2 123 3/2π − 2π νT ∗ ≤ ϕ < 3/2π − π The double line separator splits Case A from Case B K =1 sin(ϕ) > x sin(2π νT ∗ ) > y Table 3 Equations on the amplitude of the input signal 3/4 < νT ∗ < 7/2 f (t ◦ ) − f (0) > σ sin(ϕ) < −y 5/2π − 2π νT ∗ ≤ ϕ < 5/2π − π ∩1 K =2 f (t ◦ ) − f (T ∗ ) > 3σ sin(ψ) < −x sin(ϕ) < −x
90.
+ ∪2 0 < νT < 1/2 0 < νT ∗ < 3/2 sin(2π νT ∗ ) > x K =2 νT ∗ ∗ 0 ∗ − 2π + ∩2 sin(2π νT ∗ ) > y − π νT ∗ 5/4 −π/2 νT 9/2 ϕ ≤ ϕ < − π νT − ≤ ∗ 5/2π< νT ><2π νT ∗< 5/2π −π/2 − π νT ∗ K =K = 0 3 + +∪2 ∩1 νT ∗ > 0 π/2 − 2π νT ∗ ≤ ϕ < π/2 − π νT ∗ ∗ 0 < νT ∗ < 3/2 K = −1 7/2ππ/2 −νTνT≤ ≤ < < π/2 π νT ∗ − 2π π ∗ sin(ϕ) > x ϕ 7/2π − ϕ K =0 5/4 < νT ∗ < 9/2 sin(ϕ) < −x∗ νT ∗ ≤ ϕ < π/2 ∗ π/2 − 2π νTπ ≤ ϕ < π/2 − π νT + ∩2 K =3 sin(2π νT ∗ ) > y + ∪2 π/2 − νT ∗ > 1/2 Phase and frequency− π νT ∗ K =K = 1 4 7/4 − 2π < < νT ∗ 3/2 ∗ ≤ ϕ < 7/2π − π νT ∗ 7/2π< νT ∗νT 11/2 0− 2π νT< ≤ ∗ < < π 9/2π3/2π − π∗νT ϕ≤ ϕ9/2π − π νT ∗ ∗ 7/4 < νT − < 11/2 ≤ ϕ < π/2 − π νT ∗ π/2 2π νT ∗ + +∩2 ∪1 K =0 K =0 sin(2π νT ∗ ) < −y sin(ϕ) < −x< −y sin(ϕ) sin(2π ∗ ) 1/4 < νT∗∗ < 5/2< −x sin(2π νT ) νT −y < 3/2π − ∗ νT ∗ ≤ ϕ < 3/2π − π νT ∗ + ∪2 K =4 + ∩2 2π− π νT ∗ νT ∗ K = double line separator > 3/2 Case A9/2π − π νT ∗ The 0 9/2π sin(ϕ)splits≤ ϕ < from Case B ∗ 1/4 νT ∗> 1/2 νT < 5/2 < K =2 − 2π νT −x < + ∩1 K =1 sin(ϕ) > x − π νT ∗ ≤ ϕ < π 5/2π ∗ ≤ 3/2π 3/2π νT ∗ ≤ ϕ < 3/2π − π νT ∗ − 2π sin(2π− π∗νT −yϕ < πCase B The double line separator splits ) < A from νT Case sin(2π νT ∗ ) > y K =1 =1 K + ∩2 sin(ϕ) sin(ϕ) > x > y − π νT ∗ Table 3 Equations on << amplitude of the input signal 0 the νT ∗ 1/2 1/4 νT ∗ << 5/2 3/4 < νT∗∗ < 7/2> x sin(2π > ∗ sin(2π νT ) νT y ) K = −1 3/2π − 2π νT ∗∗≤ ϕ < 3/2π − π νT ∗ ∗ (t ◦ ) − f (0) > σ ≤ ϕ + ∪2 Case B Table 3 Equationsf on the amplitude of the< −π/2 − πsin(ϕ) < −y −π/2 − 2π νT input signal νT 5/2π − ∗ ∗ νT ∗ ≤ ϕ < 5/2π − π νT ∗ 2π Artiﬁcial ∩= K1 1 K =2 3/4 νT > 3/2 νT < 7/2 < f sin(ϕ) (T (t◦◦ ) − f> x∗ ) > 3σ sin(ψ) < −x sin(ϕ) < −x − f ∗ < 3/2 f (t0)< νT(0) > σ sin(ϕ) < −y − 2π − ∗ ≤ ∗ < 5/2π 5/2π 5/2π νTπ νT ϕ ≤ ϕ < π− π νT ∗ Touch ∩1 K = 0 sin(2π νT ∗ ) > y sin(2π νT ∗ ) < −y ◦ ) − f (T ∗ ) > 3σ + ∩2K = 2 = 2 K put signal ∪1 f (tπ/2 − 2π∗ ∗ ≤ ϕ < π/2 − π νT sin(ψ) < −x (t ◦ νT 7/2 ∗ sin(ϕ) > y sin(ϕ) < −x< −y sin(ϕ) (0) −<f νT ) > σ f 3/4 < 5/4 < νT ∗ < 9/2 sin(2π ∗ ) (T ∗ ) − f (t ◦ ) > 3σ f sin(ψ) > x sin(2π νT ∗ ) νT −y< −x < L. Ascari sin(ϕ) < −y 5/2π − 2π νT ∗ ≤ ϕ < 5/2π − π νT ∗ 1/4 νT ∗ < ∪1 f (0) − <(t ◦ ) > σ5/2 f sin(ϕ) > y 7/2π − 2π νT ∗ ≤ ϕ < 7/2π − π νT ∗ K =2 =1 K + ∪2 K = 3 < νT ∗ < 5/4 0νT ∗ < 9/21/2 < sin(ψ) < −x ∗ ) − f (t−x> ∗ ◦) f sin(ϕ) − 2π νT 3σ ϕ < 3/2π − π νT sin(ψ) > x ∗ (t ◦ ) − f< > 3σ≤ (T 3/2π (0) sin(ϕ) < −x sin(ϕ) > x 7/2π −π/2νT ∗ ≤ ϕ ∗ ≤ ϕ < − π νT− π νT ∗ ∩2 f sin(2π νT ∗ ) < −y − 2π − 2π νT < 7/2π −π/2 ∗ ◦ Introduction (T ∗ > σ (t◦ ) − fνT ∗ )< 7/2 f 3/4 sin(ψ) < −y K = 3 = −1 K sin(2π νT ∗ ) > y sin(ϕ) > y ∩2 f (t5/4−< νT ∗ > 3σ sin(ϕ) < −x sin(ϕ) > x sin(ϕ) > x∗ K =2 ) <f (0) < 9/2 + ∩2 7/4 < νT sin(2π νT ∗ ) > sin(ψ) > x 5/2π − 2π νT ∗ ϕ < 5/2π − π νT ∗ < −y ◦ ) − f ◦ >∗σ sin(2π νT ∗ ) < 11/2 y >y ff(t7/2π −(T ∗ )> 3σ≤ ϕ < 7/2π − π νT ∗ (0) − f (t2π νT ≤ sin(ψ) ∪2 ) sin(ϕ) > x < νT ∗ < 3/2 9/2π − 2π νT ∗ ≤ ϕ < 9/2π − π νT ∗ The tactile K =3 f ∗ ) − f (t ◦ ) > σ sin(ψ) > y K =4 7/4 0νT ∗ < 11/2 < sin(ϕ) < −x sin(ϕ) > x∗ < 9/2 (T system ∪2 K = 3 5/4 f νT f (0) − <(t ◦ ) > 3σ sin(ϕ) > x + sin(ϕ) < −x ∗ νT ∗ ≤ ϕ < π/2 − π νT ∗ 9/2π π/2 − 2π ≤ ϕ < 9/2π − π νT ∗ − 2π νT sin(ψ) < −y 7/2π − amplitude ϕ < 7/2π − π νT ∗ detected is ∪2 sin(2π νT ∗ ) > ∗y sin(2π νT ∗ ) < −y For all cases, the fminimum 2π > (T ∗ ) − f (t ◦ )νT σ≤ of the signal to be sin(ψ) > y K =4 =0 K sin(ϕ) < −x< −x sin(ϕ) a = 3/2σ 7/4 < νT ∗ < 11/2 sin(2π νT ∗ ) < −y Modelling 7/4 < amplitude For all cases, the minimum νT ∗ < ∗ of the signal to be∗ detected is sin(2π νT ∗ ) < −y sin(ϕ) > x 9/2π − 2π νT 11/2 < 9/2π − π νT ≤ϕ K = a = 3/2σ 4 + ∩2 1/4 < νT ∗ < 5/2 sin(ψ) > y K =4 9/2π − 2π νT ∗ ≤ ϕ < 9/2π − π νT ∗ sin(ϕ) < −x ∗ ∗ Validation signal to be detected is The 2 lists sin(2π νT ) < −y A from Case B Tabledouble line separator∗splitsobtained inserting the corres- 10 systems Case the in t − 2π νT ≤ ϕ < 3/2π − π νT mum and a minimum 3/2π◦ , respectively: symbols ∪1,2 and K =1 ponding values of K into Eqs. 13 (4 values) and 14 sin(ϕ) > x ∩1,2 are used to indicate a minimum and maximum conditions Table 2 lists the 10 systems obtained inserting the corres- mum and a minimum in t ◦ , νT ∗ ) > y respectively: symbols ∪1,2 and (6 values). From simple geometrical considerations (see sin(2π associated with each case, the number stands for the ﬁrst or Conclusions ponding values of K the amplitude13 the input signal and 14 Table 3 and odd on into Eqs. associated with a maxi- (4 values) ∩1,2 are used to indicate a minimum and maximum conditions Eq. 2), even Equationsvalues of K are of second half-period of 3/4 < νT ∗ < 7/2 occurrence. and Future (6 values). From simple geometrical considerations (see ◦ − (0) associated with each 5/2πthe2π νT case, number stands for the ﬁrst ord inserting the corres- mum∩ even minimum(tin) t ◦ ,of K > σ associated with sin(ϕ) < −y second half-period of occurrence. ∗ ≤ ϕ < 5/2π − π νT ∗ and a and odd fvalues frespectively: symbols ∪a maxi- Eq. 2),1 are 1,2 and − Options f (t ◦a − f (T ∗ ) >and maximum conditions −x K =2 (4 values) and 14 ∩1,2 are used to indicate 123 ) minimum 3σ sin(ψ) < sin(ϕ) < −x considerations (see associated with each case, the number stands for theBiol Cybern ﬁrst or sin(2π νT ∗ ) < −y Referencessociated with a maxi- 123 ∪1 second half-period of(0) − f (t ◦ ) > σ f occurrence. sin(ϕ) > y 5/4 < νT ∗ < 9/2 f (T ∗ ) − f (t ◦ ) > 3σ sin(ψ) > x or a signal of given phase ϕ Table 4 Final set of equations 7/2π − 2π νT ∗ ≤ ϕ < 7/2π − π νT ∗ation by the ﬁlter ∗ K =3 νT◦ > 0 sin(ϕ) < −x sin(ϕ) > x ∩2 f (t ) − f (0) > 3σ −π/2 − π νT ∗ ≤ ϕ < −π/2 f (t ◦ ) − f (T ∗ ) > σ sin(2π νT ∗ ) > y + ∪1 sin(ψ) < −yπ/2 K = −1 ∗ sin(ϕ) > y 7/4 < νT < 11/2 f (0) − f (t∗ ) > 3σ sin(2π νT ◦) > x sin(ϕ) > x ∪2 9/2π − 2π νT ∗ ≤ ϕ < 9/2π − π νT ∗ + ∩1 f (T ∗ ) − f (t ◦ ) > σ ∗ sin(ψ) > y K =4 νT > 0 sin(ϕ) < −x π/2 − π amplitude π/2 For all cases, the minimumνT ∗ ≤ ϕ < of the signal to be detected is sin(2π νT ∗ ) < −y K a= 03/2σ = + ∪1 sin(ϕ) < −y sin(2π νT ∗ ) < −x ∗ νT > 1/2 Table 2 lists the 10 systems obtained inserting the corres- mum and a minimum in t ◦ , respectively: symbols ∪1,2 and + ∩1 3/2π − π νT ∗ ≤ ϕ < π K ponding values of K into Eqs. 13 (4 values) and 14 =1 ∩1,2 are used to indicate a minimum and maximum conditions sin(ϕ) > y (6 values). From simple geometrical considerations (see associated with each case, the number stands for the ﬁrst or ∗ + ∪2 Eq. 2), even and sin(2π νT ) > x K are associated with a maxi- odd values of second half-period of occurrence. π/2 − π νT ∗ νT ∗ > 3/2
91.
of Table 4. A sinusoid with amplitude a = 10σ (roughly amplitude a = 10σ (roughly 0.1 0.1 51 σ) 51 σ) 0.1 0.1 Signal Amplitude aa(⋅(⋅51 σ) Signal Amplitude aa(⋅(⋅51 σ) corresponding to 0.2 in the y corresponding to 0.2 in the y 0.15 0.15 0.15 0.15 axis of the pictures, given the axis of the pictures, given the 0.2 0.2Phase and frequency current choice of σ = 5/255) is 0.2 0.2 Signal Amplitude Signal Amplitude current choice of σ = 5/255) is recognized as oscillation in 50% 0.25 0.25 0.25 0.25 recognized as oscillation in 50% of the cases if L = 8 (a), in 81% of the cases if L = 8 (a), in 81% 0.3 0.3 0.3 0.3 if L = 10 (b), depending on its if L = 10 (b), depending on its 0.35 0.35 0.35 0.35 phase ϕ phase ϕ 0.4 0.4 0.4 0.4 0.45 0.45 0.45 0.45 0.5 0.5 −3 0.5 0.5 −3 −2 −1 0 1 2 3 −2 −1 0 1 2 3 −3 −2 −1 0 Phase φ (rads) 1 2 3 −3 −2 −1 0 Phase φ (rads) 1 2 3 Artiﬁcial Phase φ (rads) Phase φ (rads) a a b Touch b L. Ascari Sinusoid Amplitude:10σ. Sinusoid Amplitude:10σ. After direct substitution of Eq. 1 into Eq. 15 we obtain: 0.2 After direct substitution of Eq. 1 into Eq. 15 we obtain: 0.2 a − a sin(ϕ) > σ a − a sin(ϕ) > σ Introduction 0.15 0.15 (16) ∗ (16) a − a sin(2π νT ∗ + ϕ) > 3σ a − a sin(2π νT + ϕ) > 3σ 0.1 0.1 The tactile and then and then Probability of amplitude (⋅51σ) system amplitude (⋅51σ) 0.05 0.05 a−σ sin(ϕ) < sin(ϕ) < a−σ a 0 a (17) (17) Modelling < detection! a−3σ 0 sin(2π νT ∗ + ϕ) sin(2π νT ∗ + ϕ) < a−3σ a a −0.05 −0.05 σ −a 3σ −a Deﬁning y = Deﬁning y = x = x = ψ = σ −a , a , 3σ −a , ψ = 2π νT ∗ , 2π νT ∗ , the ﬁnal Validation −0.1 −0.1 consistent withﬁnal equation for case 1 is obtained: equation for case 1 is obtained: athe a , a Conclusions −0.15 sin(ϕ) < −y −0.15 −0.2 sin(ϕ) < −y sin(ψ) < −x biological (18) (18) and Future −0.2 0 0.005 0.01 0.015 0.02 sin(ψ) < −x Options 0 0.005 0.01 0.015 0.02 time (s) time (s) counterpart the condi- In order for Eq. 18 to have at least one solution, the condi- In order for Eq. 18 to have at least one solution, References Fig. 17 Comparison between to sinusoidal inputs having the same fre- tions: tions: Fig. 17 Comparison between to sinusoidal inputs having the same fre- quency but different phase. The dashed signal does not match the requi- quency but different phase. The dashed signal does not match the requi- rements to be considered an oscillation rements to be considered an oscillation −y −y > −1 ≡ > −1 ≡ taking advantage y y <1 <1 (19) (19) −x −x > −1 ≡ > −1 ≡ x x <1 <1 Up to this point, only phase and frequency of the input of parallelism that must be satisﬁed: the condition on signal amplitude Up to this point, only phase and frequency of the input must be satisﬁed: the condition on signal amplitude that signal have been considered in the model. Symbols ∪1 , ∩1 , signal have been considered in the model. Symbols ∪1 , ∩1 , directly follows is a > 3/2σ . Equation 18 models the case directly follows is a > 3/2σ . Equation 18 models the case ∪2 , ∩2 contain additional relations linking amplitude a and ∪2 , ∩2 contain additional relations linking amplitude a and ∩1 . Analogous equations can be derived with the same pas- ∩1 . Analogous equations can be derived with the same pas- the σ parameter to ϕ, ν and T ∗ , as it will be shown the σ parameter to ϕ, ν and T ∗ , as it will be shown sages for the other three cases, as summarized in Table 3: sages for the other three cases, as summarized in Table 3: now. now. in the central column the descriptive equation of the signal in the central column the descriptive equation of the signal (analogous to Eq. 15) is indicated, while in the right column, (analogous to Eq. 15) is indicated, while in the right column, A.2 Amplitude the ﬁnal equations (analogous to Eq. 18) are reported. the ﬁnal equations (analogous to Eq. 18) are reported. A.2 Amplitude The possibility for the peak of being located in the ﬁrst or in The possibility for the peak of being located in the ﬁrst or in A.3 Solution of the joint equations A.3 Solution of the joint equations the second half of the period, and of being a maximum or a the second half of the period, and of being a maximum or a
92.
of the cases if L = 8 (a), in 81% 0.3 0.3 Signal Am Signal Am Up to this point, only phase and0.35 if L = 10 (b), depending on its frequency of the input must be satisﬁed:0.35 condition on signal amplitude that the signalϕhave been considered in the model. Symbols ∪1 , ∩1 , phase 0.4 directly follows is a > 3/2σ . Equation 18 models the case 0.4Amplitude ∪2 , ∩2 contain additional relations linking amplitude a and 0.45 the σ parameter to ϕ, ν and T ∗ ,0.5as it will be shown ∩1 . Analogous equations can be derived with the same pas- 0.45 sages for the other0.5three cases, as summarized in Table 3: now. −3 −2 −1 0 Phase φ (rads) in the central column−3 descriptive equation1 of the signal 1 2 3 the −2 −1 0 Phase φ (rads) 2 3 a (analogous to Eq. 15) is indicated, while in the right column, b A.2 Amplitude the ﬁnal equations (analogous to Eq. 18) are reported. Artiﬁcial Sinusoid Amplitude:10σ. A.3 Solution ofsubstitution of Eq. 1 into Eq. 15 we obtain: After direct the joint equations The possibility for the peak of being located in the ﬁrst or in 0.2 Touch the second half of the period, and of being a maximum or a a − a sin(ϕ) > σ L. Ascari 0.15 minimum, originates four cases: for the sake of brevity, we The set of equations to be solved is the combination of(16) the describe completely one of them, namely the case in which a − a sin(2π νT ∗ + ϕ) > corresponding ones in Table 3, equations in Table 2 with the 3σ 0.1 the input tactile signal has the shape shown in Fig. 15, i.e. a as summarized in Table 4. and then Introduction amplitude (⋅51σ) maximum in 0 < t ◦ < T ∗ /2. 0.05 Visualizing results requires some attention, as solutions a−σ From the deﬁnition given in Sect. 5.1 an oscillation is aresin(ϕ) < a in a multidimensional space: a, σ , ϕ, ν, duration of(17) the The tactile 0 contained in the period if: period T ∗νT ∗ + ϕ) < a−3σ number of considered frames. sin(2π , expressed by L, the a system −0.05 To better understand the role of each parameter, let us analyze f (t ◦ ) − f (0) > σ Deﬁning y = σ −a , x = 3σa , ψ = sinusoids at ν = −a ∗ a simpliﬁed case, by considering two pure 2π νT , the ﬁnal a Modelling −0.1 (15) f (t ◦ ) − f (T ∗ ) > 3σ 20 Hz andfor= 28 1 is and a period of L = 8 frames sampled equation ν case Hz, obtained: Validation −0.15 sin(ϕ) < −y Biol Cybern (18) Conclusions −0.2 0 0.005 0.01 0.015 0.02 sin(ψ) < −x 123 and Future time (s) Fig. 16 White color indicates ν=20 Hz , L=8 . φ coverage at a=10σ:order for Eq. 18 to have at least . φ coverage at a=10σ: 81% In 50% ν=28 Hz , L=8 one solution, the condi- Options solutions of the equation system tions: Fig. 17 Comparison between to sinusoidal inputs having the same fre- of Table 4. A sinusoid with 0.05 0.05 References quency but different phase. The dashed signal does not match the requi- amplitude a = 10σ (roughly 0.1 −y > −1 ≡ y < 1 0.1 Signal Amplitude a (⋅ 51 σ) rements to be considered an oscillation Signal Amplitude a (⋅ 51 σ) corresponding to 0.2 in the y (19) axis of the pictures, given the 0.15 −x > −1 ≡ x < 1 0.15 0.2 0.2 current choice of σ = 5/255) is must be satisﬁed: 0.25 condition on signal amplitude that the recognizedthis point, only phase and 0.25 Up to as oscillation in 50% frequency of the input signalcases ifbeen 8 (a), in 81% in the model. Symbols ∪1 , ∩1 , of the have L = considered 0.3 directly follows is a > 3/2σ . Equation 18 models the case 0.3 ∪2L ∩210 (b), depending on its if , = contain additional relations linking amplitude a and 0.35 ∩1 . Analogous equations can be derived with the same pas- 0.35 phase the σ ϕ parameter to ϕ, ν and T ∗ , 0.4 it will be shown as sages for the other 0.4 three cases, as summarized in Table 3: now. 0.45 in the central column the descriptive equation of the signal 0.45 0.5 (analogous to Eq. 15) is indicated, while in the right column, 0.5 −3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3 A.2 Amplitude Phase φ (rads) the ﬁnal equations (analogous to Eq.Phase φ (rads) 18) are reported. a b The possibility for the peak of being located in the ﬁrst or in A.3 Solution of the joint equations the second half of the period, and of being a maximum or a
93.
sin(ϕ) < a Up to this point, only phase and frequency of the input must be satisﬁed: the condition on signal amplitude thatL=5 ν, an oscillation is rec (17) signal have beenϕ) < a−3σ in the model. Symbols ∪1 , ∩1 , sin(2π νT ∗ + considered L=6 directly follows is a > 3/2σ . Equation 18 models the case 80 samples acquired every Detection Probability(%) a L=7 ∪2 , ∩2 contain additional relations linking amplitude a and ∩1 . Analogous equations can be derived with the same pas- detect with a probabilit L=8 Amplitude σ −a σ parameter a , 3σ −a equation for case 1 is obtained: now. ψ = 2π νT ∗ , the ﬁnal theDeﬁning y = to ϕ,x ν=anda T,∗ , as it will be shown sages for the other three cases, as summarized in Table 3: 60 L=9 in the central column the descriptive equation of the signal L=10 having minimum ampl L = 10, the cut-off freq sin(ϕ) < −y (analogous to Eq. 15) is indicated, while in the right column, to 30 Hz. For higher am A.2 Amplitude (18) the ﬁnal equations (analogous to Eq. 18) are reported. 40 lowers. sin(ψ) < −x02 To better clarify the The possibilityEq. 18 to have atbeing one solution, the condi- In order for for the peak of least located in the ﬁrst or in A.3 Solution of the joint equations L, Fig. Artiﬁcial a 19 contains 20 Touch function for a = 10σ a re- tions: the second half of the period, and of being a maximum or a ui- minimum, originates four cases: for the sake of brevity, we The set of equations to be solved is the combination of the for several values of L. −y > −1 ≡ y < 1 L. Ascari describe completely one of them, namely the case in which (19) equations in Table 2 with the corresponding ones in Table 3, 0 10 ms when L = 4 to the input −1 ≡ x < 1 has the shape shown in Fig. 15, i.e. a −x > tactile signal 0 10 20 as summarized in Table 4. 30 40 50 60 70 80 corresponding to samp Signal Frequency (Hz) maximum in 0 < ◦ < T ∗ /2. must be satisﬁed:t the condition on signal amplitude that Visualizing results requires some attention, as solutions case.Introduction ut directly follows is a > given. in Sect. 5.1 an oscillation is From the deﬁnition 3/2σ Equation 18 models the case are in a Detection probability space: a, = , ϕ, andduration of the Fig. 19 multidimensional plot for a σ 10σ ν, several values of∩1 , The tactile Acknowledgments The a contained in the period if: ∩1 . Analogous equations can be derived with the same pas- period T ∗values of L cause lowernumberfrequency of the frames. L. Higher , expressed by L, the cut-off of considered ﬁler and nd longer latency Manfredi, for the precious system sages for the other three cases, as summarized in Table 3: To better understand the role of each parameter, let us analyzewn f (t ◦ ) − f (0) > σ and the fruitful discussions. in the central column the descriptive equation of the signal a simpliﬁed case, by considering two pure sinusoids at ν = on in Modelling of the the framework (15) f (t ◦ ) − f Eq. > is ∗ (analogous to(T )15)3σ indicated, while in the right column, 20 Hz and ν = 28 Hz, and a period of L = 8 frames sampled project, supported by Toyota at 400 Hz (st = 2.5 ms). Figure 16 shows a projection of the Mr. Hiromichi Yanagihara a Validation the ﬁnal equations (analogous to Eq. 18) are reported. Motor Europe for the man solution space in the two cases on the [ϕ, a] plane: signals 123 support. The authors are ﬁ with phase and amplitude corresponding to a white area are Conclusions for their helpful comments. in A.3 Solution of the joint equations recognized as oscillations by the ﬁlter, whilst black areas and Futurera indicate waves non classiﬁed as oscillations. Biol Cybern Optionswe The set of equations to be solved is the combination of the Fig. 18 Detection Probability Figure 17 exempliﬁes the concept showing two sinusoidsch equations in Table 2 with the corresponding ones in Table 3, References References (solid and dashed lines) with equal frequency and different.a functions for L = 6 Table as summarized in (a) and 4. L = 10 (b). Darker regions initial phase: ϕsolid = 0.5 rad, ϕdashed = 2 rad over a period. Visualizing results requires some attention, as solutions mean lower detection Ascari L, Ziegenmeyer M, C In the ﬁrst one an oscillation is recognized, while the dashed R, Dario P (2006) Can is are in a multidimensional space: a, σ , ϕ, ν, duration of the probability. Values in the colorbar ∗ , in percentage line (ϕ = 2rad) is correctly discarded, no matter how large terrain roughness? pla period T areexpressed by L, the number of considered frames. rations. In: Prooceedin its amplitude is. To better understand the role of each parameter, let us analyze Space Technologies fo With reference to Fig. 16, as the phase ϕ was supposed to a simpliﬁed case, by considering two pure sinusoids at ν = ESTEC, Noordwijk, T 5) be a uniformly distributed random variable, the cumulative Ascari L, Corradi P, Bec 20 Hz and ν = 28 Hz, and a period of L = 8 frames sampled length of a horizontal white segment (corresponding to a and ﬂexible optoelec certain amplitude a) over the whole phase interval 2π can be J Micromech Mic 0960-1317/17/11/016 123 regarded as the probability of a signal having that amplitude ArticleID=4A9A98E0 a to be recognized as containing an oscillation in the current Asuni G, Teti G, Laschi C period. On the base of this observation, plots like the ones in inspired sensory-moto
94.
L = 10 (b). Darker regions mean lower detection probability. Values in the colorbar are in percentageAmplitude Artiﬁcial Latency lat at sampling Touch Detection/Frequency plot for a=10 σ. frequency of 400Hz as Fig. 18 can be obtained, in which the gray level indicates the L. Ascari 100 probability that in a signal of given amplitude a and frequency ν, an oscillation our case. in is recognized in the current period of L L=4 L=5 Introduction 80 L=6 samples acquired every st seconds. LL·= 6, the ﬁlter can lat = If st = 10ms Detection Probability(%) L=7 The tactile L=8 detect with a probability of 100% all oscillations in signals L=9 having minimum amplitude (a = 3/2σ ) = ν4 50 Hz. If when L and > system 60 L=10 L = 10, the cut-off frequency at minimum amplitude lowers to 30 Hz. For higher amplitudes the cut-off frequency slightly Modelling 40 lowers. lat = L · st = 25ms Validation To better clarify the importance of the correct choice of 20 when L = 10 L, Fig. 19 contains a section of the detection probability Conclusions function for a = 10σ as a function of the signal frequency for several values of L. The latency lat = L · st ranges from and Future 0 10 ms when L = 4 to 25 ms when L = 10 if st = 2.5 ms, Options 0 10 20 30 40 50 60 70 80 corresponding to sampling frequency of 400 Hz as in our Signal Frequency (Hz) case. References Fig. 19 Detection probability plot for a = 10σ and several values of L. Higher values of L cause lower cut-off frequency of the ﬁler and Acknowledgments The authors express their gratitude to Mr. Luigi longer latency Manfredi, for the precious help in setting up the biomechatronic hand and the fruitful discussions. The work described in this paper was carried on in the framework of the BIOMETH (Biomimetic Touch and Sight) project, supported by Toyota Motor Corporation. A special thank goes to at 400 Hz (st = 2.5 ms). Figure 16 shows a projection of the Mr. Hiromichi Yanagihara and Mr. Jonas Ambeck-Madsen from Toyota Motor Europe for the many fruitful discussions and the continuous solution space in the two cases on the [ϕ, a] plane: signals support. The authors are ﬁnally grateful to the anonymous reviewers, with phase and amplitude corresponding to a white area are for their helpful comments. recognized as oscillations by the ﬁlter, whilst black areas indicate waves non classiﬁed as oscillations. Figure 17 exempliﬁes the concept showing two sinusoids (solid and dashed lines) with equal frequency and different References initial phase: ϕsolid = 0.5 rad, ϕdashed = 2 rad over a period. Ascari L, Ziegenmeyer M, Corradi P, Gaßmann B, Zoellner M, Dillmann In the ﬁrst one an oscillation is recognized, while the dashed R, Dario P (2006) Can statistics help walking robots in assessing
95.
Outline Artiﬁcial Touch1 Introduction L. Ascari Touch in Robotics Touch in Prosthetics - Commercial SoA Introduction Approach The tactile system The pick and lift task Modelling Bioinspiration Validation Conclusions2 The tactile system and Future Options Hardware References Software3 Modelling4 Validation5 Conclusions and Future Options
96.
Validation on polymeric array Artiﬁcial Touch Biol Cybern Biol Cybern Biol Cybern Tactile input and oscillation detected (row:21, col:24). Tactile input and oscillation detected (row:21, col:24). Power Spectral Density Estimate (row 21, col 7)7) Power Spectral Density Estimate (row 21, col L. Ascari 250 250 5050 Stimulus Stimulus Segment 1 1 Segment Event Event Segment 2 2 Segment 4040 200 Power/frequency (dB/Hz) 200 Power/frequency (dB/Hz) 3030 Introduction 150 150 2020 The tactile 100 100 1010 system Fig. 9 Aluminum-made ﬁnger mock-up covered with a ﬂexible array of QTC sensors 5050 00 −10 −10 0 Modelling 0 1 2 1 3 2 4 3 5 4 5 Comb electrodes were patterned on a ﬂexible substrate of 0 0 Frequency (Hz) 1240 1240 1260 1260 1280 1280 1300 1300 1320 1320 1340 1340 Frequency (Hz) LF9150R Pyralux (DuPont, USA) material consisting of a 127-mm thick Kapton sheet with a 35mm layer of copper, Fig. 1111Dashed line tactile stimulus, asas provided by the experimenter Fig. 1313Power spectral densities estimate ofof“segment 1”1”and Fig. Dashed line tactile stimulus, provided by the experimenter Fig. Power spectral densities estimate “segment and Validation ﬁnger pushing oscillatory on the sensor array (intensity plot over time “segment 2”2” sections (with reference to Fig. 12) by means of a lithographic process. QTC sensors were ﬁxed ﬁnger pushing oscillatory on the sensor array (intensity plot over time “segment sections (with reference to Fig. 12) as recorded by the CNN chip), in correspondence of the pixel in row to the surface by means of UV curable adhesive Electro-Lite as21 and column 24 (see chip), in correspondenceoutput (corresponding recorded by the CNN of the pixel in row (see Fig. 10); solid line ﬁlter Fig. 10); solid line ELC-4481 on the perimeter. A protective rubber layer was 21pixel column 24 on the output image) ﬁlter output (corresponding and brightness pixel brightness on the output image) Conclusions wrapped around the sensors; Fig. 9 shows the array structure and the aluminum ﬁnger mock-up. Each sensor edge is 3-mm 6.2 MEMS sensors: grasp task 6.2 MEMS sensors: grasp task and Future long. ock-up covered with a ﬂexible array The sensors were polarized so as to generate voltage Tactile input and oscillation detected in in position (row 21, col 7) 8080 Tactile input and oscillation detected position (row 21, col 7) This experiment, differently form the previous one, involved This experiment, differently form the previous one, involved Options Stimulus signals proportional to the pressure applied; the nine signals Stimulus Event a real time task, such asas grasping and lifting several objects, a real time task, such grasping and lifting several objects, were fed into the electro-optical converter presented in Ascari 7070 Event thus requiring much higher frame rates. Given the bandwidth thus requiring much higher frame rates. Given the bandwidthterned on a ﬂexible substrate of et al. (2007), so that tactile images were produced and pro- 6060 limitations ofof the current HW system in image transfer illus- limitations the current HW system in image transfer illus- References USA) material consisting of a jected onto the ACE16K processor. trated above, all images were acquired and processed inside trated above, all images were acquired and processed inside Intensity 5050 Intensity with a 35mm layer of copper, The acquisition rate on the ACE16K processor, limited by the Bi-I camera, and thus non visible outside; this limiting the Bi-I camera, and thus non visible outside; this limitingrocess. QTC sensors were ﬁxed Fig. 10 Sequence of four tactile input frames (on the left half ), with factor, besides making the tuning of the ﬁlters’ parameters for transmission delay of the images over the network, was set 40 the corresponding output image generated by the ACE16K processor factor, besides making the tuning of the ﬁlters’ parameters for 40V curable adhesive Electro-Lite around 10 Hz and a modiﬁcation of the deﬁnition of oscil- when looking for oscillations, on the right real time tasks more difﬁcult, should be removed inin the next real time tasks more difﬁcult, should be removed the next A protective rubber layer was lation was introduced, to restrict the sensitivity of the ﬁlter 3030 releases ofof the HW platform. Results of the experiments can releases the HW platform. Results of the experiments can Fig. 9 shows the array structure only to macroscopic changes in the stimuli (for visualization be evaluated only inin terms of success or failure of the task. be evaluated only terms of success or failure of the task. 2020k-up. Each sensor edge is 3-mm purposes): an oscillation is deﬁned here as the sequence of The pick and lift experiments were performed using the The pick and lift experiments were performed using the two opposite variations having minimum amplitudes of 3σ 1010 robotic arm, the biomechatronic hand and the tactile sensory ted on the periphery of the central area, indicating that the robotic arm, the biomechatronic hand and the tactile sensoryzed so as to generate voltage and 4σ or vice versa (in contrast with the sequence 3σ -σ or intensity of the light600 500 emitted from the sensor was decreasing. system described inin Sect. 3, and consisted in picking and 700 800 900 system described Sect. 3, and consisted in picking and 500 600 700 800 900 essure applied; the nine signals vice versa adopted in Sect. 5.1). σ was set to 9 gray levels lifting three different objects using a standard pinched grasp: A close look to a single pixel helps in understanding the lifting three different objects using a standard pinched grasp: Frames Framescal converter presented in Ascari (out of 255 coding the whole input dynamics of the ACE16K behaviour of the ﬁlter: Fig. 11 contains a plot of the light a plastic bottle, a soft sponge sphere, and a piece ofof Japanese a plastic bottle, a soft sponge sphere, and a piece Japanese mages were produced and pro- Fig. 12Dashed line intensity plot over time the pixel in in row 21 Fig. intensity online intensity plot over time ofof theinstants at which soft tofu. 12 Dashed pixel row 21 soft tofu. processor). one pixel, together with the time and column 7 (see Fig. 10); solid line ﬁlter output highlights detectedcessor. Tactile stimuli were provided by the experimenter ﬁn- and column 7 (see were10); solid line calledoutput aroundwhile frames oscillations Fig. detected. Oscillations highlights detected oscillations. Frames before 560 are ﬁlter “segment 1”, frames 1280 These objects were chosen toto make the proposed approach These objects were chosen make the proposed approach oscillations. Frames before 560 are called “segment 1”, while frames ACE16K processor, limited by facing with different tactile situations: the plastic bottle has ger, pressing on the array in various ways: continuously, between 820 and 900 belong to “segment 2” ﬁlter, given the more facing with different tactile situations: the plastic bottle has are correctly not recognized by the Fig. 10 Sequence of four tactile input frames (on the left half ), with between 820 and 900 belong to “segment 2” ages over the network, was set the corresponding output image generated by the ACE16K processorto slowly and rapidly oscillatory. This solution was preferred stringent requirements in terms of minimum amplitude set- a a smooth surface, a little deformable; the sponge sphere smooth surface, a little deformable; the sponge sphere ation of the deﬁnition of oscil- when looking formechanical on the right because it offered more a controlled oscillations, stimulation up for this experiment (see above). is is much softer and deformable, and has a rugged surface; much softer and deformable, and has a rugged surface; trict the sensitivity of the ﬁlter realistic situations. The ﬁlter is intrinsically stochastic (see Appendix A for a the Japanese tofu is is extremely slippery, soft, and smooth. the Japanese tofu extremely slippery, soft, and smooth. in the stimuli (for visualization A tactile image sequence of 4 frames is shown in Fig. 10. frames 540 and 560 (let usus call these frames segmentexample. Moreover, its fragility is is extremely pronounced, as an off- frames 540 and 560 (let as shown frames following1), the complete description), call these by the segment 1), the Moreover, its fragility extremely pronounced, as an off-deﬁned here as the sequence of The area on the ACE16K processor interested by these signals ﬁlter seems toto fail between frames 820 andframe 1320 should, line preliminary experiment showed: one ofof the ﬁngers of ﬁlter seems fail between frames 820 and 900 (segment 2). The amplitude of the oscillation around 900 (segment 2). line preliminary experiment showed: one the ﬁngers of ng minimum amplitudes of 3σ ted on the periphery of the centrallong. On the left side are the The power spectral densitybyestimate of the two signalcouple the antropomorphic robotic hand was instrumented with a a is a square whose edge is 29 pixel area, indicating that the at a ﬁrst sight, be detected estimate of thethis case, the seg- The power spectral density the ﬁlter: in two signal seg- the antropomorphic robotic hand was instrumented with rast with the sequence 3σ -σ or intensity of the lightright side of each sensor was decreasing. ments (shown inin Fig.of thereveals corresponds behind that: six-components load cell (ATI NANO 17 F/T, Apex, NC, stimuli, while the emitted from the picture contains the cor- ments (shown phase) 13) reveals the reasons behinddetection six-components load cell (ATI NANO 17 F/T, Apex, NC, (amplitude, Fig. 13) signal the reasons to a that:5.1). σ was set to 9 gray levels A close look to a of the CNN when searching for the event segment 1 1 hasless than stronger component at “higher”period USA), covered with a protective layer identical toto that cove- responding output single pixel helps in understanding the segment has a a much 100% component at “higher” fre- probability much stronger (see Fig. 18). A longer fre- USA), covered with a protective layer identical that cove- input dynamics of the ACE16K behaviour of the ﬁlter: Fig. 11in the third and fourth frames quency (around been necessary in this case,whose main contri- ring the sensors, and a a piece of tofu was grasped at several oscillation: non black areas contains a plot of the light quency (around 1.3 Hz than segment 2,2, although introducing ring the sensors, and piece of tofu was grasped at several would have 1.3 Hz ) ) than segment whose main contri- intensity on one pixel, together with the time instants at which indicate detected oscillations; it is worth noting that the four bution is at 0.5Hz. absent in words, segment 2 is below the latency, virtually In other this example. levels of grasping force; an increase in the grasping force of
97.
Validation on MEMS sensors and grasp task Artiﬁcial Touch Biol Cybern Sampling rate of 400 L. Ascari and adaptability of the object and the rougher surface, the Hz time-based stability condition even on a small portion of the ﬁngers area proved to be enough to start lifting; reaction Introduction time from stimulus application to motor command, always Max number of sensors: less than 20ms, allowed the system to successfully oppose The tactile to essays to pull the ball from the closed hand, producing a system 16384 very “natural-feeling” behaviour. Modelling Tofu. With this object, space-based stability check before lif- Parallel processing: ting proved to be much more effective than the time-based Validation mechanisms. The extremely slippery and smooth surface semantic features made the recognition of oscillations and vibrations extre- mely difﬁcult (successful grasping of tofu is a very challen- Conclusions and Future taught us). extraction ging task also for a human hand, as preliminary experiments Options 7 Conclusion Incipient slip detection: References successful. The use of cellular nonlinear networks (CNN) in real-time robotic grasp control with massive sensory input has been Final DEMO: soft tofu presented for the ﬁrst time, to the authors’ knowledge; while employed to model the visual sensory system (Roska et al. 1993), their use in the tactile domain is just beginning, as grasping (movie) reported in a very recent work (Kis et al. 2006) where shear Fig. 14 Pictures taken from the grasping experiments with a plastic and torque information is extracted from 2 × 2 array of bottle (a), a piece of soft Japanese tofu (b), and a soft sponge ball (c). In the third picture the tactile modules can be seen around the distal silicon-made sensors. Working with 30% of We presented a complete system for reactive real-time phalanges of the index and thumb ﬁngers, as well as the optical ﬁbres safe grasp of unknown objects, based on the detection of carrying the optically coded signals to the ACE16K CNN processor sensors spatial–temporal events, the core of the software platform being a topological analog ﬁlter, designed and implemented During the grasp experiment, the task controller hosted in in a CNN processor. The proposed ﬁlter can be completely
98.
Outline Artiﬁcial Touch1 Introduction L. Ascari Touch in Robotics Touch in Prosthetics - Commercial SoA Introduction Approach The tactile system The pick and lift task Modelling Bioinspiration Validation Conclusions2 The tactile system and Future Options Hardware References Software3 Modelling4 Validation5 Conclusions and Future Options
99.
Biological vs artiﬁcial tactile system Biol Cybern Artiﬁcial Touch Table 1 Parallelism between the biological and artiﬁcial tactile systems at sensorimotor, tactile features, and sensors levels L. Ascari Human system Artiﬁcial system Sensorimotor scheme Event-based strategy; Event-based strategy; Introduction parallel coordination mechanism; stability check; The tactile Tactile features RF-based; sudden force RF-based; detection of system change detection; oscillations and vibrations Sensors Four kinds of mechanoreceptors with One kind of sensors and CNN-based computation to mimic the main Modelling different spatial and temporal characteristics; spatial and temporal responses of the mechanoreceptors Validation Conclusions ting tactile system, housing data collection from the per- investigated and implemented in robotic systems using a bio- and Future ipheral modules, as well as the computational HW and mimetic computing platform such as CNNs. Options Main achievements SW units, was hosted in a box with external dimensions In addition, the presented hardware/software platform of 20 × 10 × 6 cm and showed a power absorption of a could be considered as a biorobotic tool allowing both to References adaptive minimum force event based grasp controller few Ws). investigate on strategies for solving complex sensory-motor tasks, and to validate neuroscientiﬁc models requiring the presence of many sensors and biomimetic processing, like parallel analog processing Taking the anatomy and physiology of the human tactile the one, for instance, recently proposed by Johansson and system as a reference and guiding model in the design of this Birznieks (2004), hypothesizing that the very ﬁrst spike in Robust artiﬁcial system has helped to approach the scenarios and ensambles of human skin afferents may encode complex satisfy these requirements. In particular, Table 1 summarizes mechanical events such as the direction of force on the ﬁn- Universal the main aspects we took inspiration from in the biological gertip or the local shape at the ﬁngertip-object interface. tactile system and how they were implemented in the propo- In conclusion, it is our opinion that this approach could sed approach, at various levels. help to ﬁll the gap between tactile sensing and tactile per- The performed experiments showed that all the computed ception, that is recognized to be still a bottleneck to the next features are important for the task to be completed, but with challenges for robot manipulation in human environments different weights, depending on the particular object being (Kemp et al. (2007)).
100.
Artiﬁcial Touch L. Ascari Introduction The tactile system Modelling Validation Conclusions and Future Options ReferencesThank you for your attention! luca.ascari@henesis.eu
101.
References I Artiﬁcial Touch [1] L Ascari et al. “A miniaturized and ﬂexible optoelectronic L. Ascari sensing system for tactile skin”. In: Journal of Introduction Micromechanics and Microengineering 17.11 (11/2007), The tactile system pp. 2288–2298. issn: 0960-1317. doi: Modelling 10.1088/0960-1317/17/11/016. url: Validation http://ejournals.ebsco.com/direct.asp? Conclusions ArticleID=4A9A98E0B7D16F0C429C. and Future Options [2] L. Ascari et al. “Bio-inspired grasp control in a robotic References hand with massive sensorial input”. In: Biological Cybernetics 100.2 (2009), p. 109. doi: 10.1007/s00422-008-0279-0. [3] J. Ayers et al. Neurotechnology for biomimetic robots. MIT Press, 2002.
102.
References II [4] K.C. Catania and J.H. Kaas. “Somatosensory Fovea in Artiﬁcial Touch the Star-Nosed Mole: Behavioral Use of the Star in L. Ascari Relation to Innervation Patterns and Cortical Introduction Representation”. In: THE JOURNAL OF The tactile COMPARATIVE NEUROLOGY 387 (1997), system pp. 215–233. Modelling Validation [5] L.O. Chua and T. Roska. Cellular Neural Networks and Conclusions Visual Computing: Foundations and Applications. and Future Options Cambridge University Press, 2002. References [6] R.G.E. Clement et al. “Bionic prosthetic hands: A review of present technology and future aspirations”. In: The Surgeon 9.6 (12/2011), pp. 336–340. issn: 1479-666X. doi: 10.1016/j.surge.2011.06.001. url: http://www.sciencedirect.com/science/article/ pii/S1479666X11000904.
103.
References III Artiﬁcial Touch [7] E.G.M. Holweg et al. “Slip detection by tactile sensors: L. Ascari algorithms and experimental results”. In: Robotics and Introduction Automation, 1996. Proceedings., 1996 IEEE International The tactile Conference on. Vol. 4. 1996, 3234–3239 vol.4. system Modelling [8] R. D. Howe. “Tactile sensing and control of robotic Validation manipulation”. In: Journal of Advanced Robotics 8 Conclusions and Future (1994), pp. 245–261. Options [9] RS Johansson and G. Westling. “Roles of glabrous skin References receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects”. In: Experimental Brain Research 56.3 (1984), pp. 550–564.
104.
References IV Artiﬁcial Touch L. Ascari Introduction The tactile system[10] WorldRobotics. World Robotics 2006. International Modelling Federation of Robotics, Statistical Department, 2006. Validation url: http://www.worldrobotics-online.org/. Conclusions and Future Options References
Be the first to comment