1. Build a human-worn fabric tactile sensor array sleeve:
• To study incidental contact with the environment and
sensory feedback
• To understand the reaction to obstacles in the
environment
• To learn the strategy of search and retrieval of buried
objects in granular media
[www.nasa.gov]
Motivation
Objectives
• Develop robot capability for:
• Clean chemical & nuclear waste sites
• Retrieve buried unexploded ordnance
• Improve the robot’s understanding of its environment
• Teach by demonstration
Visualization
Fabric Tactile Sensor Array Sleeve for Human Use in Granular Media
Zhiyuan Yao, David Baum, Veronica Santos
Department of Mechanical and Aerospace Engineering, University of California‐Los Angeles
Little research has been conducted on haptic (touch-based) search and retrieval of buried objects. Using human strategies for inspiration, robots could be programmed to semi-autonomously
perform search and retrieval tasks for applications such as explosive ordnance disposal. The goal of this project was to develop a human-worn, flexible, sensorized fabric sleeve that would
enable the study of contact forces, whole arm movements, and strategies during search and retrieval experiments. Embedded with 24 taxels (tactile pixels) and a 9-DOF inertial measurement
unit, the functionally integrated sleeve is able to measure contact force and location, and arm kinematics and orientation with respect to gravity. These sensor data can be used to characterize
human strategies that can be used as inspiration for robot feedback control algorithms and “learning from demonstration” approaches.
Abstract
Curve Fitting
Original - 24 curves
(Mean Error: 0.4051N )
Categorized - 5 groups
(Mean Error: 0.4318N)
• The hysteresis of the group of taxels is observed in the
figure, and the sample points of loading and unloading
process are separated by the fit polynomial.
• Masses are loaded and unloaded on the testbed in
specific sequence to provide stable force. Corresponding
digital inputs from Arduino are collected in real time.
Calibration Result
• Larger forces are more distinguishable than smaller ones.
Shifted by Stretch
• The sensitivity of the taxel increases when it is stretched
10%Stretch
• Tactile sensor array sleeve was tested on a human forearm
with (left) and without (right) granular media.
• Visualizers are designed and implemented in real time:
• Color-coded Box: Contact force on every taxel
• IMU: Orientation of the forearm
Results & Discussion
• Contact force in granular media, i.e. cottonwood balls, is
relatively small. Future work is need to increase the signal-
to-noise ratio.
• Sensitivity of the taxel is improved when it is stretched. In
our experiment, the taxel was stretched by 10%, while in
real usage, the deformation is usually less than 2%.
• The algorithm for visualizing the subject-fixed Cartesian
coordinate system is based on infinitesimal analysis so
that it is immune from local magnetic field. But drift need
to be taken care of in the future.
Conclusion & Future Work
• In this project, a human-worn, flexible, sensorized fabric
sleeve that consists of 24 taxels and a 9-DOF inertial
measurement unit was built. Visualizer of both contact
force and orientation were designed.
• Future work includes the development of wireless data
transmission, IRB-approved experiments with human
subjects, and sensor fusion algorithm of IMU.
Reference Acknowledgements
The project is supported in part by the Office of Naval Research (ONR) under
Grant #N00014-16-1-2468. Sincere gratitude to Cross-Disciplinary Scholars in
Science and Technology (CSST) for providing me an opportunity to do my
internship. Many appreciation to Prof. Veronica Santos for her instruction. I also
thank the members in Biomechatronics Lab, especially Jimmy Penaloza, Kenneth
Gutierrez, Jimmy Wu, Shengxin Jia, for their instructive help.
[1] Tactile Sensing over Articulated Joints with Stretchable Sensors, Tapomayukh
Bhattacharjee, Advait Jain, Sarvagya Vaish, Marc D. Killpack, and Charles C. Kemp,
World Haptics Conference (The 5th Joint EuroHaptics Conference and IEEE
Haptics Symposium), 2013.
[2] Adafruit 9-DOF IMU Breakout – adafruit learning system, Kevin Townsend, 2015
[3] Tactile Sensing—From Humans to Humanoids, Ravinder S. Dahiya, Giorgio Metta,
Maurizio Valle, and Giulio Sandini, IEEE Transactions on Robotics, 2010
[4] Reaching in Clutter With Whole-Arm Tactile Sensing, Advait Jain, Marc D Killpack,
Aaron Edsinger and Charles C Kemp, The International Journal of Robotics
Research, 2013
• 𝑽 𝒂 = 𝑽𝒊𝒏
𝟏
𝟏+
𝑹 𝒕𝒂𝒙𝒆𝒍
𝑹 𝒓𝒆𝒇
• 𝑽 𝒂 : Voltage read by
Arduino
• 𝑽𝒊𝒏: Voltage input
• 𝑹 𝒓𝒆𝒇: Constant resistor
• 𝑹 𝒕𝒂𝒙𝒆𝒍: Variable resistor
𝑮𝑵𝑫
(Arduino Ground)
𝑽 𝒂
(Digital Input)
𝑅 𝑟𝑒𝑓 = 100Ω
𝑅𝑡𝑎𝑥𝑒𝑙
𝑉𝑖𝑛=5V
R
Before
pressed
After
pressed
Wire
Cover Layer
Conductive Layer
Resistive Layer
Conductive Layer
Cover Layer 2cm*2cm
Wire
1.5cm*1.5cm
1cm*1cm
Fabric Tactile Sensor
Anterior
Medial
Lateral
Posterior
Medial
• Four key regions on the forearm were defined, namely the
anterior, the posterior, the medial and the lateral.
• The sleeve consists of three parts:
• Armband, on which the 9-
DOF IMU and the Arduino
board are attached
• Four Columns of Taxels
• Wristband
Taxel Column
Wristband
IMU
Armband
Sleeve Design
• Testbed for normal force:
• A single taxel was fixed to a 6-DOF load cell (ATI
Nano17)
• Cottonwood ball used to simulate granular media
• Masses used in calibration: 50g, 100g, 200g, 500g, 1kg
• 24 taxels, 80 trials / taxel, 5 seconds / trial
Taxel Calibration Setup
Cottonwood
Ball
Calibrated
Masses
Taxel
6-DOF Load
Cell
Deformable
Platform
• The resistance of the piezo-resistive sensor decreases when
force applied on it increases.
• The sensor has three types of stretchable fabric: cover
fabric, conductive fabric, and resistive fabric (Eeonyx)
Voltage Divider
IMU
• Accelerometer – Acceleration (3-DOF)
• Gyroscope – Angular Speed (3-DOF)
• Magnetometer – Orientation (3-DOF)
• Using the transformations below,
visualizations of rigid body
orientation are robust to errors
induced by local magnetic fields.
O
x
y
z
1x
1y
1z
1x
1y
1z






𝐿 𝑦 𝜑 =
cos 𝜑 0 − sin 𝜑
0 1 0
sin 𝜑 0 cos 𝜑
𝐿 𝑧′1
𝜀 =
cos 𝜀 sin 𝜀 0
− sin 𝜀 cos 𝜀 0
0 0 1
𝐿 𝑥1
𝛾 =
1 0 0
0 cos 𝛾 sin 𝛾
0 − sin 𝛾 cos 𝛾
𝑳 𝜸, 𝜺, 𝝋 = 𝑳 𝒙 𝟏
𝜸 𝑳 𝒛′ 𝟏
𝜺 𝑳 𝒚 𝝋
𝑂𝑥𝑦𝑧
𝑂 𝑦|𝜑
𝑂𝑥1 𝑦𝑧′1
𝑂𝑥′1 𝑦𝑧′1
𝑂 𝑧′1|𝜀
𝑂𝑥1 𝑦′1 𝑧′1
𝑂𝑥′1 𝑦′1 𝑧′1
𝑂 𝑥1|𝛾
𝑂𝑥1 𝑦1 𝑦1
cos 𝜀 cos 𝜑 − sin 𝜀 cos 𝜑 cos 𝛾 + sin 𝜑 sin 𝛾 sin 𝜀 cos 𝜑 sin 𝛾 + sin 𝜑 cos 𝛾
sin 𝜀 cos 𝜀 cos 𝛾 − cos 𝜀 sin 𝛾
− cos 𝜀 sin 𝜑 sin 𝜀 sin 𝜑 cos 𝛾 + cos 𝜑 sin 𝛾 − sin 𝜀 sin 𝜑 cos 𝛾 + cos 𝜑 sin 𝛾
• All characteristics of 24 taxels were tested and the
relationships between force and digital output were fit
into third-order polynomials.
• To reduce computational expense, 24 third-order
polynomials were categorized into five groups by their
curve shapes.
• By using only 5 third-order polynomials, mean error
increases by 8% (0.4051N to 0.4318N), but runtime per
iteration decreases by 13% (17ms to 15ms).