[CHI '18 Paper] BioFidget: Biofeedback for Respiration Training Using an Augmented Fidget Spinner (with script)

BioFidget: Biofeedback for Respiration Training
Using an Augmented Fidget Spinner
Rong-Hao Liang
Bin Yu
Mengru Xue
Jun Hu
Loe M. G. Feijs
Department of Industrial Design
Eindhoven University of Technology, the Netherlands
Hi, this is Rong-Hao. We are from
TU Eindhoven Industrial Design.
I would like to share an interesting
project with you.
It’s called BioFidget:
Biofeedback for respiration training
using an augmented fidget spinner.

This is a project about stress
management.

What were your stressful moments?

Ｗhen we’re confronted with challenges in our everyday life, we feel stressed.
Stress can be a good thing that improves our performances.
But, a long-term stress can lead serious health problem.
Therefore, we need to manage stress.

]
Lehrer, P. M., Vaschillo, E., and Vaschillo,
B. Resonant frequency biofeedback training to increase cardiac variability: Rationale and manual for training. Applied
psychophysiology and biofeedback 25, 3 (2000), 177–191.
RespirationTraining
is Clinical-Proven in Stress Reduction,
Here we introduce a clinical-
proven method in stress
management.
And you probably knew it. It’s
respiration training.

Slow and Steady Deep Breathing, 6 cycles / min
RespirationTraining
In respiration training, what you need
to do is to take slow and steady deep
breathing in 6 cycles per minute. After
3-5 minute, it will relax your central
nerve system and you will feel much
more relieved.
Well, what’s the problem then?

Slow and Steady Deep Breathing, 6 cycles / min
RespirationTraining
but People Usually Drops Out.
The problem is, people usually drops
out this process.
Honestly, it’s a boring boring exercise.
It’s neither competitive nor rewarding.
So, when there is a distraction,
people just let go.

Stretch Sensor for Breath Sensing
ImprovingEngagement
Respiration Training with Biofeedback
To engage people in respiration training, we can provide biofeedback.
Biofeedback is a therapy that has been applied for a long time.
The therapist just put a screen in front of the patient.

Respiration
Heart Rate Variability (HRV)
(The variation in the Inter-beat Interval)
The screen shows two curves.
The blue curve is the respiration level.
The violet curve is heart-rate variability, HRV,
which is the variation in the inter-beat intervals.

Respiration
Heart Rate Variability (HRV)
(The variation in the Inter-beat Interval)
A proper respiration training may synchronize
these two curves, because the deep breathing
regulates the heart rate.

ImprovingEngagement
Respiration Training with Biofeedback
of Respiration and Heart Rate Variability (HRV) Information
During the respiration training, the patient tries to control their
breathing to match the respiration curve to the HRV curve.

Sharma, N., and Gedeon, T. Objective measures, sensors and computational techniques for stress recognition and classification: A survey.
Computer methods and programs in biomedicine 108, 3 (2012), 1287–1301.
HeartRateVariability(HRV)
is a Primary Measure of StressAfter proper training, the
HRV will be increased.

Table 5 – Empirical ranking of primary measures for
measuring stress.
Rank Primary measure
1 HRV
2 GSR
3 EEG
4 PD
5 Voice
6 Eye gaze
7 Facial expression
8 BP
Sharma, N., and Gedeon, T. Objective measures, sensors and computational techniques for stress recognition and classification: A survey.
Computer methods and programs in biomedicine 108, 3 (2012), 1287–1301.
HeartRateVariability(HRV)
is a Primary Measure of StressThis phenomenon is a clear
signal of stress reduction.

HR via Music [Yokoyama 2002]SQUID [Farjadian et al. 2013]BreathTray [Moraveji et al. 2012]
ImprovingEngagement
Providing User-Friendly Biofeedback,
Previous work tried to improve
the experiences by providing
user-friendly biofeedback in
visual, haptic, and audio
channels.

HR via Music [Yokoyama 2002]SQUID [Farjadian et al. 2013]
Living Surface [Yu et al. 2016]
BreathTray [Moraveji et al. 2012]
InnerGarden [Roo et al. 2017]
ImprovingEngagement
Some further use shape
changing interfaces and virtual
reality to increase the
immersion.

SQUID [Farjadian et al. 2013]
Living Surface [Yu et al. 2016]
BreathTray [Moraveji et al. 2012]
InnerGarden [Roo et al. 2017]
ImprovingEngagement
but Requires Bio-sensors to be Worn.
HR via Music [Yokoyama 2002]
However, these systems all
require the users to wear both
heart rate and respiration sensors,
before they start the training.

ImprovingEngagement
Photoplethysmograph (PPG)
for Heart RateVariability Sensing
Stretch Sensor for
Respiration Sensing
It leads us to ask an
intriguing question.

ImprovingEngagement
Photoplethysmograph (PPG)
for Heart RateVariability Sensing
Stretch Sensor for
Respiration Sensing
In everyday life, would you put these sensors on, when you are stressed?

FidgetSpinner
One thing we do when we’re stressed is fidgeting.
Last year, Fidget Spinners went viral.
In the spinning, the visual and tactile feedback are very soothing even addictive.
It’s healing, that’s why people love it. Nonetheless, it has no biosensors, so it
does not provide biofeedback based on the biosignals.

FidgetSpinner
… “fidget spinners and other self-regulatory
occupational therapy toys have yet to be subjected to
rigorous scientific research. Thus, their alleged
benefits remain scientifically unfounded.”
Schecter, Rachel A., et al.
"Fidget spinners: Purported benefits, adverse effects and accepted alternatives."
Current opinion in pediatrics 29.5 (2017): 616-618.
Importantly, regarding stress management, the clinical
effects of a spinner is still scientifically baseless.

do not sense the user’s physiological data
[Wensveen et al. 2012] Mind Sphere, PhilipsFidgetWidget [Karlesky et al. 2014] Relax! Pen [Bruns 2010]
FidgetDevicesforSelf-Regulation
Same as fidget spinners, previous work uses tangible
user interfaces as fidgets to help the users in doing
self-regulation. They also did not provide
biofeedback based on stress-related biosignal.

HandheldDevicesforBio-Sensing
do not support rich, engaging interaction
StressEraserHandheld Spirometer Finger-based ECG
Some handheld physiological sensors, we call it
biosensors in short, do not require users to wear,
but they try to avoid user interactions that may
affect their technical validity of bio-sensing.

HandheldDevicesforBio-Sensing
do not support rich, engaging interaction
StressEraserHandheld Spirometer Finger-based ECG
We see a gap between
these two research
domains, and that's what
we are gonna bridge.

BioFidget:AugmentedFidgetSpinner
without Requiring Additional Sensors to be Worn
That Senses HRV and Respiration and Provides Biofeedback
In this project, we reinvent
the fidget spinner.

Therefore, a fidget spinner can provide the
feedback for respiration training, without
requiring the user to put additional sensor
on the body. We call it BioFidget.

RedesigningFidgetSpinner
To Meet Both Technical Validity and Playfulness
Playfulness
Technical
Validity&
The design of BioFidget needs to meet the
technical Validity of biosensing, and also preserve
the original playfulness of a fidget spinner.

Pad
Pad
Wing
First, we dis-assemble a fidget Spinner into two
parts: pads and wing.

Pad
Pad
Wing
PPG Sensor
Micro-controller
Then, we put a ppg heart-rate sensor on the top
pad, wire connected it to the micro-controller
attached at the bottom pad.

PPG Sensor
Micro-controller
Hall Sensor
Magnets
Then, we change the wing into a light-weight one
with three magnets,

PPG Sensor
Micro-controller
Hall Sensor
Magnets
N
1 north and 2 south, and mount an analog Hall-sensor
at the bottom pad to pick up the spinning movement.

Hall Sensor
Magnets
N
NeoPixel Ring
Then, we put an RGB+White LED ring,

NeoPixel Ring
Accelerometer
and an additional accelerometer as a mode switch.

By combining all together, we have the hardware design.

And the actual device looks like this.

SignalProcessing
Signal Pipeline
PPG Sensor
Hall Sensor
DisplayMicro-controller
(500Hz Sampling)
Heart RateVariability
Respiration
Action
PPG Sensor Signals
Hall Sensor Signals
PC
On signal processing, the micro-controller
samples the sensors in 500 Hz, extracts the
HRV and Respiration from the data stream,
and show them directly to the user.

HRVSensing
Algorithm
PPG Sensor
Micro-controller
(500Hz Sampling)
Heart RateVariabilitySignals
time
(diastolic point)
Dn-1
Sn
Dn
dn
TnTn-1
(systolic point)
(dicrotic notch)
Sn-1 Sn+1
Dn+1
dn+1
Tn+1
voltage
dn-1
t(Bn-1) t(Bn) t(Bn+1)
IBInIBIn-1
Robin P Smith, Jerome Argod, Jean-Louis Pepin, and Patrick A Levy. 1999.
Pulse transit time: an appraisal of potential clinical applications. Thorax 54, 5 (1999), 452–457.
On HRV sensing, an algorithm is
implemented to extract the HRV.

PPG Sensor
Micro-controller
(500Hz Sampling)
Signals
PPG Sensor
Micro-controller
(500Hz Sampling)
Heart RateVariabilitySignals
HRVSensing
Results
For example, in this 30 seconds,
the heartbeat and the Inter-beat
intervals were correctly extracted.

SignalProcessing
Algorithm (1/2)
Hall Sensor
Micro-controller
(500Hz Sampling)
Respiration
Action
Signals
voltage
time
rectiﬁed
north
south
Hall Sensor
Magnets
N raw
On respiration sensing, the hall sensor
tracks the only north magnet in 500 Hz to
pick up the rotation of the wing.

SignalProcessing
Algorithm (2/2)
Hall Sensor
Micro-controller
(500Hz Sampling)
Respiration
Action
Signals
Hall Sensor
Micro-controller
(500Hz Sampling)
Revolution Speed
Signals
Therefore, revolution speed can be
obtained from the rotation counts

SignalProcessing
Algorithm (2/2)
Hall Sensor
Micro-controller
(500Hz Sampling)
Respiration
Action
Signals
Hall Sensor
Micro-controller
(500Hz Sampling)
Revolution Speed
Signals
Revolution Acceleration
And acceleration can be obtained from the
differentiation of speed.
The acceleration shows the force exerting to the
wing, and the speed indicates the types of actions.

Results
Hall Sensor
Micro-controller
(500Hz Sampling)
Respiration
Action
Signals
Revolution
Speed
Revolution
Acceleration
Revolution
Speed
Revolution
Acceleration
SignalProcessingSpecifically, a user can play with a spinner in two ways:
flicking its wing or blowing on its wing.
The speed pattern of the two types of actions are different,

Activity Recognition
Hall Sensor
Micro-controller
(500Hz Sampling)
Respiration
Action
Signals
Revolution
Speed
Revolution
Acceleration
Revolution
Speed
Revolution
Acceleration
SignalProcessing
W
d
Vmax
so they can be distinguished using the
time to the maximum speed with a precise
segmentation by the acceleration.

For respiration training
BioFeedback
Signals
Guidance and Feedback
Now let's talk about the visual output.
The display shows the breathing timing and
feeds the breathing quality back to the user.

Breathing
GuidanceThe timing of “breathe in” is shown in a white breathing light,

Breathing
Guidanceand the timing of “breathe out” is shown in a colorful breathing light.

Exhalation quality
BioFeedback
Respiration
Display
The quality of exhalation correlates to revolution speed, which is
shown as a linear range of hue.
The more colorful it is, the better breathing quality it is.
The color also rotates to mimic the dynamics of the spinner.

HRV Status
BioFeedback
Display
Inter-Beat Interval
Heartbeat
Before and after the training, the user perceives their HRV
information through the blinking heart-beat lights and the
white bar of his last inter-beat intervals. The white bar can
display other HRV index in the same way.

PilotStudy
3-min Respiration Trainings in a static context
Flicks the spinner while exhaling Blows on the spinner by exhaling
Pilot study tests the technical validity of
BioFidget in a static context.
An example user performed respiration
training with his both arms fixed on the table.
He performed two sessions of 3-minute
respiration in two different ways.

PilotStudy
One way is Flick: flicking the spinner while exhalation.
Breathe in,

PilotStudy
and Breathe out.

PilotStudy
Another way is Blow: blowing on the wing of spinner by exhalation.
Breathe in,

PPG sensor #2
(baseline)
PilotStudy
We also fixed a secondary ppg sensor
on his index finger of the non-
dominant hand as a baseline.

PPG sensor #2
(baseline)
Accelerometer
PilotStudy
And use the accelerometer data for
motion analysis.

Results
0 60 120 0 60 120 (s)
revolution speed
revolution acceleration
acceleration
interbeat interval
blood volume pulse
interbeat interval
blood volume pulse
sensorsonﬁdgetspinnerbaseline
HallAPPGPPG
Accelerometer
0 60 120 0 60 120 (s)
revolution speed
acceleration
interbeat interval
blood volume pulse
interbeat interval
blood volume pulse
HallAPPGPPG
(s)
PilotStudyHere’s the sensor stream results.

Accelerometer
0 60 120 0 60 120 (s)
revolution speed
acceleration
interbeat interval
blood volume pulse
interbeat interval
blood volume pulse
HallAPPGPPG
0 60 120 0 60 120 (s)
revolution speed
acceleration
interbeat interval
blood volume pulse
interbeat interval
blood volume pulse
HallAPPGPPG
(s)
2X 2X
Results
PilotStudyThe flicking and blowing actions were
all reliably captured.

Accelerometer
0 60 120 0 60 120 (s)
revolution speed
acceleration
interbeat interval
blood volume pulse
interbeat interval
blood volume pulse
HallAPPGPPG
0 60 120 0 60 120 (s)
revolution speed
acceleration
interbeat interval
blood volume pulse
interbeat interval
blood volume pulse
HallAPPGPPG
(s)
2X 2X
Results
Corrupted
PilotStudy
The HRV data is also reliably captured in the Blow session, but the
data was “corrupted” in the Flick session. The accelerator data
shows that impacts of finger flicking seems to be the reason.
We need to deal with this.

FunctionalExtension
Adding a clip to stabilize HRV sensing
Without a clip With a clip
A simple solution is adding a clip to stabilize
HRV sensing when the user is fidgeting.

UserStudy
PPG sensor #2
(baseline)
respiration training in a personally comfortable way
Without a clip
With a clip
We test this solution in a casual setting, participant
did the training in a personally comfortable way.
Participants were divided into two groups.
One uses a biofidget without a clip, and another one
uses a biofiget with a clip.
Both groups use a secondary PPG as a baseline.

Without a clip
With a clip
PPG sensor #2
(baseline)
UserStudy
Each group was evenly divided into two subgroups.
We collected four sessions of 3-minutes HRV data.
One subgroup first has a 3-minute normal session,
in which they do nothing but browsing internet.

Without a clip
With a clip
PPG sensor #2
(baseline)
UserStudy
Then, a 3-minute blow session. Then,
another 3-minute normal session, and
finally a 3-minute flick session.

Without a clip
With a clip
PPG sensor #2
(baseline)
UserStudy
Another subgroup reversed the order
of blow and flick sessions for
counterbalancing. In the end, each
participant had an individual interview.

RMSSDLF/HF
1. Respiration Training was Eﬀective in
Both Flick and Blow Sessions.
Stress Reduced Heart-rate regulation improved
QuantitativeResultsWe first use the baseline PPG to
evaluate the effectiveness of
respiration training, and found that the
respiration training was effective in
both flick and blow sessions.

RMSSDLF/HF
1. Respiration Training was Eﬀective in
Both Flick and Blow Sessions.
Stress Reduced Heart-rate regulation improved
QuantitativeResults
Before After
The comparisons between the two HRV indexes of the first
and the third sessions, show a significant stress reduction
and better heart rate regulation after the training.

17 18 19 20 2122 23 24 2526 27 28 2930 31 32
with a clip (BioFidget)
17 18 1920 21 22 2324 25 26 2728 29 30 31 32
0%
25%
50%
75%
100%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 1516
baseline (PPG #2)
without a clip (BioFidget)
0%
25%
50%
75%
100%
1 2 3 4 5 6 7 8 9 10 11 12 1314 15 16
baseline (PPG #2)
Participant ID
QuantitativeResults
2. The Clip Stabilized the HRV Sensing
and Enabled Blowing Input
PPG sensor #2
(baseline)
Now let's check the HRV collected by the two
BioFidgets. The clip significantly improved the HRV
sensing quality in the both Flick and Blow sessions.

17 18 19 20 2122 23 24 2526 27 28 2930 31 32
17 18 1920 21 22 2324 25 26 2728 29 30 31 32
0%
25%
50%
75%
100%
1 2 3 4 5 6 7 8 9 10 11 12 13 14 1516
baseline (PPG #2)
0%
25%
50%
75%
100%
1 2 3 4 5 6 7 8 9 10 11 12 1314 15 16
baseline (PPG #2)
Participant ID
QuantitativeResults
2. The Clip Stabilized the HRV Sensing
and Enabled Blowing Input
PPG sensor #2
(baseline)
In the Blow session, the HRV collected by
BioFidget with a clip is even similarly reliable
to the one collected by the Baseline PPG,
showing that the blowing action does not
affect the validity of HRV collection.

Flicking
Detected
QuantitativeResults
Strategy of HRV data collection
The results suggest an HRV data collection strategy. When a
flicking is detected, we temporary ignore the rest of the HRV
data, until 3 consecutive heartbeats have been detected.

Flicking
Detected
Blowing
Detected
QuantitativeResults
Strategy of HRV data collection
When a blowing is detected, we just continue tracking so
that most of valid HRV data can be collected.

0%
25%
50%
75%
100%
500 400 300 200
0%
25%
50%
75%
100%
500 400 300 200
window size (ms)
accuracy
detectionrate
0%
25%
50%
75%
100%
500 400 300 200
300
270
240
210
180
150
120
90
60
window size (ms)
0%
25%
50%
75%
100%
500 400 300 200
recall
window size (ms) window size (ms)
Recall: FlickingRecall: BlowingAction Detection RateAccuracy: Overall
recall
minimal
velocity
(rpm)
QuantitativeResults
3. Actions were Reliably Recognized
(with High Recall of Blowing Recognition)
W
d
Vmax
Vmax
Resultsofactivityrecognitionusingdasfeature
W = 500ms;Vmax ≥ 60rpm: 87.9% accuracy; 96.2% detection rate.
W W W W
The data collection strategy is supported by
reliable action recognition. Post-hoc analysis
shows that 96% of actions can be detected
with an 88% accuracy using a nearest-
neighbor classifier in a 5-fold cross-
validation, when a low speed threshold and
a half-minute window size were used.

31(outof32)participants:
ThebreathingguidancethroughthelightonBioFidgetwasclearandeasytoperceive.
20participants:
Spinningthefidgetmadethemfeelrelaxedandmoreabletofocusonthebreathingguidance.
26participants:
Thefeedbackengagedandmotivatedthemtoperformbetterinbreathingtraining.
UserFeedback
General responses
The general feedback from the interview is positive.
Participants agreed that the breathing guidance was
clear and easy to perceive, the spinner made them
feel relaxed and more focused, and the feedback
engaged and motivated them to perform better.

“IfeelimmersedintheexperiencewhenIwasstaringatthedevice”(P19)
“itmademefeelengagedandthusencouragedmetospinitfaster,”(P14,P23,P25)
“Ipreferredtoseeacolorfullightinsteadoftheredone,whichmotivatedmetoblowonit
harder.”(P30,P31)
“itwasanamusingvisualizationandalsoarewardformyperformance.”(P1).
”toreleasemystress”(P8),“clearmymind”(P4),and“makememoreconsciousaboutmy
breathing”(P9).
“Ibreathedsloweranddeeperwithitsfeedback.Ibelieveitishelpfultoadjustmybreathing,”
(P9)and“ithelpsintrainingmylungcapacity”(P1).
“Ireallylikethistangiblewaytomanipulatingthisdevice”(P1).
“Thisinnovationisbasedontherightobjectandmakeitmoreuseful”(P15).
UserFeedback
Individual responsesIndividual feedback also confirms the engagement and playfulness.
The last quote echoes our motivation:
“This innovation is based on the right object and make it more useful.”

24(16withoutaclip;8withaclip)participants:
Preferflickingtoblowingbecauseitiseffortless.
But,theclipmadetheflickingactiondifficulttoperform.
15participants:
Ittookalotofefforttoblowonthefidgetspinner.
2participants:
Itwasslightlyawkwardtomovethespinnerclosetothemouth.
UserFeedback
Negative responses
Negative feedback shows the area of improvements.
Adding a clip made the flicking action harder to perform.
Blowing on the spinner took lots of efforts.
Moving the spinner close to the mouth was also slightly awkward.

Twomorefunctionalextensions
Fan-shaped wing Additional display
Consequently, we propose another two more functional extensions:
Fan-shaped wing and additional display.

Fan-ShapedWing
To increase the sensitivity to respiration
The fan-shaped wing increases the sensitivity to the respiration.

Fan-ShapedWing
To increase the sensitivity to respiration
It enables the users to easier blow on the wing and get sensed
without moving the spinner towards the mouth.

Fan-ShapedWing
Allows for static use
The wing also enables the use on the table or a display, so the HRV
sensing would be stable even without a clip, as shown in pilot study.

AdditionalDisplay
Provide historical HRV information
adequate
respiration
training
inadequate
respiration
training
The display can provide more insightful information, such as the history of respiration training.
For instance, the differences between an adequate and an inadequate trainings can be
observed from the resulting drawings, because the respiration regulates the heart rate.

Follow-UpStudy
20
11 females
9 males
1-min
Functional extensions
Both functional extensions were tested with
20 participants from the formal study,

18(outof20)participants:
Thefan-shapedBioFidgetwaseasiertoblowonthanthebasicBioFidget.
18participants:
CantellthecorrelationsbetweentheirbreathandIBIpatternsandagreedthat
respirationtrainingcouldbehelpfulforheartrateregulation.
9participants:
Wantedtotryitagainbecausetheywantedtoimprovetheirresults.
20
11 females
9 males
1-min
Follow-UpStudy
Results
and they reported that the fan-shaped one was easier to
blow on. They understood the visualization. Some of them
even wanted to try it again to improve their results.

LimitationsandFutureWork
• Tethershouldberemovedandreplacedbyabattery.
• Optimizingthepowerconsumptionisnecessary.
• ImprovingHRVsensingusingmotion-resilientECGsensor.
• MakingVisualizationmoreInformative
• Amoresophisticatedincentivemechanismforlong-termuses.
There are still some limitations in our physical design, such as the tether,
power and sensor choices. Also, the visualization and the incentive
mechanism could be improved. We leave them as future work.

Conclusion
BioFidget: a smart fidget spinner that detects stress directly and
provides a biofeedback intervention for respiration training
Conclusion. We have presented BioFidget, a smart fidget spinner that detects
stress directly and provides a biofeedback intervention for respiration training

Conclusion
We explained the design, and tested the technical validity and playfulness.
These efforts turned a popular toy into a useful stress management tool.
Now, breathe in.

BioFidget: Biofeedback for Respiration Training
Using an Augmented Fidget Spinner
Rong-Hao Liang
Bin Yu
Mengru Xue
Jun Hu
Loe M. G. Feijs
Department of Industrial Design
Eindhoven University of Technology, the Netherlands
ProjectPage:http://tinyurl.com/BioFidget
and Breathe out. Thank you.

[CHI '18 Paper] BioFidget: Biofeedback for Respiration Training Using an Augmented Fidget Spinner (with script)

More Related Content

Similar to [CHI '18 Paper] BioFidget: Biofeedback for Respiration Training Using an Augmented Fidget Spinner (with script)

More from Rong-Hao Liang

Recently uploaded

[CHI '18 Paper] BioFidget: Biofeedback for Respiration Training Using an Augmented Fidget Spinner (with script)