Stress has become a significant mental health problem of the 21st century. The number of people suffering from stress is increasing rapidly. Thus, easy-to-use, inexpensive, and accurate biomarkers are needed to detect stress during its inception. Early detection of stress-related diseases allows people to access healthcare services. This thesis focuses on the development of stress stimuli and the detection of stress induced by these stimuli. Identifying brain regions affected while exposing the subject to these stressful stimuli has also been done. Three different stimuli, viz. videos, gamified application, and a game, are investigated to study their effect as stress induction stimuli. To this end, in this thesis, a system is proposed to classify participants into stressed and non-stressed categories using machine learning, deep learning, and statistical techniques. The statistical significance between stressed and non-stressed was found using Higuchi Fractal Dimensions (HFD) feature extracted from EEG. This feature also helped identify the brain’s most affected region due to stress. Another outcome of this thesis is the extra annotation of the ground truth which further helps to validate the participant’s experience under the influence of stressful stimuli. This annotation was performed by evaluating participant performance under time pressure. In addition, a technique based on in-game analytics is presented to complement the betterment of self-reported data. Further, another dimension utilizing signatures from WiFi Media Access Control (MAC) layer traffic is presented to detect stress indicators in a device-agnostic way.

1. Development of Stress Induction and
Detection System to Study its Effect on Brain
Ph.D. Thesis Defense of
Nishtha Phutela
On Wednesday August 24,2022
Advised by: Dr. Devanjali Relan (BMU, Gurugram), Prof. Goldie Gabrani (VIPS,
New Delhi) and Prof. Ponnurangam Kumaraguru (IIIT, Hyderabad)
1

2. Motivation
➢ Increased stress result in issue with health care and
work-life balance
➢ Stress effects decision-making ability and memory
➢ Rise in stress - related disorders such as cardiovascular
disease, anxiety, sleep related problems
➢ Timely detection of stress related symptoms through
non-invasive technology for better stress management
2

3. AIM Explore and develop empirical methods
to understand the manifestation of stress
3

4. Introduction
4
[1] Carneiro D, Novais P, Augusto JC, Payne N. New methods for stress assessment and monitoring at the workplace.
IEEE Transactions on Affective Computing. 2017 Apr 28;10(2):237-54.
Fig 1: Methods to Detect Stress [1]

5. Stress Induction
● Laboratory setting
● Out of laboratory setting
5

6. Stress Detection Methods
➢ Subjective - self reports ( questionnaire and
interviews)
➢ Objective - wearable and non-wearable sensors
➢ Wearable sensor-based data from physiological
signals
6

7. Research
Objectives
● To design and use stress elicitation
material
● To create experimental setup and
collect stress related data
● To explore and propose techniques
to identify stress markers
● To classify whether a person is
stressed or not stressed, using
multiple techniques
7

8. Significant Contributions
● Acquired EEG signals from participants while watching stressful videos
● Designed a game-based stimulus (CWMT) to induce stress
● Acquired EEG signals upon exposure to CWMT
● Proposed a system to check existence of significant differences between
stressed and non-stressed groups
● Designed and developed instrumented version of the educational game
Unlock Me to:
○ record human-game interactions and validate player experience.
○ analyze the difficulty in game progression and improve learning outcomes
● Created an experimental setup and collected the data set to distinctly identify
different stress related human activities using MAC layer signatures.
8

9. Next..
● Device utilized for experiments
● Video stimulus based
● Game based stimuli – CWMT
● Statistical analysis on EEG signals
● In-game analytics approach – Case Study on Unlock Me
● Device-agnostic approach
9

10. Device Description
Mmm
10
Fig 2: Non-invasive electrodes in Muse headband and 10-20 system of electrode placement
(Source: [2])

11. Stress
classification
using video stimuli
1. Objective - Study effect of stressful
videos on human stress levels
2. Motivation - Videos have capability
to elicit a person’s emotion.
3. Methodology -
1. Stimulus: Videos
2. Participants: 20
3. Data acquired: EEG and
questionnaires (5-point rating scale)
4. Technique: Classification using
machine learning
4. Result - Classification accuracy of
95.65%
14

12. 15
Workflow for stress
detection using video
stimuli

13. Stress categorization using
Higuchi Fractal Dimension
17

14. Stress categorization using Higuchi Fractal
Dimension
1. Objective :
1. To investigate existence of significant difference between stressed and non-stressed
groups of participants
2. Which brain region gets impacted during stress.
2. Motivation :
1. A marker would be helpful for the early diagnosis of stress.
2. Availability of ground truth for labeling subject-stimulus interactions
3. Method :
1. Participants: 32
2. Stimulus: Color Word and Memory Test (CWMT)
3. Ground Truth: Game score, Questionnaire
4. Technique: Statistical analysis using HFD as the feature
18

15. Experimental design for data collection
● L1 : congruent level
● L2, L3, L4 : in-congruent levels with
varying difficulty
● L5 : in-congruent level + memory test
(multi tasking)
● L2 and L3 : low difficulty
● L4 and L5 : high difficulty
19
Fig 3: Experimental design using CWMT

16. 20
Methodology for
statistical analysis

17. Process to extract HFD features from EEG
21
Fig 4. Extraction of features from EEG while performing CWMT

18. Analysis
● Analysis I : To identify significant EEG region and frequency bands impacted
by stress
● Analysis II :
○ To validate results of Analysis I
○ Participants divided into LS and HS groups based on performance in CWMT
● Analysis III : To identify hemispheric differences in the frontal region during
stress
22

19. Stress categorization using Higuchi Fractal
Dimension
Results
a. Analysis I : Beta and Alpha frequencies from the AF8 region of the
brain are affected during stress.
b. Analysis II : Beta waves from the AF8 region are a characteristic
indicator of stress.
c. Analysis III : Significant difference between HFD value in left and
right part of brain during stress.
23

20. Results
Analysis I
24
Table 1: To identify significant EEG regions and frequencies while 32 participants perform tasks of varying
difficulty. Freq_Epos_N is frequency from certain electrode position, N is the number of participants

21. Results
Analysis II
25
Table 2: To validate results from Analysis I - participants subdivided into Low Score (LS) and High Score
(HS)

22. Results
Analysis III
26
Table 3: Significant difference found between left (AF7) and right (AF8) part of brain during stress

23. Stimuli design
considerations and
in-game analytics
Aim
○ To improve the annotation in
participant-stimulus
interaction
○ Design effective and
engaging stimuli
28

24. Stimuli design considerations and in-game analytics
1. Objective: Validate the usability of game-based stimuli through
game analytics
2. Motivation:
a. Interface is usable and operable to reduce inherent stress
due to complicated and poorly designed stimuli.
b. Playing behavior gives extra annotation
c. Moving towards multimodal approaches for human stress
detection
3. Method:
a. Participants: 141
b. Stimulus: Game
c. Ground Truth: Questionnaire, usability metrics
d. Technique: Game analytics
29

25. Effectiveness of stimulus design
• Focus areas
• Usability
• User experience
• User-centric design

26. Metrics of
MEEGA+
validated using in-
game analytics

27. Results
36
Fig 5: Usability of the game measured through various parameters

28. Utilization of WiFi sniffer traffic to detect human
activities
37

29. Utilization of WiFi sniffer traffic to detect human
activities
1. Objective- Monitoring human activities for health care
2. Motivation
a. Cost-effective, Device agnostic, easy to deploy solution, does not require
wearing sensors
b. Continuous stress detection
3. Method
a. Data: 15+ hours worth of WiFi MAC-layer traffic
b. Device: WiFi sniffer
c. Technique: Feature selection, Classification using machine learning
38

30. Proposed Solution
39

31. Data Pre-processing
40

32. Simple Activities
41
Most significant
feature - subtype

33. Awake vs Sleep
Most significant
feature - Time
Difference
between
consecutive
frames.
42

34. Limitations and Future Scope
➢ Measuring chronic stress response in real-life
➢ Better annotation of ground truth
➢ Continuous stress monitoring
➢ Generating more data to employ deep learning approaches
➢ Device agnostic techniques to complement wearable and non-
wearable devices
43

35. Key Takeaways
• Collecting behavioural data
• More comprehensive view of mental illness
• Beyond one-time assessment
• Users preferer device-agnostic methods
• Measuring real-life stress indicators is challenging
44

36. Future Scope
1. Combining self reports, WiFi, and sensors collectively to infer stress and related
mental health issues
2. Using statistical and machine learning approaches
3. Using HCI approach to formulate an application to motivate people to sleep well
because stress causes degrade in quality of sleep
45

37. Research Outcomes
1. Phutela, N.,et al (2022). Unlock Me: A Real-World Driven Smartphone Game to Stimulate
COVID-19 Awareness. International journal of human-computer studies, 102818.
2. Phutela, N., et al (2021, November). EEG Based Stress Classification in Response to Stress
Stimulus. In International Conference on Artificial Intelligence and Speech Technology (pp.
354-362). Springer, Cham.
3. Phutela, N., et al. (2022). Stress Classification Using Brain Signals Based on LSTM
Network. Computational Intelligence and Neuroscience, 2022.
4. Phutela, N. (2021, October). Measuring Stress Appraisal Through Game Based Digital
Biomarkers. In Extended Abstracts of the 2021 Annual Symposium on Computer-Human
Interaction in Play (pp. 403-404).
5. Grover, H., Jaisinghani, D., Phutela, N., et al (2022, January). ML-Based Device-Agnostic
Human Activity Detection with WiFi Sniffer Traffic. In 2022 14th International Conference
on COMmunication Systems & NETworkS (COMSNETS) (pp. 72-77). IEEE.
52

38. THANK YOU
54

39. Game parameters
58