The mood disease depression is quite severe. Those who suffer from depression are often unable to function normally and may even resort to suicide if their condition worsens. Clinical interviews and questionnaires are now used in all cases of depression diagnosis, although these procedures are very subjective and lack objectivity and physiological basis.

1. Presented By:
Diagnostic Criteria for Depression Based on Both Static and
Dynamic Visual Features
International Conference on
Intelligent Data Communication Technologies and Internet
of Things
(IDCIoT 2023)
Dhairya Vyas
Research Scholar
The Maharaja Sayajirao
University of Baroda
Vadodara, Gujarat, India
dhairya.vyas-
cse@msubaroda.ac.in
Darshanaben Dipakkumar Pandya
Assistant Professor,
Department of Computer Science,
Shri C. J Patel College of Computer
Studies (BCA), Sankalchand Patel
University,
Visnagar, Gujarat
ddpandya.fcs@spu.ac.in
Sheshang Degadwala
Associate professor
Sigma institute of engineering
Vadodara, Gujarat, India
sheshang13@gmail.com
Abhijeetsinh Jadeja
I/C Principal, Department of
Computer Science,
Shri C. J Patel College of
Computer
studies (BCA), Sankalchand Patel
University, Visnagar, gujarat
abhijit.highereducation@gmail.co
m

2. OUTLINE
 Motivation
 Introduction
 Contribution
 Proposed Work
 Results
 Analysis
 Conclusion
 References

3. MOTIVATION
 The mood disease depression is quite severe.
 Those who suffer from depression are often unable to function normally and may even
resort to suicide if their condition worsens.
 Clinical interviews and questionnaires are now used in all cases of depression diagnosis,
although these procedures are very subjective and lack objectivity and physiological
basis.

4.  A person's ideas, conduct, emotions, and overall sense of well-being may all be impacted by
depression, which is characterized by a poor mood and aversion to activity[1].
 Suicidal ideation and a wide spectrum of unpleasant feelings are common among those
suffering from depression. They may lose interest in humorous things and have cognitive
difficulties such as slower processing speed and memory loss or inability to focus.
 Moreover, insomnia, hypersomnia, exhaustion, lack of energy, discomfort, or resistance to
therapy are often experienced in conjunction with depression[2].
 Currently, a professional psychologist with years of expertise is needed to establish a clinical
diagnosis of depression based on observable symptoms.
 By calculating Beck Depression Inventory II (BDI-II) values from video data, we present an
objective and non-discriminatory technique for depression diagnosis in this research.
INTRODUCTION

5.  First, we use the LBP-TOP and EVLBP algorithms to extract a dynamic feature from each
frame of the movie separately.
 The LBP operator is applied to each frame, HOG features are extracted from the LBP
picture, and finally the LBP-HOG features are transformed into histogram vectors using
BOW.
 Finally, the Gradient Boosting Regression is used to the combined dynamic and static
characteristics to calculate the BDI-II. Using the AVEC 2014 depression dataset as an
example, our tests demonstrate the efficacy of our suggested method.
CONTINUE.

6. CONTRIBUTION
 There are four main contributions to this study.
1) A novel dynamic feature extraction approach is suggested. To adequately represent
the tilt motion of the object, the EVLBP[9] feature extraction approach is used here for
the first time in depression detection. In this work, we apply EVLBP with LBP-TOP to
extract face characteristics from each video frame.
2) A different method of extracting static features is suggested. After processing each
frame using LBP, HOG features are extracted from the resulting picture, and then
BOW is used to transform the features into a histogram.
3) Thirdly, it has been shown that the BDI-II estimate may be improved by combining a
static and dynamic feature extraction approach.
4) The BDI-II value is predicted using the Gradient Boosting Regression[10] technique.

7. PROPOSED WORK
Original Picture Grayscale Image Face Detection Face Alignment
LBP HOG Histogram Bag of Words

OUTPUT
LBP-TOP
EVLBP
[LBP-TOP,EVLBP]
Video Preprocessing
Feature extraction scheme

8. DESCRIPTION
 Faces were identified and cropped with the help of dlib, a machine learning library. We removed the frame if
no face was found during the cropping procedure. Then, we used the left and right eyes, nose, and mouth as
ESR landmarks to align the other areas of the face. As can be seen in Fig. 8, video preparation is a rather
straightforward procedure.
 first, we extracted dynamic information using LBP-TOP and EVLBP, and then we extracted static information
using LBP-HOG-BOW. Frame-by-frame, we retrieved LBP-TOP and EVLBP features for dynamic information
extraction, and then we concatenated those features to get the vector F1. To improve the local texture
presentation of the face region, we ran LBP operations on the picture frame by frame to extract static
information. The gradient information was then obtained by extracting the HOG features from the LBP
pictures. We employed BOW for secondary coding of LBP-HOG to minimize the dimensionality of HOG
features produced by an image, and we referred to the resulting feature as LBP-HOG-BOW or vector F2 to
avoid confusion with the original feature. Finally, we ran regression tests on F1, F2, and their combination.

9. PARAMETERS (WRITE)
2
1
1
ˆ
( )
i i
i
N
RMSE y y
N 
 

1
1
ˆ
| |
i i
i
N
M y y
N
AE

 


10. RELATION BETWEEN K AND RMSE

11. DEPRESSION RECOGNITION RESULTSVIA
SINGLE MODEL ON AVEC2014 (TEST SET)
Methods MAE RMSE
LBP-TOP 7.26 8.94
EVLBP 8.15 9.77
LBP-HOG-BOW 7.48 9.88

12. PREDICTED AND ACTUAL DEPRESSION SCALES ON
TEST SETWITH LBP-TOP+EVLBP+LBP-HOG-BOW

13. DEPRESSION RECOGNITION RESULTSVIA
MIXTURE MODELS ON AVEC2014 (TEST SET)
Methods
MAE RMSE
LBP-TOP + LBP-HOG-BOW 7.22 8.69
EVLBP + LBP-HOG-BOW 8.22 10.04
LBP-TOP + EVLBP 7.09 8.58
LBP-TOP + EVLBP + LBP-HOG-BOW 7.28 8.67

14. OVERALL COMPARISON
Methods MAE RMSE
Baseline [14] 8.76 10.76
Jan et al. [22] 8.34 10.40
Kaya et al. [27] 8.10 10.17
Our Method
7.20 8.77

15. CONCLUSION
 This research proposes a novel approach for detecting depression using face
feature extraction from video data. We created the EVLBP feature extraction
approach and integrated it with the LBP-TOP as a dynamic information descriptor
to collect motion information in the direction of face tilt. We isolated the LBP-
HOG-BOW feature and blended it with another feature to better capture the face's
static information. In the end, BDI-II estimation was accomplished with the aid of
gradient boosting regression. Experiments conducted on the AVEC 2014 dataset
proved the scheme's efficacy. Additionally, our suggested strategy outperformed
the majority of the strategies that rely just on visual cues. Meanwhile, the
experiments shown that the effectiveness of depression detection could be
enhanced by extracting information about face tilt motion, and that the
combination of dynamic features and static features was more successful than the
single feature extraction approach.

16. REFERENCES
[1] S. Iqbal, A.-N. Qureshi, and G. Mustafa, Hybridization of CNN with LBP for Classification of Melanoma Images. 71(3): pp. 4915--4939. 2022.
[2] D.-Q. Zeebaree, et al., Multi-Level Fusion in Ultrasound for Cancer Detection Based on Uniform LBP Features. 66(3): pp. 3363--3382. 2021.
[3] Y.-E. Almalki, et al., LBP–Bilateral Based Feature Fusion for Breast Cancer Diagnosis. 73(2): pp. 4103--4121.2022.
[4 ]S. D'Mello, and J. Kory, Consistent but modest: A meta-analysis on unimodal and multimodal affect detection accuracies from 30 studies. ICMI'12 -
Proceedings of the ACM International Conference on Multimodal Interaction, 2012.
[5] J. Wei, et al., Micro-expression recognition using local binary pattern from five intersecting planes. Multimedia Tools and Applications, 81(15): pp.
20643-20668. 2022.
[6] J.H. Friedman, Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5): pp. 1189-1232. 2001.
[7] J.R. Williamson, et al., Vocal biomarkers of depression based on motor incoordination. 2013.
[8] N. Cummins, et al., Diagnosis of depression by behavioural signals: a multimodal approach. 2013.
[9] M.F. Valstar, et al., AVEC 2013: the continuous audio/visual emotion and depression recognition challenge. 2013.
[10] M. Valstar, et al., AVEC 2014 - 3D dimensional affect and depression recognition challenge. AVEC 2014 - Proceedings of the 4th International
Workshop on Audio/Visual Emotion Challenge, Workshop of MM 2014: p. 3-10.2014.

17. CONTINUE.
[11] T.R. Almaev, and M.F. Valstar. Local Gabor Binary Patterns from Three Orthogonal Planes for Automatic Facial Expression Recognition. in 2013
Humaine Association Conference on Affective Computing and Intelligent Interaction. 2013.
[12] L. Wen, et al., Automated Depression Diagnosis Based on Facial Dynamic Analysis and Sparse Coding. IEEE Transactions on Information
Forensics and Security, 10(7): p. 1432-1441. 2015.
[13] V. Jain, et al., Depression Estimation Using Audiovisual Features and Fisher Vector Encoding. AVEC 2014 - Proceedings of the 4th International
Workshop on Audio/Visual Emotion Challenge, Workshop of MM 2014, 2014.
[14] N. Dalal, B. Triggs, and C. Schmid, Human Detection Using Oriented Histograms of Flow and Appearance. Vol. 3952. 428-441. 2006.
[15] J. Sánchez, T. Mensink, and J. Verbeek, Image Classification with the Fisher Vector: Theory and Practice. International Journal of Computer
Vision, 105. 2013.
[16] L. He, D. Jiang, and H. Sahli. Multimodal depression recognition with dynamic visual and audio cues. in 2015 International Conference on
Affective Computing and Intelligent Interaction (ACII). 2015.
[17] H. Meng, et al., Motion History Histograms for Human Action Recognition. p. 139-162. 2009.
[18] H. Meng, et al., Depression recognition based on dynamic facial and vocal expression features using partial least square regression. 21-30.
2013.
[19] H. Kaya, F. Çilli, and A. Salah, Ensemble CCA for Continuous Emotion Prediction. 2014.
[20] A. Jan, et al., Automatic Depression Scale Prediction using Facial Expression Dynamics and Regression. 73-80. 2014.

18. CONTINUE.
[21] L. Wu, S.C.H. Hoi, and N. Yu, Semantics-Preserving Bag-of-Words Models and Applications. Ieee Transactions on Image Processing, 19(7):
pp. 1908-1920. 2010.
[22] J.R.R. Uijlings, A.W.M. Smeulders, and R.J.H. Scha, Real-Time Visual Concept Classification. IEEE Transactions on Multimedia, 12(7): pp. 665-
681. 2010.
[23] M. Jazaery, and G. Guo, Video-Based Depression Level Analysis by Encoding Deep Spatiotemporal Features. IEEE Transactions on Affective
Computing, PP: p. 1-1. 2018.
[24] V.a.S. Kazemi, Josephine, One millisecond face alignment with an ensemble of regression trees. 2014 IEEE Conference on Computer Vision
and Pattern Recognition: pp. 1867-1874. 2014.
[25] A. Jan, et al., Artificial Intelligent System for Automatic Depression Level Analysis Through Visual and Vocal Expressions. IEEE Transactions
on Cognitive and Developmental Systems, PP: p. 1-1. 2017.
[26] M. Sidorov, and W. Minker, Emotion Recognition and Depression Diagnosis by Acoustic and Visual Features. 81-86. 2014.
[27] Y. Zhu, et al., Automated Depression Diagnosis Based on Deep Networks to Encode Facial Appearance and Dynamics. IEEE Transactions on
Affective Computing, PP: p. 1-1. 2017.