SlideShare a Scribd company logo
1 of 24
Download to read offline
JAMES COOK AUSTRALIA
INSTITUTE OF HIGHER
LEARNING
IN SINGAPORE
HEALTH DIAGNOSTIC BY ANALYSING FACE IMAGES USING
MOBILE DEVICES
Instructor : Dr. Insu Song
Student : Ho Thi Hoang
Yen
Email:
INTRODUCTION
Previously : A robust, highly accurate method for detecting
20 facial points in images of expressionless faces
BACKGROUND
A restricted Boltzmann machine (RBM) is a generative stochastic
artificial neural network that can learn a probability distribution over its
set of inputs.
Inventor : under the name Harmonium by Paul Smolensky in 1986.
Fast learning algorithms : mid-2000s by Geoffrey Hinton &
collaborators.
RBMs have found applications in dimensionality reduction,
classification, collaborative filtering, feature learning and topic
modelling.
INTRODUCTION
Track 26 feature points
different facial expressions, varying poses, or occlusion
INTRODUCTION
Other methods :
- track facial feature points independently or
- build a shape model to capture the variations of face shape or
- appearance regardless of the facial expressions and face poses
This method : capture the distinctions & variations of face shapes
due to facial expression and pose change in a UNIFIED
framework
CONTENT - MEDOTHOLOGY
1. Related work
2. FrontalRBM & PoseRBM
3. Facial feature tracking based on face shape prior
model
4. Experimental results
1. RELATED WORK
Facial feature localization:
 2 categories :
• Without shape prior models : track each facial feature
point independently and ignore the prior knowledge
about the face => sensitive with expression & pose
• With shape prior models : capture the dependence
between facial feature points by explicitly modeling the
general properties as well as the variations of facial
shape or appearance
1. RELATED WORK
Facial feature localization:
Recently methods:
• Active Shape Model (ASM) [2] and Active Appearance Model
(AAM) : linear generative models
• Facial point detection using boosted regression and graph
models : facial feature points are detected independently based
on the response of the support vector regressor.
• Gaussian Process Latent Variable model : a single Gaussian
is used for each facial component.
• Multi-State Facial Component Model of Tian and Cohn
• ….
1. RELATED WORK
Restricted Boltzmann Machines based shape prior
model:
• Deep Belief Networks(DBNs)-like model : S. Eslami, N.
Heess, and J. Winn. (2012) - a strong model of object
shape.
• Implicit mixture of Conditional Restricted Boltzmann
Machines : G. Taylor, L. Sigal, D. Fleet, and G. Hinton
(2010) - capture the human poses and motions (imRBM)
under different activities such as walking, running etc
• …
CONTENT - MEDOTHOLOGY
1. Related work
2. FrontalRBM & PoseRBM
3. Facial feature tracking based on face shape prior
model
4. Experimental results
2. FRONTAL-RBM & POSE-RBM
the locations
of facial
feature
points for
frontal face
when
subjects
show
different
facial
expressions
the
corresponding
locations of
facial feature
points for non-
frontal face
under the same
facial
expression
H1 & H2 are two
sets of hidden
nodes
FACIAL FEATURE TRACKING BASED ON FACE
SHAPE PRIOR MODEL
Gaussian assumption : estimate the prior probability by
calculating the mean vector μp and covariance matrix Σp
from the samples.
Kernel Density Function: to estimate the probability.
CONTENT - MEDOTHOLOGY
1. Related work
2. FrontalRBM & PoseRBM
3. Facial feature tracking based on face shape prior
model
4. Experimental results
RESULT
Experiments on synthetic data
FrontalRBM shows strong power as a face shape prior
model.
RESULT
Experiments on CK+ database: Error rate reduce.
RESULT
Experiments on MMI database: comparable to Facial
point detection using boosted regression and graph
models (rate error of 5.3 on 400 images ).
19 : Robust facial feature tracking under varying face pose and facial expression
(Y. Tong, Y. Wang, Z. Zhu, and Q. Ji. - Nov 2007)
RESULT
Tracking under occlusion : ISL db - happiness
RESULT
Tracking under occlusion : ISL db - suprised
RESULT
Tracking under occlusion : ASL database, sequence 1
RESULT
Tracking under occlusion :ASL database, sequence 2
CONCLUSION
• Improving The Accuracy And Robustness Of Facial Feature Tracking
Under Simultaneous Pose And Expression Variations
• 1st : A face shape prior model to capture the face shape patterns under
varying facial expressions for near-frontal face based on deep belief
networks
• 2nd : Extend the frontal face prior model by a 3-way RBM to capture face
shape patterns under simultaneous expression and pose variation.
• 3rd : Systematically combine the face prior models with image
measurements of facial feature points to perform facial feature point
tracking.
SWOT
• STRENGTH ?
 Experiments on many methods & do well comparing.
• WEAKNESS?
 The steps of the methods are not very clear.
 There is no specific correct detection rate.
• OPPORTUNITY ?
 Can be very useful for face detection related programs
• THREAT ?
 There are more than 6 basic expression.
The training data must be labelled manually.
OPINION
This paper has described a good method for face
detection under varying expression and specially with
occlusions.
It is valuable for all kinds of researches related to face,
to the system for interacting between human & computer , or
the face recognition and to FACE ANALYSIS FOR HEALTH
PURPOSE.
Facial Feature Tracking under Varying Facial Expressions and Face Poses based on Restricted Boltzmann Machines

More Related Content

What's hot

Single frontal face detection by finding dark pixel group and comparing xy
Single frontal face detection by  finding dark pixel group and  comparing xySingle frontal face detection by  finding dark pixel group and  comparing xy
Single frontal face detection by finding dark pixel group and comparing xyIAEME Publication
 
A Novel Mathematical Based Method for Generating Virtual Samples from a Front...
A Novel Mathematical Based Method for Generating Virtual Samples from a Front...A Novel Mathematical Based Method for Generating Virtual Samples from a Front...
A Novel Mathematical Based Method for Generating Virtual Samples from a Front...CSCJournals
 
2. 7698 8113-1-pb
2. 7698 8113-1-pb2. 7698 8113-1-pb
2. 7698 8113-1-pbIAESIJEECS
 
Corneal topographic reports
Corneal topographic reportsCorneal topographic reports
Corneal topographic reportsAl Amin
 
FACIAL LANDMARKING LOCALIZATION FOR EMOTION RECOGNITION USING BAYESIAN SHAPE ...
FACIAL LANDMARKING LOCALIZATION FOR EMOTION RECOGNITION USING BAYESIAN SHAPE ...FACIAL LANDMARKING LOCALIZATION FOR EMOTION RECOGNITION USING BAYESIAN SHAPE ...
FACIAL LANDMARKING LOCALIZATION FOR EMOTION RECOGNITION USING BAYESIAN SHAPE ...cscpconf
 
Facial landmarking localization for emotion recognition using bayesian shape ...
Facial landmarking localization for emotion recognition using bayesian shape ...Facial landmarking localization for emotion recognition using bayesian shape ...
Facial landmarking localization for emotion recognition using bayesian shape ...csandit
 
Recognition of Facial Expressions using Local Binary Patterns of Important Fa...
Recognition of Facial Expressions using Local Binary Patterns of Important Fa...Recognition of Facial Expressions using Local Binary Patterns of Important Fa...
Recognition of Facial Expressions using Local Binary Patterns of Important Fa...CSCJournals
 
F ACIAL E XPRESSION R ECOGNITION B ASED ON E DGE D ETECTION
F ACIAL  E XPRESSION  R ECOGNITION  B ASED ON  E DGE  D ETECTIONF ACIAL  E XPRESSION  R ECOGNITION  B ASED ON  E DGE  D ETECTION
F ACIAL E XPRESSION R ECOGNITION B ASED ON E DGE D ETECTIONIJCSES Journal
 
FACE DETECTION USING PRINCIPAL COMPONENT ANALYSIS
FACE DETECTION USING PRINCIPAL COMPONENT ANALYSISFACE DETECTION USING PRINCIPAL COMPONENT ANALYSIS
FACE DETECTION USING PRINCIPAL COMPONENT ANALYSISIAEME Publication
 
Facial Expression Recognition Based on Facial Motion Patterns
Facial Expression Recognition Based on Facial Motion PatternsFacial Expression Recognition Based on Facial Motion Patterns
Facial Expression Recognition Based on Facial Motion Patternsijeei-iaes
 

What's hot (14)

Single frontal face detection by finding dark pixel group and comparing xy
Single frontal face detection by  finding dark pixel group and  comparing xySingle frontal face detection by  finding dark pixel group and  comparing xy
Single frontal face detection by finding dark pixel group and comparing xy
 
A Novel Mathematical Based Method for Generating Virtual Samples from a Front...
A Novel Mathematical Based Method for Generating Virtual Samples from a Front...A Novel Mathematical Based Method for Generating Virtual Samples from a Front...
A Novel Mathematical Based Method for Generating Virtual Samples from a Front...
 
2. 7698 8113-1-pb
2. 7698 8113-1-pb2. 7698 8113-1-pb
2. 7698 8113-1-pb
 
Schematic eye
Schematic eye   Schematic eye
Schematic eye
 
Corneal topographic reports
Corneal topographic reportsCorneal topographic reports
Corneal topographic reports
 
Rules in refractive surgery
Rules  in refractive surgeryRules  in refractive surgery
Rules in refractive surgery
 
FACIAL LANDMARKING LOCALIZATION FOR EMOTION RECOGNITION USING BAYESIAN SHAPE ...
FACIAL LANDMARKING LOCALIZATION FOR EMOTION RECOGNITION USING BAYESIAN SHAPE ...FACIAL LANDMARKING LOCALIZATION FOR EMOTION RECOGNITION USING BAYESIAN SHAPE ...
FACIAL LANDMARKING LOCALIZATION FOR EMOTION RECOGNITION USING BAYESIAN SHAPE ...
 
Facial landmarking localization for emotion recognition using bayesian shape ...
Facial landmarking localization for emotion recognition using bayesian shape ...Facial landmarking localization for emotion recognition using bayesian shape ...
Facial landmarking localization for emotion recognition using bayesian shape ...
 
Rules in refractive surgery + cases presentation
Rules  in refractive surgery + cases presentationRules  in refractive surgery + cases presentation
Rules in refractive surgery + cases presentation
 
pxc3888681
pxc3888681pxc3888681
pxc3888681
 
Recognition of Facial Expressions using Local Binary Patterns of Important Fa...
Recognition of Facial Expressions using Local Binary Patterns of Important Fa...Recognition of Facial Expressions using Local Binary Patterns of Important Fa...
Recognition of Facial Expressions using Local Binary Patterns of Important Fa...
 
F ACIAL E XPRESSION R ECOGNITION B ASED ON E DGE D ETECTION
F ACIAL  E XPRESSION  R ECOGNITION  B ASED ON  E DGE  D ETECTIONF ACIAL  E XPRESSION  R ECOGNITION  B ASED ON  E DGE  D ETECTION
F ACIAL E XPRESSION R ECOGNITION B ASED ON E DGE D ETECTION
 
FACE DETECTION USING PRINCIPAL COMPONENT ANALYSIS
FACE DETECTION USING PRINCIPAL COMPONENT ANALYSISFACE DETECTION USING PRINCIPAL COMPONENT ANALYSIS
FACE DETECTION USING PRINCIPAL COMPONENT ANALYSIS
 
Facial Expression Recognition Based on Facial Motion Patterns
Facial Expression Recognition Based on Facial Motion PatternsFacial Expression Recognition Based on Facial Motion Patterns
Facial Expression Recognition Based on Facial Motion Patterns
 

Viewers also liked

Code로 이해하는 RNN
Code로 이해하는 RNNCode로 이해하는 RNN
Code로 이해하는 RNNSANG WON PARK
 
災害対策推進委員会の発足 2017年版
災害対策推進委員会の発足 2017年版災害対策推進委員会の発足 2017年版
災害対策推進委員会の発足 2017年版akihiko kondo
 
Hotels and tourism
Hotels and tourismHotels and tourism
Hotels and tourismHarish Lad
 
Angela Frucci Portfolio
Angela Frucci PortfolioAngela Frucci Portfolio
Angela Frucci PortfolioAngela Frucci
 
O lúdico na educação infantil
O lúdico na educação infantilO lúdico na educação infantil
O lúdico na educação infantilSimoneHelenDrumond
 
에너지 수확용,재생용 It융합기술의 개발 동향
에너지 수확용,재생용 It융합기술의 개발 동향에너지 수확용,재생용 It융합기술의 개발 동향
에너지 수확용,재생용 It융합기술의 개발 동향atelier t*h
 
Reinforcement learning v0.5
Reinforcement learning v0.5Reinforcement learning v0.5
Reinforcement learning v0.5SANG WON PARK
 
학습전이
학습전이학습전이
학습전이dcjin
 
ETICA Y DEONTOLOGIA PROFESIONAL
ETICA Y DEONTOLOGIA PROFESIONALETICA Y DEONTOLOGIA PROFESIONAL
ETICA Y DEONTOLOGIA PROFESIONALjuanclp14
 
Hadoop eco story 이해
Hadoop eco story 이해Hadoop eco story 이해
Hadoop eco story 이해SANG WON PARK
 
The history of the Waffen SS 1925-1945
The history of the Waffen SS 1925-1945The history of the Waffen SS 1925-1945
The history of the Waffen SS 1925-1945Odal Rune
 
텐서플로우 설치도 했고 튜토리얼도 봤고 기초 예제도 짜봤다면 TensorFlow KR Meetup 2016
텐서플로우 설치도 했고 튜토리얼도 봤고 기초 예제도 짜봤다면 TensorFlow KR Meetup 2016텐서플로우 설치도 했고 튜토리얼도 봤고 기초 예제도 짜봤다면 TensorFlow KR Meetup 2016
텐서플로우 설치도 했고 튜토리얼도 봤고 기초 예제도 짜봤다면 TensorFlow KR Meetup 2016Taehoon Kim
 
Teaching Students with Emojis, Emoticons, & Textspeak
Teaching Students with Emojis, Emoticons, & TextspeakTeaching Students with Emojis, Emoticons, & Textspeak
Teaching Students with Emojis, Emoticons, & TextspeakShelly Sanchez Terrell
 
UX, ethnography and possibilities: for Libraries, Museums and Archives
UX, ethnography and possibilities: for Libraries, Museums and ArchivesUX, ethnography and possibilities: for Libraries, Museums and Archives
UX, ethnography and possibilities: for Libraries, Museums and ArchivesNed Potter
 
Designing Teams for Emerging Challenges
Designing Teams for Emerging ChallengesDesigning Teams for Emerging Challenges
Designing Teams for Emerging ChallengesAaron Irizarry
 
Visual Design with Data
Visual Design with DataVisual Design with Data
Visual Design with DataSeth Familian
 
3 Things Every Sales Team Needs to Be Thinking About in 2017
3 Things Every Sales Team Needs to Be Thinking About in 20173 Things Every Sales Team Needs to Be Thinking About in 2017
3 Things Every Sales Team Needs to Be Thinking About in 2017Drift
 

Viewers also liked (20)

Code로 이해하는 RNN
Code로 이해하는 RNNCode로 이해하는 RNN
Code로 이해하는 RNN
 
災害対策推進委員会の発足 2017年版
災害対策推進委員会の発足 2017年版災害対策推進委員会の発足 2017年版
災害対策推進委員会の発足 2017年版
 
Hotels and tourism
Hotels and tourismHotels and tourism
Hotels and tourism
 
Angela Frucci Portfolio
Angela Frucci PortfolioAngela Frucci Portfolio
Angela Frucci Portfolio
 
O lúdico na educação infantil
O lúdico na educação infantilO lúdico na educação infantil
O lúdico na educação infantil
 
에너지 수확용,재생용 It융합기술의 개발 동향
에너지 수확용,재생용 It융합기술의 개발 동향에너지 수확용,재생용 It융합기술의 개발 동향
에너지 수확용,재생용 It융합기술의 개발 동향
 
Reinforcement learning v0.5
Reinforcement learning v0.5Reinforcement learning v0.5
Reinforcement learning v0.5
 
학습전이
학습전이학습전이
학습전이
 
ETICA Y DEONTOLOGIA PROFESIONAL
ETICA Y DEONTOLOGIA PROFESIONALETICA Y DEONTOLOGIA PROFESIONAL
ETICA Y DEONTOLOGIA PROFESIONAL
 
Project Presentation
Project PresentationProject Presentation
Project Presentation
 
Xml parser
Xml parser Xml parser
Xml parser
 
Hadoop eco story 이해
Hadoop eco story 이해Hadoop eco story 이해
Hadoop eco story 이해
 
The history of the Waffen SS 1925-1945
The history of the Waffen SS 1925-1945The history of the Waffen SS 1925-1945
The history of the Waffen SS 1925-1945
 
Kesehatan reproduksi remaja
Kesehatan reproduksi remajaKesehatan reproduksi remaja
Kesehatan reproduksi remaja
 
텐서플로우 설치도 했고 튜토리얼도 봤고 기초 예제도 짜봤다면 TensorFlow KR Meetup 2016
텐서플로우 설치도 했고 튜토리얼도 봤고 기초 예제도 짜봤다면 TensorFlow KR Meetup 2016텐서플로우 설치도 했고 튜토리얼도 봤고 기초 예제도 짜봤다면 TensorFlow KR Meetup 2016
텐서플로우 설치도 했고 튜토리얼도 봤고 기초 예제도 짜봤다면 TensorFlow KR Meetup 2016
 
Teaching Students with Emojis, Emoticons, & Textspeak
Teaching Students with Emojis, Emoticons, & TextspeakTeaching Students with Emojis, Emoticons, & Textspeak
Teaching Students with Emojis, Emoticons, & Textspeak
 
UX, ethnography and possibilities: for Libraries, Museums and Archives
UX, ethnography and possibilities: for Libraries, Museums and ArchivesUX, ethnography and possibilities: for Libraries, Museums and Archives
UX, ethnography and possibilities: for Libraries, Museums and Archives
 
Designing Teams for Emerging Challenges
Designing Teams for Emerging ChallengesDesigning Teams for Emerging Challenges
Designing Teams for Emerging Challenges
 
Visual Design with Data
Visual Design with DataVisual Design with Data
Visual Design with Data
 
3 Things Every Sales Team Needs to Be Thinking About in 2017
3 Things Every Sales Team Needs to Be Thinking About in 20173 Things Every Sales Team Needs to Be Thinking About in 2017
3 Things Every Sales Team Needs to Be Thinking About in 2017
 

Similar to Facial Feature Tracking under Varying Facial Expressions and Face Poses based on Restricted Boltzmann Machines

Improved Face Recognition across Poses using Fusion of Probabilistic Latent V...
Improved Face Recognition across Poses using Fusion of Probabilistic Latent V...Improved Face Recognition across Poses using Fusion of Probabilistic Latent V...
Improved Face Recognition across Poses using Fusion of Probabilistic Latent V...TELKOMNIKA JOURNAL
 
Survey on Facial Expression Analysis and Recognition
Survey on Facial Expression Analysis and RecognitionSurvey on Facial Expression Analysis and Recognition
Survey on Facial Expression Analysis and RecognitionIRJET Journal
 
Facial expression recognition based on local binary patterns final
Facial expression recognition based on local binary patterns finalFacial expression recognition based on local binary patterns final
Facial expression recognition based on local binary patterns finalahmad abdelhafeez
 
Facial expression identification by using features of salient facial landmarks
Facial expression identification by using features of salient facial landmarksFacial expression identification by using features of salient facial landmarks
Facial expression identification by using features of salient facial landmarkseSAT Journals
 
Face recognition tech1
Face recognition tech1Face recognition tech1
Face recognition tech1Ankit Gupta
 
Fiducial Point Location Algorithm for Automatic Facial Expression Recognition
Fiducial Point Location Algorithm for Automatic Facial Expression RecognitionFiducial Point Location Algorithm for Automatic Facial Expression Recognition
Fiducial Point Location Algorithm for Automatic Facial Expression Recognitionijtsrd
 
opencv_project_tech_computer vision .pptx
opencv_project_tech_computer vision .pptxopencv_project_tech_computer vision .pptx
opencv_project_tech_computer vision .pptxjibin63
 
A Review on Face Detection under Occlusion by Facial Accessories
A Review on Face Detection under Occlusion by Facial AccessoriesA Review on Face Detection under Occlusion by Facial Accessories
A Review on Face Detection under Occlusion by Facial AccessoriesIRJET Journal
 
A Study on Face Recognition Technique based on Eigenface
A Study on Face Recognition Technique based on EigenfaceA Study on Face Recognition Technique based on Eigenface
A Study on Face Recognition Technique based on Eigenfacesadique_ghitm
 
face recognition system
face recognition systemface recognition system
face recognition systemAnil Kumar
 
Face Verification Across Age Progression using Enhanced Convolution Neural Ne...
Face Verification Across Age Progression using Enhanced Convolution Neural Ne...Face Verification Across Age Progression using Enhanced Convolution Neural Ne...
Face Verification Across Age Progression using Enhanced Convolution Neural Ne...sipij
 
FACE VERIFICATION ACROSS AGE PROGRESSION USING ENHANCED CONVOLUTION NEURAL NE...
FACE VERIFICATION ACROSS AGE PROGRESSION USING ENHANCED CONVOLUTION NEURAL NE...FACE VERIFICATION ACROSS AGE PROGRESSION USING ENHANCED CONVOLUTION NEURAL NE...
FACE VERIFICATION ACROSS AGE PROGRESSION USING ENHANCED CONVOLUTION NEURAL NE...sipij
 
Deformable Facial Models and 3D Face Reconstruction Methods: A survey
Deformable Facial Models and 3D Face Reconstruction Methods: A surveyDeformable Facial Models and 3D Face Reconstruction Methods: A survey
Deformable Facial Models and 3D Face Reconstruction Methods: A surveyLakshmi Sarvani Videla
 
photo detection in personal photo collection
photo detection in personal photo collectionphoto detection in personal photo collection
photo detection in personal photo collectionsonalijagtap15
 
HVDLP : HORIZONTAL VERTICAL DIAGONAL LOCAL PATTERN BASED FACE RECOGNITION
HVDLP : HORIZONTAL VERTICAL DIAGONAL LOCAL PATTERN BASED FACE RECOGNITION HVDLP : HORIZONTAL VERTICAL DIAGONAL LOCAL PATTERN BASED FACE RECOGNITION
HVDLP : HORIZONTAL VERTICAL DIAGONAL LOCAL PATTERN BASED FACE RECOGNITION sipij
 

Similar to Facial Feature Tracking under Varying Facial Expressions and Face Poses based on Restricted Boltzmann Machines (20)

Improved Face Recognition across Poses using Fusion of Probabilistic Latent V...
Improved Face Recognition across Poses using Fusion of Probabilistic Latent V...Improved Face Recognition across Poses using Fusion of Probabilistic Latent V...
Improved Face Recognition across Poses using Fusion of Probabilistic Latent V...
 
Survey on Facial Expression Analysis and Recognition
Survey on Facial Expression Analysis and RecognitionSurvey on Facial Expression Analysis and Recognition
Survey on Facial Expression Analysis and Recognition
 
Facial expression recognition based on local binary patterns final
Facial expression recognition based on local binary patterns finalFacial expression recognition based on local binary patterns final
Facial expression recognition based on local binary patterns final
 
Facial expression identification by using features of salient facial landmarks
Facial expression identification by using features of salient facial landmarksFacial expression identification by using features of salient facial landmarks
Facial expression identification by using features of salient facial landmarks
 
Face recognition tech1
Face recognition tech1Face recognition tech1
Face recognition tech1
 
Fiducial Point Location Algorithm for Automatic Facial Expression Recognition
Fiducial Point Location Algorithm for Automatic Facial Expression RecognitionFiducial Point Location Algorithm for Automatic Facial Expression Recognition
Fiducial Point Location Algorithm for Automatic Facial Expression Recognition
 
Fl33971979
Fl33971979Fl33971979
Fl33971979
 
Fl33971979
Fl33971979Fl33971979
Fl33971979
 
50120140504002
5012014050400250120140504002
50120140504002
 
Hoip10 articulo reconocimiento facial_univ_vigo
Hoip10 articulo reconocimiento facial_univ_vigoHoip10 articulo reconocimiento facial_univ_vigo
Hoip10 articulo reconocimiento facial_univ_vigo
 
opencv_project_tech_computer vision .pptx
opencv_project_tech_computer vision .pptxopencv_project_tech_computer vision .pptx
opencv_project_tech_computer vision .pptx
 
A Review on Face Detection under Occlusion by Facial Accessories
A Review on Face Detection under Occlusion by Facial AccessoriesA Review on Face Detection under Occlusion by Facial Accessories
A Review on Face Detection under Occlusion by Facial Accessories
 
A Study on Face Recognition Technique based on Eigenface
A Study on Face Recognition Technique based on EigenfaceA Study on Face Recognition Technique based on Eigenface
A Study on Face Recognition Technique based on Eigenface
 
face recognition system
face recognition systemface recognition system
face recognition system
 
Face Verification Across Age Progression using Enhanced Convolution Neural Ne...
Face Verification Across Age Progression using Enhanced Convolution Neural Ne...Face Verification Across Age Progression using Enhanced Convolution Neural Ne...
Face Verification Across Age Progression using Enhanced Convolution Neural Ne...
 
FACE VERIFICATION ACROSS AGE PROGRESSION USING ENHANCED CONVOLUTION NEURAL NE...
FACE VERIFICATION ACROSS AGE PROGRESSION USING ENHANCED CONVOLUTION NEURAL NE...FACE VERIFICATION ACROSS AGE PROGRESSION USING ENHANCED CONVOLUTION NEURAL NE...
FACE VERIFICATION ACROSS AGE PROGRESSION USING ENHANCED CONVOLUTION NEURAL NE...
 
Deformable Facial Models and 3D Face Reconstruction Methods: A survey
Deformable Facial Models and 3D Face Reconstruction Methods: A surveyDeformable Facial Models and 3D Face Reconstruction Methods: A survey
Deformable Facial Models and 3D Face Reconstruction Methods: A survey
 
A017530114
A017530114A017530114
A017530114
 
photo detection in personal photo collection
photo detection in personal photo collectionphoto detection in personal photo collection
photo detection in personal photo collection
 
HVDLP : HORIZONTAL VERTICAL DIAGONAL LOCAL PATTERN BASED FACE RECOGNITION
HVDLP : HORIZONTAL VERTICAL DIAGONAL LOCAL PATTERN BASED FACE RECOGNITION HVDLP : HORIZONTAL VERTICAL DIAGONAL LOCAL PATTERN BASED FACE RECOGNITION
HVDLP : HORIZONTAL VERTICAL DIAGONAL LOCAL PATTERN BASED FACE RECOGNITION
 

Recently uploaded

Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 

Recently uploaded (20)

Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 

Facial Feature Tracking under Varying Facial Expressions and Face Poses based on Restricted Boltzmann Machines

  • 1. JAMES COOK AUSTRALIA INSTITUTE OF HIGHER LEARNING IN SINGAPORE HEALTH DIAGNOSTIC BY ANALYSING FACE IMAGES USING MOBILE DEVICES Instructor : Dr. Insu Song Student : Ho Thi Hoang Yen Email:
  • 2. INTRODUCTION Previously : A robust, highly accurate method for detecting 20 facial points in images of expressionless faces
  • 3. BACKGROUND A restricted Boltzmann machine (RBM) is a generative stochastic artificial neural network that can learn a probability distribution over its set of inputs. Inventor : under the name Harmonium by Paul Smolensky in 1986. Fast learning algorithms : mid-2000s by Geoffrey Hinton & collaborators. RBMs have found applications in dimensionality reduction, classification, collaborative filtering, feature learning and topic modelling.
  • 4. INTRODUCTION Track 26 feature points different facial expressions, varying poses, or occlusion
  • 5. INTRODUCTION Other methods : - track facial feature points independently or - build a shape model to capture the variations of face shape or - appearance regardless of the facial expressions and face poses This method : capture the distinctions & variations of face shapes due to facial expression and pose change in a UNIFIED framework
  • 6. CONTENT - MEDOTHOLOGY 1. Related work 2. FrontalRBM & PoseRBM 3. Facial feature tracking based on face shape prior model 4. Experimental results
  • 7. 1. RELATED WORK Facial feature localization:  2 categories : • Without shape prior models : track each facial feature point independently and ignore the prior knowledge about the face => sensitive with expression & pose • With shape prior models : capture the dependence between facial feature points by explicitly modeling the general properties as well as the variations of facial shape or appearance
  • 8. 1. RELATED WORK Facial feature localization: Recently methods: • Active Shape Model (ASM) [2] and Active Appearance Model (AAM) : linear generative models • Facial point detection using boosted regression and graph models : facial feature points are detected independently based on the response of the support vector regressor. • Gaussian Process Latent Variable model : a single Gaussian is used for each facial component. • Multi-State Facial Component Model of Tian and Cohn • ….
  • 9. 1. RELATED WORK Restricted Boltzmann Machines based shape prior model: • Deep Belief Networks(DBNs)-like model : S. Eslami, N. Heess, and J. Winn. (2012) - a strong model of object shape. • Implicit mixture of Conditional Restricted Boltzmann Machines : G. Taylor, L. Sigal, D. Fleet, and G. Hinton (2010) - capture the human poses and motions (imRBM) under different activities such as walking, running etc • …
  • 10. CONTENT - MEDOTHOLOGY 1. Related work 2. FrontalRBM & PoseRBM 3. Facial feature tracking based on face shape prior model 4. Experimental results
  • 11. 2. FRONTAL-RBM & POSE-RBM the locations of facial feature points for frontal face when subjects show different facial expressions the corresponding locations of facial feature points for non- frontal face under the same facial expression H1 & H2 are two sets of hidden nodes
  • 12. FACIAL FEATURE TRACKING BASED ON FACE SHAPE PRIOR MODEL Gaussian assumption : estimate the prior probability by calculating the mean vector μp and covariance matrix Σp from the samples. Kernel Density Function: to estimate the probability.
  • 13. CONTENT - MEDOTHOLOGY 1. Related work 2. FrontalRBM & PoseRBM 3. Facial feature tracking based on face shape prior model 4. Experimental results
  • 14. RESULT Experiments on synthetic data FrontalRBM shows strong power as a face shape prior model.
  • 15. RESULT Experiments on CK+ database: Error rate reduce.
  • 16. RESULT Experiments on MMI database: comparable to Facial point detection using boosted regression and graph models (rate error of 5.3 on 400 images ). 19 : Robust facial feature tracking under varying face pose and facial expression (Y. Tong, Y. Wang, Z. Zhu, and Q. Ji. - Nov 2007)
  • 17. RESULT Tracking under occlusion : ISL db - happiness
  • 18. RESULT Tracking under occlusion : ISL db - suprised
  • 19. RESULT Tracking under occlusion : ASL database, sequence 1
  • 20. RESULT Tracking under occlusion :ASL database, sequence 2
  • 21. CONCLUSION • Improving The Accuracy And Robustness Of Facial Feature Tracking Under Simultaneous Pose And Expression Variations • 1st : A face shape prior model to capture the face shape patterns under varying facial expressions for near-frontal face based on deep belief networks • 2nd : Extend the frontal face prior model by a 3-way RBM to capture face shape patterns under simultaneous expression and pose variation. • 3rd : Systematically combine the face prior models with image measurements of facial feature points to perform facial feature point tracking.
  • 22. SWOT • STRENGTH ?  Experiments on many methods & do well comparing. • WEAKNESS?  The steps of the methods are not very clear.  There is no specific correct detection rate. • OPPORTUNITY ?  Can be very useful for face detection related programs • THREAT ?  There are more than 6 basic expression. The training data must be labelled manually.
  • 23. OPINION This paper has described a good method for face detection under varying expression and specially with occlusions. It is valuable for all kinds of researches related to face, to the system for interacting between human & computer , or the face recognition and to FACE ANALYSIS FOR HEALTH PURPOSE.

Editor's Notes

  1. Good morning every body, Code : http://deeplearning.net/tutorial/DBN.html
  2. Previously on Literature Review, I have introduced a paper about A robust, highly accurate method for detecting 20 facial points in images of expressionless faces. Today, I’ll continue with a paper about “facial feature tracking under varying facial expressions”
  3. RBMs were initially invented under the name Harmonium by Paul Smolensky in 1986, but only rose to prominence after Geoffrey Hinton and collaborators invented fast learning algorithms for them in the mid-2000s. RBMs have found applications in : dimensionality reduction, classification,[3] collaborative filtering,[4] feature learning[5] and topic modelling.[6]
  4. This paper present a work that can effectively track facial feature points using face shape prior models that are constructed based on RBM. The facial feature tracker can track 26 facial feature points (Fig. 1 (a)) even if faces have different facial expressions, varying poses, or occlusion
  5. Unlike the previous works that track facial feature points independently or build a shape model to capture the variations of face shape or appearance regardless of the facial expressions and face poses the proposed model could capture the distinctions as well as the variations of face shapes due to facial expression and pose change in a unified framework.
  6. Methods without shape prior model track each facial feature point independently and ignore the prior knowledge about the face. As a result, they usually are sensitive to facial expression change, face poses change, occlusion etc. (etcetera) On the other hand, methods with shape prior model capture the dependence between facial feature points by explicitly modeling the general properties as well as the variations of facial shape or appearance
  7. In this paper, authors have researched on other methods in recently article. In real world situations, faces usually vary in facial expressions and poses. These natural movements make facial feature tracking even more difficult. To solve this problem, Tian and Cohn [9] propose a multi-state facial component model, where the state is selected by tracking a few control points. As a result, the accuracy of their method critically relies on how accurately and reliably the control points are tracked. Tong et al. [19] propose a model to capture the different states of facial components like mouth open and mouth closed. In addition, they project the frontal face to face with poses to handle the varying poses problem. However, during tracking, they need to dynamically and explicitly estimate the state of local components and switch between different models. In [3], Dantone and Van Gool take into account the pose variations and build sets of conditional regression forests on different poses. In [1], instead of using the parametric model, Belhumeur present methods to represent the face shape variations with non-parametric training data.
  8. Eslami et al. propose a strong model of object shape based on Boltzmann Machines. Specifically, they build a Deep Belief Networks(DBNs)-like model but with only locally shared weights in the first hidden layer to represent the shape of horse and motorbikes. The sampling results from the model look realistic and have a good generalization The mixture nature of im-RBM makes it possible to learn a single model to represent the human poses and motions under different activities such as walking, running, etc.
  9. By experiments, authors have found that there are exist patterns for human face shapes, but these patterns depend on the facial expressions. These are 6 basic facial expressions To capture these patterns, they have proposed a face shape prior model based on Deep Belief Networks which we call ForntalRBM in this paper : use two-layer DBNs to explicitly capture the face shape patterns under different facial expressions Part 2 : the 3-way RBM model captures the transition between the facial feature locations for frontal face and corresponding non-frontal face. The two part can be trained seperately
  10. Gausian assumption and Kernel Density Function are used to tracking facial feature points based on face shape prior model.
  11. We show the experiments using synthetic data, sequences from the extended Cohn-Kanade database (CK+) [10], the MMI fa- cial expression database [15], the American Sign Lan- guage (ASL) database [24] and the ISL Facial Expression database Synthetic data are used in the process of data mining. Testing and training fraud detection systems This synthetic data assists in teaching a system how to react to certain situations or criteria and (c) are faces with outlier (left eyebrow tip) and corrupted points on the left half face. (b) and (d) are the results after correction. FrontalRBM shows strong power as a face shape prior model.
  12. The CK+ database contains facial behavior videos of 123 subjects showing 7 basic facial expressions including anger, disgust, fear, happiness, sadness, surprise, and contempt. They compared the result with the Active Appearance Models of Matthews & Baker (2004).
  13. MMI database : there are 196 sequences of 27 subjects with 6 basic facial expressions. Some subjects may wear glasses. By incorporating the frontalRBM as face shape prior model, the overall errors decrease by 16.50% and 15.13% when using Gaussian assumption and KDE to combine the measurement and the mode. This result is comparable to the state of art research (Facial point detection using boosted regression and graph models), which reports an average detection error of 5.3 on 400 images selected from not only the MMI database but also the FERET database
  14. Gabor approach for local feature extraction outperformed PCA (Principal Component Analysis), FLD (Fisher’s Linear Discriminant) and LFA (Local Feature Analysis)