SlideShare a Scribd company logo
Beyond Nouns: Exploiting Prepositions and Comparative Adjectives for Learning Visual Classifiers Abhinav Gupta and Larry S. Davis University of Maryland, College Park Proceedings of ECCV 2008 Presented by: DebaleenaChattopadhyay
Presentation Outline - The Problem Definition - The Novelty - The Problem Solution - The Results
The Problem Definition To learn visual classifiers for object recognition from weakly labeled data Input: Labels: city, mountain, sky, sun sun sky Expected Output: mountain city
 Novelty To learn visual classifiers for object recognition from weakly labeled data utilizing additional language constructs Input: Labels:  (Nouns)         city, mountain, sky, sun (Relations)     below(mountain, sky), below(mountain, sun)   above(sky, city),  above(sun, city)       brighter(sun, mountain), brighter(sun, city)      behind(mountain, city), convex(sun, city) in(sun, sky), smaller(sun, sky)       sun sky Expected Output:  mountain city
 Related Work Some Previous Works: ,[object Object],[Ferrari et. al] ,[object Object],     [Bernard et. al] Some After Works: ,[object Object],[Fei-Fei Li et. al, CVPR 09] ,[object Object],[ Forsyth et. al, ICCV 2009]
Overview Pairs of Nouns: Nouns: (SEA, SUN) SEA (SEA, SKY)  (SKY, SEA) SKY  (SKY, SUN) SUN  (SUN, SKY)  (SUN, SEA) Relationships:  in, above, below
Proposed Algorithm ,[object Object]
Algorithm:
Each image represented into a set of image regions.
 Each image region is represented by a set of features
Classifiers for nouns are based on these features (CA)
Classifiers for relationships are based on differential features extracted from pairs of regions (CR)
EM-approach is used to learn noun and relationship models simultaneously
 E-step: Update assignments of nouns to image regions, given CA and CR
M-step: Update model parameters,(CA  and CR ) given updated assignments,[object Object]
Learning the Model EM-approach: Simultaneously solve for the correspondence problem and learn the parameters of classifiers (noun and relationship) E-step: Compute the noun assignment using parameters from the previous iteration.  P( noun iassigned to region j) = Where,
Learning the Model
Learning the Model EM-approach: Simultaneously solve for the correspondence problem and learn the parameters of classifiers (noun and relationship) M-step: Update the model parameters depending on the updated assignments in the E-step. The Maximum Likelihood parameters depends upon the classifier used. To utilize contextual information for labeling test-images, priors on relationship ,P(r|ns,np), are also learnt from a co-occurrence table after the relationship annotations are generated.
Inference- Labeling ,[object Object]
 We know Ij and we have to estimate nj.

More Related Content

Similar to Beyond nouns eccv_2008

M.E Computer Science Image Processing Projects
M.E Computer Science Image Processing ProjectsM.E Computer Science Image Processing Projects
M.E Computer Science Image Processing Projects
Vijay Karan
 
M.Phil Computer Science Image Processing Projects
M.Phil Computer Science Image Processing ProjectsM.Phil Computer Science Image Processing Projects
M.Phil Computer Science Image Processing Projects
Vijay Karan
 
M.Phil Computer Science Image Processing Projects
M.Phil Computer Science Image Processing ProjectsM.Phil Computer Science Image Processing Projects
M.Phil Computer Science Image Processing Projects
Vijay Karan
 
Automatic face naming by learning discriminative
Automatic face naming by learning discriminativeAutomatic face naming by learning discriminative
Automatic face naming by learning discriminative
jpstudcorner
 
SINGLE IMAGE SUPER RESOLUTION: A COMPARATIVE STUDY
SINGLE IMAGE SUPER RESOLUTION: A COMPARATIVE STUDYSINGLE IMAGE SUPER RESOLUTION: A COMPARATIVE STUDY
SINGLE IMAGE SUPER RESOLUTION: A COMPARATIVE STUDY
csandit
 
Download
DownloadDownload
Downloadbutest
 
Download
DownloadDownload
Downloadbutest
 
CCA.ppt
CCA.pptCCA.ppt
CCA.ppt
dizonjermae
 
Automatic face naming by learning discriminative
Automatic face naming by learning discriminativeAutomatic face naming by learning discriminative
Automatic face naming by learning discriminative
nexgentech15
 
AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...
AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...
AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...
Nexgen Technology
 
AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...
AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...
AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...
nexgentechnology
 
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
ijcax
 
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
ijcax
 
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
ijcax
 
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image AnnotationA Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
ijcax
 
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image AnnotationA Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
ijcax
 
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
ijcax
 
Image resolution enhancement via multi surface fitting
Image resolution enhancement via multi surface fittingImage resolution enhancement via multi surface fitting
Image resolution enhancement via multi surface fitting
International Journal of Science and Research (IJSR)
 
fuzzy LBP for face recognition ppt
fuzzy LBP for face recognition pptfuzzy LBP for face recognition ppt
fuzzy LBP for face recognition ppt
Abdullah Gubbi
 
Asilomar09 compressive superres
Asilomar09 compressive superresAsilomar09 compressive superres
Asilomar09 compressive superres
Hoàng Sơn
 

Similar to Beyond nouns eccv_2008 (20)

M.E Computer Science Image Processing Projects
M.E Computer Science Image Processing ProjectsM.E Computer Science Image Processing Projects
M.E Computer Science Image Processing Projects
 
M.Phil Computer Science Image Processing Projects
M.Phil Computer Science Image Processing ProjectsM.Phil Computer Science Image Processing Projects
M.Phil Computer Science Image Processing Projects
 
M.Phil Computer Science Image Processing Projects
M.Phil Computer Science Image Processing ProjectsM.Phil Computer Science Image Processing Projects
M.Phil Computer Science Image Processing Projects
 
Automatic face naming by learning discriminative
Automatic face naming by learning discriminativeAutomatic face naming by learning discriminative
Automatic face naming by learning discriminative
 
SINGLE IMAGE SUPER RESOLUTION: A COMPARATIVE STUDY
SINGLE IMAGE SUPER RESOLUTION: A COMPARATIVE STUDYSINGLE IMAGE SUPER RESOLUTION: A COMPARATIVE STUDY
SINGLE IMAGE SUPER RESOLUTION: A COMPARATIVE STUDY
 
Download
DownloadDownload
Download
 
Download
DownloadDownload
Download
 
CCA.ppt
CCA.pptCCA.ppt
CCA.ppt
 
Automatic face naming by learning discriminative
Automatic face naming by learning discriminativeAutomatic face naming by learning discriminative
Automatic face naming by learning discriminative
 
AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...
AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...
AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...
 
AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...
AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...
AUTOMATIC FACE NAMING BY LEARNING DISCRIMINATIVE AFFINITY MATRICES FROM WEAKL...
 
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
 
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
 
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
 
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image AnnotationA Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
 
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image AnnotationA Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
 
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
A Multi Criteria Decision Making Based Approach for Semantic Image Annotation
 
Image resolution enhancement via multi surface fitting
Image resolution enhancement via multi surface fittingImage resolution enhancement via multi surface fitting
Image resolution enhancement via multi surface fitting
 
fuzzy LBP for face recognition ppt
fuzzy LBP for face recognition pptfuzzy LBP for face recognition ppt
fuzzy LBP for face recognition ppt
 
Asilomar09 compressive superres
Asilomar09 compressive superresAsilomar09 compressive superres
Asilomar09 compressive superres
 

More from Debaleena Chattopadhyay

Trusted Drug-Drug Interaction Alerts: From Critique to Collaboration
Trusted Drug-Drug Interaction Alerts: From Critique to CollaborationTrusted Drug-Drug Interaction Alerts: From Critique to Collaboration
Trusted Drug-Drug Interaction Alerts: From Critique to Collaboration
Debaleena Chattopadhyay
 
Touchless Interaction from an Embodied Perspective
Touchless Interaction from an Embodied PerspectiveTouchless Interaction from an Embodied Perspective
Touchless Interaction from an Embodied PerspectiveDebaleena Chattopadhyay
 
Touchless Circular Menus
Touchless Circular MenusTouchless Circular Menus
Touchless Circular Menus
Debaleena Chattopadhyay
 
Experimental evaluation of five methods for collecting emotions in field sett...
Experimental evaluation of five methods for collecting emotions in field sett...Experimental evaluation of five methods for collecting emotions in field sett...
Experimental evaluation of five methods for collecting emotions in field sett...Debaleena Chattopadhyay
 
Keeping things in context a comparative evaluation of focus plus context scre...
Keeping things in context a comparative evaluation of focus plus context scre...Keeping things in context a comparative evaluation of focus plus context scre...
Keeping things in context a comparative evaluation of focus plus context scre...Debaleena Chattopadhyay
 
Supporting mobility for the blind a broad lit review
Supporting mobility for the blind   a broad lit reviewSupporting mobility for the blind   a broad lit review
Supporting mobility for the blind a broad lit reviewDebaleena Chattopadhyay
 
Defocus magnification
Defocus magnificationDefocus magnification
Defocus magnification
Debaleena Chattopadhyay
 
Estimating natural illumination from a single outdoor scene final
Estimating natural illumination from a single outdoor scene   finalEstimating natural illumination from a single outdoor scene   final
Estimating natural illumination from a single outdoor scene final
Debaleena Chattopadhyay
 
Exploiting Hierarchical Context on a Large Database of Object Categories
Exploiting Hierarchical Context on a Large Database of Object Categories Exploiting Hierarchical Context on a Large Database of Object Categories
Exploiting Hierarchical Context on a Large Database of Object Categories
Debaleena Chattopadhyay
 

More from Debaleena Chattopadhyay (10)

Trusted Drug-Drug Interaction Alerts: From Critique to Collaboration
Trusted Drug-Drug Interaction Alerts: From Critique to CollaborationTrusted Drug-Drug Interaction Alerts: From Critique to Collaboration
Trusted Drug-Drug Interaction Alerts: From Critique to Collaboration
 
Touchless Interaction from an Embodied Perspective
Touchless Interaction from an Embodied PerspectiveTouchless Interaction from an Embodied Perspective
Touchless Interaction from an Embodied Perspective
 
Touchless Circular Menus
Touchless Circular MenusTouchless Circular Menus
Touchless Circular Menus
 
Think aloud protocol a reflection
Think aloud protocol  a reflectionThink aloud protocol  a reflection
Think aloud protocol a reflection
 
Experimental evaluation of five methods for collecting emotions in field sett...
Experimental evaluation of five methods for collecting emotions in field sett...Experimental evaluation of five methods for collecting emotions in field sett...
Experimental evaluation of five methods for collecting emotions in field sett...
 
Keeping things in context a comparative evaluation of focus plus context scre...
Keeping things in context a comparative evaluation of focus plus context scre...Keeping things in context a comparative evaluation of focus plus context scre...
Keeping things in context a comparative evaluation of focus plus context scre...
 
Supporting mobility for the blind a broad lit review
Supporting mobility for the blind   a broad lit reviewSupporting mobility for the blind   a broad lit review
Supporting mobility for the blind a broad lit review
 
Defocus magnification
Defocus magnificationDefocus magnification
Defocus magnification
 
Estimating natural illumination from a single outdoor scene final
Estimating natural illumination from a single outdoor scene   finalEstimating natural illumination from a single outdoor scene   final
Estimating natural illumination from a single outdoor scene final
 
Exploiting Hierarchical Context on a Large Database of Object Categories
Exploiting Hierarchical Context on a Large Database of Object Categories Exploiting Hierarchical Context on a Large Database of Object Categories
Exploiting Hierarchical Context on a Large Database of Object Categories
 

Recently uploaded

Brand Identity For A Sportscaster Project and Portfolio I
Brand Identity For A Sportscaster Project and Portfolio IBrand Identity For A Sportscaster Project and Portfolio I
Brand Identity For A Sportscaster Project and Portfolio I
thomasaolson2000
 
132. Acta Scientific Pharmaceutical Sciences
132. Acta Scientific Pharmaceutical Sciences132. Acta Scientific Pharmaceutical Sciences
132. Acta Scientific Pharmaceutical Sciences
Manu Mitra
 
欧洲杯买球平台-欧洲杯买球平台推荐-欧洲杯买球平台| 立即访问【ac123.net】
欧洲杯买球平台-欧洲杯买球平台推荐-欧洲杯买球平台| 立即访问【ac123.net】欧洲杯买球平台-欧洲杯买球平台推荐-欧洲杯买球平台| 立即访问【ac123.net】
欧洲杯买球平台-欧洲杯买球平台推荐-欧洲杯买球平台| 立即访问【ac123.net】
foismail170
 
DIGITAL MARKETING COURSE IN CHENNAI.pptx
DIGITAL MARKETING COURSE IN CHENNAI.pptxDIGITAL MARKETING COURSE IN CHENNAI.pptx
DIGITAL MARKETING COURSE IN CHENNAI.pptx
FarzanaRbcomcs
 
皇冠体育- 皇冠体育官方网站- CROWN SPORTS| 立即访问【ac123.net】
皇冠体育- 皇冠体育官方网站- CROWN SPORTS| 立即访问【ac123.net】皇冠体育- 皇冠体育官方网站- CROWN SPORTS| 立即访问【ac123.net】
皇冠体育- 皇冠体育官方网站- CROWN SPORTS| 立即访问【ac123.net】
larisashrestha558
 
Chapters 3 Contracts.pptx Chapters 3 Contracts.pptx
Chapters 3  Contracts.pptx Chapters 3  Contracts.pptxChapters 3  Contracts.pptx Chapters 3  Contracts.pptx
Chapters 3 Contracts.pptx Chapters 3 Contracts.pptx
Sheldon Byron
 
The Impact of Artificial Intelligence on Modern Society.pdf
The Impact of Artificial Intelligence on Modern Society.pdfThe Impact of Artificial Intelligence on Modern Society.pdf
The Impact of Artificial Intelligence on Modern Society.pdf
ssuser3e63fc
 
欧洲杯投注网站-欧洲杯投注网站推荐-欧洲杯投注网站| 立即访问【ac123.net】
欧洲杯投注网站-欧洲杯投注网站推荐-欧洲杯投注网站| 立即访问【ac123.net】欧洲杯投注网站-欧洲杯投注网站推荐-欧洲杯投注网站| 立即访问【ac123.net】
欧洲杯投注网站-欧洲杯投注网站推荐-欧洲杯投注网站| 立即访问【ac123.net】
foismail170
 
DOC-20240602-WA0001..pdf DOC-20240602-WA0001..pdf
DOC-20240602-WA0001..pdf DOC-20240602-WA0001..pdfDOC-20240602-WA0001..pdf DOC-20240602-WA0001..pdf
DOC-20240602-WA0001..pdf DOC-20240602-WA0001..pdf
Pushpendra Kumar
 
Personal Brand exploration KE.pdf for assignment
Personal Brand exploration KE.pdf for assignmentPersonal Brand exploration KE.pdf for assignment
Personal Brand exploration KE.pdf for assignment
ragingokie
 
How to create an effective K-POC tutorial
How to create an effective K-POC tutorialHow to create an effective K-POC tutorial
How to create an effective K-POC tutorial
vencislavkaaa
 
Widal Agglutination Test: A rapid serological diagnosis of typhoid fever
Widal Agglutination Test: A rapid serological diagnosis of typhoid feverWidal Agglutination Test: A rapid serological diagnosis of typhoid fever
Widal Agglutination Test: A rapid serological diagnosis of typhoid fever
taexnic
 
How to Master LinkedIn for Career and Business
How to Master LinkedIn for Career and BusinessHow to Master LinkedIn for Career and Business
How to Master LinkedIn for Career and Business
ideatoipo
 
133. Reviewer Certificate in Advances in Research
133. Reviewer Certificate in Advances in Research133. Reviewer Certificate in Advances in Research
133. Reviewer Certificate in Advances in Research
Manu Mitra
 
New Explore Careers and College Majors 2024.pdf
New Explore Careers and College Majors 2024.pdfNew Explore Careers and College Majors 2024.pdf
New Explore Careers and College Majors 2024.pdf
Dr. Mary Askew
 
太阳城娱乐-太阳城娱乐推荐-太阳城娱乐官方网站| 立即访问【ac123.net】
太阳城娱乐-太阳城娱乐推荐-太阳城娱乐官方网站| 立即访问【ac123.net】太阳城娱乐-太阳城娱乐推荐-太阳城娱乐官方网站| 立即访问【ac123.net】
太阳城娱乐-太阳城娱乐推荐-太阳城娱乐官方网站| 立即访问【ac123.net】
foismail170
 
Operating system. short answes and Interview questions .pdf
Operating system. short answes and Interview questions .pdfOperating system. short answes and Interview questions .pdf
Operating system. short answes and Interview questions .pdf
harikrishnahari6276
 
欧洲杯投注app-欧洲杯投注app推荐-欧洲杯投注app| 立即访问【ac123.net】
欧洲杯投注app-欧洲杯投注app推荐-欧洲杯投注app| 立即访问【ac123.net】欧洲杯投注app-欧洲杯投注app推荐-欧洲杯投注app| 立即访问【ac123.net】
欧洲杯投注app-欧洲杯投注app推荐-欧洲杯投注app| 立即访问【ac123.net】
foismail170
 
131. Reviewer Certificate in BP International
131. Reviewer Certificate in BP International131. Reviewer Certificate in BP International
131. Reviewer Certificate in BP International
Manu Mitra
 
15385-LESSON PLAN- 7TH - SS-Insian Constitution an Introduction.pdf
15385-LESSON PLAN- 7TH - SS-Insian Constitution an Introduction.pdf15385-LESSON PLAN- 7TH - SS-Insian Constitution an Introduction.pdf
15385-LESSON PLAN- 7TH - SS-Insian Constitution an Introduction.pdf
gobogo3542
 

Recently uploaded (20)

Brand Identity For A Sportscaster Project and Portfolio I
Brand Identity For A Sportscaster Project and Portfolio IBrand Identity For A Sportscaster Project and Portfolio I
Brand Identity For A Sportscaster Project and Portfolio I
 
132. Acta Scientific Pharmaceutical Sciences
132. Acta Scientific Pharmaceutical Sciences132. Acta Scientific Pharmaceutical Sciences
132. Acta Scientific Pharmaceutical Sciences
 
欧洲杯买球平台-欧洲杯买球平台推荐-欧洲杯买球平台| 立即访问【ac123.net】
欧洲杯买球平台-欧洲杯买球平台推荐-欧洲杯买球平台| 立即访问【ac123.net】欧洲杯买球平台-欧洲杯买球平台推荐-欧洲杯买球平台| 立即访问【ac123.net】
欧洲杯买球平台-欧洲杯买球平台推荐-欧洲杯买球平台| 立即访问【ac123.net】
 
DIGITAL MARKETING COURSE IN CHENNAI.pptx
DIGITAL MARKETING COURSE IN CHENNAI.pptxDIGITAL MARKETING COURSE IN CHENNAI.pptx
DIGITAL MARKETING COURSE IN CHENNAI.pptx
 
皇冠体育- 皇冠体育官方网站- CROWN SPORTS| 立即访问【ac123.net】
皇冠体育- 皇冠体育官方网站- CROWN SPORTS| 立即访问【ac123.net】皇冠体育- 皇冠体育官方网站- CROWN SPORTS| 立即访问【ac123.net】
皇冠体育- 皇冠体育官方网站- CROWN SPORTS| 立即访问【ac123.net】
 
Chapters 3 Contracts.pptx Chapters 3 Contracts.pptx
Chapters 3  Contracts.pptx Chapters 3  Contracts.pptxChapters 3  Contracts.pptx Chapters 3  Contracts.pptx
Chapters 3 Contracts.pptx Chapters 3 Contracts.pptx
 
The Impact of Artificial Intelligence on Modern Society.pdf
The Impact of Artificial Intelligence on Modern Society.pdfThe Impact of Artificial Intelligence on Modern Society.pdf
The Impact of Artificial Intelligence on Modern Society.pdf
 
欧洲杯投注网站-欧洲杯投注网站推荐-欧洲杯投注网站| 立即访问【ac123.net】
欧洲杯投注网站-欧洲杯投注网站推荐-欧洲杯投注网站| 立即访问【ac123.net】欧洲杯投注网站-欧洲杯投注网站推荐-欧洲杯投注网站| 立即访问【ac123.net】
欧洲杯投注网站-欧洲杯投注网站推荐-欧洲杯投注网站| 立即访问【ac123.net】
 
DOC-20240602-WA0001..pdf DOC-20240602-WA0001..pdf
DOC-20240602-WA0001..pdf DOC-20240602-WA0001..pdfDOC-20240602-WA0001..pdf DOC-20240602-WA0001..pdf
DOC-20240602-WA0001..pdf DOC-20240602-WA0001..pdf
 
Personal Brand exploration KE.pdf for assignment
Personal Brand exploration KE.pdf for assignmentPersonal Brand exploration KE.pdf for assignment
Personal Brand exploration KE.pdf for assignment
 
How to create an effective K-POC tutorial
How to create an effective K-POC tutorialHow to create an effective K-POC tutorial
How to create an effective K-POC tutorial
 
Widal Agglutination Test: A rapid serological diagnosis of typhoid fever
Widal Agglutination Test: A rapid serological diagnosis of typhoid feverWidal Agglutination Test: A rapid serological diagnosis of typhoid fever
Widal Agglutination Test: A rapid serological diagnosis of typhoid fever
 
How to Master LinkedIn for Career and Business
How to Master LinkedIn for Career and BusinessHow to Master LinkedIn for Career and Business
How to Master LinkedIn for Career and Business
 
133. Reviewer Certificate in Advances in Research
133. Reviewer Certificate in Advances in Research133. Reviewer Certificate in Advances in Research
133. Reviewer Certificate in Advances in Research
 
New Explore Careers and College Majors 2024.pdf
New Explore Careers and College Majors 2024.pdfNew Explore Careers and College Majors 2024.pdf
New Explore Careers and College Majors 2024.pdf
 
太阳城娱乐-太阳城娱乐推荐-太阳城娱乐官方网站| 立即访问【ac123.net】
太阳城娱乐-太阳城娱乐推荐-太阳城娱乐官方网站| 立即访问【ac123.net】太阳城娱乐-太阳城娱乐推荐-太阳城娱乐官方网站| 立即访问【ac123.net】
太阳城娱乐-太阳城娱乐推荐-太阳城娱乐官方网站| 立即访问【ac123.net】
 
Operating system. short answes and Interview questions .pdf
Operating system. short answes and Interview questions .pdfOperating system. short answes and Interview questions .pdf
Operating system. short answes and Interview questions .pdf
 
欧洲杯投注app-欧洲杯投注app推荐-欧洲杯投注app| 立即访问【ac123.net】
欧洲杯投注app-欧洲杯投注app推荐-欧洲杯投注app| 立即访问【ac123.net】欧洲杯投注app-欧洲杯投注app推荐-欧洲杯投注app| 立即访问【ac123.net】
欧洲杯投注app-欧洲杯投注app推荐-欧洲杯投注app| 立即访问【ac123.net】
 
131. Reviewer Certificate in BP International
131. Reviewer Certificate in BP International131. Reviewer Certificate in BP International
131. Reviewer Certificate in BP International
 
15385-LESSON PLAN- 7TH - SS-Insian Constitution an Introduction.pdf
15385-LESSON PLAN- 7TH - SS-Insian Constitution an Introduction.pdf15385-LESSON PLAN- 7TH - SS-Insian Constitution an Introduction.pdf
15385-LESSON PLAN- 7TH - SS-Insian Constitution an Introduction.pdf
 

Beyond nouns eccv_2008

  • 1. Beyond Nouns: Exploiting Prepositions and Comparative Adjectives for Learning Visual Classifiers Abhinav Gupta and Larry S. Davis University of Maryland, College Park Proceedings of ECCV 2008 Presented by: DebaleenaChattopadhyay
  • 2. Presentation Outline - The Problem Definition - The Novelty - The Problem Solution - The Results
  • 3. The Problem Definition To learn visual classifiers for object recognition from weakly labeled data Input: Labels: city, mountain, sky, sun sun sky Expected Output: mountain city
  • 4. Novelty To learn visual classifiers for object recognition from weakly labeled data utilizing additional language constructs Input: Labels: (Nouns) city, mountain, sky, sun (Relations) below(mountain, sky), below(mountain, sun)   above(sky, city), above(sun, city)       brighter(sun, mountain), brighter(sun, city)      behind(mountain, city), convex(sun, city) in(sun, sky), smaller(sun, sky)       sun sky Expected Output: mountain city
  • 5.
  • 6. Overview Pairs of Nouns: Nouns: (SEA, SUN) SEA (SEA, SKY) (SKY, SEA) SKY (SKY, SUN) SUN (SUN, SKY) (SUN, SEA) Relationships: in, above, below
  • 7.
  • 9. Each image represented into a set of image regions.
  • 10. Each image region is represented by a set of features
  • 11. Classifiers for nouns are based on these features (CA)
  • 12. Classifiers for relationships are based on differential features extracted from pairs of regions (CR)
  • 13. EM-approach is used to learn noun and relationship models simultaneously
  • 14. E-step: Update assignments of nouns to image regions, given CA and CR
  • 15.
  • 16. Learning the Model EM-approach: Simultaneously solve for the correspondence problem and learn the parameters of classifiers (noun and relationship) E-step: Compute the noun assignment using parameters from the previous iteration. P( noun iassigned to region j) = Where,
  • 18. Learning the Model EM-approach: Simultaneously solve for the correspondence problem and learn the parameters of classifiers (noun and relationship) M-step: Update the model parameters depending on the updated assignments in the E-step. The Maximum Likelihood parameters depends upon the classifier used. To utilize contextual information for labeling test-images, priors on relationship ,P(r|ns,np), are also learnt from a co-occurrence table after the relationship annotations are generated.
  • 19.
  • 20. We know Ij and we have to estimate nj.
  • 21. The labeling problem is constrained by priors on relationships between pairs of nouns.
  • 22.
  • 23. For training, 850 images with nouns and hand-labelled relationships between subset of pairs of nouns.
  • 24. Nearest neighbor and Gaussian Classifier based likelihood model for nouns is used.
  • 25. Decision stump based likelihood model for relationships is used.
  • 27. 19 relationships: above, behind, below, beside, more textured, brighter, in, greener, larger, left, near, far from, ontopof, more blue, right, similar, smaller, taller, shorter
  • 28.
  • 29. Compared with human labeling
  • 31. Range of semantics identified- Both algorithm give similar performance (L)
  • 32. Frequency Correct- Later algorithm performs better in number of times a noun is identified (R)Nouns only Nouns & Relationships (Human) Nouns & Relationships (learned) Proposed EM algorithm bootstrapped by IBM Model 1 Proposed EM algorithm bootstrapped by Duygulu et. al
  • 33. Experimental Results Reducing Correspondence Ambiguity Duygulu et. al Beyond Nouns
  • 34.
  • 36.
  • 37. Experimental Results Precision-Recall: Precision Ratio- The ratio of number of images that have been correctly annotated with that word to the number of images which were annotated with the word by the algorithm. (Respect to Human Observers) Recall Ratio: The ratio of the number of images correctly annotated with that word using the algorithm to the number of images that should have been annotated with that word. (Respect to Corel Annotations)
  • 38.
  • 39. This algorithm proposes an EM based method to simultaneously learn visual classifiers for nouns, prepositions and comparative adjectives.
  • 40.
  • 41.
  • 42. We know Ij and we have to estimate nj.
  • 43. The labeling problem is constrained by priors on relationships between pairs of nouns.
  • 44. Bayesian Network is used to represent the labeling problem and belief propagation for inference.
  • 45. The word likelihood in an image is given as:

Editor's Notes

  1. We are to determine the correspondence between image regions and semantic object classes Problem: Significant ambiguities in correspondence of visual features and object class
  2. Instead of using only co-occurrence of nouns and image features over large databases of images to determine the correspondence, additional language constructs are considered like “prepositions” and “comparative adjectives”. This paper simultaneously learns the visual features defining “nouns” and the differential visual features defining “binary-relationships” using EM approach
  3. Not applicable for binary relationships if models for nouns not givenhave used spatial relationships between image patches for scene recognition. The paper applies a feature mining approach to get discriminative image patches and the relationship between them is interpreted as adjectives or prepositions. The authors mined relationships between more than two image patches too. They used SVM to train the data mining problem with different types of adjectives and prepositions encoded. Encoding is based on image representation of multi-scale local patches and the spatial pyramid representation. SIFT descriptors are used to represent each appearance patch. At first the visual code words are recognized in an image and then relationships are extracted using Apriori mining algorithm.introduces an approach to learn jointly detectors for object classes and attributes (color and texture) based on a co-training algorithm. Object to attribute is a one way association here i.e. a red table or a metallic table; but not both. Here also the image is divided into a number of windows and joint multiple instance learning is used to force learners for both the object class and the attribute class to co-operate on labeling windows that must contain both the object and attribute. They have focused on windows that are salient and homogenous to select candidate windows.In most of the cases, the object detection average precision is better than the separate learning approach and moreover “visual attribute object” not in the training set can also be detected by combining visual attribute and object detectors learned from the other categories.
  4. Visual features based on appearance and shapeInitialization with random assignements
  5. Word sense disambiguation is not taken into context
  6. Aij refers to the subset of the set of all possible assignments for animage in which noun i is assigned to region j.
  7. Aij refers to the subset of the set of all possible assignments for animage in which noun i is assigned to region j.
  8. For a Gaussian classifier we estimate the mean and varianceInitialization random Authors use the result of Bernard’s paper, translation based model. Any image annotation approach with localization shall workAfter learning the maximum likelihood parameters, weuse the relationship classifier and the assignment to find possible relationshipsbetween all pairs of words. Using these generated relationship annotations weform a co-occurrence table which is used to compute P
  9. For each region, we have two nodes corresponding tothe noun and image features from that region. For all possible pairs of regions,we have another two nodes representing a relationship word and differentialfeatures from that pair of regions.An example of a Bayesian network with 3 regions. The rjk represent the possiblewords for the relationship between regions (j, k). Due to the non-symmetric nature ofrelationships we consider both (j, k) and (k, j) pairs (in the figure only one is shown).The magenta blocks in the image represent differential features (Ijk).
  10. Relationship model is based one differential features.The parameterlearning M-step therefore also involves feature selection for relationshipclassifiers.
  11. The first measure counts the number of words that are labeled properly bythe algorithm. In this case, each word has similar importance regardless of thefrequency with which it occurs. In the second case, a word which occurs morefrequently is given higher importance.Using the first measure, both algorithms have similar performance becausethey can correctly label one word each. However, using the second measurethe latter algorithm is better as sky is more common and hence the number ofcorrectly identified regions would be higher for the latter algorithm.a co-occurrence based translation model [ibm model 1]and translation based model with mixing probabilities [duygulu et. al] form the baseline algorithms.
  12. For each region, we have two nodes corresponding tothe noun and image features from that region. For all possible pairs of regions,we have another two nodes representing a relationship word and differentialfeatures from that pair of regions.An example of a Bayesian network with 3 regions. The rjk represent the possiblewords for the relationship between regions (j, k). Due to the non-symmetric nature ofrelationships we consider both (j, k) and (k, j) pairs (in the figure only one is shown).The magenta blocks in the image represent differential features (Ijk).