hi

1. Show and Tell: A
Neural Image
Caption Generator
May 17 , 2021
Hee Dae Kwon

2. Contents
• Introduction
• Model
• Experiments
• Results

3. Introduction
Image description 문제를
푸는 end-to-end 시스템을
제안함.
논문에서 제안하는 모델은
vision과 language 모델
중에 SOTA인 모델들의
구조의 일부를 조합

4. Introduction
encoder RNN →→
encoder CNN
image I
Target sequnce of
Words S
S=S1,S2,…S=S1,S2,…
maximizing p(S|I) 하도
록 학습

5. Model

6. Model
Encoder
: CNN (pre-trained)
Decorder
: RNN(LSTM)

7. Experiments
Over fiiting을 막기 위한 techniques
Pre-Train 된 Deep CNN 사용.
Pre-Train 된 Word embedding vetor 사용. (효과 적음)
Dropout & Ensembling

8. Results
• How dataset size affects generalization
• What kinds of transfer learning it would be able to achieve
• How it would deal with weakly labeled examples

9. Results
• Generation Results

10. Results
• Flickr30k -> Flickr8k
BLEU 4 증가
• MSCOCO -> Flickr8k
BLEU 10 감소,
• MSCOCO -> SBU
BLEU 16 감소

11. Results
• Generation Diversity
Discussion
=> generating model 이 새롭고
다양하고 높은 퀄리티의
문장생성

12. Results
• Ranking Results

13. Results
• Human Evaluation

14. Results
• Analysis of Embedding

15. •감사합니다