2. ncat.edu
ProgressReport
2
What is Zero-shot learning
.
Zero-shot classification refers to the problem setting where we want to recognize objects from
classes that our model has not seen during training..
•Seen classes: These are classes for which we have labelled images during training
•Unseen classes: These are classes for which labelled images are not present during the
training phase.
•Auxiliary information: This information consists of descriptions/semantic attributes/word
embeddings for both seen and unseen classes at train time. This information acts as a
bridge between seen and unseen classes
3. ncat.edu
ProgressReport
3
Example:
.
Suppose,
We have a classification problem with class label : Dog, Cat, Mouse, Car, Bus, Train
Seen Class: Dog, Fox, Bus, Train
UnSeen Class: Wolf, Car
Auxiliary Information: word2vec of all 6 class
4. ncat.edu 4
Suppose,
word2vec in 2d (x,y)
Dog : (2,10)
Wolf : (4,8)
Fox : (2,7)
Car : (6,4)
Bus : (6,3)
Train : (7,3)
Word2Vec as Auxiliary Information
6. ncat.edu 6
Dog
Wolf
Fox
Bus
Train
Car
Testing the Neural Net without unseen Class
Output vector value closest word2vec value from seen and unseen class measure using Euclidian distance
Word2vec
Value V
7. ncat.edu
ProgressReport
7
Example 2:
.
Suppose,
We have a Alphabet classification problem with English Alphabet:
Seen Class: A,B,C,E,F,G,I,J,K,L,M,N,O,P,Q,S,T,U,W,X,Y
UnSeen Class: “ J”, “D”, “H”, “R”, “Z”
Auxiliary Information: Attributes
8. ncat.edu 8
10 out of 15 manually designed features
Auxiliary Information