Fcv rep darrell

283 views
200 views

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
283
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Introduce transformations; show why symmetric transformation isn’t enough
  • End of Kate’s part
  • Why is this nontrivial?
  • Todo: make into bar plot
  • Fcv rep darrell

    1. 1. Learning visual representations for unfamiliar environments Kate Saenko, Brian Kulis, Trevor Darrell UC Berkeley EECS & ICSI
    2. 2. The challenge of large scale visual interaction Last decade has proven the superiority of models learned from data vs. hand engineered structures!
    3. 3. Large-scale learning• “Unsupervised”: Learn models from “found data”; often exploit multiple modalities (text+image) … The Tote is the perfect example of two handbag design principles that ... The lines of this tote are incredibly sleek, but ... The semi buckles that form the handle attachments are ...
    4. 4. E.g., finding visual senses Artifact sense: “telephone” DICTIONARY 1: (n) telephone, phone, telepho ne set (electronic equipment that converts sound into electrical signals that can be transmitted over distances and then converts received signals back into sounds) 2: (n) telephone, telephony (transmitting speech at a distance) [Saenko and Darrell ’09] 4
    5. 5. Large-scale Learning• “Unsupervised”: Learn models from “found data”; often exploit multiple modalities (text+image) … The Tote is the perfect example of two handbag design principles that ... The lines of this tote are incredibly sleek, but ... The semi buckles that form the handle attachments are ...• Supervised: Crowdsource labels (e.g., ImageNet)
    6. 6. Yet…• Even the best collection of images from the web and strong machine learning methods can often yield poor classifiers on in-situ data! ?• Supervised learning assumption: training distribution == test distribution• Unsupervised learning assumption: joint distribution is stationary w.r.t. online world and real world Almost never true! 6
    7. 7. “What You Saw Is Not What You Get” SVM:20% NBNN:19%SVM:54%NBNN:61% The models fail due to domain shift
    8. 8. Examples of visual domain shifts digital SLR webcam Close-up Far-away amazon.com FLICKR CCTV Consumer images
    9. 9. Examples of domain shift:change in camera, feature type, dimension digital SLR webcam SURF SIFT VQ to 300 Different VQ to 1000 dimensions
    10. 10. Solutions?• Do nothing (poor performance)• Collect all types of data (impossible)• Find out what changed (impractical)• Learn what changed
    11. 11. Prior Work on Domain Adaptation• Pre-process the data [Daumé ’07] : replicate features to also create source- and domain- specific versions; re-train learner on new features• SVM-based methods [Yang’07], [Jiang’08], [Duan’09], [Duan’10] : adapt SVM parameters• Kernel mean matching [Gretton’09] : re-weight training data to match test data distribution
    12. 12. Our paradigm: Transform-basedDomain Adaptation Example: “green” and “blue” domainsPrevious methods’ drawbacks• cannot transfer learned shift to new categories• cannot handle new featuresWe can do both by learning W domain transformations* * Saenko, Kulis, Fritz, and Darrell. Adapting visual category models to new domains. ECCV, 2010
    13. 13. Limitations of symmetric transforms Symmetric assumption fails!Saenko et al. ECCV10 used metric learning:• symmetric transforms• same features WHow do we learn more general shifts?
    14. 14. Latest approach*: asymmetric transforms Asymmetric transform (rotation)• Metric learning model no longer applicable• We propose to learn asymmetric transforms – Map from target to source – Handle different dimensions *Kulis, Saenko, and Darrell, What You Saw is Not What You Get: Domain Adaptation Using Asymmetric Kernel Transforms, CVPR 2011
    15. 15. Latest approach: asymmetric transforms Asymmetric transform (rotation)• Metric learning model no longer applicable• We propose to learn asymmetric transforms W – Map from target to source – Handle different dimensions
    16. 16. Model Details W• Learn a linear transformation to map points from one domain to another – Call this transformation W – Matrices of source and target:
    17. 17. Loss FunctionsChoose a point x from thesource and y from thetarget, and consider innerproduct:Should be “large” for similarobjects and “small” for dissimilarobjects
    18. 18. Loss Functions• Input to problem includes a collection of m loss functions• General assumption: loss functions depend on data only through inner product matrix
    19. 19. Regularized Objective Function• Minimize a linear combination of sum of loss functions and a regularizer:• We use squared Frobenius norm as a regularizer – Not restricted to this choice
    20. 20. The Model Has Drawbacks• A linear transformation may be insufficient• Cost of optimization grows as the product of the dimensionalities of the source and target data• What to do?
    21. 21. Kernelization• Main idea: run in kernel space – Use a non-linear kernel function (e.g., RBF kernel) to learn non-linear transformations in input space – Resulting optimization is independent of input dimensionality – Additional assumption necessary: regularizer is a spectral function
    22. 22. Kernelization Kernel matrices for source and targetOriginal TransformationLearning Problem New Kernel ProblemRelationship betweenoriginal and new problemsat optimality
    23. 23. Summary of approach Input Input space space 1. Multi-Domain Data 2. Generate Constraints, Learn W Test point y1 y2 Test point 3. Map via W 4. Apply to New Categories
    24. 24. Multi-domain dataset
    25. 25. Experimental Setup• Utilized a standard bag-of-words model• Also utilize different features in the target domain – SURF vs SIFT – Different visual word dictionaries• Baseline for comparing such data: KCCA
    26. 26. Novel-class experiments Our Method (linear) Our Method• Test method’s ability to transfer domain shift to unseen classes• Train transform on half of the classes, test on the other half
    27. 27. Extreme shift example Query from target Nearest neighbors in source using KCCA+KNN Nearest neighbors in source using transformation
    28. 28. Conclusion• Should not rely on hand-engineered features any more than we rely on hand engineered models!• Learn feature transformation across domains• Developed a domain adaptation method based on regularized non-linear transforms – Asymmetric transform achieves best results on more extreme shifts – Saenko et al ECCV 2010 and Kulis et al CVPR 2011; journal version forthcoming

    ×