Fcv hum mach_perona

276 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
276
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Fcv hum mach_perona

  1. 1. Humans & Machines collaborating on vision Pietro Perona California Institute of Technology NSF Workshop - Frontiers in Vision Cambridge, 23 Aug 2011Friday, August 26, 2011
  2. 2. “Collaborative vision’’ ? Pietro Perona California Institute of Technology NSF Workshop - Frontiers in Vision Cambridge, 23 Aug 2011Friday, August 26, 2011
  3. 3. Objectives • Sketch new area of research • Sampler of initial work • Drawing lessons • Brainstorm: potential, way forwardFriday, August 26, 2011
  4. 4. Plan • Define area (10’) • Presentations (50’): Perona, Geman, Grauman, Berg, Belongie • Discussion (15’)Friday, August 26, 2011
  5. 5. DefinitionFriday, August 26, 2011
  6. 6. 6Friday, August 26, 2011
  7. 7. ? 6Friday, August 26, 2011
  8. 8. 7Friday, August 26, 2011
  9. 9. Friday, August 26, 2011
  10. 10. Friday, August 26, 2011
  11. 11. Friday, August 26, 2011
  12. 12. 9Friday, August 26, 2011
  13. 13. Lessons: • Visual queries • Easy for humans • Difficult for machines • Much information is available on line • Pictures are digital dark matter • Experts not providing visual knowledge 10Friday, August 26, 2011
  14. 14. Unsupervised learning? 11 [Fergus et al., CVPR03]Friday, August 26, 2011
  15. 15. Unsupervised learning? 11 [Fergus et al., CVPR03]Friday, August 26, 2011
  16. 16. 12Friday, August 26, 2011
  17. 17. Friday, August 26, 2011
  18. 18. ThroatFriday, August 26, 2011
  19. 19. ThroatFriday, August 26, 2011
  20. 20. Visual knowledge Categorical (experts) 14 Task-oriented (practitioners)Friday, August 26, 2011
  21. 21. World Ob n ser io va at tio rv n se Ob Science, Shared Education Users expertise knowledge Experts ers sw es An eri Models Image Qu annotations Machine vision Annotators Automata scientists 15Friday, August 26, 2011
  22. 22. World Ob n ser io va at tio rv n se Ob Science, Shared Education Users expertise knowledge Experts ers sw es An eri Models Image Qu annotations Machine vision Annotators Automata scientists 15Friday, August 26, 2011
  23. 23. World Ob n ser io va at tio rv n se Ob Science, Shared Education Users expertise knowledge Experts ers sw es An eri Models Image Qu annotations Machine vision Annotators Automata scientists 15Friday, August 26, 2011
  24. 24. World Ob n ser io va at tio rv n se Ob Science, Shared Education Users expertise knowledge Experts ers sw es An eri Models Image Qu annotations Machine vision Annotators Automata scientists 15Friday, August 26, 2011
  25. 25. Some progress...Friday, August 26, 2011
  26. 26. DUCKS Waterbirds Mallard American Black Duck Canada Goose Red Necked Grebe ClutterFriday, August 26, 2011
  27. 27. Multidimensional signals and annotators p(xi | zi = 1) 2 xi 1 2 xi = (xi , xi ) 1 xi p(xi | zi = 0)Friday, August 26, 2011
  28. 28. Multidimensional signals and annotators p(xi | zi = 1) 2 xi 1 2 xi = (xi , xi ) 1 xi p(xi | zi = 0)Friday, August 26, 2011
  29. 29. Multidimensional signals and annotators p(xi | zi = 1) 2 xi 1 2 xi = (xi , xi ) 1 xi τj 1 2 wj = (wj , wj ) p(xi | zi = 0)Friday, August 26, 2011
  30. 30. α γ Full model annotators σj wj τj β θz M zi xi yij lij Ji ij images N labels |Lij | [Welinder et al., NIPS2010]Friday, August 26, 2011
  31. 31. Is there a duck in the image? 2 xi 1 xiFriday, August 26, 2011
  32. 32. Is there a duck in the image? 2 xi 1 xiFriday, August 26, 2011
  33. 33. Is there a duck in the image? 2 xi 1 xiFriday, August 26, 2011
  34. 34. Is there a duck in the image? 2 xi 1 xiFriday, August 26, 2011
  35. 35. Is there a duck in the image? 2 xi 1 xiFriday, August 26, 2011
  36. 36. Is there a duck in the image? 2 xi 1 xiFriday, August 26, 2011
  37. 37. Is there a duck in the image? 2 xi 1 xiFriday, August 26, 2011
  38. 38. Concluding...Friday, August 26, 2011
  39. 39. Collaborative vision 100% Automation 0% Performance 100%Friday, August 26, 2011
  40. 40. Collaborative vision 100% Automation 0% Performance 100%Friday, August 26, 2011
  41. 41. Collaborative vision 100% Automation 0% Performance 100%Friday, August 26, 2011
  42. 42. Collaborative vision 100% Automation 0% Performance 100%Friday, August 26, 2011
  43. 43. Collaborative vision 100% + Automation Applications Training data - Complexity Cost 0% Performance 100%Friday, August 26, 2011
  44. 44. World Ob n ser io va at tio rv n se Ob Science, Shared Education Users expertise knowledge Experts ers sw es An eri Models Image Qu annotations Machine vision Annotators Automata scientists 24Friday, August 26, 2011
  45. 45. New research directions • Incremental learning • Models of human vision, decision, attention • Systems composed of machines and humans • Performance bounds (humans, machines) • Representations (human-machine-friendly) • Extracting visual knowledge from expertsFriday, August 26, 2011

×