Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Accidentally
Becoming
A
Digital Librarian
John Resig
Providing Access to Knowledge.
Educating and Empowering Others.
1. Solve your own problems
2. Adapt and solve other’s
problems
Talking about 4 problems
Chazen Museum: 1980.2386
Problem 1:

Searching by Image
imgSeek
• Compares entire image.
• Finds similar images, not
exact.
• Does not find parts of
an image.
• Color sensitive.
•...
https://services.tineye.com/MatchEngine
http://pastec.io/
Problem 2: No one agrees on
names/titles
Japanese Names
• Utagawa Hiroshige
• Ando Hiroshige
• Andō Hiroshige
• Hiroshige
• 歌川広重
• 広重
Similar Images
Different photo, same work of art.
Similar Images
Different photo, slightly different cropping.
Alternate Images
Partial Image vs. Much Larger Image
Alternate Images
Color vs. Black-and-White
Conservation
Conservation
Repairs and possibly removal of later additions.
Conservation/Destruction
Analysis even spots dramatic conservation work.
Copies
Copies
Copies
Copies
Fondazione Zeri
PHAROS: An International Consortium of Photo Archives
• Bibliotheca Hertziana, Rome (1,065,000)

• Bildarchiv Foto Marburg...
PHAROS Images: Coming June 2016
Problem 3: Bad Images
Idyll: Offline Image Cropping
• Crop and annotate images offline and on a
mobile device.

• Saves the selections back to a se...
ComputerVision
• Unsupervised (requires no labeling):
• Comparing an entire image
• Categorizing an image
• Supervised (re...
Unsupervised Training
• Requires little-to-no prepping of data
• Can just give the tool a set of images and
have it produc...
Supervised Training
• Need lots of training data
• Needs to be pre-selected/categorized
• Think:Thousands of images.
• If ...
General Computer
Vision
• Ideal for some supervised training problems
• CCV
• http://libccv.org/
• https://github.com/liul...
Object Detection
Problem 4: Education
Does your person have a bald spot at the top of their head?
Does your person have a *hidden* bald spot at the top of their head?
Does your person have a *hidden* bald spot at the top of their head?
Does your person have a cloth covering the front of their head?
Does your person have a comb in their hair?
• http://ejohn.org/research/

• http://ukiyo-e.org/
• https://github.com/jeresig
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Accidentally Becoming a Digital Librarian
Upcoming SlideShare
Loading in …5
×

Accidentally Becoming a Digital Librarian

395 views

Published on

Given at Catskills Conf 2015

Published in: Art & Photos
  • Be the first to comment

  • Be the first to like this

Accidentally Becoming a Digital Librarian

  1. 1. Accidentally Becoming A Digital Librarian John Resig
  2. 2. Providing Access to Knowledge. Educating and Empowering Others.
  3. 3. 1. Solve your own problems
  4. 4. 2. Adapt and solve other’s problems
  5. 5. Talking about 4 problems
  6. 6. Chazen Museum: 1980.2386
  7. 7. Problem 1:
 Searching by Image
  8. 8. imgSeek • Compares entire image. • Finds similar images, not exact. • Does not find parts of an image. • Color sensitive. • Open Source
  9. 9. https://services.tineye.com/MatchEngine
  10. 10. http://pastec.io/
  11. 11. Problem 2: No one agrees on names/titles
  12. 12. Japanese Names • Utagawa Hiroshige • Ando Hiroshige • Andō Hiroshige • Hiroshige • 歌川広重 • 広重
  13. 13. Similar Images Different photo, same work of art.
  14. 14. Similar Images Different photo, slightly different cropping.
  15. 15. Alternate Images Partial Image vs. Much Larger Image
  16. 16. Alternate Images Color vs. Black-and-White
  17. 17. Conservation
  18. 18. Conservation Repairs and possibly removal of later additions.
  19. 19. Conservation/Destruction Analysis even spots dramatic conservation work.
  20. 20. Copies
  21. 21. Copies
  22. 22. Copies
  23. 23. Copies
  24. 24. Fondazione Zeri
  25. 25. PHAROS: An International Consortium of Photo Archives • Bibliotheca Hertziana, Rome (1,065,000) • Bildarchiv Foto Marburg, Germany (2,000,000) • Courtauld Institute of Art, London (4,173,500) • Fondazione Federico Zeri, Bologna (290,000) • Frick Art Reference Library, New York (1,346,000) • Getty Research Institute, Los Angeles (2,086,000) • Villa I Tatti, Florence (239,000) • Institut National d’Histoire de l’Art, Paris (750,000) • Kunsthistorisches Institut, Florence (650,000) • National Gallery of Art, Washington (7,600,000) • Paul Mellon Centre, London (185,000) • Rijksbureau, The Hague (7,000,000 • Warburg Institute, London (3,500,000) • Yale Center for British Art, New Haven (132,000)
  26. 26. PHAROS Images: Coming June 2016
  27. 27. Problem 3: Bad Images
  28. 28. Idyll: Offline Image Cropping • Crop and annotate images offline and on a mobile device. • Saves the selections back to a server.
  29. 29. ComputerVision • Unsupervised (requires no labeling): • Comparing an entire image • Categorizing an image • Supervised (requires labeling): • Finding parts of an image • Finding and categorizing parts of an image
  30. 30. Unsupervised Training • Requires little-to-no prepping of data • Can just give the tool a set of images and have it produce results • Extremely easy to get started, results aren’t always as interesting. • Unsupervised: MatchEngine, PasteC
  31. 31. Supervised Training • Need lots of training data • Needs to be pre-selected/categorized • Think:Thousands of images. • If your collection is smaller than this, perhaps it may not benefit. • Or you may need crowd sourcing. • Results can be more interesting: • “Find all the people in this image”
  32. 32. General Computer Vision • Ideal for some supervised training problems • CCV • http://libccv.org/ • https://github.com/liuliu/ccv • OpenCV • http://opencv.org/
  33. 33. Object Detection
  34. 34. Problem 4: Education
  35. 35. Does your person have a bald spot at the top of their head?
  36. 36. Does your person have a *hidden* bald spot at the top of their head?
  37. 37. Does your person have a *hidden* bald spot at the top of their head?
  38. 38. Does your person have a cloth covering the front of their head?
  39. 39. Does your person have a comb in their hair?
  40. 40. • http://ejohn.org/research/ • http://ukiyo-e.org/ • https://github.com/jeresig

×