This document discusses how computer vision techniques can be applied to art history. It provides an overview of different computer vision approaches such as optical character recognition (OCR), face recognition, and image similarity/categorization. Unsupervised techniques like OCR and image similarity require little labeling of data but may not provide as interesting results. Supervised techniques can more precisely locate parts of images or categorize images but require large labeled datasets. The document recommends several free and open-source computer vision libraries and tools that can be used to explore applying these techniques to art history, along with some caveats about training data requirements.

1. Applying Computer
Vision to Art History
John Resig - http://ejohn.org/research/
Visiting Researcher, Ritsumeikan University

2. What “Works” Today
Reading license plates, zip codes, checks

3. Optical Character
Recognition
• Tesseract
• https://
code.google.com/
p/tesseract-ocr/

4. What “Works” Today
Face recognition

5. Face Matching
• OpenBR
• http://openbiometrics.org/

6. What “Works” Today
Recognition of flat, textured, objects

7. Computer Vision
• Unsupervised (requires no labeling):
• Comparing an entire image
• Categorizing an image
• Supervised (requires labeling):
• Finding parts of an image
• Finding and categorizing parts of an image

8. Unsupervised Training
• Requires little-to-no prepping of data
• Can just give the tool a set of images and
have it produce results
• Extremely easy to get started, results aren’t
always as interesting.

9. Supervised Training
• Need lots of training data
• Needs to be pre-selected/categorized
• Think: Thousands of images.
• If your collection is smaller than this, perhaps
it may not benefit.
• Or you may need crowd sourcing.
• Results can be more interesting:
• “Find all the people in this image”

10. Image Similarity
• imgSeek (Open Source)
• http://www.imgseek.net/
• TinEye’s MatchEngine
• http://services.tineye.com/MatchEngine
• Both are completely unsupervised. No
training data is required.

11. imgSeek
• Compares entire
image.
• Finds similar images,
not exact.
• Does not find parts of
an image.
• Color sensitive.

12. • Compares
portions of
images.
• Finds exact
matches.
• Finds images
inside other
images.
• Color
Ukiyo-e.org (Using MatchEngine)
insensitive.

17. Anonymous Italian Art (Frick PhotoArchive)
Using MatchEngine

18. Conservation

19. Copies

20. Image Portion
Partial Image vs. Much Larger Image

21. Image Categorization
• Deep neural networks
• Requires minimal categorization
• Very little user-input required.
• Ersatz
• http://ersatz1.com/

22. Requires a lot of training
data (thousands of images)
Takes a lot of computers
(Not cheap)
The less categories you
have, the better.

26. General Computer
Vision
• Ideal for some supervised training problems
• CCV
• http://libccv.org/
• https://github.com/liuliu/ccv
• OpenCV
• http://opencv.org/

27. Object Detection

28. Training Caveats
• Requires thousands (if not 10s of
thousands) of images
• Will take at least a week to run on a very
powerful computer
• Does not work with 3D objects

29. Learn More about
Computer Vision
• Learn more:
• http://cs.brown.edu/courses/csci1430/
• Just published paper on Frick Computer
Vision work:
• http://ejohn.org/research/