This document discusses using computer vision and machine learning to enable visual search of historical buildings and locations. It describes how semantic segmentation can be used to extract foreground objects like buildings from photos and create compressed representations that can be compared to a catalog of historical images to find similar matches. The document encourages participation by having people create and contribute Wikipedia pages and photos of historical sites in their communities to help build training data and test the visual search approach.

1. History Visual Search:
Take a Photo, Look up History
Patty Ryan, July 16, 2019
Match to Wikipedia page
for this historical place

2. Introduction
Who am I?
• Applied Machine Learning Engineer, 24 years in technology
• Working with enterprises across industries applying machine learning
• Computer Vision, Natural Language Processing, Sound, Tabular
• Grew up in National Historic Park ‘Keweenaw’
What’s motivating this talk?
• Desire to preserve the historical stories from my father and others
• Applied a visual search approach to clothing using semantic segmentation
• Realized we could do the same for historical buildings

3. How does Visual Search Work?
Given: Find:
Semantic
Segmentation
Item Snapshot Top ‘N’ most similar
results
Segmented
Snapshot
(foreground
only)
Compressed
Representation of
Item without
Background
Compare to compressed
image catalogue and
derive Similarity
Measure
High Level Sequence of Steps
Same approach can be applied to
photos of historical places!

4. How can you participate?
• Create Wikipedia pages for historical buildings or locations in your
community.
• Upload 10 or more photos of that building from various angles.
• Fill in the historical Wikipedia index to image on our site.
• Test historical visual search and report bugs.
• Provide more training photos of your historical building if needed.