This document discusses using machine learning to recognize animal species in images taken by motion-triggered camera traps. It explores using convolutional neural networks (CNNs) to classify images and combining image recognition data with metadata like temperature, date and time. Two CNN models (Caffe and TensorFlow) were tested on labeled camera trap images, with retraining improving accuracy from initially inaccurate results. Optical character recognition was also used to recognize temperatures in images and combining data sources could help analyze animal species trends related to time and temperature.

1. Implementing Machine Learning Based Image Recognition
For Animal Detection
Andy Rosales-Elias, Nevena Golubovic, Chandra Krintz, Rich Wolski
{andyrosales, nevena, ckrintz, rich}@cs.ucsb.edu
• Animal surveillance is error-prone, time consuming and expensive.
• Using a computer to count animals and observe trends is faster, more reliable and
cheaper.
• We use different machine learning technologies to recognize and categorize animal
species in an image, which is taken by motion-triggered camera traps.
• We combine data from the machine learning model with metadata such as: date, time
and temperature in order to learn more about animal species
Introduction
Using Convolutional Neural Networks
Image Recognition With Metadata
No Retraining
0
0.2
0.4
0.6
0.8
1
1.2
Accuracy%
Previous OCR New OCR
0
50
100
150
200
250
300
Time(s)
Previous OCR New OCR
Methods
For the classification task we used Convolutional Neural Networks (CNN) – a machine
learning framework that has been proven to be significantly better at image recognizing
tasks¹
The open-source Convolutional Neural Networks come pre-trained with data from
ImageNet². However this can be a problem because the default, pre-trained model is
trained on classes that are irrelevant to us (figure 1) therefore we need to modify our CNN
in order to obtain only relevant results. We modified our CNN by adding an extra layer that
filters only the results that are irrelevant to our purposes. (Figure 2)
as
OCR Performance Results
The next step is to combine the image recognition data with information like temperature,
time and date. Time and date are automatically generated with every picture but
temperature needs to be recognized using OCR techniques. This information can help us
recognize trends and answer questions like “what time/temperature do animals come out
the most?”
To compare both types of OCR,
we picked 20 random images and
analyzed them with each method.
The “new OCR” (figure 8) proved
to be significantly better in all
aspects of accuracy and time
Shared Classifier vs Tailored Classifier
CNN #1: Caffe CNN #1: TensorFlow
The first experiment consisted of testing two different Convolutional Neural Network
models – Caffe and TensorFlow. Both models were trained on the same dataset by default:
ImageNet. The pie charts figure 4 and figure 5 show the most common recognition results
for both models. These results were incredibly inaccurate due to the under fitting training
datasets.
The second experiment (figure 6) shows results after re-training the default models with
labeled images from camera traps. With these trained labeled images, the model was able
to significantly increase the accuracy of recognition.
Recognition results of 12210 images
? ?
The computer does know what
these numbers mean, so we have to
‘teach’ it to recognize them (figure 7
and figure 8)
Time/date can be easily obtained with
the image’s EXIF data
Retrained in Four Categories
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Results
Figure 6
Re-training dataset:
• Bear: 170 images
• Deer: 489 images
• Coyote: 259 images
• Empty/other: 21 images
Figure 7 Figure 8
The diagrams below show two different forms of recognizing images using optical character
recognition. This is necessary in order to know the temperature of the image as shown in
figure 3. The first method depicted in figure 7 shows our original approach. However,
because of the shared “unknown classifier”, misclassification was common (e.g. 7 classified
as 1). With the new technique of using a “tailored classifier”, classification is always
accurate as proven in figure 9.
Figure 9 Figure 10
Recognition results of 12210 images
Re-trained TensorFlow
References:
¹ Zhong, Zhuoyao et al. “High Performance Offline Handwritten Chinese Character Recognition Using GoogLeNet and Directional Feature Maps.”
²ImageNet Large Scale Visual Recognition Challenge. IJCV, 2015.
Funded in part by NSF CCF-1539586
CSEP