This document summarizes research on developing an algorithm for thermal-to-visible light face recognition. The algorithm performs preprocessing of images, uses a convolutional neural network to extract features from thermal and visible light images of the same individuals, and compares the images using a cost function to identify matches across modalities. The algorithm is still being developed and evaluated on its ability to accurately match faces between visible light and thermal images. The goal is to refine the cost function and neural network to improve the identification rate for cross-modal face recognition.

1. Thermal-to-Visible Light Face Recognition
Malik Majette
Dr. Xinyu Huang
North Carolina Central University
Abstract
Identifying face images across different modalities is a pressing issue in computer vision.
Face recognition contains broad applications with the military and civilian sectors. With a database
of images from different people in visible light and thermal settings we created an algorithm to
identify the underlying structural similarities between the two modalities. The algorithm addresses
cross-modal face recognition through preprocessing, the convolutional neural network (CNN),and
minimization. In Python we completed the preprocessing which included cropping, resizing, and
pairing with an image of the same individual from a different modality. We also normalized each
image by changing the image type to ‘float64’ and dividing by its bit specification to tailor to the
CNN supported Theano library. Using CNN the thermal and visible light images are then filtered,
max-pooled, and convolved separately and then compared using the cost function. Currently the
algorithm for the feature extraction and minimization is still being developed. The performance of
the proposed algorithm is evaluated through rank 1 identification between visible light and thermal
images of an identical face. We hope to continually increase the identification rate through the cost
function and backpropagation to develop a consistent algorithm for cross-modal face
identification.