This document summarizes an approach for unconstrained activity recognition in an office environment. Two PTZ cameras are used to track faces and recognize people, keeping them centered in the frame. An action dataset of 863 samples covering five actions was collected from two viewpoints. DenseTrack features including HOG, HOF, and MBH were extracted and an SVM classifier with k-fold cross validation achieved an average accuracy of 67% for action classification. Facial recognition using Fisherfaces was more reliable than other algorithms for tagging actions with people.

1. We use bag-of-words (BoW) to encode our DenseTrack interest points. Using k-means
over all five actions, we create a codebook of size 2000. We used SVM with Radio Bases
Function (RBF) kernel to classify our dataset. SVM finds the optimal hyper-plane by
maximizing the margin from the supporting vectors of each action dataset. Using k-fold
cross validation we split the data into train and test and achieved an average F1-score of
%67.
In order to continuously track a person we run
Viola Jones once every 20 frames and use
Camshift method for the remaining frames to
track more efficiently. Camshift continuously
uses and adapts the Mean-Shift algorithm which
creates a confidence map in the new image based
on the color histogram of the previous object in
the previous image.
We present a systematic approach to automatically detect and classify limited number
of human actions in an office environment. Our setup includes two Pan-Tilt-Zoom
(PTZ) network cameras that track faces and recognize people using linear discriminant
analysis (LDA). The subjects are tracked in the image plane and the PTZ camera
parameters are updated in real time to keep the person at the center of the image. Our
office dataset includes 863 samples covering five actions (Interaction between two or
more people, walking, sitting, writing on a whiteboard, and getting coffee) from two
different viewpoints. A set of spatio-temporal visual features are computed to represent
these actions. DenseTrack features include Histogram of Oriented Gradients (HOG),
Histogram of Optical Flow (HOF), Motion Boundary Histogram (MBH) and trajectory
data. Using support vector machine (SVM) we train and test DenseTrack features with
k-fold cross validation to achieve an average accuracy of 67%.
The purpose of face recognition is to tag actions with corresponding people. Three
different algorithms (Eigenface, Fisherface, and Local Binary Pattern Histogram) are
implemented, however Fisherface shows to be more reliable in our setup. This is due to
the fact that it uses LDA to differentiate features
Both PTZ-213 cameras used in our experiment are installed in the back corner of the
office ceiling. Cameras are connected to a secured network and are controlled with
TCP/IP. Each camera is initialized at a different predefined location (entrance, near the
coffee machine). Once a person is detected, the tracker computes the pan, tilt, and zoom
to keep the person in the center of the screen in real time until there is no motion for a
predefined duration of time (10 sec).
Unconstrained Activity Recognition in an Office Environment
Christopher Ray Ramirez, Parker James Paul Sankey, California State University San Bernardino
Amir M. Rahimi, B.S. Manjunath, Department of Electrical and Computer Engineering
Abstract
Face Detection
Acknowledgements
For face detection we used Viola
Jones/Haar Training algorithm. This
algorithm has a bag of features that is
used to compare darker and lighter
regions in frame. This selects the
region to be used in tracking the face.
Conclusion
Camera Setup
Tracking
This work was supported in part by the National Science Foundation
through grant number IIS-0808772 and by the U.S. Office of Naval
Research N00014-12-1-0503.
We implemented a systematic framework
that automatically detects, recognize, track,
and classify human actions in an
unconstrained office environment. Each
module (detection, recognition,
classification) is optimized separately for
the most reliable performance. In future
work higher order interactions can be
modeled among actions to model and
predict long sequence of actions (Activities).
Facial Recognition
Action Dataset
DenseTrack contains dense points and tracks them based off movements from a dense
optical flow field. By having HOG, HOF, and MBH, DenseTrack is robust to fast,
irregular movements as well as a efficient solution to remove camera motion.
Collecting/Annotating Dataset
Wang, Heng, Alexander Kläser, Cordelia Schmid, and Cheng-Lin Liu. "Dense Trajectories and Motion Boundary Descriptors for Action Recognition."
International Journal of Computer Vision 103.1 (2013): 60-79. Web. 11 Aug. 2014.
Performance (Face Detection/Recognition)
Feature Encoding and Classification