The authors use system event monitoring to distinguish between the behavioral characteristics of normal and anomalous computer system users. Identifying anomalous behavior at the system event level diminishes privacy concerns and supports the identification of cross-application behavioral patterns.

1. 34 IT Pro July/August 2013 P u b l i s h e d by t h e I E E E C o m p u t e r S o c i e t y 1520-9202/13/$31.00 © 2013 IEEE
Jeffery Payne, Mark Fenner, and Jonathan Kauffman, Coveros
The authors use system event monitoring to distinguish between the
behavioral characteristics of normal and anomalous computer system
users. Identifying anomalous behavior at the system event level
diminishes privacy concerns and supports the identification of
cross-application behavioral patterns.
A
nomaly detection with system events
is used in a variety of intrusion-
detection systems.1–4,8
Our project
for DARPA’s Active Authentication
program uses system event monitoring to dis-
tinguish between the behavioral characteristics
of normal and anomalous computer system us-
ers. Identifying anomalous behavior in system
events has advantages over doing so at other lev-
els. First, there are fewer privacy concerns with
the information collected, because system event
streams don’t include the acted-on content. Sec-
ond, information over all applications and pro-
cesses is available to identify cross-application
behavioral patterns.
Our Approach
We perform active authentication by detecting
anomalies in a stream of operating system events
generated by a computer user. Figure 1 shows
how our system operates.
The process in Figure 1a begins when a comput-
er user generates system events as he or she per-
forms application-level operations. These events
are captured using either Microsoft’s Procmon or
Detours utilities. Procmon monitors a variety of
system events, including file system events, reg-
istry access, network events, and process/thread
activity. Detours can intercept Windows system
events individually. System events are filtered
based on their process type and event class. Initial
System Event
Monitoring
for Active
Authentication
Security: DARPA

2. computer.org/ITPro 35
experimentation led us to remove
system events from background
tasks. We filter events by class to
keep similar events together so
that detectors can evaluate a single
type of event.
Once filtered, system events
are analyzed using one or more
anomaly-detection learning sys-
tems—including one-class sup-
port vector machine (1SVM),
replicator neural network (RNN),
probabilistic similarity (PS), and
similarity matching. Anomaly-
detection learning systems5
dif-
fer from standard classification
learning systems, because they’re
trained on instances of a single
class and then determine wheth-
er new instances are members of
that original class. 1SVM is a spe-
cific implementation of SVMs to
handle anomaly detection.6
RNNs
use backpropagation training cou-
pled with a specific architecture
to reconstruct input examples as
output.7
The error in reconstruc-
tion is interpreted as an anomaly
score. PS, based on an approach
first proposed by Salvatore Stolfo
and his colleagues,7
is an anomaly
detector that identifies anomalies
based on the marginal distribu-
tions of tuples in a training set of
features. Similarity matching is a
baseline algorithm that matches
system event streams with previ-
ously seen streams.
In our approach, we break the
raw system event stream into
fixed-time chunks called blocks.
Each block provides a sequence
of overlapping subsequences, called windows, of
system events. The learning algorithms are applied
directly to these windows. Each window is labeled
normal or anomalous. We then compare the percent
of anomalous windows in the larger block against
a threshold and render an accept or reject decision
for that block. The results of multiple detectors are
combined using Boolean or counting operations.
At both the window and block levels, we can apply
a leaky bucket algorithm to smooth the detection
stream. When the system recognizes an anomaly,
it either sends an alert to a centralized monitoring
system or requires a manual re-authentication.
Figure 1b shows how we evaluated the accu-
racy of our anomaly-detection approach. After
performing training on a training block, we fed
our system both authorized user data (additional
system events from the same training user) and
Figure 1. A high-level depiction of the system-event-monitoring
approach: the (a) system-event-monitoring process and (b) approach for
determining false accept and false reject rates.
A computer user generates system calls. The system calls are captured and filtered.
Then the calls are analyzed
Procmon/Detours
Subsets
noise
Multilayer
perceptron
or support
vector
machine
Noise
reduction
rating
identity
Leaky bucket
thresholds
voting
Login prompt
centralized monitor
Authorized user Unauthorized user
Training block Self-test block Other-test block
Windows
Blocks
Streams
Anomaly
detector
(learn)
Anomaly
detector
(detect)
Anomaly
detector
(detect)
Threshold Compare number of
anomalous windows
to threshold
Compare number of
anomalous windows
to threshold
False reject Accept Reject False accept
(a)
(b)
to make a decision whether to re-authenitcate.

3. 36 IT Pro July/August 2013
Security: DARPA
unauthorized user data (system events from a dif-
ferent user). If a block from the training user was
rejected, the result was a false reject. If a block from
a different user was accepted, the result was a false
accept. Otherwise, the result was true (correct).
Empirical Results
To identify a set of system parameters that ex-
hibits good performance for a variety of users,
we developed an experimentation framework in
Python that evaluates various sets of parameters
for event streams produced by multiple users. To
collect these streams, we deployed Procmon on
seven test subjects’ workstations and collected
two hours of system events during their normal,
daily computer usage. Their typical tasks includ-
ed communication, software development and
configuration, and document creation.
Each participant produced an event stream that
was 2–4 Gbytes in size and comprised 3–8 million
records. We split each of these event streams into
two sets. We used the first set to tune the system
parameters. The experimentation framework uses
this first set as normal data for a particular partici-
pant while using the second set of each participant’s
event stream to represent anomalous behavior.
During initial experimentation, we explored
a variety of parameter settings to measure their
­effect on system performance. For most of the
system parameters, we didn’t observe a significant
change in performance as we varied their values.
However, using the registry subset of events, as
well as enhancing event representation to include
both the elapsed time since the previous event and
the name of the process that invoked the system
event, noticeably improved system performance.
Figure 2 presents our best results using a detec-
tion error trade-off (DET) curve for both individ-
ual users and all users as a whole. We produced
the graph by applying cross-validation to the first
dataset. DET curves show how the performance
of a single system changes as the threshold for
a block of anomalies varies. Each point on the
graph represents system performance, both in
terms of the false reject rate (FRR) and false ac-
cept rate (FAR) at a particular threshold value.
The unconnected points of the same shape show
the DET curve for each individual user, while the
thick line shows the DET curve for all users. To
quantify a DET curve, we measure its area under
the curve (AUC). A DET curve that’s pushed to-
ward the origin will have a low AUC measure, in-
dicating that the system is more accurate.
Our best results were obtained by aggregating
the verdicts produced by two separate detectors.
We tried several methods of combining verdicts and
found that a method based on our relative trust of
the two detectors produced the best results. In this
approach, one detector more reliably accepts an au-
thorized user, and another more reliably rejects an
unauthorized user. When one detector produces a
trusted verdict and the other produces an untrusted
verdict, we use the trusted verdict as the aggregate.
In the case where both detectors produce a
trusted verdict, we choose the verdict from the
detector we trust (a priori) the most. The detec-
tor we trust the most in this scenario is the one
that gave the most accurate verdicts during indi-
vidual testing of each detector. When both detec-
tors produce an untrusted verdict, we declare the
result as unknown. Depending on the threshold
value, this means that anywhere from 0–20 per-
cent of the testing results were removed. By con-
sidering the curves for individual users, we reach
an FRR of 0 percent and a FAR of approximately
10–15 percent for several users.
O
ur future research will focus on improv-
ing these results when applied to differ-
ent types of users. Experimentation will
examine the degree to which normal behavior
can be distinguished from malicious behavior as
opposed to simply normal versus abnormal use.
Figure 2. Results from multisensor analysis. The
unconnected points of the same shape show the
detection error trade-off (DET) curve for each
individual user, while the thick line shows the
DET curve for all users.
100
80
60
40
20
0
0 20 40 60 80
FRR
FAR
100

4. computer.org/ITPro 37
We’re currently collecting additional field test
data so we can compare it to malicious use-case
scenarios developed from real-world attacks. We
also plan on researching mechanisms for dynam-
ically modifying threshold values on the basis of
system performance.
References
1. A.K. Ghosh, J. Wanken, and F. Charron, “Detect-
ing Anomalous and Unknown Intrusions Against
Programs,” Proc. Computer Security Applications Conf.,
IEEE, 1999, pp. 259–267.
2. A.K. Ghosh, A. Schwartzbard, and M. Schatz, “Using
Program Behavior Profiles for Intrusion Detection,”
Proc. SANS Conf. and Workshop on Intrusion Detection and
Response, 1999.
3. A.K. Ghosh, A. Schwartzbard, and M. Schatz,
“Learning Program Behavior Profiles for Intrusion
Detection,” Proc. 1st Workshop on Intrusion Detection and
Network Monitoring (ID 99), Usenix, 1999, p. 6.
4. V. Chandola, A. Banerjee, and V. Kumar. “Anomaly
Detection: A Survey,” ACM Computing Surveys, vol. 41,
no. 3, 2009, article 15.
5. S. Hawkins et al., “Outlier Detection Using Replica-
tor Neural Networks,” Proc. 4th Int’l Conf. Data Ware-
housing and Knowledge Discovery (DaWaK 00), Springer,
2002, pp. 170–180.
6. B. Scholkopf et al., “Estimating the Support of a
High-Dimensional Distribution,” Neural Computation,
vol. 13, no. 7, 2001, pp. 1443–1471.
7. S. Stolfo et al., “A Comparative Evaluation of Two Al-
gorithms for Windows Registry Anomaly Detection,”
J. Computer Security, vol. 13, no. 4, 2005, pp. 659–693.
8. S. Forrest et al.,“A Sense of Self for Unix Processes,” Proc.
IEEE Symp. Security and Privacy, IEEE, 1996, pp. 120–128.
Jeffery Payne is CEO and founder of Coveros and is a princi-
pal investigator on an Active Authentication project ­sponsored
by DARPA. Contact him at jeff.payne@coveros.com.
Mark E. Fenner is a research scientist at Coveros. Contact
him at mark.fenner@coveros.com.
Jonathan Kauffman is a consultant at Coveros, where
he’s performing research in Active Authentication. Contact
him at jonathan.kauffman@coveros.com.
handles the details
so you don’t have to!
Professional management and production of your publication
Inclusion into the IEEE Xplore and CSDL Digital Libraries
Access to CPS Online: Our Online Collaborative Publishing System
Choose the product media type that works for your conference:
Books, CDs/DVDs, USB Flash Drives, SD Cards, and Web-only delivery!
www.computer.org/cps or cps@computer.org
Contact CPS for a Quote Today!