An overview of two popular methods to detect Botnets: BotSniffer and DNSBL.

1. Botnet
Detection
A brief Incursion into Botnet Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
Anant Narayanan
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
Advanced Topics in Computer and Network Security
October 5, 2009
Botnet Detection

2. What We’re Going To Cover
Botnet
Detection
1
Introduction
Introduction
2
BotSniffer
Control Channels
Architecture
Algorithms
Results
3
DNSBL Method
Counter-intelligence
Reconnaissance
4
Conclusion
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
Botnet Detection

3. What Are Botnets?
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
Networks of “zombie” computers
The perpetrator compromises a series of systems using
various tools on existing security holes
Then, he simply controls these bots to do his bidding
Botnet Detection

4. Why Are They Bad?
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
Botnet Detection

5. How Do They Work?
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
PULL
HTTP(S) is the most commonly used protocol
A simple GET request at regular interval to receive
commands
PUSH
IRC(S) is the most commonly used protocol
All bots join a chat room and wait for commands
Botnet Detection

6. How Can We Stop Them?
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
Prevent computer from being infected in the first place?
Impractical, given the thousands of vulnerable machines
that will probably never be patched
Actively prevent commands from reaching bots, or prevent
bots from acting on those commands (use the network)
Passively detect a botnet’s presence and take offline action
Botnet Detection

7. Detecting C&C Traffic
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
Botnet C&C Traffic is difficult to detect because:
Uses normal protocols in ordinary ways
Traffic volume is low
Number of bots in a monitored network may be small
Traffic may use encrypted channels
Botnet Detection

8. Spatial-Temporal Correlation!
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Pre-programmed response activities
Command is sent to all bots around the same time
(especially true for PUSH models)
Bots process and usually perform some network operation
in response
Ordinary network traffic is unlikely to demonstrate such
synchronized or correlated behavior
Conclusion
Response Types
Message response: Execution result, status or progress
Activity response: Actual (malicious) network activity
Botnet Detection

9. Message Response (e.g., IRC PRIVMSG)
Activity Response (binary downloading)
(a) Message response crowd.
(b) Activity response crowd.
BotSniffer: Architecture
Figure 2. Spatial-temporal correlation and similarity in bot responses (message response an
response).
Botnet
Detection
Monitor Engine
Introduction
Reports
Activity
Response
Detection
Scan
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Malicious Activity
Events
Spam
Network
Traffic
Preprocessing
(WhiteList
WatchList)
Correlation
Engine
Binary
Downloading
Protocol
Matcher
HTTP/IRC
Connection
Records
HTTP
Activity Log
IRC
Conclusion
Network Traffic of
IRC PRIVMSG
Message Records
Message
Response
Detection
Incoming
PRIVMSG Analyzer
Outgoing
PRIVMSG Analyzer
Figure 3. BotSniffer Architecture.
Botnet Detection
Reports

10. Monitor Engine
Botnet
Detection
Preprocessing:
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
Unlikely protocols
White lists
Protocol Matcher
Currently focuses on IRC/HTTP
Message Response Detection
IRC PRIVMSG responses
Activity Response Detection
Abnormally high scan rates
Weighted failed connection rates
SMTP connections
Botnet Detection

11. Correlation Engine
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
First, the BotSniffer groups clients according to their
destination IPs and ports
Then, it perform correlation analysis on these groups
Botnet Detection

12. Group Activity Response
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
Response-Crowd-Density-Check
H0 → “Not Botnet”, H1 → “Botnet”, Yi → i th group member
Pr (Y1 , . . . , Yn |H1 )
Yi |H1
=
ln
Pr (Y1 , . . . , Yn |H0 )
Pr |H0
i
User chooses α (false positive rate) and β (false negative rate)
∧n = ln
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
Threshold Random Walk
When Yi = 1, increment by ln θ1
θ0
1−θ1
When Yi = 0, decrement by ln 1−θ0
If the walk reaches ln 1−β it is a botnet
α
β
If it reaches ln 1−α it is not
Otherwise, we watch the next round
Botnet Detection

13. Group Message Response
Botnet
Detection
Instead of looking at density, let’s look at homogeneity
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
Response-Crowd-Homogeneity-Check
Let Yi denote if the i th crowd is homogenous or not
Homogeneity is decided by the Dice factor
Dice(X , Y ) =
2|ngrams(X ) ∩ ngrams(Y )|
|ngrams(X )| + |ngrams(Y )|
Now, for q clients in the crowd, compare all unique pairs and
calculate their Dice distances. If (for eg.) > 50% are within a
threshold t, the crowd is marked as homogenous
Botnet Detection

14. Botnet
Detection
i.e., P r(X = i) = m pi (1 − p)m−i . Then the probability
i
of having more
Selecting q and tthan k similar pairs is P r(X ≥ k) =
m
m i
m−i
. If we pick k = mt where t is
i=k i p (1 − p)
the threshold to decide whether a crowd is homogeneous,
we obtain the probability θ(q) = P r(X ≥ mt).
1
Introduction
0.9
BotSniffer
is eve
reaso
thoug
the T
[17, 2
In
botne
round
0.8
Control
Channels
Architecture
Algorithms
Results
Counterintelligence
Reconnaissance
Conclusion
0.6
θ(q)
DNSBL
Method
0.7
where
negat
is no
messa
0.5
0.4
q=2,t=0.5
q=4,t=0.5
q=6,t=0.5
q=4,t=0.6
q=6,t=0.6
0.3
0.2
0.1
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
p
Figure 4. θ(q), the probability of crowd homogeneity with q responding clients, and
Botnet Detection
These

15. Single client detection
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
IRC
We can make use of the fact that IRC is a broadcast protocol
and apply the homogeneity check on incoming messages to a
single client
HTTP
Bots have strong periodical visiting patterns (to connect and
retrieve commands)
Botnet Detection

16. Did it Work?
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
Trace
IRC-1
IRC-2
IRC-3
IRC-4
IRC-5
IRC-6
IRC-7
IRC-8
All-1
All-2
All-3
All-4
All-5
trace size
54MB
14MB
516MB
620MB
3MB
155MB
60MB
707MB
4.2GB
6.2GB
7.6GB
15GB
24.5GB
duration
171h
433h
1,626h
673h
30h
168h
429h
1,010h
10m
10m
1h
1.4h
5h
Pkt
189,421
33,320
2,073,587
4,071,707
19,190
1,033,318
393,185
2,818,315
4,706,803
6,769,915
16,523,826
21,312,841
43,625,604
TCP flows
10,530
4,061
4,577
24,837
24
6,981
717
28,366
14,475
28,359
331,706
110,852
406,112
(IRC/Web) servers
2,957
335
563
228
17
85
209
2,454
1,625
1,576
1,717
2,140
2,601
FP
0
0
6
3
0
1
0
1
0
0
0
0
0
Table 1. Normal traces statistics (left part) and detection results (right columns).
Botnet Detection

17. Did it Work?
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
BotTrace
B-IRC-G
B-IRC-J-1
B-IRC-J-2
V-Rbot
V-Spybot
V-Sdbot
B-HTTP-I
B-HTTP-II
trace size
950k
26MB
15MB
66KB
6MB
37MB
duration
8h
1,267s
1,931s
533s
3.6h
19h
Pkt
4,447
143,431
262,878
347,153
180,822
474
65,695
395,990
TCP flow
189
103,425
147,921
14
237
790
Detected
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Table 2. Botnet traces statistics and detection results.
Botnet Detection

18. Passive Detection
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
DNSBL
DNS Blackhole Lists contain IP addresses that are sources
of spam. Botmasters sell bots not on any DNSBL at a
premium price
Thus, Botmasters themselves perform lookups on DNSBLs
to determine the status of their bots. Can we use this?
Botnet Detection

19. Heuristics
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
Spatial
A legitimate mail server will perform queries and be the object
of queries. Bots will only perform queries, they will be not be
queried for by other hosts
Botnet Detection

20. Heuristics
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
Temporal
Legitimate lookups are typically driven automatically when
emails arrive at the mail server and will this arrive at a rate
that mirrors arrival rates of emails
Botnet Detection

21. Types
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
Self Lookup: Each bot looks up it’s own DNSBL record.
Usually a dead giveaway, thus not used
Third-party Lookup: All bots are looked up by a single
dedicated machine. If that machine isn’t a mail server, we
can simply use Spatial heuristics and detect botnet
membership
Distributed Lookups: Each bot looks up a set of records
for other bots in the network. Complicated to implement
and spatial heuristics will fail. Temporal heuristics,
however, may help in detection
Botnet Detection

22. Thanks for Listening
Botnet
Detection
Introduction
BotSniffer
Control
Channels
Architecture
Algorithms
Results
DNSBL
Method
Counterintelligence
Reconnaissance
Conclusion
Detecting botnets is hard work, but certainly possible!
Questions?
Botnet Detection