Linux Intrusion Discovery v. 0.4 May 2005 Anton Chuvakin, Ph.D., GCIA, GCIH, GCFA Security Strategist http://www.chuvakin.org

Linux Intrusion Discovery

v. 0.4

May 2005

Anton Chuvakin, Ph.D., GCIA, GCIH, GCFA

Security Strategist

http://www.chuvakin.org

Outline Linux Overview : Battleground Linux Common Attacks and Intruder Behavior : What they will hit you with? First Suspicions : Traces and anomalies Confirming the Intrusion : Oh, it is REALLY “owned”! Conclusion : What to do after the panic subsides? 

Linux Overview : Battleground Linux

Common Attacks and Intruder Behavior : What they will hit you with?

First Suspicions : Traces and anomalies

Confirming the Intrusion : Oh, it is REALLY “owned”!

Conclusion : What to do after the panic subsides? 

Linux Linux ”profile”: Free Open source Widely deployed Great for servers Easy to use * Somewhat poorly coded** Result: great target for attackers from “script kiddiez” to pros * - somewhat  ** - at least, according to the xBSD fans 

Linux ”profile”:

Free

Open source

Widely deployed

Great for servers

Easy to use *

Somewhat poorly coded**

Result: great target for attackers from “script kiddiez” to pros

* - somewhat 

** - at least, according to the xBSD fans 

Common Linux Attacks Vulnerable network daemons RPC FTP HTTP/HTTPS Brute forcing passwords Web application and CGI attacks Sniffing Local console abuse See SANS “UNIX Top 10 Weaknesses” for more details

Vulnerable network daemons

RPC

FTP

HTTP/HTTPS

Brute forcing passwords

Web application and CGI attacks

Sniffing

Local console abuse

See SANS “UNIX Top 10 Weaknesses” for more details

What the attackers do? Close the holes Backdoors Trojans IRC Scanning and exploitation DoS attacks Sniffing Storing “warez” and pirated content Searching for credit cards

Close the holes

Backdoors

Trojans

IRC

Scanning and exploitation

DoS attacks

Sniffing

Storing “warez” and pirated content

Searching for credit cards

What do we want? Give you or your subordinates/colleagues tools and methods to tell that a system is likely compromised Not require any advanced security knowledge while still be effective Focus on performing simple actions and looking at their results Use locally run built-in commands (and some free tools) Likely not effective against advanced attackers  which is OK!

Give you or your subordinates/colleagues tools and methods to tell that a system is likely compromised

Not require any advanced security knowledge while still be effective

Focus on performing simple actions and looking at their results

Use locally run built-in commands (and some free tools)

Likely not effective against advanced attackers  which is OK!

Hack Omens Summary Groups of intrusions signs covered on the next slides: Resource waste System failures Unusual objects and traces Unusual networking “Something just doesn’t feel right!” 

Groups of intrusions signs covered on the next slides:

Resource waste

System failures

Unusual objects and traces

Unusual networking

“Something just doesn’t feel right!” 

Omens: Resource waste Slow system [anton@bmw anton]$ uptime 11:53pm up 41 days, 8:54, 1 user, load average: 12.14, 9.12, 7.09 Excessive memory use [anton@bmw anton]$ free total used free shared buffers cached Mem: 127820 108856 18964 38636 13860 21684 -/+ buffers/cache: 73312 54508 Swap: 336504 43788 292716 Missing disk space [anton@bmw anton]$ df Filesystem 1k-blocks Used Available Use% Mounted on /dev/hda1 2016016 2016000 1193 99% / Slow network connectivity [anton@bmw anton]$ ping

Slow system

[anton@bmw anton]$ uptime

11:53pm up 41 days, 8:54, 1 user, load average: 12.14, 9.12, 7.09

Excessive memory use

[anton@bmw anton]$ free

total used free shared buffers cached

Mem: 127820 108856 18964 38636 13860 21684

-/+ buffers/cache: 73312 54508

Swap: 336504 43788 292716

Missing disk space

[anton@bmw anton]$ df

Filesystem 1k-blocks Used Available Use% Mounted on

/dev/hda1 2016016 2016000 1193 99% /

Slow network connectivity

[anton@bmw anton]$ ping

Omens: Misc Failures Reboots [anton@bmw anton]$ uptime 10:05pm up 3 hours , 1:54, 2 user, load average: 0.14, 0.12, 0.09 Application crashes and errors VM: killing process spamassassin Application restarts Mar 14 05:22:32 bmw syslogd 1.3-3 : restart. Authentication failures Mar 14 19:02:04 bmw PAM_unix[29426]: authentication failure ; evil(uid=500) -> root for system-auth service Spontaneous system unavailability

Reboots

[anton@bmw anton]$ uptime

10:05pm up 3 hours , 1:54, 2 user, load average: 0.14, 0.12, 0.09

Application crashes and errors

VM: killing process spamassassin

Application restarts

Mar 14 05:22:32 bmw syslogd 1.3-3 : restart.

Authentication failures

Mar 14 19:02:04 bmw PAM_unix[29426]: authentication failure ; evil(uid=500) -> root for system-auth service

Spontaneous system unavailability

Omens: Unusual Objects Files/directories [root@bmw /tmp]# ls -la total 35 drwxrwxrwt 5 root root 15360 Mar 16 00:22 . drwx------ 2 root root 1024 Mar 16 00:22 ... Processes Accounts Connections From server, to client, too many Command output “ Hmm, why does it do that ?”  Log entries

Files/directories

[root@bmw /tmp]# ls -la

total 35

drwxrwxrwt 5 root root 15360 Mar 16 00:22 .

drwx------ 2 root root 1024 Mar 16 00:22 ...

Processes

Accounts

Connections

From server, to client, too many

Command output

“ Hmm, why does it do that ?” 

Log entries

Action Plan What do the above signs indicate? Nothing really ?  Maybe so, but let’s check! How to quickly confirm an intrusion? Using default system tools Open source programs And some built-in intelligence 

What do the above signs indicate? Nothing really ?  Maybe so, but let’s check!

How to quickly confirm an intrusion?

Using default system tools

Open source programs

And some built-in intelligence 

Actions Look for suspicious files Look for suspicious accounts Look for system corruption Look for suspicious networking Look for suspicious processes Look for weird log entries Look for misc other “weirdness”

Look for suspicious files

Look for suspicious accounts

Look for system corruption

Look for suspicious networking

Look for suspicious processes

Look for weird log entries

Look for misc other “weirdness”

Look for suspicious files Large files # find / -size +10000k –print Or # find / -size +10000k –mtime +7 -print Nobody’s files # find / -nouser -print SUID root files # find / -uid 0 –perm -4000 –print Weird file names (“. “,” “,”…”, etc) # find / -name “...“ –print

Large files

# find / -size +10000k –print

Or

# find / -size +10000k –mtime +7 -print

Nobody’s files

# find / -nouser -print

SUID root files

# find / -uid 0 –perm -4000 –print

Weird file names (“. “,” “,”…”, etc)

# find / -name “...“ –print

Look for suspicious accounts Privileged Accounts grep :0: /etc/passwd [root@bmw /tmp]# grep :0: /etc/passwd root:x:0:0:root:/root:/bin/bash sync:x:5:0:sync:/sbin:/bin/sync shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown halt:x:7:0:halt:/sbin:/sbin/halt operator:x:11:0:operator:/root: rewt:x:0:0:root:/dev/…:/bin/bash

Privileged Accounts

grep :0: /etc/passwd

Look for system corruption Installed software integrity # rpm –qa | sort # rpm –Va | sort File integrity: AIDE # aide --check File integrity: Tripwire # tripwire --check System integrity : Chkrootkit # chkrootkit

Installed software integrity

# rpm –qa | sort

# rpm –Va | sort

File integrity: AIDE

# aide --check

File integrity: Tripwire

# tripwire --check

System integrity : Chkrootkit

# chkrootkit

Look for suspicious networking Promiscuous / sniffers # ip link | grep PROMISC or # /sbin/ifconfig or # dmesg | grep promisc Listeners (to) # lsof –i # netstat –nap Connections ( from) # netstat –na ARP # arp –a bmw.chuvakin.org (10.10.230.12) at 00:90:27:9F:B5:8C [ether] on eth0

Promiscuous / sniffers

# ip link | grep PROMISC

or

# /sbin/ifconfig

or

# dmesg | grep promisc

Listeners (to)

# lsof –i

# netstat –nap

Connections ( from)

# netstat –na

ARP

# arp –a

bmw.chuvakin.org (10.10.230.12) at 00:90:27:9F:B5:8C [ether] on eth0

Look for suspicious processes Process list # ps –aux (./daemons, strange names, etc) Process details # cat /proc/13555 Utilized system components # lsof –p 13555 Daemons and services # chkconfig --list Kernel module list # /sbin/lsmod

Process list

# ps –aux

(./daemons, strange names, etc)

Process details

# cat /proc/13555

Utilized system components

# lsof –p 13555

Daemons and services

# chkconfig --list

Kernel module list

# /sbin/lsmod

Look for weird log entries RPC exploit attempts Oct 19 05:27:43 bmw rpc.statd[560]: gethostbyname error for ^X÷ÿ¿^X÷ÿ¿^Z÷ÿ¿^Z÷ÿ¿%8x%8x%8x%8x%8x%8x%8x%8x%8x%62716x%hn%51859x% HTTP attacks /scripts/..%2f../winnt/system32/cmd.exe?/c+dir SSL attacks [error] mod_ssl: SSL handshake failed (server bmw.chuvakin.org 443, client 10.0.0.10) (OpenSSL library error follows) [error] OpenSSL: error:1406908F:lib(20):func(105):reason(143) Auth failures (SSH, telnet, HTTP, FTP, POP3, IMAP, SQL, etc) Large quantities of errors Large/small log files

RPC exploit attempts

Oct 19 05:27:43 bmw rpc.statd[560]: gethostbyname error for ^X÷ÿ¿^X÷ÿ¿^Z÷ÿ¿^Z÷ÿ¿%8x%8x%8x%8x%8x%8x%8x%8x%8x%62716x%hn%51859x%

HTTP attacks

/scripts/..%2f../winnt/system32/cmd.exe?/c+dir

SSL attacks

[error] mod_ssl: SSL handshake failed (server bmw.chuvakin.org 443, client 10.0.0.10) (OpenSSL library error follows) [error] OpenSSL: error:1406908F:lib(20):func(105):reason(143)

Auth failures (SSH, telnet, HTTP, FTP, POP3, IMAP, SQL, etc)

Large quantities of errors

Large/small log files

Misc other “weirdness” Contents of .rhosts / .shost .forward /etc/inetd.conf or /etc/xinetd.* ~/.ssh/authorized_keys /tmp and /var/tmp Suspicious cron jobs (esp. “root”) Suspicious logged on users (“system”, “bin”, etc) File attributes (“lsattr –R /”)

Contents of

.rhosts / .shost

.forward

/etc/inetd.conf or /etc/xinetd.*

~/.ssh/authorized_keys

/tmp and /var/tmp

Suspicious cron jobs (esp. “root”)

Suspicious logged on users (“system”, “bin”, etc)

File attributes (“lsattr –R /”)

What the attackers do II Close the holes : system changes, application restarts Backdoors : system changes, broken commands, new servers Trojans : new programs, new application behavior IRC : network connections, servers Scanning and exploitation : network connections, new programs DoS attacks : network connections, system slow Sniffing : promiscuous, missing disk space Storing “warez” and pirated content: missing disk space, slow networking

Close the holes : system changes, application restarts

Backdoors : system changes, broken commands, new servers

Trojans : new programs, new application behavior

IRC : network connections, servers

Scanning and exploitation : network connections, new programs

DoS attacks : network connections, system slow

Sniffing : promiscuous, missing disk space

Storing “warez” and pirated content: missing disk space, slow networking

What have we learned? We can quickly look for known signs of intrusions We have a plan for doing that! It doesn’t require any expensive “security tools” Many regular computer users can be trained to do that

We can quickly look for known signs of intrusions

We have a plan for doing that!

It doesn’t require any expensive “security tools”

Many regular computer users can be trained to do that

Conclusion Is Linux Secure? Just “securable”! Let’s just help it a bit by looking for intrusion signs! Similar methods are available for Windows!

Is Linux Secure?

Just “securable”!

Let’s just help it a bit by looking for intrusion signs!

Similar methods are available for Windows!

Additional Resources SANS resources – Intrusion Discovery Checklists http://www.sans.org/score/checklists/ID_Linux.pdf http://www.sans.org/score/checklists/ID_Windows.pdf

SANS resources – Intrusion Discovery Checklists

http://www.sans.org/score/checklists/ID_Linux.pdf

http://www.sans.org/score/checklists/ID_Windows.pdf

Thanks for Viewing the Presentation Anton Chuvakin, Ph.D., GCIA, GCIH, GCFA http://www.chuvakin.org Author of “Security Warrior” (O’Reilly) – http://www.securitywarrior.org Read my blog at http:// chuvakin.blogspot.com Book on logs is coming soon! See http://www.info-secure.org for my papers, books, reviews and other security resources related to logs

Anton Chuvakin, Ph.D., GCIA, GCIH, GCFA

http://www.chuvakin.org

Author of “Security Warrior” (O’Reilly) – http://www.securitywarrior.org

Read my blog at http:// chuvakin.blogspot.com

Book on logs is coming soon!

See http://www.info-secure.org for my papers, books, reviews and other security resources related to logs