SANS @Night There's Gold in Them Thar Package Management Databases
Upcoming SlideShare
Loading in...5
×
 

SANS @Night There's Gold in Them Thar Package Management Databases

on

  • 201 views

 

Statistics

Views

Total Views
201
Views on SlideShare
152
Embed Views
49

Actions

Likes
0
Downloads
1
Comments
0

1 Embed 49

http://stuffphilwrites.com 49

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

SANS @Night There's Gold in Them Thar Package Management Databases SANS @Night There's Gold in Them Thar Package Management Databases Presentation Transcript

  • THERE’S GOLD IN 
 THEM THAR PACKAGE MANAGEMENT DATABASES! Phil Hagen phil@lewestech.com
 @PhilHagen
 plus.google.com/+PhilHagen Image: flickr.com/matthigh
  • WHY ARE WE HERE? Goals: Describe the value of package management databases during the course of a Linux system forensic examination Provide information with immediate benefit! 2
  • WHO IS THIS GUY? Forensic/infosec consultant Former DoD/IC contractor, USAF Comm Officer (USAFA CompSci) Course lead, FOR572: Advanced Network Forensics & Analysis Linux guy since Slackware needed a stack of floppies and an unsupported SCSI controller meant bootstrapping a kernel compilation 3
  • OPEN SOURCE: INFINITE POWER (ITTY BITTY LIVING SPACE) Downloading and compiling source code is an amazingly powerful aspect of most open-source software Not viable for the large scale of a production environment Does not address dependencies No common install/uninstall process Fun for the lab or a hobbyist environment, but a headache in an operational environment 4
  • PACKAGE MANAGEMENT SOFTWARE TO THE RESCUE! Dependencies, file manifests, install/uninstall/upgrade scripts Generally makes software management less of a headache! Many solutions in widespread use among various distributions: opkg (fork of ipkg): Embedded devices like QNAP NAS, etc. dpkg: Debian and Ubuntu tgz: Slackware RPM: RedHat, CentOS, Fedora,many more
 (part of Linux Standard Base) 5
  • …STILL NOT END-ALL/BE-ALL Most incorporate higher-layer software to handle automatic inclusion of dependencies and other “meta” functions and avoid the much-feared “dependency hell” or “RPM hell” apt-get / aptitude YUM RedHat Network Not focusing on this higher-layer software Some useful artifacts available - bad guys 
 known to “yum install nmap” 6
  • SCOPE FOR THIS PRESENTATION Just looking at RPM Most concepts apply to other package management standards Notably dpkg All examples created and tested on CentOS 6.5 Ideally: Same OS as subject (RPM library versions, etc) Technically: Same version of RPM, BDB, related libraries May have some success with unmatched versions, but beware!! Presentation notes (including all commands) published soon http://stuffphilwrites.com 7
  • SOURCES FOR RPM EVIDENCE Populates Berkeley database Lives in /var/lib/rpm/ Includes metadata for every RPM-controlled file User/Group ownership Mode (aka permissions) MD5 (er… SHA256) checksum File size Major/minor number (For entries in /dev/) Symbolic link string (aka “target”) for symlinks Modification time /var/log/yum.log May have useful timestamped history of install/remove/ upgrade actions /var/log/rpmpkgs* 8
  • PRACTICAL USAGE 1. Mount all partitions from subject filesystem under /mnt/subject/ $ mount | grep subject
 /dev/mapper/vg_centos6vm-lv_root on /mnt/ subject type ext4 (ro,noload)
 /dev/mapper/loop0p1 on /mnt/subject/boot type ext4 (ro,noload) 2. Run RPM commands with “--root /mnt/subject” option Don’t trust the rpm(1) binary from a suspect system! Note: This performs a chroot(2) for all operations - needs root user privileges for validation (not query) actions 9
  • USE CASE:
 FILE ORIGIN $ rpm -qf <filename> Identifies what package owns the specified file Useful to answer “where did this file come from?” or to identify a file as package-less $ rpm --root /mnt/subject -qf /usr/sbin/sshd
 openssh-server-5.3p1-94.el6.x86_64 $ rpm --root /mnt/subject -qf /etc/mail.rc
 Remember chroot! mailx-12.4-7.el6.x86_64 $ rpm --root /mnt/subject -qf /etc/crypttab
 file /etc/crypttab is not owned by any package 10
  • USE CASE:
 PACKAGE VALIDATION (1) $ rpm -V <packagename> Verifies contents of specified package Compares expected (database) to actual (filesystem) Displays files that failed >=1 check, noting which checks failed SM5DLUGT (Size, mode, MD5 (or SHA256!), major/minor nos, link target, user, group, mtime) Shows “?” in output if user running command lacks permission to check (e.g. read access to generate checksums) 11
  • USE CASE:
 PACKAGE VALIDATION (2) WARNING!!! Packages can include “verification scripts” which will execute when the “-V” option is used! You’re not planning to run arbitrary, unknown code on your forensic workstation/VM, are you? The chroot action needs root - these will execute as a child to the sudo process! Use the “--noscripts" option with “-V” to prevent this 12
  • USE CASE: PACKAGE VALIDATION (3) Some files are expected to change after installation: config files! Still show changes after installation, but denoted with a “c” character Missing files also noted in output as such $ sudo rpm --root /mnt/subject 
 -V openssh-server --noscripts
 <no output> $ sudo rpm --root /mnt/subject 
 -V sudo --noscripts
 S.5....T. c /etc/sudoers 13
  • USE CASE: PACKAGE VALIDATION (4) $ sudo rpm S.5....T. .M....... S.5....T. ....L.... ....L.... ....L.... ....L.... ..5....T. --root /mnt/subject -Va --noscripts
 c /etc/sudoers
 /proc
 c /etc/maven/maven2-depmap.xml
 c /etc/pam.d/fingerprint-auth
 c /etc/pam.d/password-auth
 c /etc/pam.d/smartcard-auth
 c /etc/pam.d/system-auth
 c /usr/lib64/security/
 classpath.security $ rpm -root /mnt/subject —V postfix --noscripts
 missing c /etc/postfix/master.cf 14
  • COOL FEATURE ALERT! Many packages are GPG-signed Independently verify package without using compromised or untrusted system However: RPM database contents not signed: Trojaned RPM package reports no anomalies! How can we use this great feature to our advantage? 15
  • VALIDATE FILESYSTEM
 AGAINST A PACKAGE FILE (1) The “-p” option runs validation checks between filesystem contents and RPM package file contents Signed package files can be GPG-verified! Avoids an untrusted RPM database entirely Prevents false negative validation from compromised RPM installations Relatively simple process: 1. Download trusted binary RPM file 2. Validate RPM file using GPG 3. Validate filesystem contents against package contents 16
  • VALIDATE FILESYSTEM
 AGAINST A PACKAGE FILE (2) Consider a system with a compromised RPM database, or a trojaned installation of the Apache web server software $ rpm --root /mnt/subject -V httpd --noscripts
 S.5....T c /etc/httpd/conf/httpd.conf $ wget "http://mirror.centos.org/centos-5/5.10/os/ x86_64/CentOS/httpd-2.2.3-82.el5.centos.x86_64.rpm"
 $ rpm -K httpd-2.2.3-82.el5.centos.x86_64.rpm
 httpd-2.2.3-82.el5.centos.x86_64.rpm: (sha1) dsa sha1 md5 gpg OK
 $ rpm --root /mnt/subject -Vp httpd-2.2.3-82.el5.centos.x86_64.rpm --noscripts
 S.5....T c /etc/httpd/conf/httpd.conf
 S.5....T /usr/sbin/httpd 17
  • REAL-WORLD USE CASES (1) Find all non-config files owned by an RPM that fails a verification check $ sudo rpm --root /mnt/subject -Va --noscripts | 
 grep -v " c "
 ...
 S.5....T /var/www/awstats/lang/awstats-tt-tr.txt
 S.5....T /var/www/awstats/lang/awstats-tt-tw.txt
 S.5....T /var/www/awstats/lang/awstats-tt-ua.txt
 .......T /var/www/awstats/lib/blacklist.txt
 S.5....T /var/www/awstats/lib/browsers.pm
 S.5....T /var/www/awstats/lib/browsers_phone.pm
 ... 18
  • REAL-WORLD USE CASES (2) File only config files owned by an RPM, which fail checksum verification $ sudo rpm --root /mnt/subject -Va --noscripts | 
 grep "^..5..... c "
 S.5....T c /etc/pam.d/sshd
 S.5....T c /etc/ssh/sshd_config
 S.5....T c /etc/openldap/slapd.conf
 S.5....T c /etc/sysconfig/ldap
 S.5..... c /etc/sysconfig/saslauthd
 S.5..... c /etc/security/limits.conf
 S.5....T c /etc/logrotate.conf
 SM5....T c /etc/snmp/snmpd.conf
 S.5....T c /etc/sysconfig/snmpd.options
 ... 19
  • REAL-WORLD USE CASES (3A) Find all files not owned by an RPM (This is going to be slow!) $ cat find_orphans.sh
 for file in $( sudo find /mnt/subject/etc -type f ); do
 file=$( echo "$file" | sed -e 's//mnt/subject//' )
 rpm --root /mnt/subject -qf "$file" 2>&1 | 
 grep 'package$|directory$' | 
 sed -E 's/^(error: )?file (.*)(: No such file or directory| is not owned by any package)/2/'
 done $ ./find_orphans.sh
 /etc/crypttab
 /etc/sysconfig/network
 /etc/sysconfig/keyboard
 /etc/sysconfig/iptables
 ... 20
  • REAL-WORLD USE CASES (3B) Find all files not owned by an RPM (This is going to be slow!) $ ./find_orphans.sh
 /etc/crypttab
 /etc/sysconfig/network
 /etc/sysconfig/keyboard
 /etc/sysconfig/iptables
 ... 21
  • COOL PARTY TRICK (1) ! Use the “--queryformat” option to output only relevant/useful fields from the RPM database Provides >150 different tags that can be output for package or for each file in a package Available tags vary by version - online documentation is terrible Use “--querytags” for listing specific to your version of RPM ! Consider “RPMDBtoTimeline”… 22
  • COOL PARTY TRICK (2)* $ for pkg in $( rpm --root /mnt/subject -qa ) ; do
 rpm --root /mnt/subject -q $pkg --queryformat 
 "[%{FILEDIGESTS}|%{FILENAMES}|0|%{FILEMODES:perms}|
 %{FILEUSERNAME}|%{FILEGROUPNAME}|%{FILESIZES}|0|
 %{FILEMTIMES}|0|0n]" | sed -e 's/^|/0|/' 
 -e 's/|0|d/|0|d/d/' -e ’s/|0|-/|0|r/r/' 
 done
 ...
 4398551f...a13988cf|/usr/share/doc/gamin-0.1.10/ callbacks.gif|0|r/rrw-r—r—|root|root|4514|0|1183556209|0|0
 0|/usr/lib64/libmenuw.so.5|0|lrwxrwxrwx|root|root|15|0| 1282146079|0|0
 0|/usr/share/cracklib|0|d/drwxr-xr-x|root|root|4096|0| 1308983949|0|0
 ... * “Cool” claim not valid at all parties. YMMV. 23
  • IN CONCLUSION RPM is a pretty cool way to eliminate known files from a Linux system examination Know the shortcomings in the RPM package database so you can mitigate them With a little shell scripting,
 you can develop useful tools to
 quickly and consistently
 minimize input data