1. Your Inner Sysadmin
Chris Tankersley
@dragonmantank
MidwestPHP 2015
MidwestPHP 2015 1

2. Who Am I
• PHP Programmer for over
10 years
• Sysadmin/DevOps for
around 8 years
• Using Linux for more than
15 years
• https://github.com/dragon
mantank
MidwestPHP 2015 2

3. Here Be Dragons
MidwestPHP 2015 3

4. Traditional Lamp Stack
MidwestPHP 2015 4

5. Our Players
MidwestPHP 2015 5

6. And of course…
MidwestPHP 2015 6

7. The Server
• /bin - Essential user executable files
• /boot - Stuff that makes the OS boot up!
• /dev - Special device stuff you probably won't
touch
• /etc - Configuration files
• /home - User home directories
• /sbin - System binaries
• /usr - Multi-user apps and utilities
• /var - Data usually lives here
MidwestPHP 2015 7

8. Installing Software
• Compile software from scratch
• Use the package manager (yum/apt)
MidwestPHP 2015 8

9. Learn to love the Command
Line
MidwestPHP 2015 9

10. Learn a CLI text editor
• vi/vim
• emacs
• nano
MidwestPHP 2015 10

11. Authentication and
Authorization
MidwestPHP 2015 11

12. sudo
You can give admin access to users (or groups of
users) without giving them root.
MidwestPHP 2015
12
# Add sudo access to a single user to run as root
dragonmantank ALL=(ALL) ALL
# Add sudo access to a full group
%admin ALL=(ALL) ALL
You can even restrict what commands the users can run
# Restrict web developers to only restart Apache and MySQL
%webdevs 192.168.1.0/255.255.225.0=(root)
NOPASSWD:/usr/sbin/service apache2 restart, /usr/sbin/service
mysql restart

13. Jailing Users
Keeps people from getting to things they shouldn't.
Protects the users from themselves.
MidwestPHP 2015 13

14. Jailed Shells
Gives users a full shell but not the entire file system.
You can pick and choose what programs the user can
have access too. Jailkit makes this incredibly easy to
set up.
MidwestPHP 2015 14

15. Jailed SFTP
Locks the user to a specific base path, but doesn’t
give them a shell, much like FTP. You get the security
of SSH though! It does require a system user
however.
MidwestPHP 2015 15

16. Jailing SFTP
# In /etc/ssh/sshd_config
Subsystem ftp sftp-internal
# At the bottom of the file
Match User jailedsftp
ChrootDirectory /some/path
AllowTCPForwarding no
X11Forwarding no
ForceCommand internal-sftp
MidwestPHP 2015 16

17. Docker
MidwestPHP 2015 17
If you do it the non-Docker way

18. Scripting Languages
MidwestPHP 2015 18

19. Bash
Most servers use bash as the default shell. Most
shells understand bash's syntax. If you find yourself
running the same commands over and over, throw it
in a bash script.
MidwestPHP 2015 19

20. Python
Ships with most distros.
Great for when you need
more power than what
bash has.
MidwestPHP 2015 20

21. PHP!
Leverage your PHP skills to write shell scripts.
• Symfony Console Component
• Aura CLI
MidwestPHP 2015 21

22. Locking Down your Code
MidwestPHP 2015 22

23. Running Apache as a different
user
MPM-ITK
MidwestPHP 2015 23
MOD_RUID2
<IfModule mpm_itk_module>
AssignUserId [user] [user]
</IfModule>
RMode config
RUidGid myuser mygroup
RDocumentChRoot
/var/www/vhosts/domain.com/
www/public

24. PHP-FPM
user = myuser
group = mygroup
chroot = /path/to/my/chroot
MidwestPHP 2015 24

25. Logs
MidwestPHP 2015 25

26. Logrotate
Rotates logs out for organization (or other purposes)
MidwestPHP 2015 26
weekly
rotate 4
create
include /etc/logrotate.d
/var/log/wtmp {
monthly
minsize 1M
create 0664 root utmp
rotate 1
}

27. Logwatch
Script that runs every so often and scans a bunch of logs
so you get a pretty e-mail with a summary of events
MidwestPHP 2015 27
--------------------- httpd Begin ------------------------
0.17 MB transferred in 792 responses (1xx 0, 2xx 786, 3xx 0, 4xx 6, 5xx 0)
199 Content pages (0.09 MB),
593 Other (0.09 MB)
Requests with error response codes
400 Bad Request
/w00tw00t.at.ISC.SANS.DFind:): 1 Time(s)
404 Not Found
/MyAdmin/scripts/setup.php: 1 Time(s)
/phpmyadmin/scripts/setup.php: 1 Time(s)
/w00tw00t.at.blackhats.romanian.anti-sec:): 1 Time(s)
/webdav/: 2 Time(s)
---------------------- httpd End -------------------------

28. OSSEC
Actually a Host Intrusion Detection system, but it does
this by watching logs. Will alert you immediately to
problems, and even shut down the attacks.
MidwestPHP 2015 28
OSSEC HIDS Notification.
2012 Oct 24 11:38:10
Received From: maple->/var/log/auth.log
Rule: 5712 fired (level 10) -> "SSHD brute force trying to get
access to the system."
Portion of the log(s):
Oct 24 11:38:09 maple sshd[1062]: Failed password for invalid
user alias from 199.167.138.44 port 59988 ssh2
Oct 24 11:38:07 maple sshd[1062]: Invalid user alias from
199.167.138.44
Oct 24 11:38:06 maple sshd[1059]: Failed password for invalid
user recruit from 199.167.138.44 port 59884 ssh2

29. Preventing Intruders
MidwestPHP 2015 29

30. hosts.deny and hosts.allow
Set of files to allow or deny access to the machine or
certain apps/ports on the machine
MidwestPHP 2015 30

31. IPTables
A firewall that is generally available on Linux
machines that can be configured many different ways
to allow or block or mangle traffic
MidwestPHP 2015 31

32. OSSEC
IDS that was logs and will use hosts.deny and
iptables to block stuff automatically for you!
MidwestPHP 2015 32

33. Configuration Management
MidwestPHP 2015 33

34. What is Configuration
Management?
Process by which you figure out what goes on your
servers, how you want them set up, and keeping
track of that information. Files are usually stored in
source control on one server and pushed to clients.
MidwestPHP 2015 34

35. Why do you need it?
• Ever needed to keep track of when files get
changed?
• Ever needed to roll back a change?
• Ever needed to push the same change to a bunch
of servers
• Ever needed to set up a server exactly the same
way as another server?
MidwestPHP 2015 35

36. General CM Workflow
MidwestPHP 2015 36
Write a Manifest
file
Client checks and
compiles the
manifests
Client makes
changes based on
manifests

37. Ansible
• https://serversforhackers.com/getting-started-
with-ansible/
MidwestPHP 2015 37

38. Puppet
• http://www.erikaheidi.com/page/vagrant
MidwestPHP 2015 38

39. Server Monitoring
MidwestPHP 2015 39

40. Quick Poll
• Who here knows that their server is up right now?
• Are all of the required services running?
• Are there enough resources currently available?
MidwestPHP 2015 40

41. Service Monitoring with Monit
MidwestPHP 2015 41

42. Host Monitoring with Icinga
MidwestPHP 2015 42

43. Software Tools
MidwestPHP 2015 43

44. tmux/screen
Command line multiplexer
MidwestPHP 2015 44

45. tail
Look at the newest entries in a log, or even watch log
files as they are generated
MidwestPHP 2015 45

46. curl
Command line program for transferring data via a URL
MidwestPHP 2015 46

47. iftop
Displays a breakdown of bandwidth usage by host
MidwestPHP 2015 47

48. htop
Slightly better interface for checking memory and
CPU usage
MidwestPHP 2015 48

49. tcpdump
Allows you to view and record data transmitted over
the network. Couple this with wireshark and you can
inspect the packets!
MidwestPHP 2015 49

50. Servers for Hackers
Chris Fidao
@fideloper
http://serversforhackers.
com
MidwestPHP 2015 50

51. Questions?
MidwestPHP 2015 51

52. Thank You!
http://ctankersley.com
chris@ctankersley.com
@dragonmantank
https://joind.in/13069
MidwestPHP 2015 52