One thing that most programmers do not take the time to understand is the servers that their application lives on. Most know a smattering of Apache configs, PHP configs, and basic information about the OS. This talk will deal with looking at tools that can help you quickly set up a server and how it can help you be a better developer. We'll look at tools like puppet for server management, OSSEC for log management, different command line tools, and nagios/monit for system monitoring.
2. Who Am I
• PHP
Programmer
for
over
10
years
• Sysadmin/DevOps
for
around
8
years
• Using
Linux
for
more
than
15
years
• hFps://github.com/dragonmantank
LonestarPHP
2015
2
7. The Server
•
/bin
-‐
EssenPal
user
executable
files
•
/boot
-‐
Stuff
that
makes
the
OS
boot
up!
•
/dev
-‐
Special
device
stuff
you
probably
won't
touch
•
/etc
-‐
ConfiguraPon
files
•
/home
-‐
User
home
directories
•
/sbin
-‐
System
binaries
•
/usr
-‐
MulP-‐user
apps
and
uPliPes
•
/var
-‐
Data
usually
lives
here
LonestarPHP
2015
7
12. SSH Keys
• SSH
generally
uses
a
Username/Password
• SSH
Keys
pass
a
public
key
to
the
server
• Can
use
a
single
key
for
mulPple
machines,
or
mulPple
keys
for
mulPple
machines
• More
secure
since
‘passwords’
cannot
be
stolen
LonestarPHP
2015
12
13. sudo
You
can
give
admin
access
to
users
(or
groups
of
users)
without
giving
them
root.
LonestarPHP
2015
13
#
Add
sudo
access
to
a
single
user
to
run
as
root
dragonmantank
ALL=(ALL)
ALL
#
Add
sudo
access
to
a
full
group
%admin
ALL=(ALL)
ALL
You
can
even
restrict
what
commands
the
users
can
run
#
Restrict
web
developers
to
only
restart
Apache
and
MySQL
%webdevs
192.168.1.0/255.255.225.0=(root)
NOPASSWD:/usr/sbin/service
apache2
restart,
/usr/sbin/service
mysql
restart
14. Jailing Users
Keeps
people
from
geang
to
things
they
shouldn't.
Protects
the
users
from
themselves.
LonestarPHP
2015
14
15. Jailed Shells
Gives
users
a
full
shell
but
not
the
enPre
file
system.
You
can
pick
and
choose
what
programs
the
user
can
have
access
too.
Jailkit
makes
this
incredibly
easy
to
set
up.
LonestarPHP
2015
15
16. Jailed SFTP
Locks
the
user
to
a
specific
base
path,
but
doesn’t
give
them
a
shell,
much
like
FTP.
You
get
the
security
of
SSH
though!
It
does
require
a
system
user
however.
LonestarPHP
2015
16
17. Jailing SFTP
#
In
/etc/ssh/sshd_config
Subsystem
ftp
sftp-‐internal
#
At
the
bottom
of
the
file
Match
User
jailedsftp
ChrootDirectory
/some/path
AllowTCPForwarding
no
X11Forwarding
no
ForceCommand
sftp-‐internal
LonestarPHP
2015
17
20. Bash
Most
servers
use
bash
as
the
default
shell.
Most
shells
understand
bash's
syntax.
If
you
find
yourself
running
the
same
commands
over
and
over,
throw
it
in
a
bash
script.
LonestarPHP
2015
20
21. Python
Ships
with
most
distros.
Great
for
when
you
need
more
power
than
what
bash
has.
LonestarPHP
2015
21
22. PHP!
Leverage
your
PHP
skills
to
write
shell
scripts.
• Symfony
Console
Component
• Aura
CLI
LonestarPHP
2015
22
27. Logrotate
Rotates
logs
out
for
organizaPon
(or
other
purposes)
LonestarPHP
2015
27
weekly
rotate
4
create
include
/etc/logrotate.d
/var/log/wtmp
{
monthly
minsize
1M
create
0664
root
utmp
rotate
1
}
28. Logwatch
Script
that
runs
every
so
oXen
and
scans
a
bunch
of
logs
so
you
get
a
preFy
e-‐mail
with
a
summary
of
events
LonestarPHP
2015
28
-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐
httpd
Begin
-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐
0.17
MB
transferred
in
792
responses
(1xx
0,
2xx
786,
3xx
0,
4xx
6,
5xx
0)
199
Content
pages
(0.09
MB),
593
Other
(0.09
MB)
Requests
with
error
response
codes
400
Bad
Request
/w00tw00t.at.ISC.SANS.DFind:):
1
Time(s)
404
Not
Found
/MyAdmin/scripts/setup.php:
1
Time(s)
/phpmyadmin/scripts/setup.php:
1
Time(s)
/w00tw00t.at.blackhats.romanian.anti-‐sec:):
1
Time(s)
/webdav/:
2
Time(s)
-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐
httpd
End
-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐
29. OSSEC
Actually
a
Host
Intrusion
DetecPon
system,
but
it
does
this
by
watching
logs.
Will
alert
you
immediately
to
problems,
and
even
shut
down
the
aFacks.
LonestarPHP
2015
29
OSSEC
HIDS
Notification.
2012
Oct
24
11:38:10
Received
From:
maple-‐>/var/log/auth.log
Rule:
5712
fired
(level
10)
-‐>
"SSHD
brute
force
trying
to
get
access
to
the
system."
Portion
of
the
log(s):
Oct
24
11:38:09
maple
sshd[1062]:
Failed
password
for
invalid
user
alias
from
199.167.138.44
port
59988
ssh2
Oct
24
11:38:07
maple
sshd[1062]:
Invalid
user
alias
from
199.167.138.44
Oct
24
11:38:06
maple
sshd[1059]:
Failed
password
for
invalid
user
recruit
from
199.167.138.44
port
59884
ssh2
31. hosts.deny and hosts.allow
Set
of
files
to
allow
or
deny
access
to
the
machine
or
certain
apps/
ports
on
the
machine
LonestarPHP
2015
31
32. IPTables
A
firewall
that
is
generally
available
on
Linux
machines
that
can
be
configured
many
different
ways
to
allow
or
block
or
mangle
traffic
LonestarPHP
2015
32
33. OSSEC
IDS
that
was
logs
and
will
use
hosts.deny
and
iptables
to
block
stuff
automaPcally
for
you!
LonestarPHP
2015
33
35. What is Configuration Management?
Process
by
which
you
figure
out
what
goes
on
your
servers,
how
you
want
them
set
up,
and
keeping
track
of
that
informaPon.
Files
are
usually
stored
in
source
control
on
one
server
and
pushed
to
clients.
LonestarPHP
2015
35
36. Why do you need it?
• Ever
needed
to
keep
track
of
when
files
get
changed?
• Ever
needed
to
roll
back
a
change?
• Ever
needed
to
push
the
same
change
to
a
bunch
of
servers
• Ever
needed
to
set
up
a
server
exactly
the
same
way
as
another
server?
LonestarPHP
2015
36
37. General CM Workflow
LonestarPHP
2015
37
Write
a
Manifest
file
Client
checks
and
compiles
the
manifests
Client
makes
changes
based
on
manifests
41. Quick Poll
• Who
here
knows
that
their
server
is
up
right
now?
• Are
all
of
the
required
services
running?
• Are
there
enough
resources
currently
available?
LonestarPHP
2015
41
50. tcpdump
Allows
you
to
view
and
record
data
transmiFed
over
the
network.
Couple
this
with
wireshark
and
you
can
inspect
the
packets!
LonestarPHP
2015
50