Containing the Gear: Deep Dive on SELinux, Multi-tenancy, Containers & Security with Dan Walsh Presenter: Dan Walsh In this talk, Dan will do a deep dive into the Origin PaaS use of SELinux and containerization. He will discuss how SELinux being utilized to ensure that Origin is the the most secure PAAS available today. He will address some of his ideas for the future of Origin and SELinux. From 2013-04-14 OpenShift Origin Community Day in Portland, Oregon

1. OpenShift & SELinux
Dan Walsh
Twitter: #rhatdan
Blog: danwalsh.livejournal.com
Email: dwalsh@redhat.com

2. SELinux is a LABELING System
● Everything has a label
– Process,file,dir, chr_file, blk_file, port, node.
● SELinux Policy defines that access between
process labels and all other labels.
● The Kernel controls the access.

3. Security Goals
http://en.wikipedia.org/wiki/Maginot_line

4. SELinux is Type Enforcement
● system_u:system_r:openshift_t:s0:c1,c2
● SELinux is Type Enforcement
seinfo -t | grep openshift
openshift_mail_tmp_t, httpd_openshift_content_t, openshift_cgroup_read_tmp_t,
openshift_initrc_tmp_t, openshift_var_lib_t, openshift_var_run_t, openshift_app_t,
openshift_min_t, openshift_net_t, openshift_tmp_t, openshift_min_app_t,
openshift_net_app_t, openshift_cgroup_read_t, httpd_openshift_script_exec_t,
openshift_cron_tmp_t, openshift_initrc_t, httpd_openshift_script_t,
openshift_cron_exec_t, openshift_initrc_exec_t, openshift_rw_file_t,
openshift_log_t, openshift_cron_t, openshift_mail_t, openshift_port_t,
httpd_openshift_ra_content_t, httpd_openshift_rw_content_t,
httpd_openshift_htaccess_t, openshift_cgroup_read_exec_t, openshift_t,
openshift_tmpfs_t

5. SELinux is Type Enforcement
● Process Labels can be on Files
● File Labels can not on Processes
● openshift_t -> Process
● openshift_var_lib_t -> File

6. SELinux is MCS
● system_u:system_r:openshift_t:s0:c1,c2
● Multi Category System
● MCS Separation is for like types, but totally separated
● openshift_t:s0:c1,c2 -> openshift_var_lib_t:s0:c1,c2
● openshift_t:s0:c3,c4 -> openshift_var_lib_t:s0:c3,c4

7. Libvirt – Dynamic Labeling in action
openshift_t:MCS1
openshift_t:MCS2
Kernel
SELinux
Host Hardware
memory, storage, etc.
openshift_t:MCS1 openshift_t:MCS2

8. MCS Labeling based on UID
def gen_level(uid):
SETSIZE=1023
TIER=SETSIZE
ORD=uid;
while ORD > TIER:
ORD = ORD - TIER;
TIER= TIER - 1;
TIER = SETSIZE - TIER;
ORD = ORD + TIER;
return "s0:c%d,c%d" % (TIER, ORD)

9. How do the labels get on gears
● Host receives packet for a gear
– OpenShift server
● launches application with correct SELinux label.
● Sends packet to application
● If connection comes in via git or ssh
– Ssh uses pam_openshift
● Launch sh with correct context
● Launch git with correct context

10. DEMO

11. Monitoring Logs

12. Problems with OpenShift Security
● Gear Application == Administrator of Gear
– Same UID
– Same SELinux Label openshift_t
● Solution:
– openshift_t
● Administrator of gear
– openshift_app_t
● Type of the application
– openshift_var_lib_t
● openshift_t can read/write/execute
● openshift_app_t can read/execute
– openshift_rw_file_t
● openshift_t & openshift_app_t can read/write/execute

13. Problem with OpenShift Security
● All gears run as openhift_t
– All have same network access.
● openshift_t/openshift_app_t
● openshift_net_t/openshift_net -app_t
● openshift_min_t/openshift_min_app_t

14. What about trust between nodes.
● IPTables not enough
● Node1:Gear1 can not attack Node1:gear2
● Node1:Gear1 can attack Node2:gear2
● Labeled Networking between Nodes
– Based on MLS CIPSO Labels
● Labeled Networking SELinux rules
– Node1:Gear1 can use Node2:gear1
– Node1:Gear1 attacking Node2:gear2 blocked
● Requires UID being the same between nodes.

15. Problems with SELinux
Confinement
● Node Separation
– 127.0.0.1 blocked to all.
– We do not want multiple Domains binding to
127.0.0.1:8080
● First one wins
● Apps trying to do SELinux stuff
● SELinux blocks access to processes but it
knows they are there.

16. Secure Linux Containers

17. Containers != Security
● Running root in a container, machine pwned
● Local Privilege Escalation, machine pwned
● Much of the system is not containerized.
– Audit
– /sys
● selinuxfs, cgroupfs, sysfs
– Need to block mount
– Need to block mknod

18. Linux Namespaces
● Mount : mounting/unmounting filesystems
– Currently used by Openshift for /tmp, /var/tmp and /dev/shm
● UTS : hostname, domainname
● IPC : SysV message queues, semaphore/shared memory
segments
● Network: IPv4/IPv6 stacks, routing, firewall, proc/net
/sys/class/net directory trees, sock
– Critical to fix localhost problem
● Pid: Private /proc, multiple pid 1's
● UID: Just showing up in the Kernel now..

19. Libvirt-lxc
● Boot “init” binary
● SELinux Types + MCS
● Firewall ebtables/ip[6]tables
● Host FS passthrough bind mounts
● CGroups resource control
● Available in RHEL6.4
– But your on your own...

20. virt-sandbox
● Package to help managing Linux Containers

21. DEMO