Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

OpenShift & SELinux with Dan Walsh @rhatdan

3,414 views

Published on

Containing the Gear: Deep Dive on SELinux, Multi-tenancy, Containers & Security with Dan Walsh

Presenter: Dan Walsh
In this talk, Dan will do a deep dive into the Origin PaaS use of SELinux and containerization. He will discuss how SELinux being utilized to ensure that Origin is the the most secure PAAS available today. He will address some of his ideas for the future of Origin and SELinux.

From 2013-04-14 OpenShift Origin Community Day in Portland, Oregon

Published in: Technology
  • Be the first to comment

OpenShift & SELinux with Dan Walsh @rhatdan

  1. 1. OpenShift & SELinuxDan WalshTwitter: #rhatdanBlog: danwalsh.livejournal.comEmail: dwalsh@redhat.com
  2. 2. SELinux is a LABELING System● Everything has a label – Process,file,dir, chr_file, blk_file, port, node.● SELinux Policy defines that access between process labels and all other labels.● The Kernel controls the access.
  3. 3. Security Goals http://en.wikipedia.org/wiki/Maginot_line
  4. 4. SELinux is Type Enforcement● system_u:system_r:openshift_t:s0:c1,c2● SELinux is Type Enforcementseinfo -t | grep openshiftopenshift_mail_tmp_t, httpd_openshift_content_t, openshift_cgroup_read_tmp_t,openshift_initrc_tmp_t, openshift_var_lib_t, openshift_var_run_t, openshift_app_t,openshift_min_t, openshift_net_t, openshift_tmp_t, openshift_min_app_t,openshift_net_app_t, openshift_cgroup_read_t, httpd_openshift_script_exec_t,openshift_cron_tmp_t, openshift_initrc_t, httpd_openshift_script_t,openshift_cron_exec_t, openshift_initrc_exec_t, openshift_rw_file_t,openshift_log_t, openshift_cron_t, openshift_mail_t, openshift_port_t,httpd_openshift_ra_content_t, httpd_openshift_rw_content_t,httpd_openshift_htaccess_t, openshift_cgroup_read_exec_t, openshift_t,openshift_tmpfs_t
  5. 5. SELinux is Type Enforcement● Process Labels can be on Files● File Labels can not on Processes● openshift_t -> Process● openshift_var_lib_t -> File
  6. 6. SELinux is MCS● system_u:system_r:openshift_t:s0:c1,c2● Multi Category System● MCS Separation is for like types, but totally separated● openshift_t:s0:c1,c2 -> openshift_var_lib_t:s0:c1,c2● openshift_t:s0:c3,c4 -> openshift_var_lib_t:s0:c3,c4
  7. 7. Libvirt – Dynamic Labeling in action openshift_t:MCS1 openshift_t:MCS2 Kernel SELinux Host Hardware memory, storage, etc. openshift_t:MCS1 openshift_t:MCS2
  8. 8. MCS Labeling based on UIDdef gen_level(uid): SETSIZE=1023 TIER=SETSIZE ORD=uid; while ORD > TIER: ORD = ORD - TIER; TIER= TIER - 1; TIER = SETSIZE - TIER; ORD = ORD + TIER; return "s0:c%d,c%d" % (TIER, ORD)
  9. 9. How do the labels get on gears● Host receives packet for a gear – OpenShift server ● launches application with correct SELinux label. ● Sends packet to application● If connection comes in via git or ssh – Ssh uses pam_openshift ● Launch sh with correct context ● Launch git with correct context
  10. 10. DEMO
  11. 11. Monitoring Logs
  12. 12. Problems with OpenShift Security● Gear Application == Administrator of Gear – Same UID – Same SELinux Label openshift_t● Solution: – openshift_t ● Administrator of gear – openshift_app_t ● Type of the application – openshift_var_lib_t ● openshift_t can read/write/execute ● openshift_app_t can read/execute – openshift_rw_file_t ● openshift_t & openshift_app_t can read/write/execute
  13. 13. Problem with OpenShift Security● All gears run as openhift_t – All have same network access. ● openshift_t/openshift_app_t ● openshift_net_t/openshift_net -app_t ● openshift_min_t/openshift_min_app_t
  14. 14. What about trust between nodes.● IPTables not enough● Node1:Gear1 can not attack Node1:gear2● Node1:Gear1 can attack Node2:gear2● Labeled Networking between Nodes – Based on MLS CIPSO Labels● Labeled Networking SELinux rules – Node1:Gear1 can use Node2:gear1 – Node1:Gear1 attacking Node2:gear2 blocked● Requires UID being the same between nodes.
  15. 15. Problems with SELinux Confinement● Node Separation – 127.0.0.1 blocked to all. – We do not want multiple Domains binding to 127.0.0.1:8080 ● First one wins● Apps trying to do SELinux stuff● SELinux blocks access to processes but it knows they are there.
  16. 16. Secure Linux Containers
  17. 17. Containers != Security● Running root in a container, machine pwned● Local Privilege Escalation, machine pwned● Much of the system is not containerized. – Audit – /sys ● selinuxfs, cgroupfs, sysfs – Need to block mount – Need to block mknod
  18. 18. Linux Namespaces● Mount : mounting/unmounting filesystems – Currently used by Openshift for /tmp, /var/tmp and /dev/shm● UTS : hostname, domainname● IPC : SysV message queues, semaphore/shared memory segments● Network: IPv4/IPv6 stacks, routing, firewall, proc/net /sys/class/net directory trees, sock – Critical to fix localhost problem● Pid: Private /proc, multiple pid 1s● UID: Just showing up in the Kernel now..
  19. 19. Libvirt-lxc● Boot “init” binary● SELinux Types + MCS● Firewall ebtables/ip[6]tables● Host FS passthrough bind mounts● CGroups resource control● Available in RHEL6.4 – But your on your own...
  20. 20. virt-sandbox● Package to help managing Linux Containers
  21. 21. DEMO

×