This document summarizes the Linux management practices at the William States Lee College of Engineering at UNC Charlotte. It discusses that the college IT team manages over 140 Linux machines including servers and desktops using Red Hat Enterprise Linux and automation tools like Kickstart, CFEngine, and scripts. They use OpenAFS for network storage and aim to provide high levels of customer service while automating management tasks and ensuring all systems are disposable to handle growth. Upcoming goals include transitioning to Red Hat Satellite and exploring new technologies like Docker.

1. Managing Linux in an
Engineering College
A Case Study

2. My opinions are my own.
The Lawyers Made Me Do It

3. About Me
Jason Edgecombe (@edgester)
● Linux Administrator in the College of
Engineering at UNC Charlotte
● OpenAFS contributor & buildbot admin
● http://engrmosaic.uncc.edu
● https://github.com/edgester
● http://rampaginggeek.com

4. The William States Lee College of
Engineering at UNC Charlotte
● 5 Departments
o Civil and Environmental Engineering
o Electrical and Computer Engineering
o Engineering Technology
o Mechanical Engineering and Engineering Science
o Systems Engineering and Engineering Management
● Various research centers (EPIC)

5. College Stats
● 3322 Students who major in COE (Fall 2014)
o 2710 undergrad, 612 grad
● 154 faculty (140 FTE), 60 staff FTE
● 4342 login accounts (May 2015)
● 10 person IT team (full-time) + student
interns
● 2 Linux admins

6. What We Support
● 2 Managed platforms
o Mosaic Windows
 1200+ Mosaic windows machines
o Mosaic Linux
 41 production servers
(mix of bare-metal and VM’s)
 65 Linux desktops (offices and research labs)
 141 total Linux machines (that we manage)
● Assorted standalone boxes (research)

7. What We DON’T Maintain
● The main campus network
● Primary departmental/college web sites
● Research compute cluster
● Other stuff provided by university-level IT
● Moodle
● Campus ERP systems

8. Mosaic Linux Platform
● 50+ commercial and open source software
suites
● RedHat Enterprise Linux 5 (Client & Server)
● Only IT staff have root/sudo
● OpenAFS for network storage
(Linux & Windows)

9. Key Philosophies
● High level of customer service
● Lots of automation
o Scripted installation and management.
(little or no imaging)
● Store everything in AFS (Windows & Linux)
o Roaming profiles and home directories
● Workstations and servers are disposable
o Primary disks must not contain persistent user data

10. Remote Access
● Current: 10 servers, 8C/16T, 96GB RAM
● Future: 2 servers, 8C/64T, 256GB RAM
● Remote graphical and ssh access
● Today: FreeNX over globally-accessible ssh
port
● Near Future: Xrdp and ssh behind the VPN
● Used by on-site and off-site users

11. Current Automation
● Kickstart automated installation
● Masterless cfengine w/AFS to distribute files
● Scripts (Bash, Perl, Python)
● Git for VCS and moving code changes
between environments
● Folders of RPMs to designate patchsets
(separate from git)

12. Future Automation/Platform
● RHEL7
● RedHat Satellite (Puppet, Katello, Foreman, Pulp)
○ Use lifecycle environments to promote
Dev→Test→Prod
○ Content views to synchronize puppet modules and
RPMs
○ Reduce the dependency on a shared file system for
our management tools

13. Ongoing Challenges & Responses
● Usual suspects: budget & staff time
○ Automate and streamline
○ Engage stakeholders to set priorities
○ Interns for some stuff
● Growth (12% student growth 2009-2014)
○ Automate
○ Set priorities
● Bimodal user populations
○ Set expectations per application

14. Ongoing Challenges & Responses 2
● Users want root/sudo (especially Elec. Eng.)
○ Better customer service
○ Make the managed offering irresistible
● Need to support centralized and distributed
usage models
○ Users can choose their support model
○ Support both with a hybrid team

15. Ongoing Challenges & Responses 3
● Divergent application requirements
○ Install dependencies in separate folders with
applications
○ Can’t always fix this. Can Docker help?
● Semi-trusted network
○ Assume the attackers are inside the machines!
○ Run fail2ban on workstations
○ Be wary of applications that have little/no security

16. Challenges Yet To Be Conquered!
● Research
○ Special requirements that can’t be handled by the
managed offering. (security & technical)
● Storage (More!)
● Security
○ Risk of more one-offs
○ Belligerent users
● Consolidation?

17. The Future Is Uncertain
● Looking to replace our:
o Network file system
o User Directory
o Backup system
● Docker
● Cloud
● Continuous Delivery
○ automated testing, logging, metrics

18. Discussion
http://engrmosaic.uncc.edu