Presented at All Things Open 2023
Presented by Peter Czanik - One Identity
Title: Sudo – Giving access while staying in control
Abstract: Sudo is used by millions to control and log administrator access to systems, but using the default configuration only, there are plenty of blind spots. Using the latest features in sudo let you watch some previously blind spots and control access to them. Here are four major new features, which arrived since the 1.9.0 release, allowing you see your blind spots:
- configuring a working directory or chroot within sudo often makes full shell access redundant
- JSON-formatted logs give you more details on events and are easier to act on
- relays in sudo_logsrvd make session recording collection more secure and reliable
- you can log and control sub-commands executed by the command run through sudo
Let us take a closer look at each of these.
Previously, there were quite a few situations where you had to give users full shell access through sudo. Typical examples include when you need to run a command from a given directory, or running commands in a chroot environment. You can now configure the working directory or the chroot directory and give access only to the command the user really needs.
Logging is a central role of sudo, to see who did what on the system. Using JSON-formatted log messages gives you even more information about events. What is even more: structured logs are easier to act on. Setting up alerting for suspicious events is much easier when you have a single parser to configure for any kind of sudo logs. You can collect sudo logs not only by local syslog, but also by using sudo_logsrvd, the same application used to collect session recordings.
Speaking of session recordings: instead of using a single central server, you can now have multiple levels of sudo_logsrvd relays between the client and the final destination. This allows session collection even if the central server is unavailable, providing you with additional security. It also makes your network configuration simpler.
Finally, you can log sub-commands executed from the command started through sudo. You can see commands started from a shell. No more unnoticed shell access from text editors. Best of all: you can also intercept sub-commands.
These are just a few of the most prominent features helping you to watch and control previous blind spots on your systems. See these and other possibilities in action in some live demos during our presentation.
Find more info about All Things Open:
On the web: https://www.allthingsopen.org/
Twitter: https://twitter.com/AllThingsOpen
LinkedIn: https://www.linkedin.com/company/all-things-open/
Instagram: https://www.instagram.com/allthingsopen/
Facebook: https://www.facebook.com/AllThingsOpen
Mastodon: https://mastodon.social/@allthingsopen
Threads: https://www.threads.net/@allthingsopen
2023 conference: https://2023.allthingsopen.org/
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Sudo – Giving access while staying in control
1.
2. Peter Czanik
Open Source Evangelist
One Identity
@PCzanik
Security Starts Here | One Identity - Restricted - Confidential
Sudo: Giving
access while
staying in control
3. • Working at the Budapest office of One Identity
(formerly known as Balabit)
• syslog-ng & sudo upstream
• Helping in RPM and FreeBSD packaging
• Blogger and speaker
About me
Security Starts Here | One Identity - Restricted - Confidential
4. • What is sudo?
• Some lesser-known features
• What’s new?
• JSON-formatted logging
• Relays
• Chroot, working directory
• Logging sub-commands
• Masking passwords
• Getting more precise information
Overview
Security Starts Here | One Identity - Restricted - Confidential
5. - Is sudo a prefix for administrative commands?
- Yes, but also a lot more:
• Control and log access
• Record and play back terminal input and output
• Modular: extend with your own code, now even in
Python
• It even has humor!
What is sudo?
6. • Default config
%wheel ALL=(ALL) ALL
• (Almost) all permissions to the wheel group
• Still useful:
• Controls access
• No shared password
• You see who did what
A basic /etc/sudoers
7. • Recording the terminal
• Playback
• Difficult to modify (not cleartext)
• Saved locally, therefore easy to delete with
unlimited access
• Sudo 1.9: central session recording using sudo_logsrvd
Session recording
8. • Propagates in real-time
• Can’t be modified locally
• Many limitations (aliases, etc.)
LDAP for central management
9. • Extending sudo using Python
• Using the same APIs as C plugins
• API: https://www.sudo.ws/man/sudo_plugin.man.html
• Python plugin documentation:
https://www.sudo.ws/man/sudo_plugin_python.man.html
• No development environment or compilation needed
Python support
10. • Accessing input and output from user sessions
• Python example:
IO logs API
11. • Fun, but not always politically correct :)
Defaults insults
czanik@linux-mewy:~> sudo ls
[sudo] password for root:
Hold it up to the light --- not a brain in sight!
[sudo] password for root:
My pet ferret can type better than you!
[sudo] password for root:
sudo: 3 incorrect password attempts
czanik@linux-mewy:~>
Insults
12. • Until now I talked about sudo <= 1.9.0 (most
“enterprise” & “LTS” distros)
• Since late 2020 most distros changed to 1.9.0 or
later
• Current version is 1.9.14
• Some 1.9.15 features mentioned at the end:
coming soon ☺
• Sudo installers:
https://www.sudo.ws/getting/packages/
Sudo versions
13. • Introduced in sudo 1.9.4
• Traditionally plain-text logs with minimal information
• Feature introduced due to old syslog constraints
• Nov 18 12:31:33 centos7sudo sudo[30666]: czanik : 3
incorrect password attempts ; TTY=pts/0 ; PWD=/home/czanik ;
USER=root ; COMMAND=/bin/bash
• Nov 18 12:31:43 centos7sudo sudo[30670]: czanik :
TTY=pts/0 ; PWD=/home/czanik ; USER=root ;
COMMAND=/bin/bash
• Nov 18 12:31:49 centos7sudo sudo[30670]: czanik : command
rejected by I/O plugin ; TTY=pts/0 ; PWD=/home/czanik ;
USER=root ; COMMAND=/bin/bash
JSON-formatted logs
Security Starts Here | One Identity - Restricted - Confidential
14. • JSON-formatted logs have more information in a structured format
Defaults log_format=json
Nov 18 12:40:30 centos7sudo sudo[30891]:
@cee:{"reject":{"reason":"command rejected by I/O
plugin","server_time":{"seconds":1605699630,"nanoseconds":9332939
11,"iso8601":"20201118114030Z","localtime":"Nov 18
11:40:30"},"submit_time":{"seconds":1605699620,"nanoseconds":130
500349,"iso8601":"20201118114020Z","localtime":"Nov 18
11:40:20"},"submituser":"czanik","command":"/bin/bash","runuser":"ro
ot","runcwd":"/home/czanik","ttyname":"/dev/pts/0","submithost":"cent
os7sudo.localdomain","submitcwd":"/home/czanik","runuid":0,"columns"
:118,"lines":60,"runargv":["/bin/bash"]}}
JSON-formatted logs
Security Starts Here | One Identity - Restricted - Confidential
15. • Use jq or similar to make logs more human readable on the terminal:
{
"sudo": {
"accept": {
"uuid": "616bc9efcf-b239-469d-60ee-deb5af8ce6",
"server_time": {
"seconds": 1643374700,
"nanoseconds": 222446715,
"iso8601": "20220128125820Z",
"localtime": "Jan 28 13:58:20"
},
"submituser": "czanik",
[…]
JSON-formatted logs
Security Starts Here | One Identity - Restricted - Confidential
16. • Logging:
• Syslog
• Audit plugin API – reachable also from Python for custom
logging
• Sudo 1.9.4 added logging to sudo_logsrvd
Defaults log_servers=172.16.167.150
Logging to sudo_logsrvd
Security Starts Here | One Identity - Restricted - Confidential
17. • Sudo_logsrvd sends logs to syslog
• “HOST” field shows where logs are coming from
Nov 18 12:40:16 centos8splunk.localdomain sudo[21028]: czanik : 3 incorrect
password attempts ; HOST=centos7sudo.localdomain ; TTY=pts/0 ;
PWD=/home/czanik ; USER=root ; COMMAND=/bin/bash
Nov 18 12:40:23 centos8splunk.localdomain sudo[21028]: czanik :
HOST=centos7sudo.localdomain ; TTY=pts/0 ; PWD=/home/czanik ; USER=root
; TSID=00000A ; COMMAND=/bin/bash
Nov 18 12:40:30 centos8splunk.localdomain sudo[21028]: czanik : command
rejected by I/O plugin ; HOST=centos7sudo.localdomain ; TTY=pts/0 ;
PWD=/home/czanik ; USER=root ; COMMAND=/bin/bash
• JSON formatting available
Logging to sudo_logsrvd
Security Starts Here | One Identity - Restricted - Confidential
18. • Sudo_logsrvd collects session recordings to a central
location
• Originally all sudo clients sent recordings directly
• Sudo version 1.9.7 introduced relay mode
• You can have multiple levels of relays to structure your
network
Using sudo_logsrvd in relay mode
Security Starts Here | One Identity - Restricted - Confidential
19. • Collect recordings even when central server is
unavailable (maintenance or network problem)
• Have a single network connection through the firewall
instead of granting each host access
• Run it on a gateway host to relay from networks
without direct Internet access, like AWS private
networks
Why relay mode?
Security Starts Here | One Identity - Restricted - Confidential
20. • Configuring the client or the central server is the same
• On the relay:
• Where to forward
• In case of unreliable networks, store first (default: false)
relay_host = 172.16.167.161
store_first = true
• TLS encryption available
Configuring relay mode
Security Starts Here | One Identity - Restricted - Confidential
21. • Previously, full root shell access was needed to start an
application from a user-inaccessible directory
• Full root access easily gained using chroot
• Starting with sudo 1.9.3, both can be configured from
/etc/sudoers
Using chroot and cwd
Security Starts Here | One Identity - Restricted - Confidential
22. • The chroot command needs root privileges
• Using with sudo, it is still possible to sudo chroot /
• Chroot support must be explicitly enabled in sudoers
Using chroot
Security Starts Here | One Identity - Restricted - Confidential
23. • If directory is not restricted in sudoers:
Defaults:%wheel runchroot=*
• sudo --chroot / -s can do the same ☺
• But at least it is nicely logged:
Sep 24 15:58:55 centos7sudo sudo[8149]: czanik :
TTY=pts/0 ; CHROOT=/ ; PWD=/home/czanik ;
USER=root ; TSID=00001G ; COMMAND=/bin/bash
Using chroot
Security Starts Here | One Identity - Restricted - Confidential
24. • Directory can be restricted in sudoers:
Defaults:%wheel runchroot=/var/lib/mock/epel7-
x86_64/root
• If chroot or a given directory is not allowed, it is logged:
Sep 25 08:43:32 centos7sudo sudo[2640]: czanik : user
not allowed to change root directory to
/an/interesting/directory ; TTY=pts/0 ;
CHROOT=/an/interesting/directory ; PWD=/home/czanik ;
USER=root ; COMMAND=/bin/bash
Using chroot
Security Starts Here | One Identity - Restricted - Confidential
25. • Before sudo 1.9.8, only session recording helped in
case of shell or editor access
• Watching recordings is boring and time consuming
• 1.9.8 introduced:
• Logging
• Intercepting
• Works in most cases (does not work for built-in
commands, etc.)
Logging and intercepting sub-commands
Security Starts Here | One Identity - Restricted - Confidential
26. • Enable with:
Defaults log_subcmds
• Turn on JSON formatting:
Defaults log_format=json
Logging sub-commands
Security Starts Here | One Identity - Restricted - Confidential
27. I Unnamed (Modified) Row 14 Col 1
czplaptop:/home/czanik # id
uid=0(root) gid=0(root) groups=0(root)
czplaptop:/home/czanik # ls /usr/share/syslog-ng/include/scl/
apache ewmm logmatic snmptrap
cee fortigate mbox solaris
checkpoint graphite netskope sudo
cim graylog2 nodejs sumologic
cisco iptables osquery syslogconf
collectd junos pacct system
default-network-drivers linux-audit paloalto telegram
discord loadbalancer rewrite websense
elasticsearch loggly slack windowseventlog
czplaptop:/home/czanik # exit
Logging sub-commands: editor screenshot
Security Starts Here | One Identity - Restricted - Confidential
28. • Log without logging subcommands:
Aug 30 13:03:00 czplaptop sudo[10150]: Czanik :
TTY=pts/1 ; PWD=/home/Czanik ; USER=root ;
COMMAND=/usr/bin/joe
Logging sub-commands
Security Starts Here | One Identity - Restricted - Confidential
30. • Log with JSON formatting:
Aug 30 13:29:28 czplaptop sudo[11740]:
@cee:{"sudo":{"accept":{"uuid":"18f25b2438-0c44-ddaf-a264-
c70998d319","server_time":{"seconds":1630322968,"nanoseconds":12453428
3,"iso8601":"20210830112928Z","localtime":"Aug 30
11:29:28"},"submit_time":{"seconds":1630322965,"nanoseconds":357407987,
"iso8601":"20210830112925Z","localtime":"Aug 30
11:29:25"},"submituser":"czanik","command":"/usr/bin/joe","runuser":"root","r
uncwd":"/home/czanik","ttyname":"/dev/pts/1","submithost":"czplaptop","subm
itcwd":"/home/czanik","runuid":0,"columns":80,"lines":24,"runargv":["joe","/et
c/issue"],"runenv":["LANG=en_US.UTF-
8","COLORTERM=truecolor","TERM=xterm-
256color","MAIL=/var/mail/root","PATH=/usr/sbin:/usr/bin:/sbin:/bin:/usr/local
/bin:/usr/local/sbin","LOGNAME=root","USER=root","HOME=/root","SHELL=/bin
/bash","SUDO_COMMAND=/usr/bin/joe
/etc/issue","SUDO_USER=czanik","SUDO_UID=1000","SUDO_GID=100"]}}}
Logging sub-commands
Security Starts Here | One Identity - Restricted - Confidential
31. • Can prevent applications from running
• Enabling is a two-step process in sudoers
Defaults intercept
• And the actual rule:
czanik ALL = (ALL) ALL, !/usr/bin/who
Intercepting sub-commands
Security Starts Here | One Identity - Restricted - Confidential
32. • Even if running a shell with full root access:
czanik@czplaptop:~> sudo -s
czplaptop:/home/czanik # who
Sorry, user czanik is not allowed to execute
'/usr/bin/who' as root on czplaptop.
bash: /usr/bin/who: Permission denied
Intercepting sub-commands
Security Starts Here | One Identity - Restricted - Confidential
33. • Visibility is not always good
• Session recordings can include passwords
Hiding passwords in recordings
Security Starts Here | One Identity - Restricted - Confidential
34. • Recordings are saved under /var/log/sudo-io/
• No sudo tool to display input
• Files are compressed
zless /var/log/sudo-io/00/00/01/ttyin
passwd bla^Mblabla^Mblabla^M^D
Hiding passwords in recordings
Security Starts Here | One Identity - Restricted - Confidential
35. • In /etc/sudoers:
Defaults !log_passwords
• Passwords are masked in session recordings:
passwd bla^M********^M********^M^D
Hiding passwords in recordings
Security Starts Here | One Identity - Restricted - Confidential
36. • The list pseudo-command allows regular users to list
other user’s privileges
• Audit without full admin access
• Introduced in sudo version 1.9.13
bla ALL=(ALL) list
Listing privileges
37. bla@czplaptop:~> sudo -l
bla's password:
Matching Defaults entries for bla on czplaptop:
always_set_home,
secure_path=/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:/usr/local/sbin,
env_reset, env_keep="LANG LC_ADDRESS LC_CTYPE LC_COLLATE LC_IDENTIFICATION
LC_TELEPHONE LC_ATIME LC_ALL LANGUAGE LINGUAS XDG_SESSION_COOKIE",
!insults, ignore_iolog_errors, log_output, log_input
User bla may run the following commands on czplaptop:
(ALL) list
bla@czplaptop:~> sudo -U czanik -l
Matching Defaults entries for czanik on czplaptop:
always_set_home,
secure_path=/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:/usr/local/sbin,
env_reset, env_keep="LANG LC_ADDRESS LC_CTYPE LC_COLLATE LC_IDENTIFICATION
LC_TELEPHONE LC_ATIME LC_ALL LANGUAGE LINGUAS XDG_SESSION_COOKIE",
!insults, ignore_iolog_errors, log_output, log_input
User czanik may run the following commands on czplaptop:
(ALL) ALL
List privileges
38. • The long list (-ll) option now also prints the file name
• Arrives in sudo version 1.9.15
List privileges
39. leap154b:~ # sudo -U mytest -ll
Matching Defaults entries for mytest on leap154b:
always_set_home, secure_path=/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin:/usr/local/sbin, env_reset,
env_keep="LANG LC_ADDRESS LC_CTYPE LC_COLLATE LC_IDENTIFICATION LC_MEASUREMENT LC_MESSAGES LC_MONETARY LC_NAME
LC_NUMERIC LC_PAPER LC_TELEPHONE LC_TIME LC_ALL LANGUAGE LINGUAS XDG_SESSION_COOKIE", insults, targetpw
User mytest may run the following commands on leap154b:
Sudoers entry: /etc/sudoers
RunAsUsers: ALL
Commands:
ALL
Sudoers entry: /etc/sudoers.d/foo
RunAsUsers: root
Commands:
ALL
List privileges
40. • The file name and line number for the rule in the
SOURCE filed of JSON-formatted logs
• Arrives in sudo version 1.9.15
"source":"/etc/sudoers:66:23"
"source":"/etc/sudoers.d/foo:1:15"
SOURCE in JSON logs
41. • Recent versions of sudo let you see and control a lot
more activities:
• More detailed, easier to use log messages
• Log sessions through relays
• Less need for root shells: chroot & cwd support
• Track and intercept sub-commands
• Password masking in logs
• Listing privileges
Summary
Security Starts Here | One Identity - Restricted - Confidential