Richard Cunningham provides tips for diagnosing problems on Linux servers. He outlines basic commands like ps, top, and lsof that can help identify issues. Top shows running processes and their resource usage, while lsof lists open files and network ports. Logs in /var/log/ contain troubleshooting information. Iostat and sar help monitor I/O and system performance over time. Strace shows system calls made by a process. Wireshark can analyze captured network traffic for troubleshooting.

1. How to Diagnose Problems Quickly on Linux Servers Richard Cunningham (@rythie)

3. Running Slow Two Types of Problems

4. Basics ps – List of processes (use ps -ef) top – Similar to ps but shows memory usage and is interactive dmesg – Messages directly from the kernel lsof – list open file and network ports /var/log/ - Where most/all of the logs live du – you may be out of disk space!

5. Understanding 'top' top - 17:59:20 up 272 days, 22:17, 1 user, load average: 0.27, 0.33, 0.29 Tasks: 142 total, 1 running, 141 sleeping, 0 stopped, 0 zombie Cpu(s): 1.4%us, 0.5%sy, 0.3%ni, 95.3%id, 1.9%wa, 0.0%hi, 0.1%si, 0.5%st Mem: 2051088k total, 1986640k used, 64448k free, 48156k buffers Swap: 4194300k total, 129028k used, 4065272k free, 810744k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1156 mysql 20 0 793m 467m 5060 S 2.0 23.4 16960:15 /usr/local/mysql-5.1 4130 richie 20 0 10916 940 668 R 2.0 0.0 0:00.01 top -cbn1 17311 backend 20 0 115m 40m 6240 S 2.0 2.0 12:54.17 /usr/local/bin/php 1 root 20 0 10408 508 480 S 0.0 0.0 6:32.75 init [3] 2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 [kthreadd] 3 root 20 0 0 0 0 S 0.0 0.0 5:56.83 [ksoftirqd/0]

6. /var/log ls -ltr /var/log – see most recently modified tail -f /var/log/message – watch for new lines at the end of a log file grep – most of those logs have too much irrelevant stuff in them, grep helps

7. Slowness iostat – I/O device statistics free – how much free memory you have sar – historical logs of performance

8. Understanding 'free' $ free -m total used free shared buffers cached Mem: 3019 2908 111 0 225 1187 -/+ buffers/cache: 1495 1524 Swap: 0 0 0

9. What are System Calls? open(), read(), write()... 011100101101010... Compiled Program (including libraries) Kernel Hardware

10. strace – typical output open("/etc/configfile.conf", O_RDONLY) = 3 read(3, "#this is the config file for...”, 4096) = 32 write(1, "hello", 5) = 5

11. strace - invocation strace -p <process id> strace <program name>

12. Wireshark

13. Wireshark

14. Wireshark

15. Wireshark Use tcpdump -s0 -w filename To capture network traffic to a file for later analysis in wireshark (i.e. copy it to you local machine) Wireshark available for OSX, Windows and Linux and is Open Source

16. The End Richard Cunningham (@rythie)