<ul><li>Thorsten Kellermann </li></ul><ul><li>Sun Microsystems </li></ul>X64 Work Shop  Linux Information Gathering
Agenda <ul><li>Linux Support Overview </li></ul><ul><ul><li>Support model and structure </li></ul></ul><ul><li>Data Collec...
Agenda (cont) <ul><li>Advanced Troubleshooting </li></ul><ul><ul><li>System Core Dump Capturing </li></ul></ul><ul><ul><li...
Linux Support Overview <ul><li>Linux Support from System TSC organization: </li></ul><ul><ul><li>EMEA: System TSC VSP </li...
Linux Support Overview (cont) <ul><li>Supported Linux Versions: </li></ul><ul><ul><li>Red Hat Enterprise Linux (RHEL) </li...
Linux Support Overview (cont) <ul><li>What is covered by support? </li></ul><ul><ul><li>Bugs within the OS or with Core ap...
Data Collection <ul><li>Entitlement information </li></ul><ul><ul><li>We need the entitlement for the Linux the customer i...
Data Collection (cont) <ul><li>Samples: </li></ul><ul><ul><li>Customer has a working and a non working system </li></ul></...
Data Collection (cont) <ul><li>Red Hat:  </li></ul><ul><ul><li>sysreport </li></ul></ul><ul><ul><ul><li>Mandatory for esca...
Data Collection (cont) <ul><li>Others </li></ul><ul><ul><li>Linux Explorer </li></ul></ul><ul><ul><ul><li>Most complete da...
Data Analyzing <ul><li>There is no automatic tool! </li></ul><ul><li>This presentation isn't complete at all. </li></ul><u...
Data Analyzing (cont) <ul><li>What packages are installed? Which version? </li></ul><ul><ul><li>RPM is the packages manage...
Data Analyzing (cont) <ul><li>Hardware/Firmware information </li></ul><ul><ul><li>lspci [-v[v[v[v[v]]]]] </li></ul></ul><u...
Data Analyzing (cont) <ul><li>Overview </li></ul>
System Core Dump Capturing <ul><li>No standard at the moment </li></ul><ul><ul><li>Kdump has find it's way into the mainst...
System Core Dump Capturing (cont) <ul><li>SLES 8 / 9 uses LKCD </li></ul><ul><ul><li>Based on an IBM/SGI implementation. <...
Setting up RHEL 3 & 4 Netdump <ul><li>Install Netdump Server </li></ul><ul><ul><li>install package netdump-server </li></u...
Setting up RHEL 5 Kdump <ul><li>Installed by Default </li></ul><ul><li>Configuration Dialog </li></ul><ul><ul><li>enable /...
Setting up SLES 8/9 LKCD <ul><li>Install required package: </li></ul><ul><ul><li>lkcdutils </li></ul></ul><ul><li>Edit /et...
Seting up SLES 10 Kdump <ul><li>Install needed packages: </li></ul><ul><ul><li>kexec-tools </li></ul></ul><ul><ul><li>kern...
Checking dump setup <ul><li>Check if everything fit together: </li></ul><ul><ul><li>Enable Magic SysRq feature temporarily...
Linux SysRq Feature <ul><li>The Magic SysRq Feature is somewhat similar to Stop-A on Solaris </li></ul><ul><li>It can forc...
Linux SysRq Feature (cont) <ul><li>Disabled by default, need to be enabled </li></ul><ul><ul><li>temporarily until next re...
Linux SysRq Feature (cont) <ul><li>Some Hotkeys: </li></ul><ul><ul><li>K </li></ul></ul><ul><ul><ul><li>call the Secure At...
Linux SysRq Feature (cont) <ul><li>Some Hotkeys (cont): </li></ul><ul><ul><li>b </li></ul></ul><ul><ul><ul><li>boots the s...
Crash Dump Analyzing <ul><li>Crash utility </li></ul><ul><ul><li>Support varios dump fomats </li></ul></ul><ul><ul><ul><li...
Crash Dump Analyzing (cont) <ul><li>You need to have the debug information of the kernel </li></ul><ul><li>Crash package n...
Troubleshoot a Hanging System <ul><li>Hard to troubleshoot due to lack of information </li></ul><ul><li>If a deadlocked ke...
Links <ul><li>External Sources: </li></ul><ul><ul><li>Linux Explorer http://www.unix-consultants.co.uk/examples/scripts/li...
Links <ul><li>Did you know http://www.google.com/linux? </li></ul>
X64 Work Shop  Linux Information Gathering <ul><li>Thorsten Kellermann </li></ul><ul><ul><li>[email_address] </li></ul></ul>
Upcoming SlideShare
Loading in...5
×

X64 Workshop Linux Information Gathering

2,370

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,370
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
41
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Effectively from the 1 st of October, 2006 the Partnership between Red Hat and Sun moves from an OEM partner to a Reseller. This means Sun only sells Red Hat products, but didn&apos;t provide Support for this products. Customers need to engage Red Hat directly to get Support. Sun has certified their Hardware so this is fully supported by Red Hat. At the 3 rd level we do not have any escalation path to Red Hat for subscriptions sold after the 30 th of September 2006.
  • It&apos;s always a question from Customer if to use drivers shipped with the Hardware from Sun or uses the build in drivers from the distribution. From the prospective of the Linux support Customer should use the build in drivers, otherwise it may end up in an unsupported installation. The sustaining of the Drivers on the other hand isn&apos;t really working perfectly, so it makes it easier to get Issues fixed by the Distribution vendors.
  • With newer releases, sysreport has a command line switch to collect dmidecode information. This should be enabled if available. Config.sh has several command line switches, please use always the switch &amp;quot;-A&amp;quot; to gather the complete possible set. There are several command line switches to exclude sensible data from the report. All data gathering tools may collect security information. The customer should be aware about this fact. But none of the tools collect user passwords from the /etc/shadow. For SuSE&apos;s siga report we also need to requested at least the /var/log/message* files as in the siga report is only a tail of 250 lines; This is unusable as even the boot process creates more lines in the messages.
  • There are many updates and security fixes for both, Red Hat and SuSE to being able to automatically check if all necessary patches are applied. So normally we first identify where the issue is related to and than start investigation of the packages and it change logs Red Hat Network RHN is a usable web interface for it, but you need to have an account to access this suite: https://rhn.redhat.com . Customer needs to get register at RHN to get online update (up2date).
  • The kernel version is a highly important information: Is this the latest release? Is it a custom compiled kernel? Fit the architecture to the machine? etc. RPM provides you with information about the the installed packages and the versions. It also have the capability to find changes to the system or if there are missing files etc. This is done by &amp;quot;rpm -qaV&amp;quot; S.5....T. c /etc/sysconfig/iptables-config S.5....TC /usr/share/system-switch-mail/functions.pyc S.5....TC /usr/share/system-switch-mail/switchmail_tui.pyc The flags in the first column showing what is different to that file: S file Size differs M Mode differs (includes permissions and file type) 5 MD5 sum differs D Device major/minor number mismatch L readLink(2) path mismatch U User ownership differs G Group ownership differs T mTime differs C selinux Context differs If available also the data from the sar report may be interesting (/var/log/sa/*) as it provides statistical information about the system load and usage etc.
  • dmidecode should be available by default on all Red Hat systems, it&apos;s part of the kernel-utils package. The hardware.py is a script which is used to provide the RHN with information of this system. It basically use dmidecode to get information and format them to a database friendly format. Sometimes the hardware.py is included into the sysreport, sometimes not. Having no idea why it makes sense to look if this file is included into the sysreport before asking the customer for this information. If you what to see the output of this script, just run the command /usr/share/rhn/up2date_client/hardware.py
  • Kdump is the dumping facility which has find it&apos;s way in the mainstream kernel. This will be used by many distributions in the future. At the moment it supports many architectures like i386 and x86_64. There is no support for IA64 architectures at the moment.
  • A short overview about the Kdump on SLES 10 could be found at http://ftp.suse.com/pub/people/tiwai/kdump-training/kdump-training.pdf
  • The dumps on the server will be written to the directory /var/crash. There should be enough disk space available, special when dumping machines with a huge amount of memory. As the core files normally are not that small, we suggest to use bzip2 to compress them before uploading. Also should be a md5sum provided to see if the dump is getting corrupted during up-/download. The communication between the server and the client is secured by using the ssh protocol, default port for communication is 6666 for the dump capture and 514 for syslog. Firewalled systems may need to adjust the configuration. The communication between the client and server is working on key authentication, make sure to run the command &amp;quot;service netdump propagate&amp;quot; on the client after installation (is also a good test if the configuration is set up correctly). If you need to start the services as system boot you need to enable this by using &amp;quot;chkconfig&amp;quot;. Did you know this command? Try &amp;quot;chkconfig --list&amp;quot; to see which services are started and stopped in which run level. You can change this by using &amp;quot;chkconfig &lt;service_name&gt; [on|off]&amp;quot;.
  • A complete description on setting up LKCD on SLES could be found at http://www.novell.com/coolsolutions/feature/15284.html
  • Some basic informations about the configuration could be again find in http://ftp.suse.com/pub/people/tiwai/kdump-training/kdump-training.pdf
  • We will discuss the magic SysRq feature on the next slides.
  • A complete description about the SysRq could be find in the kernel documentation (/usr/share/doc/kernel-docs/sysrq.txt or in the ./Documentation directory of the kernel sources )
  • The value 1 for /proc/sys/kernel/sysrq means all features are supported. Values &gt; 1 are bit flags for special features. Please see the sysrq.txt for more information.
  • The debug information are sometimes hard to find: On installation medium you find the debug information for the installed packages; if the system is updated by RHN, you may find the debug information there; When there are newer but not applied updates on RHN available you need to find the debug infos on Red Hat FTP server. There is a web page available which provides you with help in searching debug information on the Red Hat FTP server: http://people.redhat.com/duffy/debuginfo/index-js.html In case the customer has compiled his own kernel, you need to ask the customer for the kernel sources; If he is using rpmbuild for building, he also obtain the package with the debug information.
  • With nmi_watchdog enabled, the kernel may detect the hang, if any CPU in the system does not execute the period local timer interrupt for more than 5 seconds, then the NMI handler generates an oops and kills the process. This will create some messages and will force the kernel to dump, so that more information for analyze the root cause for the lockup.
  • X64 Workshop Linux Information Gathering

    1. 1. <ul><li>Thorsten Kellermann </li></ul><ul><li>Sun Microsystems </li></ul>X64 Work Shop Linux Information Gathering
    2. 2. Agenda <ul><li>Linux Support Overview </li></ul><ul><ul><li>Support model and structure </li></ul></ul><ul><li>Data Collection </li></ul><ul><ul><li>Red Hat sysreport </li></ul></ul><ul><ul><li>SuSE siga / config.sh </li></ul></ul><ul><ul><li>Linux explorer </li></ul></ul><ul><li>Data Analyzing </li></ul>
    3. 3. Agenda (cont) <ul><li>Advanced Troubleshooting </li></ul><ul><ul><li>System Core Dump Capturing </li></ul></ul><ul><ul><li>Linux SysRq </li></ul></ul><ul><ul><li>Hanging System </li></ul></ul><ul><li>Linux Analysis </li></ul><ul><ul><li>CDA </li></ul></ul>
    4. 4. Linux Support Overview <ul><li>Linux Support from System TSC organization: </li></ul><ul><ul><li>EMEA: System TSC VSP </li></ul></ul><ul><ul><ul><li>coverage from 9am - 5pm </li></ul></ul></ul><ul><ul><li>AMER/APAC: System TSC OS </li></ul></ul>
    5. 5. Linux Support Overview (cont) <ul><li>Supported Linux Versions: </li></ul><ul><ul><li>Red Hat Enterprise Linux (RHEL) </li></ul></ul><ul><ul><ul><li>Version 3 and 4 </li></ul></ul></ul><ul><ul><ul><li>AS, WS, ES, DESKTOP </li></ul></ul></ul><ul><ul><ul><li>Only existing contracts, no new contracts after the 30.09.2006. </li></ul></ul></ul><ul><ul><li>Novell/SuSE Linux Enterprise (SLES) </li></ul></ul><ul><ul><ul><li>Version 8, 9 and 10 </li></ul></ul></ul><ul><li>Back line support from Vendor available </li></ul><ul><ul><li>We have a path to escalate issue to Red Hat or Novell/SuSE. </li></ul></ul>
    6. 6. Linux Support Overview (cont) <ul><li>What is covered by support? </li></ul><ul><ul><li>Bugs within the OS or with Core applications </li></ul></ul><ul><li>What is not covered? </li></ul><ul><ul><li>Configuration of the system </li></ul></ul><ul><ul><li>HowTo questions </li></ul></ul><ul><ul><li>3 rd Party applications </li></ul></ul><ul><li>Other limitations </li></ul><ul><ul><li>Own compiled Kernels, tainted Modules </li></ul></ul><ul><ul><li>Sun do not fix bugs within any distribution, its up to the Vendor. </li></ul></ul>
    7. 7. Data Collection <ul><li>Entitlement information </li></ul><ul><ul><li>We need the entitlement for the Linux the customer installed </li></ul></ul><ul><li>General thoughts about data collection </li></ul><ul><ul><li>The issue must be visible within the data. </li></ul></ul><ul><ul><li>Data must be current. </li></ul></ul><ul><ul><ul><li>Anything changed to the system? New data! </li></ul></ul></ul><ul><ul><li>And it must be understandable. </li></ul></ul><ul><ul><ul><li>if not, try SGRT. </li></ul></ul></ul>
    8. 8. Data Collection (cont) <ul><li>Samples: </li></ul><ul><ul><li>Customer has a working and a non working system </li></ul></ul><ul><ul><ul><li>Collect data from both systems </li></ul></ul></ul><ul><ul><li>Customer has changed the configuration by the advice of Sun Support, but this doesn't work. </li></ul></ul><ul><ul><ul><li>Collect again all relevant data from the system to see what was changed. </li></ul></ul></ul><ul><ul><li>Customer applies online updates to the system, but the issue isn't fixed. </li></ul></ul><ul><ul><ul><li>We need again the data from the system to see what updates are applied. </li></ul></ul></ul>
    9. 9. Data Collection (cont) <ul><li>Red Hat: </li></ul><ul><ul><li>sysreport </li></ul></ul><ul><ul><ul><li>Mandatory for escalating to Red Hat </li></ul></ul></ul><ul><ul><ul><li>File system hierarchy </li></ul></ul></ul><ul><ul><ul><li>Lack of some interesting information. </li></ul></ul></ul><ul><li>SuSE: </li></ul><ul><ul><li>siga </li></ul></ul><ul><ul><ul><li>Insufficient messages etc. </li></ul></ul></ul><ul><ul><li>config.sh (preferred) </li></ul></ul><ul><ul><ul><li>Collect much more infos than siga. </li></ul></ul></ul><ul><ul><ul><li>Encapsulate siga report </li></ul></ul></ul>
    10. 10. Data Collection (cont) <ul><li>Others </li></ul><ul><ul><li>Linux Explorer </li></ul></ul><ul><ul><ul><li>Most complete data collection </li></ul></ul></ul><ul><ul><ul><li>Not a Sun tool </li></ul></ul></ul><ul><ul><ul><li>We are in discussion with SuSE to also accept this data set instead of siga/config.sh </li></ul></ul></ul><ul><ul><ul><li>Not accepted by Red Hat for escalation </li></ul></ul></ul>
    11. 11. Data Analyzing <ul><li>There is no automatic tool! </li></ul><ul><li>This presentation isn't complete at all. </li></ul><ul><li>Determinate the Linux Version: </li></ul><ul><ul><li>uname -a </li></ul></ul><ul><ul><li>/etc/*release* </li></ul></ul><ul><li>Looking up Messages: </li></ul><ul><ul><li>messages </li></ul></ul><ul><ul><li>dmesg </li></ul></ul><ul><ul><li>boot.log </li></ul></ul>
    12. 12. Data Analyzing (cont) <ul><li>What packages are installed? Which version? </li></ul><ul><ul><li>RPM is the packages manager of RHEL and SLES. </li></ul></ul><ul><ul><ul><li>rpm -qa </li></ul></ul></ul><ul><ul><ul><li>rpm -qaV (takes some time) </li></ul></ul></ul><ul><li>SAR report </li></ul><ul><ul><li>looking in the sar data (package sysstat) shows the load of the system at the time when an issue occurs </li></ul></ul>
    13. 13. Data Analyzing (cont) <ul><li>Hardware/Firmware information </li></ul><ul><ul><li>lspci [-v[v[v[v[v]]]]] </li></ul></ul><ul><ul><li>lsusb </li></ul></ul><ul><ul><li>dmidecode </li></ul></ul><ul><ul><ul><li>not part of sysreport! </li></ul></ul></ul><ul><ul><ul><li>hardware.py </li></ul></ul></ul><ul><ul><ul><ul><li>Python script wrapping dmidecode (RH only, may included in sysreport) </li></ul></ul></ul></ul><ul><ul><li>dmesg or /proc releated </li></ul></ul><ul><ul><ul><li>e.g. firmware of SCSI disk in /proc/scsi/scsi </li></ul></ul></ul>
    14. 14. Data Analyzing (cont) <ul><li>Overview </li></ul>
    15. 15. System Core Dump Capturing <ul><li>No standard at the moment </li></ul><ul><ul><li>Kdump has find it's way into the mainstream kernel. </li></ul></ul><ul><li>RHEL 3 / 4 uses it's own stuff </li></ul><ul><ul><li>netdump (preferred) </li></ul></ul><ul><ul><li>diskdump </li></ul></ul><ul><li>RHEL 5 uses Kdump </li></ul><ul><ul><li>An resident own kernel with small footprint </li></ul></ul><ul><ul><li>highly flexible and reliable </li></ul></ul>
    16. 16. System Core Dump Capturing (cont) <ul><li>SLES 8 / 9 uses LKCD </li></ul><ul><ul><li>Based on an IBM/SGI implementation. </li></ul></ul><ul><li>SLES 10 uses kdump </li></ul><ul><ul><li>An resident own kernel with small footprint </li></ul></ul><ul><ul><li>highly flexible and reliable </li></ul></ul>
    17. 17. Setting up RHEL 3 & 4 Netdump <ul><li>Install Netdump Server </li></ul><ul><ul><li>install package netdump-server </li></ul></ul><ul><ul><li>normally no configuration needed. </li></ul></ul><ul><ul><li>start service </li></ul></ul><ul><li>Install Netdump Client </li></ul><ul><ul><li>install package netdump-client </li></ul></ul><ul><ul><li>configure /etc/sysconfig/netdump </li></ul></ul><ul><ul><li>&quot;service netdump propagate&quot; </li></ul></ul><ul><ul><li>start service </li></ul></ul>
    18. 18. Setting up RHEL 5 Kdump <ul><li>Installed by Default </li></ul><ul><li>Configuration Dialog </li></ul><ul><ul><li>enable / disable kdump </li></ul></ul><ul><ul><li>configure dump locations </li></ul></ul><ul><ul><ul><li>local: file </li></ul></ul></ul><ul><ul><ul><li>net: nfs / ssh </li></ul></ul></ul><ul><ul><ul><li>partitions: ext2 / ext3 / raw </li></ul></ul></ul><ul><li>Quite easy to setup with the GUI dialog </li></ul>
    19. 19. Setting up SLES 8/9 LKCD <ul><li>Install required package: </li></ul><ul><ul><li>lkcdutils </li></ul></ul><ul><li>Edit /etc/sysconfig/dump </li></ul><ul><li>Write configuration </li></ul><ul><ul><li># lkcd config </li></ul></ul><ul><li>Activate service </li></ul><ul><ul><li># insserv /etc/init.d/boot.lkcd </li></ul></ul>
    20. 20. Seting up SLES 10 Kdump <ul><li>Install needed packages: </li></ul><ul><ul><li>kexec-tools </li></ul></ul><ul><ul><li>kernel-kdump </li></ul></ul><ul><ul><li>kernel-*-debuginfo </li></ul></ul><ul><li>Edit /etc/sysconfig/kdump </li></ul><ul><li>Enable kdump init service </li></ul><ul><ul><li>via YaST runlevel editor </li></ul></ul><ul><ul><li>&quot;chkconfig kdump on&quot; </li></ul></ul><ul><li>Add boot option &quot;crashkernel=64M@16M&quot; </li></ul>
    21. 21. Checking dump setup <ul><li>Check if everything fit together: </li></ul><ul><ul><li>Enable Magic SysRq feature temporarily </li></ul></ul><ul><ul><ul><li>echo &quot;1&quot; > /proc/sys/kernel/sysrq </li></ul></ul></ul><ul><ul><li>Force the system to dump </li></ul></ul><ul><ul><ul><li>echo &quot;c&quot; > /proc/sysrq-trigger </li></ul></ul></ul>
    22. 22. Linux SysRq Feature <ul><li>The Magic SysRq Feature is somewhat similar to Stop-A on Solaris </li></ul><ul><li>It can force the kernel to printout or dump information about the system </li></ul><ul><li>Sometimes really helpful for trouble shouting </li></ul><ul><li>May even work if the system seems to hang </li></ul>
    23. 23. Linux SysRq Feature (cont) <ul><li>Disabled by default, need to be enabled </li></ul><ul><ul><li>temporarily until next reboot </li></ul></ul><ul><ul><ul><li>echo &quot;1&quot; > /proc/sys/kernel/sysrq </li></ul></ul></ul><ul><ul><li>permanently </li></ul></ul><ul><ul><ul><li>edit /etc/sysctl.conf to add the line: kernel.sysrq = 1 </li></ul></ul></ul><ul><li>Issue locally on keyboard by Alt+SysRQ+<hotkey> </li></ul><ul><li>Issue remote by &quot;echo <hotkey> > /proc/sysrq-trigger&quot; </li></ul>
    24. 24. Linux SysRq Feature (cont) <ul><li>Some Hotkeys: </li></ul><ul><ul><li>K </li></ul></ul><ul><ul><ul><li>call the Secure Attention function (SAK). SAK terminate every process running on the actual console, to cleanup the terminal. </li></ul></ul></ul><ul><ul><li>s </li></ul></ul><ul><ul><ul><li>Synchronized all hard disks. </li></ul></ul></ul><ul><ul><li>u </li></ul></ul><ul><ul><ul><li>Remounts all hard disks in read only mode. This will prevent dataloss, when the system is in an unstable situation. </li></ul></ul></ul><ul><ul><li>t </li></ul></ul><ul><ul><ul><li>Shows the actual task list. </li></ul></ul></ul>
    25. 25. Linux SysRq Feature (cont) <ul><li>Some Hotkeys (cont): </li></ul><ul><ul><li>b </li></ul></ul><ul><ul><ul><li>boots the system immediately. You should synchronize and remount the hard disks read only before restarting the system. </li></ul></ul></ul><ul><ul><li>p </li></ul></ul><ul><ul><ul><li>Prints out the actual register content. </li></ul></ul></ul><ul><ul><li>m </li></ul></ul><ul><ul><ul><li>Prints out the memory information. </li></ul></ul></ul><ul><li>For a complete list lookup sysrq.txt in the Kernel documentation </li></ul>
    26. 26. Crash Dump Analyzing <ul><li>Crash utility </li></ul><ul><ul><li>Support varios dump fomats </li></ul></ul><ul><ul><ul><li>Kdump, LKCD, Net/Disk dump </li></ul></ul></ul><ul><ul><li>Integrated GDB </li></ul></ul><ul><ul><li>Can examinate live system Kernel </li></ul></ul><ul><li>http://people.redhat.com/~anderson/ </li></ul>
    27. 27. Crash Dump Analyzing (cont) <ul><li>You need to have the debug information of the kernel </li></ul><ul><li>Crash package need to be installed </li></ul><ul><li>Load vmcore for analyzing </li></ul><ul><ul><li>crash System.map vmlinux vmcore </li></ul></ul><ul><ul><ul><li>dmesg </li></ul></ul></ul><ul><ul><ul><li>ps list </li></ul></ul></ul><ul><ul><ul><li>stack traces </li></ul></ul></ul><ul><ul><ul><li>etc. </li></ul></ul></ul>
    28. 28. Troubleshoot a Hanging System <ul><li>Hard to troubleshoot due to lack of information </li></ul><ul><li>If a deadlocked kernel, NMI watchdog may help </li></ul><ul><ul><li>add Kernel boot cmd nmi_watchdog=1 to grub configuration. </li></ul></ul><ul><ul><li>When system look is detected, a kernel panic will be initiated. </li></ul></ul><ul><li>There might be a chance to force a dump (SysRq) when system hanging </li></ul>
    29. 29. Links <ul><li>External Sources: </li></ul><ul><ul><li>Linux Explorer http://www.unix-consultants.co.uk/examples/scripts/linux/linux-explorer/ </li></ul></ul><ul><ul><li>LKCD Setup on SLES http://www.novell.com/coolsolutions/feature/15284.html </li></ul></ul><ul><ul><li>Crash Utility http://people.redhat.com/~anderson/ </li></ul></ul><ul><li>Internal Sources </li></ul><ul><ul><li>System TSC Linux pages http://systems-tsc/twiki/bin/view/Teams/LinuxDataGathering </li></ul></ul><ul><ul><li>PTS Linux pages (outdated) http://barentz.germany.sun.com/ptsvs/Wiki.jsp?page=LinuxHowTos </li></ul></ul>
    30. 30. Links <ul><li>Did you know http://www.google.com/linux? </li></ul>
    31. 31. X64 Work Shop Linux Information Gathering <ul><li>Thorsten Kellermann </li></ul><ul><ul><li>[email_address] </li></ul></ul>
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×