This document contains a summary of system monitoring data from a Splunk installation, including statistics on forwarders, indexers, searches, and a detailed view of configuration and activity for a specific host. Key metrics shown include top forwarders and indexers by throughput, longest and most frequent searches, and CPU/memory usage over time for the selected host. The host details view contains in-depth information on applications, open ports, running processes, configuration files, tickets, and recent changes for the localhost.
1. splunk > search Logged in as Username | Logout | Manager | Jobs (3) | Apps
Dashboards Search Views Saved Searches Help | Preferences | About
Splunk.com
Downloads and Downloaders last refreshed 01/01/08 2:00:43 Pageviews and Visitors last refreshed 01/01/08 2:00:43
Downloads by Platform last refreshed 01/01/08 2:00:43 Usage by License last refreshed 01/01/08 2:00:43
Title last refreshed 01/01/08 2:00:43 Usage by Version last refreshed 01/01/08 2:00:43
OVERLAY: Heat map High/Low values
2. splunk > Enterprise Manager Logged in as Username | Logout | Manager | Jobs (3) | Apps
Dashboards Search Views Saved Searches Help | Preferences | About
Forwarders
Top 10 average tcp thruput by forwarder last refreshed 01/01/08 2:00:43
VIEW: past hour past 24 hours past week
random html (text, images, links)
show details for: past 24 hours Top 10 forwarders >
Forwarders down in the past 24 hours last refreshed 01/01/08 2:00:43
Host
1 blogs.int.splunk.com
2 web1.int.splunk.com
3 web2.int.splunk.com
3. splunk > Enterprise Manager Logged in as Username | Logout | Manager | Jobs (3) | Apps
Dashboards Search Views Saved Searches Help | Preferences | About
Indexers
Indexer Volume last refreshed 01/01/08 2:00:43 Throughput for all Indexers last refreshed 01/01/08 2:00:43
VIEW: past hour past 24 hours past week VIEW: past hour past 24 hours past week last refreshed, next
scheduled run
show details for: past 24 hours Top 10 indexers > show details for: past 24 hours Top 10 indexers >
Top 10 sourcetypes across all Indexers last refreshed 01/01/08 2:00:43
VIEW: past hour past 24 hours past week
show details for: past 24 hours Top 10 forwarders >
4. splunkEnterprise Manager
> Enterprise Manager Logged in in as Username Logout | |Manager || Jobs (3)
Logged as Username | | Logout Manager Jobs | Apps
Help
Dashboards | | Search | | Reports | | Visualizations | | Application Settings
Dashboards Search Views Saved Searches Help | Preferences | About
Searches
10 Longest Running Searches last refreshed 01/01/08 2:00:43 | next run: tomorrow 12:00
VIEW: past hour past 24 hours past week
show details for past 24 hours Top 10 searches >
10 Most Frequent Searches & Time Taken last refreshed 01/01/08 2:00:43
VIEW: past hour past 24 hours past week
show details for past 24 hours Top 10 searches >
Number of Searches by Hour last refreshed 01/01/08 2:00:43
Top Search Users last refreshed 01/01/08 2:00:43
VIEW: past hour past 24 hours past week VIEW: past hour past 24 hours past week
show details for: past 24 hours select search > show details for: past 24 hours select search >
5. splunk > Change Management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Change Detection Change Investigation Change Validation Change Auditing Change Reporting Help | Preferences | About
Overview
last refreshed 01/01/08 2:00:43
Unauthorized Changes in the last 24 hours last refreshed 01/01/08 2:00:43 New software in the last 24 hours
VIEW BY: tyoe hour VIEW BY: role and host software
Role Host Software
Mail Host_1 MS Internet Explorer 1.0
Splunk 4.0
VMWare
Web Host_456 Adobe Acrobat
Host_188 Adobe Acrobat
Desktop Host_189 Adobe Acrobat
6. splunk > Change Management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Change Detection Change Investigation Change Validation Change Auditing Change Reporting Help | Preferences | About
Variances
last refreshed 01/01/08 2:00:43
Variances by Host Class in the past week last refreshed 01/01/08 2:00:43 Count of Variances the past week last refreshed 01/01/08 2:00:43
VIEW BY: role tyoe location application
show details for: past week All Host Classes > show details for: past week All Roles >
7. splunk > Change Management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Change Detection Change Investigation Change Validation Change Auditing Change Reporting Help | Preferences | About
Violations
last refreshed 01/01/08 2:00:43 last refreshed
last refreshed 01/01/08 2:00:43
Policy violations in the last 24 hours last refreshed 01/01/08 2:00:43 Change window violations in the last 24 hours 01/01/08 2:00:43
show details for: past 24 hours All policy violations >
8. splunk > Change Management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Change Detection Change Investigation Change Validation Change Auditing Change Reporting Help | Preferences | About
Host Details
Location Host
All locations localhost >
localhost 01/01/2008 08:00:00 - 01/07/2008 08:00:00
Overview Events Tickets Ports Processes Configuration
IP Address: 91.782.918.678 Role: Web. Other Location: San Jose, CA Owner:Jef Bekes OS / Patch Level ???
Applications last refreshed 01/01/08 2:00:43 | next run: tomorrow 12:00
Application 1 Application 1 Application 1 Application 1 Application 1 Application 1
Application 2 Application 2 Application 2 Application 2 Application 2 Application 2
Splunk 3.4 Splunk 3.4 Splunk 3.4 Splunk 3.4 Splunk 3.4 Splunk 3.4
Splunk 4.0 Splunk 4.0 Splunk 4.0 Splunk 4.0 Splunk 4.0 Splunk 4.0
Photoshop CS3 Photoshop CS3 Photoshop CS3 Photoshop CS3 Photoshop CS3 Photoshop CS3
compare to: All Web Hots >
CPU Utilization last refreshed 01/01/08 2:00:43 Memory Usage last refreshed 01/01/08 2:00:43
VIEW BY: hour day week VIEW BY: hour day week
Ports Open | Search
compare to: All Web Hots > compare to: All Web Hots >
9. splunk > Change Management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Change Detection Change Investigation Change Validation Change Auditing Change Reporting Help | Preferences | About
Host Details
Location Host
All locations localhost >
localhost 01/01/2008 08:00:00 - 01/07/2008 08:00:00
Overview Events Tickets Ports Processes Configuration
Summary
IP Address: 91.782.918.678 Role: Web. Other Location: San Jose, CA Owner:Jef Bekes OS / Patch Level ???
Applications last refreshed 01/01/08 2:00:43 | next run: tomorrow 12:00
Application 1 Application 1 Application 1 Application 1 Application 1 Application 1
Application 2 Application 2 Application 2 Application 2 Application 2 Application 2
Splunk 3.4 Splunk 3.4 Splunk 3.4 Splunk 3.4 Splunk 3.4 Splunk 3.4
Splunk 4.0 Splunk 4.0 Splunk 4.0 Splunk 4.0 Splunk 4.0 Splunk 4.0
Photoshop CS3 Photoshop CS3 Photoshop CS3 Photoshop CS3 Photoshop CS3 Photoshop CS3
CPU Utilization last refreshed 01/01/08 2:00:43 Memory Usage last refreshed 01/01/08 2:00:43
VIEW BY: hour day week VIEW BY: hour day week
Ports Open | Search
compare to: All Web Hots > compare to: All Web Hots >
10. splunk > Change Management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Change Detection Change Investigation Change Validation Change Auditing Change Reporting Help | Preferences | About
Host Details
Location Host
All locations select host >
11. splunk > Change Management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Change Detection Change Investigation Change Validation Change Auditing Change Reporting Help | Preferences | About
Host Details
Location Host
All locations localhost >
localhost 01/01/2008 08:00:00 - 01/07/2008 08:00:00
IP Address: 91.782.918.678 Role: Web. Other Location: San Jose, CA Owner: Jef Bekes OS / Patch Level: FreeBSD 6.2-RELEASE-p1
Overview Events Tickets Ports Processes Configuration Recent Changes Connections
Applications last refreshed 01/01/08 2:00:43 | next run: tomorrow 12:00
DirectX 9 Preview iPhoto Nessus Automator
FireFox Mail.app Garage Band Airfoil QuickTime Player
Internet Explorer iCal twhirl Skype DivX Player
Safari Photo Booth iMovie Adium Flock
PhotoShop Microsoft Office 2008 iTunes Adobe Acrobat Internet Explorer 6.0
CPU Utilization last refreshed 01/01/08 2:00:43 Memory Usage last refreshed 01/01/08 2:00:43
VIEW BY: hour day week VIEW BY: hour day week
Ports Open | Search
compare to: All Web Hots > compare to: All Web Hots >
12. splunk > Change Management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Change Detection Change Investigation Change Validation Change Auditing Change Reporting Help | Preferences | About
Host Details
Location Host
All locations localhost >
localhost 01/01/2008 08:00:00 - 01/07/2008 08:00:00
IP Address: 91.782.918.678 Role: Web. Other Location: San Jose, CA Owner: Jef Bekes OS / Patch Level: ???
Overview Events Tickets Ports Processes Configuration Recent Changes Connections
get rid of fields
13. splunk > Change Management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Change Detection Change Investigation Change Validation Change Auditing Change Reporting Help | Preferences | About
Host Details
Location Host
All locations localhost >
localhost 01/01/2008 08:00:00 - 01/07/2008 08:00:00
IP Address: 91.782.918.678 Role: Web. Other Location: San Jose, CA Owner: Jef Bekes OS / Patch Level: ???
Overview Events Tickets Ports Processes Configuration Recent Changes Connections
Tickets
Table, link each ticket to? What info do we have? Ticket 4582
14. splunk > Change Management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Change Detection Change Investigation Change Validation Change Auditing Change Reporting Help | Preferences | About
Host Details
Location Host
All locations localhost >
localhost 01/01/2008 08:00:00 - 01/07/2008 08:00:00
IP Address: 91.782.918.678 Role: Web. Other Location: San Jose, CA Owner: Jef Bekes OS / Patch Level: ???
Overview Events Tickets Ports Processes Configuration Recent Changes Connections
Open Ports
22 SSHD 22 <process name>
25 unknown 25 unknown
80 HTTPD 80 <process name>
8000 SplunkD 8000 <process name>
22 <process name> 22 <process name>
25 unknown 25 unknown
80 <process name> 80 <process name>
8000 <process name> 8000 <process name>
Show details for Enter port name > Compare to: All Web Hosts >
15. splunk > Change Management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Change Detection Change Investigation Change Validation Change Auditing Change Reporting Help | Preferences | About
Host Details
Location Host
All locations localhost >
localhost 01/01/2008 08:00:00 - 01/07/2008 08:00:00
IP Address: 91.782.918.678 Role: Web. Other Location: San Jose, CA Owner: Jef Bekes OS / Patch Level: ???
Overview Events Tickets Ports Processes Configuration Recent Changes Connections
Processes Currently Running
SSHD Root <Process ID> 0 - 65000
HTTPD Root
SYSLOGD Root
SPLUNKD Splunk
SSHD Root
HTTPD <User>
SYSLOGD <User>
SPLUNKD <User>
Show details for Enter process name > Compare to: All Web Hosts >
16. splunk > Change Management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Change Detection Change Investigation Change Validation Change Auditing Change Reporting Help | Preferences | About
Host Details
Location Host
All locations localhost >
localhost 01/01/2008 08:00:00 - 01/07/2008 08:00:00
IP Address: 91.782.918.678 Role: Web. Other Location: San Jose, CA Owner: Jef Bekes OS / Patch Level: ???
Overview Events Tickets Ports Processes Configuration Recent Changes Connections
Confuguration
Files
Details for File 1
File 1 *.err;kern.*;auth.notice;authpriv,remoteauth,install.none;mail.crit
/dev/console
File 2
File 3 *.notice;authpriv,remoteauth,ftp,install.none;kern.debug;mail.crit
/var/log/system.log
Another file
# Send messages normally sent to the console also to the serial port.
Yet another file # To stop messages from being sent out the serial port, comment out this line.
#*.err;kern.*;auth.notice;authpriv,remoteauth.none;mail.crit
More files
/dev/tty.serial
# The authpriv log file should be restricted access; these messages shouldn't go to terminals or publically-readable files.
auth.info;authpriv.*;remoteauth.crit
/var/log/secure.log
lpr.info /var/log/lpr.log
mail.* /var/log/mail.log
ftp.* /var/log/ftp.log
install.* /var/log/install.log
Compare to: All Web Hosts >
17. splunk > Change Management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Change Detection Change Investigation Change Validation Change Auditing Change Reporting Help | Preferences | About
Host Details
Location Host
All locations localhost >
localhost 01/01/2008 08:00:00 - 01/07/2008 08:00:00
IP Address: 91.782.918.678 Role: Web. Other Location: San Jose, CA Owner: Jef Bekes OS / Patch Level: ???
Overview Events Tickets Ports Processes Configuration Recent Changes Connections
Top 10 Recent Changes
Change Date Authorized? User
/etsy/hosts
/etsy/passwd
ets/somethingelse
All changes... Compare to: All Web Hosts >
Compare to: All Web Hosts >
18. splunk > Change Management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Dashboards Search Views Saved Searches Change Reporting Help | Preferences | About
Overview | Change Audit | Change Validation | Change Detection | Change Investigation |
Change Management Overview
last refreshed 01/01/08 2:00:43 Top 5 High Severity Tickets
Status last refreshed 01/01/08 2:00:43
Change Validation Change Detection Ticket Details
1 ticket_01928 Ticket body text goes here...
2 ticket_01967 Ticket body text goes here...
3 ticket_01990 Ticket body text goes here...
4 ticket_01987 Ticket body text goes here...
5 ticket_01877 Ticket body text goes here...
Changes in the past week last refreshed 01/01/08 2:00:43 Tickets by Severity for the past week last refreshed 01/01/08 2:00:43
VIEW BY: authorization change window tyoe
authorized unauthorized normal medium authorized unauthorized
Top 10 Change Policy Violations by location (past week) last refreshed 01/01/08 2:00:43
VIEW: past week past month past year
show details for: past 24 hours All Locations >
19. splunk > Change Management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Dashboards Search Views Saved Searches Help | Preferences | About
IT Operations Management
last refreshed
Changes in the past week last refreshed 01/01/08 2:00:43 Host trends in the past week 01/01/08 2:00:43
VIEW BY: authorization change window tyoe VIEW BY: type changes location
authorized unauthorized
last refreshed
Changes in the last 24 hours last refreshed 01/01/08 2:00:43 Hosts changed in the past week by hour of day 01/01/08 2:00:43
VIEW BY: tyoe host severity VIEW BY: location host tyoe
view report.... view report....
Changes by hour of day last refreshed 01/01/08 2:00:43 Hosts by change type in the past week last refreshed 01/01/08 2:00:43
TIME PERIOD: day week month
view report....
last refreshed
Count of change types in the last 24 hrs 01/01/08 2:00:43
VIEW BY: host severity
view report....
20.
21. splunk > Windows Management Logged in as Username | Logout | Manager | Jobs (3)
Overview System Management Configuration Management Performance Tutorial Help | Preferences | About
Welcome to Splunk for Windows Management refreshed 01/01/08 2:00:43
last
Splunk can run many apps Splunk > Search
Index and search any IT data Get Started using Splunk for Windows Management
Switch to other apps using
the menu above. Splunk > Windows
You are here Search your local Windows IT data using the search bar above.
Get more apps... Other apps
Check out some information about your Windows environment.
last refreshed 01/01/08 2:00:43
System status in the past 24 hours last refreshed 01/01/08 2:00:43
Short description of where this data is coming from and what it means...
100,000 5 23
events warnings errors
System performance in the last hour last refreshed 01/01/08 2:00:43
More performance information...
last refreshed 01/01/08 2:00:43
Available data sources last refreshed 01/01/08 2:00:43
View All... View All... View All...
See how you can start working with this and other data now. Take the tutorial...
22. splunk > Windows Management Logged in as Username | Logout | Manager | Jobs (3)
Dashboards Search Views Saved Searches Help | Preferences | About
Getting Started | Dashboard 2 | Dashboard 3 hide dashboard
Welcome to Splunk > Windows Management refreshed 01/01/08 2:00:43
last
Get Started...
Splunk Apps Splunk > Search
Index and search any IT data Search for your windows environment data using the search bar
Switch to other apps using
the menu above. Splunk > Windows or any of the search views in the menu above.
You are here Take me to my default search view...
Get more apps... Other apps
Check out some information about your Windows environment
last refreshed 01/01/08 2:00:43
System Status last refreshed 01/01/08 2:00:43 Configuration Status last refreshed 01/01/08 2:00:43
in the past 24 hours in the past 24 hours
450 5 23 3
events warnings errors registered changes
Short description of where this data is coming from and what it means... Short description of where this data is coming from and what it means...
CPU performance in the past 24 hours last refreshed 01/01/08 2:00:43
last refreshed 01/01/08 2:00:43
All indexed data last refreshed 01/01/08 2:00:43
View All... View All... View All...
See how you can start working with this and other data now. Take the tutorial...
23. splunk > Windows Management Logged in as Username | Logout | Manager | Jobs (3)
Dashboards Search Views Saved Searches Help | Preferences | About
Getting Started | Dashboard 2 | Dashboard 3 hide dashboard
Welcome to Splunk > Windows Management refreshed 01/01/08 2:00:43
last Other installed apps
Search for your windows environment data using any of the search views Splunk > Search Index any data and search it using Splunk.
or saved searches in the mensu above. Splunk > Change Management
Search anything using Splunk > Search Application 3
Check out some information about your Windows environment Get more applications...
last refreshed 01/01/08 2:00:43
System management last refreshed 01/01/08 2:00:43 Configuration management last refreshed 01/01/08 2:00:43
450 23 3
events errors registered changes
in the past
24 hours
in the past 24 hours
CPU performance last refreshed 01/01/08 2:00:43 Learn how to use Splunk
Tutorials Other references:
Search your IT data using Splunk Wiki article
Wiki article 2
Add search views and dashboards
Create charts and reports
Add inputs to collect more data
last refreshed 01/01/08 2:00:43
All indexed data last refreshed 01/01/08 2:00:43
24. splunk > windows management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Dashboards
Dashboards Search Views Saved Searches Help | Preferences | About
Getting Started | Troubleshooting | Change Management | Performance | Data Summary
Getting Started Hide dashboard
What are you trying to do?
Troubleshooting Change Management Performance
Go to Dashboard Go to Dashboard Go to Dashboard
Other ways to get started
Add more inputs Take a tutorial
Go to Splunk Manager Play >
25. splunk > windows management Logged in as Username | Logout | Manager | Jobs (3) | Apps
Dashboards
Dashboards Search Views Saved Searches Help | Preferences | About
Troubleshooting | Change Management | Performance | Data Summary
Troubleshooting Hide dashboard
Troubleshooting Module
Go to Splunk Manager Go to ?? Go to ??
Welcome to Splunk!
Discover the many ways you can use splunk to manage your IT data.
Troubleshooting Change Management Performance
Another TroubleshootingDashboard
Go to Module Go to Dashboard Go to Dashboard
Or... Add more inputs Take a tutorial
Go to Splunk Manager Play Tutorial >
26. Enterprise Manager Logged in as Username | Logout | Admin | Jobs (3) | Help
Dashboards | | Search | | Reports | | Visualizations | | Application Settings
Enterprise Manager Overview
??? Errors
Top forwarder errors
Errors in the last 24 hours / JoeSmith
Errors in the last 24 hours / JoeSmith
Errors in the last 24 hours / JoeSmith
Errors in the last 24 hours / JoeSmith
Top indexer errors
Errors in the last 24 hours / JoeSmith
Errors in the last 24 hours / JoeSmith
Top search errors
Errors in the last 24 hours / JoeSmith
Errors in the last 24 hours / JoeSmith
Errors in the last 24 hours / JoeSmith
VIEW Timeframe Series Fields Errors in the last 24 hours / JoeSmith
last 7 days All <series name> >
Top 10 average tcp thruput by forwarder
random html (text, images, links)
VIEW Timeframe Forwarder
last 7 days All >
ERRORS / FEEDBACK
unable to display because:
- not configured
- no data for specified time range