A simple yet powerful and secure monitoring daemon for Windows operating systems, this is what a NSClient++ aims to be. It is built for Nagios, but nothing in the daemon is actually Nagios specific and with little or no change NSClient++ could be also integrated into any monitoring software that supports running user tools for polling. In this presentation we are going to look at the new features of NSClient++ 0.3.x. But perhaps more importantly looking at what’s coming in the 0.4.x version!

1. “Monitoring” SQL
Server
Workshop
Open Source Monitoring Conferrence
Presentation © Michael Medin

2. These slides represent the work and opinions
of the author and do not constitute official
positions of any organization sponsoring the
author’s work
This material has not been peer reviewed and
is presented here as-is with the permission of
the author.
The author assumes no liability for any
content or opinion expressed in this
presentation and or use of content herein.
Disclaimer!

3. Developer (not system manager)
Not working with Nagios
Accidentally ended up in our NOC
Hated BB
2003: The birth of NSClient++
NSClient sucked (Broke Exchange)
NRPE_NT was to hard to use
2004: The open source of NSClient++
“just for fun”
My Background

4. Introduction 5 minutes
Part 1: Remote access 15 minutes
Part 2: Tools of the Trade 40 minutes
Break 20 minutes
Part 3: Monitoring 20 minutes
Part 4: Scripting 10 minutes
Conclusion 0 minutes
Agenda

5. Who are you?
What are your experiences with monitoring?
What are your experiences with SQL Server?
What are your expectations?
…
Introductions…

6. Split into groups
2-3 persons

7. Assignment
Where do we start?

8. When you manage a SQL Server:
What’s the most important thing to monitor?
Task/Question

9. It is not something I will speak about…
Hint 1

10. It is not something I will speak about…
It is not SQL Server related
Hint 2

11. If the infrastructure goes down
so will SQL Server
Always start by monitoring the infrastructure
Hardware
●
Clusters
●
Raid arrays
●
Network infrastructure
●
Temperature/HW Status
●
…
Software
(My) Answer

12. This depends on what environment you have
NOT related to SQL Server
Thus... Your on your own…
Monitoring Infrastructure?

13. Part 1: Remote access
Click to edit Master text styles
Second level
●
Third level
●
Fourth level
●
Fifth level

14. Remote access
SQL Server from perl

15. Perl
Install perl and DBI with ODBC (and TDS
driver)
●
You need (for debian):
– perl
– unixodbc unixodbc-dev tdsodbc
●
Then you need to configure a odbc driver
(/etc/odbcinst.ini)
– [FreeTDS]
– Description = TDS driver (Sybase/MS SQL)
– Driver = /usr/lib/odbc/libtdsodbc.so
– Setup = /usr/lib/odbc/libtdsS.so
– CPTimeout =
– CPReuse =
●
Perl and SQL Server

16. On Windows:
Install ActiveState perl
● Everything you need comes included
Install a ODBC driver for SQL Server
● http://www.microsoft.com/downloads/en/details.aspx?
familyid=78CAC895-EFC2-4F8E-A9E0-3A1AFBD5922E&displaylang=en
Verify:
● perl -e "use DBI 1.6;"
● perl –e "use DBD::ODBC;"
● perl -MDBD::ODBC -e "print $DBD::ODBC::VERSION;"
● perl -MDBI -e "DBI->installed_versions;"
Perl and SQL Server

17. Unix:
Edit: /etc/odbc.ini
●
[osmc]
●
Driver = FreeTDS
●
Server = <IP>
Windows:
Data sources -> System data source
●
Add -> {next, next, next} -> done
Name: osmc
Server: <IP>
Add a datasource

18. #!/usr/bin/perl -w
use strict;
use DBI;
my $dbh = DBI->connect('dbi:ODBC:osmc','osmc','osmc') or die "Can't connect to osmc: $DBI::errstr";
my $sql = "SELECT 1 AS test_col";
my $sth = $dbh->prepare($sql) or die "Can't prepare statement: $DBI::errstr";
$sth->execute();
print "$sth->{NAME}->[0]n";
while ( my @row = $sth->fetchrow_array ) {
print "@rown";
}
$dbh->disconnect;
$ perl test.pl
test_col
1
Connecting to a database

19. Assignment
Setting up perl

20. Configure perl for database connectivity
Connect to the following database
Database = osmc
Ip= ?.?.?.?
Task/Question

21. ●
apt-get install perl
●
apt-get install unixodbc unixodbc-dev
●
apt-get install tdsodbc
●
cat /usr/share/tdsodbc/odbcinst.ini >> /etc/odbcinst.ini
●
echo -e '[osmc]nDriver = FreeTDSnServer = ?.?.?.?' >>
/etc/odbc.ini cpan –i DBI
●
cpan –i DBI
●
cpan –i DBD::ODBC
Hint 1

22. $ cat > test.pl <<EOF
#!/usr/bin/perl -w
use strict;
use DBI;
my $dbh = DBI-
>connect('dbi:ODBC:osmc','osmc','osmc
') or die "Can't connect to osmc:
$DBI::errstr";
my $sql = "SELECT 1 AS test_col";
my $sth = $dbh->prepare($sql) or
Hint 2

23. Hopefully you can now connect 
(My) Answer

24. Remote access
Performance counters

25. We will mainly use this for counters
check_nrpe
●
–H <ip> -c <command> -a <arguments>
Or if you are on windows:
nsclient++ -noboot NRPEClient
●
–H <ip> -c <command> -a <arguments>
Command:
CheckCounter
check_nrpe

26. Assignment
Checking performance
counters

27. Access performance counters from <my ip>
Command is:
CheckCounter
Counter is:
Processor(_Total)% Processor Time
Task/Question

28. Command is:
CheckCounter
Counter is:
Processor(_Total)% Processor Time
Unix:
●
check_nrpe
●
–H <ip> -c <command> -a <arguments>
Or if you are on windows:
●
nsclient++ -noboot NRPEClient
Hint 1

29. Command is:
CheckCounter
Counter is:
Processor(_Total)% Processor Time
Unix:
●
check_nrpe
●
–H <ip> -c CheckCounter -a ‘Processor(_Total)% Processor Time’
Or if you are on windows:
●
nsclient++ -noboot NRPEClient
Hint 2

30. Hopefully you can now connect 
OK all counters within bounds.
|'Processor(_Total)% processortid'=14.204358
(My) Answer

31. perl run.pl <query>
For instance:
perl run.pl “SELECT 1 AS test_col”
Or
perl test.pl "sp_monitor“
Or
perl test.pl "SELECT r.session_id, r.wait_type
AS wait_category, r.wait_type, r.wait_time
FROM sys.dm_exec_requests AS r INNER JOIN
Running SQL Queries

32. Part 2: The tools of the
Trade Click to edit Master text styles
Second level
●
Third level
●
Fourth level
●
Fifth level

33. DML Triggers
Fires T-SQL code on table events
PerfMon/SysMon
Listens for hundreds of Windows/SQL
Server events
SQL Trace and Profiler
Monitors internal events
Wait States
Tracks system waits (processes waiting
for resources)
Useful tools in SQL Server

34. A trigger is a custom “event” which fires when
data changes.
For instance:
You want to be notified when someone does:
●
insert into orders (…)
DML Triggers

35. Nothing “out of the box”.
Can be extended (via external stored
procedures) and submit to for instance NSCA
or what not.
There are probably better/simpler ways unless
you really need this…
Monitoring DML Triggers

36. PDH (Performance Data Helper)
A framework with metrics for “various things”
Format:
Group(Instance)Metric
●
Group = “area”
●
Instance = “instance identifier”
●
Metric = “value”
Processor(_Total)% Processor Time
Performance Counters

37. Performance Viewer
Click to edit Master text styles
Second level
●
Third level
●
Fourth level
●
Fifth level

38. Two (ish) options:
PDH Counters
●
nsclient++, nc_net, nsclient, and everything else
WMI (Windows Management Instrumentation)
●
nsclient++ and modern solutions (including
agentless)
Many ways to monitor
We will use nsclient++ for simplicity
Monitoring Performance
Counters

39. SQL Server Profiler shows how SQL Server
resolves queries internally
This allows administrators to see exactly what
statements are submitted to the server and
how the server accesses the database to
return result sets.
Used to find an error or resolve a problem
Not monitor the server
SQL Server Trace Profiler

40.  Nothing you should monitor though…
 Can be viewed from the database (normal sql)
SELECT TextData, Duration, CPU
 FROM trace_table_name
 WHERE EventClass = 12 AND CPU < (Duration*1000)
Normal way to use it though is from inside the
tool
Monitoring Trace Profiler

41. System view with statuses for processes
running
Deprecated in 2005 (and beyond)
Shows;
processes waiting for resources
deadlocks
etc
Wait States

42. Can be viewed from the database (normal sql)
select * from sysprocesses
Monitoring Wait States

43. configured through;
SQL Server Management Studio
c2 audit mode option in sp_configure
the server records both failed and successful
attempts to access statements and objects
Can help profile system activity and track possible
security policy violations
Superseded by Common Criteria Certification
C2 Auditing

44. Nothing out of the box.
Generates “log files” (unsure of format)
Can be viewed from the database (normal sql)
SELECT * FROM ::fn_trace_gettable('C:Program FilesMicrosoft SQL
ServerMSSQLDataaudittrace_20040822191554.trc', default)
Monitoring C2 Auditing

45. Good stuff:
PerfMon/SysMon
●
System metrics
Wait States
●
SQL Process metrics (ish)
Useful:
DML Triggers
●
Database events
Not very usefull (for monitoring):
SQL Trace and Profiler
Summary

46. Assignment
Server Performance

47. Do we have a problem?
Task/Question

48. What’s the server CPU load?
Hint 1

49. What’s the server CPU load?
Check PDH (counter) object:
Processor(_Total)% Processor Time
Hint 2

50. What’s the server CPU load?
Check PDH (counter) object:
Processor(_Total)% Processor Time
Use check_nrpe (or NSClient++)
check_nrpe –H <IP> -c CheckCounter
●
-a ”Processor(_Total)% Processor Time”
Hint 3

51. No, but most will say so
Why do you buy that expensive server?
●
If not to use it?
But beware!
If load increases you will be in trouble!
(My) Answer

52. Common Criteria Compliance
Combines SQL Trace and security settings for
compliance
DMVs
dynamic management views, catalog views
(metadata details and current status)
Management Studio Reports
●
easy way to view DMV data
Performance Dashboard
●
downloadable add-on with extra reports
New tools added in 2005

53. Builds on c2 audit mode and is mainly a
certification thing as I understand it
Common Criteria
Compliance

54. I don’t really know :P
But I think it is much same as before…
Monitoring “C3”

55. Database views with metrics about server
status
Can be combined with reports to form a nice
monitoring/management view
In short:
This is the god stuff!
Dynamic management
views

56. Regular database view (normal sql)
For instance listing waiting sessions
SELECT
r.session_id, r.wait_type AS wait_category, r.wait_type, r.wait_time
FROM sys.dm_exec_requests AS r
INNER JOIN sys.dm_exec_sessions AS s ON r.session_id = s.session_id
WHERE
r.wait_type IS NOT NULL AND s.is_user_process = 0x1
Monitoring DMVs

57. Same as DML triggers but for DDL
Can show you;
When/if someone alters the database
(structure)
DDL Triggers

58. Same as with DML triggers
Nothing “out of the box”.
Can be extended (via external stored
procedures) and submit to for instance NSCA
or what not.
Monitoring DDL triggers

59. Good stuff:
DMVs
●
Lots of SQL Server metrics
●
Really nice reports
●
Downloadable add-on with extra reports
Somewhat useful:
DDL Triggers
●
Database events
Not very useful (for monitoring):
SQL Trace and Profiler
Summary

60. Assignment
Dynamic Management Views

61. Does the server have a problem?
Use your new found skills to diagnose the
system and find any errors and/or problems
Task/Question

62. Use Dynamic Management Views
Hint 1

63. Use Dynamic Management Views
For instance:
run_sql
SELECT status, wait_type, wait_time
FROM sys.dm_exec_requests
WHERE wait_type IS NOT NULL
(or)
SELECT
r.session_id, r.wait_type AS wait_category, r.wait_type, r.wait_time
FROM sys.dm_exec_requests AS r
INNER JOIN sys.dm_exec_sessions AS s ON r.session_id = s.session_id
WHERE
Hint 2

64. Absolutely!
Now we see we have tasks waiting!
Waiting means slow clients: not good!
(My) Answer

65. Change Tracking
synchronously records which rows have been
updated
Change Data Capture
asynchronously reads T-Log and records data
changes
Extended Events
lightweight windows monitoring technology
(similar to SQL Trace Event)
SQL Auditing
More goodies from 2008

66. Part 3: Monitoring
Click to edit Master text styles
Second level
●
Third level
●
Fourth level
●
Fifth level

67. Monitoring
Existing tools

68. MSSQL Job Monitoring
Monitors “scheduled jobs”
Written in java (server)
DBTuna
Some commercial thing
Haven’t checked
check_mssql_nt
Monitors “many things” (w32 binary)
Written in debian (called via nrpe)
Existing Nagios scripts

69. check mssql databases status by snmp
Queries status via snmp
Check MSSQL Database Mirroring
Queries specific tables (sys schema)
Written in perl
check_mssql_health
Queries many aspects of the server
Written in perl
check_mssql_write.py
Existing Nagios scripts

70. check_mssql_sproc
Monitors “things” by executing a procedure
Written in perl
check_mssql_monitor
Wrapper for sp_monitor
Written in perl
check_sql
Runs various queries and checks various things
Written in perl
Existing Nagios scripts

71. Best ones:
check_sql
Good ones:
check_sql
Check MSSQL Database Mirroring
check_mssql_health
check_mssql_write.py
check_mssql_sproc
check_mssql_monitor
Existing Nagios scripts

72. If you are using ODBC
check_sql is sort of broken (I think) 
At least I do this to connect
my $cs = "DBI:$driver:$hostname";
#my $cs = "DBI:$driver:" . ($database ? "database=$database;" : '') .
"host=$hostname" . ($port ? ";port=$port" : '');
But I am not a Perl DBI expert…
WARNING!

73. There are nice tools to work with this…
use them…
Ask you DBA and your developers
They know what requires monitoring
Don’t believe there is a “boxed solution”
There seldom is
Don’t “finish” once the server is monitored.
Monitor your application as well!
Don’t forget “non-events”
Don’t do this from
“Nagios”

74. Assignment
Existing scripts

75. Use check_sql to check that
select count(*) from orders is 0
Task/Question

76. perl check_sql -h
Hint 1

77. perl check_sql -h
perl check_sql
-H osmc -d ODBC -U osmc -P osmc
●
-q "select count(*) from orders“
●
-e 0
Hint 2

78. Hopefully it worked out…
(My) Answer

79. Monitoring
Performance Counters

80. IO
Pages / second
●
MemoryPages/sec
Physical disk time
●
PhysicalDisk(_Total)% Disk Time
Database files:
Database file size
●
SQLServer:Databases(???)Data File(s) Size (KB)
Database log file size
●
SQLServer:Databases(???)Log File(s) Size (KB)
A few Good Counters

81. Monitoring
DMVs

82. dm_db_*:
Databases and database objects
dm_exec_*:
Execution of user code and associated
connections
dm_os_*:
Memory, locking, and scheduling
dm_tran_*:
Transactions and isolation
Dynamic Management
Views

83. CPU
sys.dm_exec_query_stats
Memory:
sys.dm_os_sys_info
I/O:
sys.dm_os_wait_stats
Blocked sessions:
sys.dm_os_waiting_tasks
Technet link with sample queries:
A few good DMVs

84. Monitoring
The art of stealing...

85. …the fifth of november
One of the best things about DVM:
..are the reports…
●
Management Studio Reports
– easy way to view DMV data
●
Performance Dashboard
– downloadable add-on with extra reports
These are premade reports with a nice gui
●
Something like what your “friends” with that “big expensive
alternative” to OSS keep selling right?
●
Why not steal it?
Remember, remember,

86. Are;
XML files (*.rdl)
With predefined queries (good)
And a lot of nice boxes and what not (crappy)
Look for:
//DataSets/DataSet
Or if you prfer:
●
<DataSets>
– <DataSet Name=”<WHATEVER>">
Management Studio
Reports

87. Assignment
Dissecting a report

88. Open a report and steal the query…
For instance the historical_io.rdl report
Task/Question

89. The PDB (Performance Dash Board) folder has
the reports…
Hint 1

90. The PDB (Performance Dash Board) folder has
the reports…
The file I called historical_io.rdl
Hint 2

91. The PDB (Performance Dash Board) folder has
the reports…
The file I called historical_io.rdl
Look for DataSet (and then Query)
Hint 3

92. The PDB (Performance Dash Board) folder has
the reports…
The file I called historical_waits.rdl
Look for DataSet (and then Query)
No it is not CHECK_DEPENDENCIES
Hint 4

93. Master thief I dub thee
(My) Answer

94. Monitoring
Applications

95. Dont forget to monitor your applications
Check that tables and columns are ”writable”
Check that counts (rows)
Check that new rows ”are coming”
●
(recent item count)
Check that ”things” work.
Talk to your DBA and developers!
Applications!

96. Assignment
Monitoring my order system

97. Create a query which makes sure that orders
are coming in
For instance:
Select count(*) from orders
●
(in real life you probably want;
– WHERE order_date > DATEADD(DD,-
1,CONVERT(CHAR(8),CURRENT_TIMESTAMP,112))
●
Or something similar)
Task/Question

98. Use the “run.pl” perl script
Hint 1

99. Use the “run.pl” perl script
perl run.pl "select count(*) from orders"
Hint 2

100. Whoops… we are not gonna be rich afterall 
(My) Answer

101. Part 4: Scripting
Click to edit Master text styles
Second level
●
Third level
●
Fourth level
●
Fifth level

102. Server side;
Remote access from the monitoring server
Requires more ”firewall openings”
More configuration server side
Your DBA might disprove
Client side;
Local access to SQL Server (remote access to agent)
Requires less firewall openings
● And you probably require this anyway
More configuration client side
More work ”distributing” your scripts
Client or Server side?

103. Perl is a good candidate
Support SQL queries
Fairly simple driver handling (via ODBC)
Has “helpers” for database access
What we have been using all this time
Most languages can be used though
Used by various existing scripts
●
Python, PHP
●
Java (but it has a very high overhead)
Which language? (Server
side)

104. Perl is (still) a good candidate
Makes your compatible
Requires installation (ActiveState or similar)
VB is a popular choice
With the Library from op5 you get much “for free”
PowerShell
Probably your best bet if you wanna do cool stuff
Many other choices as well
Which language? (Client
side)

105. The good;
Support SQL queries
Fairly simple driver handling (via ODBC)
Has “helpers” for database access
Makes your compatible
The bad;
Requires installation (ActiveState or similar)
Some issues with 64bit and databases
● Requires the SAME as the ODBC libraries
Perl (Client side)

106. The good;
Support SQL queries - ???
Simple driver handling - ???
Has helpers “Nagios API”
Can do a lot of “other things”
●
Ie. a “check_all” is possible
The bad;
No “Nagios API” available
Script will not run on Linux
VB (VisualBasic)

107. The good;
Support SQL queries
Simple driver handling
Has helpers for database (SQL) access
Can (re)configure the database
The bad;
Somewhat arcane syntax
No “Nagios API” available
Script will not run on Linux
PowerShell

108. SQL Server Management Objects (SMO)
Replication Management Objects (RMO)
Analysis Services Management Objects (AMO)
Just a quick PowerShell
slide

109. Click to edit Master text styles
Second level
●
Third level
●
Fourth level
●
Fifth level

110. Michael Medin
michael@medin.name
http://www.linkedin.com/in/mickem
Information about NSClient++
http://nsclient.org
Slides, and examples at:
Thank You!