Smarter Technology
for Better Business
How to Build a
Centralized Monitoring Solution:
Never Miss a Backup
Building a Centralized Monitoring Solution
Agenda
Introductions
Today’s Goals
The Traditional Method
New Centralized Metho...
Introductions and Qualifications:
Presenting Today…
• Over 20 years of solid project leadership and technical experience i...
Building a Centralized Monitoring Solution
Today’s Goals
Outline a Better Approach to Monitoring to Ensure:
Databases Avai...
Building a Centralized Monitoring Solution
Traditional Maintenance and
Alerting Methods
Utilize Maintenance Plans
Utilize ...
Building a Centralized Monitoring Solution
Typical Alert Generating Activities:
Check Frequency
DB Integrity
Daily
(At lea...
Building a Centralized Monitoring Solution
Pitfalls with Traditional Methods:
Missed Backups:
• Maintenance Plans are modi...
Building a Centralized Monitoring Solution
Pitfalls:
Missed Backups
Too Many Alerts
No Notice of Failed Jobs
Undetected of...
Building a Centralized Monitoring Solution
A Better Method:
What is
Centralized
Monitoring?
All Monitoring Components
Run ...
Building a Centralized Monitoring Solution
A Better Method:
Why
Centralized
Monitoring is
Better
 Scalable and Less Labor...
Building a Centralized Monitoring Solution
A Better Method:
Why
Centralized
Monitoring is
Better
 Proactive, Not Reactive...
Single Tier
Approach
Components Running on
One centralized DB Server:
• Meta Data to hold info
about servers, connection
s...
Multi-Tier
Approach
Adds App Server and
Additional Components:
• Same Data Model:
adds additional tables for
scheduling jo...
Building a Centralized Monitoring Solution
Key Tables
Included in Data Model
Server
Holds server configuration information...
Create
Multiple
Data Gather
Jobs
Sends Single
Alert Email
Generates
• Trends
• Forecasts
• Consolidation
Opportunities
Sto...
Building a Centralized Monitoring Solution
Implement Key Queries to Detect:
Building a Centralized Monitoring Solution
SQL Query: Gather Database Backup Info
Building a Centralized Monitoring Solution
SQL Query: Gather Database Failed Jobs
Building a Centralized Monitoring Solution
SQL Query: Gather Database File Size
Building a Centralized Monitoring Solution
SQL Procedure: Gather Disk Free Space
Utilize SQL Hidden Procedure:
EXEC master...
Building a Centralized Monitoring Solution
SQL Procedure: Gather Access Information
Building a Centralized Monitoring Solution
Detect and Alert SP: Failed Backups/Jobs
 SELECT Last Set of Records from Data...
Building a Centralized Monitoring Solution
Detect and Alert SP: Disk Space/File Size
 SELECT Current Set of Records from ...
Building a Centralized Monitoring Solution
Detect and Report Inactivity
 SELECT Current Set of Records from Data
Results ...
Building a Centralized Monitoring Solution
Trending and Forecasting
Forecasting
Stored
Procedure
Collects trending data,
a...
Building a Centralized Monitoring Solution
Add Jobs to Assist with Tuning
For Example:
Monitor Index Usage
Monitor Memory ...
Building a Centralized Monitoring Solution
The Aspect Solution: Prodative® WatchDog EMS
WatchDog
EMS
Centralizes
the Right...
Monitor Critical Information and Alerts:
 Database Availability
 Missed Backups
 Critical Disk Space & File System Usa...
 View Trending Data, Analyze
and Generate Reports
 Track Server and Job Statistics
 Track Disk Space and Database
Growt...
 Maintain and Extend Monitoring
using Flexible Controls:
 Extend and Add Monitoring Jobs
 Set Alerting Parameters
 Add...
Building a Centralized Monitoring Solution
Questions…
Upcoming SlideShare
Loading in …5
×

How to Build a Centralized Database Monitoring Solution - Never Miss a Backup

1,118 views

Published on

As every DBA knows, the one question you want to be able to answer affirmatively is “Can you recover that data?” Monitoring is critical, but monitoring methods can be imperfect. Traditional methods are difficult to set up and maintain across your entire environment, resulting in incomplete monitoring and missed alerts, so that it’s difficult to be sure of your answer to that all important question. This presentation will: • Show you how to implement a better way to monitor your database environment that is more efficient, easier to maintain, and guarantees that you never miss an alert. • Share the methodology, framework, and key syntax, so that you are certain the databases you are responsible for are always up, always backed up, and never run out of disk/file space. So that your answer to that all important question is always YES!

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,118
On SlideShare
0
From Embeds
0
Number of Embeds
420
Actions
Shares
0
Downloads
76
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

How to Build a Centralized Database Monitoring Solution - Never Miss a Backup

  1. 1. Smarter Technology for Better Business How to Build a Centralized Monitoring Solution: Never Miss a Backup
  2. 2. Building a Centralized Monitoring Solution Agenda Introductions Today’s Goals The Traditional Method New Centralized Method The Approach Aspect Solution
  3. 3. Introductions and Qualifications: Presenting Today… • Over 20 years of solid project leadership and technical experience in Data Management, Data Integration, Database Administration. • Extensive expertise with logical and physical DBA activities with both SQL Server and Oracle. • Currently providing remote database administration and monitoring services for multiple clients across industries. John Abrams Senior DBA/Data Architect Mike McDonnell Senior DBA/Data Architect
  4. 4. Building a Centralized Monitoring Solution Today’s Goals Outline a Better Approach to Monitoring to Ensure: Databases Available Backups Occurring as Scheduled No Issues with Disk Space Usage/File Size And… Achieve All This in the Most Efficient Way Possible
  5. 5. Building a Centralized Monitoring Solution Traditional Maintenance and Alerting Methods Utilize Maintenance Plans Utilize Third-Party Backup Agents Utilize Custom Scripts with Alert Notifications Or….no maintenance/monitoring at all
  6. 6. Building a Centralized Monitoring Solution Typical Alert Generating Activities: Check Frequency DB Integrity Daily (At least weekly) Reorganize Index Weekly (Daily if time) Rebuild Index Weekly (Daily if time) Update Statistics Weekly (Daily if time) Execute Agent Job Depends Backup DB (Full) Weekly Backup DB (Diff) Daily Backup DB (Transaction) Hourly Maintenance Cleanup Daily Clean Up History Weekly
  7. 7. Building a Centralized Monitoring Solution Pitfalls with Traditional Methods: Missed Backups: • Maintenance Plans are modified from All User to selected databases, potentially leaving new databases without backup. No Notice of Failed Jobs: • SQL Agent does not restart, no jobs run-without your knowledge. Missed Alerts: • Too many alerts: Multiple success/failure emails buried in inbox. Undetected Unavailable Database Instance: • Monitoring not centralize; difficult to detect.
  8. 8. Building a Centralized Monitoring Solution Pitfalls: Missed Backups Too Many Alerts No Notice of Failed Jobs Undetected offline DB instances Resolution: Proactive Centralized Monitoring …that pulls data out of the environment.
  9. 9. Building a Centralized Monitoring Solution A Better Method: What is Centralized Monitoring? All Monitoring Components Run on a Centralized Server:  Monitoring Jobs to collect and store critical info.  Connection information used to link to monitored servers.  Procedures to run jobs and generate notifications.
  10. 10. Building a Centralized Monitoring Solution A Better Method: Why Centralized Monitoring is Better  Scalable and Less Labor Intensive: Single set of monitoring jobs, rather than deploying the same set of jobs across 100 servers.  Nothing Slips Through the Cracks: You do not miss when jobs do not run.  One Single Email: Email displays results across entire environment-rather than multiple emails lost in your inbox.  Historical Data Readily Available: Data can be used for trending and alerting.
  11. 11. Building a Centralized Monitoring Solution A Better Method: Why Centralized Monitoring is Better  Proactive, Not Reactive: Trend and forecast collected data so you can predict issues before they occur.  Discover Opportunities for Consolidation: Easy to collect data that allows you to  Retire inactive databases  Consolidate servers  Reclaim disk space  Flexible Framework: The sky is the limit—you can easily expand & extend monitoring & trending.
  12. 12. Single Tier Approach Components Running on One centralized DB Server: • Meta Data to hold info about servers, connection strings, jobs to send alerts, results tables, etc. • Linked servers to connect to all destination databases. • Dynamic SQL Generating Procedure to run jobs, store results, and send alerts. Building a Centralized Monitoring Solution Two Approaches: Single-Tier
  13. 13. Multi-Tier Approach Adds App Server and Additional Components: • Same Data Model: adds additional tables for scheduling jobs to run on instances. • Utilizes Windows Service Agent to connect, execute SQL, and send alerts. • Adds Front End to manage job maintenance, view alerts and reports. Building a Centralized Monitoring Solution Two Approaches: Multi-Tier
  14. 14. Building a Centralized Monitoring Solution Key Tables Included in Data Model Server Holds server configuration information. DB Instance Stores database connection string information for each instance. Jobs Stores job description and SQL. Job Schedule Stores scheduling information. Results Tables Multiple tables store results gathered from each job.
  15. 15. Create Multiple Data Gather Jobs Sends Single Alert Email Generates • Trends • Forecasts • Consolidation Opportunities Stored Procedures Analyze Data Building a Centralized Monitoring Solution Centralized Monitoring Components: Data Stored in Results Tables
  16. 16. Building a Centralized Monitoring Solution Implement Key Queries to Detect:
  17. 17. Building a Centralized Monitoring Solution SQL Query: Gather Database Backup Info
  18. 18. Building a Centralized Monitoring Solution SQL Query: Gather Database Failed Jobs
  19. 19. Building a Centralized Monitoring Solution SQL Query: Gather Database File Size
  20. 20. Building a Centralized Monitoring Solution SQL Procedure: Gather Disk Free Space Utilize SQL Hidden Procedure: EXEC master..xp_fixeddrives
  21. 21. Building a Centralized Monitoring Solution SQL Procedure: Gather Access Information
  22. 22. Building a Centralized Monitoring Solution Detect and Alert SP: Failed Backups/Jobs  SELECT Last Set of Records from Data Results Table.  Use MAX(LASTMOD) value to select records.  IF detects SOME records (i.e., where backup date is older than 24 hours OR where job status indicates failure)  THEN  Generate alert email.  Write records to an Alert table. Alerting Stored Procedure Detects failures, sends alert, and stores alert history
  23. 23. Building a Centralized Monitoring Solution Detect and Alert SP: Disk Space/File Size  SELECT Current Set of Records from Data Results Table AND Set of Records inserted X hours (e.g. 24 hours) ago.  Use MAX(LASTMOD) value to select both sets of records.  COMPARE file size/disk space against rolling average (specified time period, yesterday, per month)  IF detects SOME records where size increased by X%...  THEN  Generate alert email.  Write records to alert table. Alerting Stored Procedure Detects out of range conditions, sends alert, and stores alert history
  24. 24. Building a Centralized Monitoring Solution Detect and Report Inactivity  SELECT Current Set of Records from Data Results Table.  COMPARE today’s date with last access date from current record.  IF date is more than X days ago...  THEN  Add to report.  Write records to history table. Reporting Stored Procedure Detects inactivity, sends report, and stores history
  25. 25. Building a Centralized Monitoring Solution Trending and Forecasting Forecasting Stored Procedure Collects trending data, analyze over time, and forecasts to prevent issues from occurring  SELECT  Data Set (Time Period-Value) for analysis.  Metric:  Free Disk Space  Data File Size  Record Counts  Table Size  ANALYZE:  Determine: # of Periods to Analyze  Determine: # of Periods to Forecast  CALCULATE & FORECAST:  Calculate & apply historical slope to create forecast.
  26. 26. Building a Centralized Monitoring Solution Add Jobs to Assist with Tuning For Example: Monitor Index Usage Monitor Memory Utilization Monitor Distribution of Data within Tables Monitor Fact Table Record Count Growth Utilize PowerShell to collect Server Metrics Flexible Framework Lets You Easily Extend Monitoring
  27. 27. Building a Centralized Monitoring Solution The Aspect Solution: Prodative® WatchDog EMS WatchDog EMS Centralizes the Right Info & Alerts Across Platforms Capture Key Data for Trending & Forecasting Easy to Configure and Use Displays info and alerts (e.g. availability, backup, and file size)via a web portal; no need to go to the server for info. Solution is simple to configure and use for immediate impact; no need for full-time staff for support. Enables you to analyze, predict, prepare, and address issues before users are impacted.
  28. 28. Monitor Critical Information and Alerts:  Database Availability  Missed Backups  Critical Disk Space & File System Usage (at both the device and subdirectory level)  Problematic Indexing and Queries Monitor Info & Alerts Introducing Prodative® WatchDog EMS Features
  29. 29.  View Trending Data, Analyze and Generate Reports  Track Server and Job Statistics  Track Disk Space and Database Growth Over Time Analyze & Report Introducing Prodative® WatchDog EMS Features
  30. 30.  Maintain and Extend Monitoring using Flexible Controls:  Extend and Add Monitoring Jobs  Set Alerting Parameters  Add Reports  Maintain Users Maintain & Extend Introducing Prodative® WatchDog EMS Features
  31. 31. Building a Centralized Monitoring Solution Questions…

×