• Save
thwackCamp 2013: Building a Large-Scale SolarWinds Installation
 

thwackCamp 2013: Building a Large-Scale SolarWinds Installation

on

  • 403 views

SolarWinds IT management products are designed to scale to IT infrastructure of all sizes, from SMBs with less than 50 devices, to large enterprise deployments with over 100K devices. Join SolarWinds ...

SolarWinds IT management products are designed to scale to IT infrastructure of all sizes, from SMBs with less than 50 devices, to large enterprise deployments with over 100K devices. Join SolarWinds customer Cardinal Health and SolarWinds Product Management as they discuss some of the considerations for deploying SolarWinds in a large, distributed environment.

Statistics

Views

Total Views
403
Views on SlideShare
403
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

thwackCamp 2013: Building a Large-Scale SolarWinds Installation thwackCamp 2013: Building a Large-Scale SolarWinds Installation Presentation Transcript

  • Building a Large-Scale SolarWinds® Installation With Leon Adato, Monitoring Architect at Cardinal Health® © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
  • © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED. Enterprise – Bigger Than a Breadbox Leon’s not-so-humble-opinion: » Over 2,000 devices » More than 20 locations » 10,000 elements
  • Most Projects Fail at the Top The Real OSI Model Politics Legal/Compliance Finance Application Presentation Session Transport Network Data© 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
  • REPLACE TITLE CANT USE SONG LYRICS » A complete, accurate, meaningful inventory. » Network connectivity to all devices part 1: ping » Network connectivity part 2: Firewall » A domain or local account that has permission to pull WMI/RPC » SNMP enabled with a known RO string on all devices – And your polling engines PERMITTED to SNMP poll © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
  • SNMP WTF? © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
  • ALERTS: The Four Questions » What will you do when you get this alert? » Do you have a documented procedure? » Can you make this problem happen on purpose (for testing) » What will happen (what is the cost) if you don’t get this alert? © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
  • Standard Enterprise Design © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
  • © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED. Kick It Up a Notch
  • © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED. Portocalypse Now* For SNMP and RPC, you will need: » ICMP bi-directional » TCP 135 » TCP 139 » TCP 445 » TCP 161 » UDP 137 » UDP 138 » For WMI? All ports over 1024, or go read “WMI Portocalypse” on thwack *Tip of the Thwack-cap to aLTeReGo
  • © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED. DMZ? Make sure these are open » ICMP bi-directionally – And that DNS works in both directions » TCP 17777 » TCP 17778 » TCP 80 (or whatever your SolarWinds portal runs on) » TCP 1433 (or whatever your database listens on) – Named instances will not work here.
  • © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED. Account Permissions » RPC – local or domain user that is a member of: – Performance Monitor Users group – Distributed COM users group – And has the “Allow logon locally” permission – (NOT a member of allow login through remote desktop) » WMI – local or domain user that is a member of: – Performance Monitor Users group – Distributed COM users group – Remote desktop users group – Has “Allow logon locally” permission – Has been added to the “root” and “CIMv2” branches of the WMI settings. » Or just give it local admin already!
  • © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED. Security Hates Me* So you can’t get them to permit WMI » Most SAM items that use WMI can be obtained via RPC – Most processes can be monitored via SNMP with CPU and RAM stats – Windows eventlog can be gathered via RPC – Windows process and/or services – Perfmon has always used RPC What you are giving up (ie: How To Make Your Case): » Virtual Mount Points » Restarting a service as an alert action » Realtime process/service explorer » Windows® script monitoring (VBScript™ or PowerShell®) » WMI Monitors (duh!) *Or: “I have to monitor through a NAT-ed VPN”
  • Server Spec’s » Separate the Database and primary poller. » Database is physical. “Need More Power” should be your request. Mine has 12 CPU and 128Gb RAM. » Pollers can be virtual. I tend to run with 8 CPU, 12Gb ram to start. » Disk is a big deal. You want lots of spindles. You want RAID 10 or you want the SAN team to tell you they can load your entire database into the memory of the storage array. » Everything has to be in the same time zone. » Primary poller and database must be in the same location © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
  • © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED. Element Counts – The Real story » Sure, you have elements (~10,000)* – Ping-able IP’s – Interfaces – Disks » But you also have UnDP’s » And SAM components » And the number of DOWN items is going to affect how much you can handle » Upshot: There is no formula. *Ultimate limit is ~110,000 per Solarwinds instance
  • Know Thy Pollers! © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
  • Processes You Have to Nail Down » Device Lifecycle – Who, how, where, when devices are added – Ditto for elements – Ditto for SAM items » Devices (and volumes, and interfaces) missing key custom property information » Decom devices » Down Devices » Devices not SNMP polling » Applications in “unknown” status » Duplicate Nodes » Bad application report © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
  • © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED. Device Down Report
  • © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED. Applications with Problems
  • © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED. Handy Documents to Have on Hand SolarWinds Run Book » Overall design » Accounts used for installation, connectivity (Poller to db) etc. » Additional software installed (Perl®, ImageMagik®, PuTTY™, etc) » How to install Windows patches » How to install SolarWinds patches » Common troubleshooting techniques » Common validation/error-checking techniques » How to open a ticket with SolarWinds (with your account ID)
  • Gotchas » Event correlation » Where does scalability end? » No DBA? That’s gonna hurt. » Time zone © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
  • © 2013 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED. Thank You! The SOLARWINDS and SOLARWINDS & Design marks are the exclusive property of SolarWinds Worldwide, LLC, are registered with the U.S. Patent and Trademark Office, and may be registered or pending registration in other countries. All other SolarWinds trademarks, service marks, and logos may be common law marks, registered or pending registration in the United States or in other countries. All other trademarks mentioned herein are used for identification purposes only and may be or are trademarks or registered trademarks of their respective companies.