I turned to NTA today and found the front page couldn't show the graphs I usually see.
Then corporate Security asked me to check NPM's syslog for a few specific events and it timed out, unable to display anything.
A review of the database showed it was far bigger than expected. And it was 99% fragmented.
A short Support case later with Solarwinds, and after a review of the logs and database, I added to my learning about certain Cisco products, their syslog exports, and NPM's native syslog solution:
- Cisco 5508 Wireless Controllers can send VAST amounts of syslog info. I knew this from a previous message of concern from a DBA,when he informed me that Orion's database was growing far too quickly. At that time I stopped our WLC's from sending their syslogs directly to Orion, and pointed them to our Splunk solution.
- Cisco ASA's whose duty is isolating a Cisco 5508 Wireless Anchor Controller in a DMZ ALSO can send COPIOUS amounts of syslog data--apparently mostly about the 5508's traffic. I had NOT redirected the ASA's output to Splunk, and it was killing my Orion syslog solution.
- NPM (according to Solarwinds Tech Support) had the syslog module added in as a nice feature, but not as a robust one. Tech Support recommended I purchase/install Kiwi Syslog and point my high volume Cisco sylsog devices at it. It should safely handle up to 2 million messages per time period (per minute? per hour? I missed that part while mulling over the concept that Orion NPM syslog had limitations).
After my DBA set a scheduled job to re-index and defragment my Orion database, and after stopping the ASA's and WLC's from overloading my Orion syslog solution, things appear much better. I honestly didn't understand the amount of syslog data the ASA's and 5508's were sending to Orion Syslog.
Learn from my learning.
And swift packets to you and your clients!
Rick Schroeder