I am currently able to log my messages to a MS SQL database, so I am not really asking how to accomplish that task.
I would like to know how to build the database itself. I know it will ultimately contain a massive amount of data, so I am looking for the little details that would allow the database to survive the lessons of time... at least until it becomes another person's problem.
Should I send everything to a single table, or break it up by IP, hostname, etc..?
Do I need to have the data "rolled up", similar to the way NPM/Orion retains hour/day/week/month/year data?
If so, how would I accomplish this, via scheduled task/management, triggers, procedures?
Currently, I have various rules within the Kiwi console, separating messages by similar device types and purposes. Each of those rules dumps the messages into a different table in the same database, as well as dumping the messages to different log files. This has been working fairly decently, however, after approximately 6 months, some of those tables have so many rows in there, it is taking a very long time to read.
I am really not looking to do anything fancy with Kiwi. I simply want to store the data, and I need it to be accessed quickly. I am not really concerned, so much, with the size of the overall storage, as much as I am with eventually having 5+ years of data stored in a single table of the database.
Thank you,
-Will
