THWACK: Message List - Kiwi Syslog

Hi Jeff,

Thank you, I changed the language setting.

It was in french, i configured it to english us.

I already configured this.....but i don't reboot it.

After reboot , It's Ok

I see very well the logs coming this VM.

Typically, when I am successful on a longer SolarWinds project I like to document that success on Thwack. My thought is that if I couldn't find the solution when I started then someone else might come to Thwack with the same question I had. By posting my solution, I hope I'm paying it forward. This is a pay it forward post.

We have a VM that runs the Kiwi Syslog Server and also the Kiwi Syslog Web Access application. I think the Web Access instructions may have asked for a large local drive but our Systems team gave us a max of 5GB.

I have about 30 network devices sending data to my Kiwi server as part of a pilot project. I will add over 300 more eventually but during the pilot I could already see that my hard drive was maxed out. The first thing I learned is that Kiwi Syslog Web Access keeps its data in a separate database from Kiwi Syslog Server. And the Web Access application puts a 4GB cap on the amount of data it will store. Between the files that run Kiwi and the Web Access log files, I filled the hard drive to capacity in 2-3 weeks! The solution there was to increase the size of my local D: drive from 5GB to 10GB. That gave me some breathing room. I now had enough space to store data from Kiwi Syslog.

I considered writing scripts in DOS or Python to move the daily syslog files from the Kiwi server over to the network share but I eventually realized that there is a feature in the Kiwi Syslog Service Manager (File -> Setup) that will archive the data for me. Below the Rules section is a grouping called Schedules that had nothing within it. I right clicked on Schedules and told it Add New Schedule. I now had a GUI I could configure that would back up files based on date or size. It will even append a date to the name of the *.zip file that is one day older than that of the datafile. Let me share why this is important ...

Let's say that today is January 1. Kiwi writes all of its data to a file and at the end of the day it closes that file and starts a new file for the next day. The problem is that the file is written at midnight ... on January 2. Thus, if I look at the file creation date for my log file, I will find data for Jan 1 in the file created on Jan 2. The archive process has the option to append a date (today's or yesterday's) to the filename making it easier to identify when the logs were created.

I ran into a problem using this renaming feature, however, which I haven't been able to resolve. I have Kiwi append the date to my filename each day. If I didn't then the Jan 1 data would be in the same file as the Jan 2 data. That's good. But when the archive job runs, it appends the date (today's or yesterday's) to the log file name and that becomes the zip file name. So if the data collected on Jan 1 is written into file Syslog-2020-01-02.txt, the archive name becomes Syslog-2020-01-02_2020-01-01.zip. I haven't found a way to strip today's date out of the log file name before applying yesterday's date to the archive file name. Even if I didn't add a date to the log file name so that the archive file had the correct name ... and archived daily ... the system would have a conflict when trying to create a new file for Jan 2 with the same name as the Jan 1 file name that hadn't yet been archived and erased. I haven't solved this challenge yet though admittedly, it is very a very minor issue.

The bigger challenge that I faced was that I could get my archive job to run when I kicked it off manually but it would not run via the scheduler. It turns out that the issue was that the Kiwi Syslog Server application was being launched by the Local System account, and that account did not have authorization to write to my network share. To solve the problem, I went into the Windows settings for Services and told it to run that application using a network service account (an Active Directory account created for just this sort of thing) that has admin access to my Kiwi server and also has write access to the network share. Now, when I schedule a job, it runs the way I expect: moving syslog files that are at least 7 days old each into their own zip file and deleting the original datafile. It also now writes a report to the network share telling me which files were moved.

For those who want a little more detail (like my future self!), to get to the Windows settings for Services, here are the steps:

1. MAKE SURE YOU EXIT KIWI SYSLOG SERVER CONSOLE BEFORE YO MAKE CHANGES TO THE SERVICES so that you don't corrupt the application.

2. TAKE A SNAPSHOT OF THE VM before you start. If you have to roll back, you'll only lose the data sent to Kiwi between the time you took the snapshot and when you finish the roll-back.

3. Open the Windows start menu and start typing Services. You are looking for an application called Services (Desktop App) with an icon of two gears. Click on it.

4. Once it's open, scroll down to the Ks and find Kiwi Syslog Server. Right click and go to Properties.

5. On the Properties page, go to the Log On tab.

6. Select the radio button for This account.

7. For the username field, I added the domain in front of the username for the service account. I later tested successfully. I don't know whether the domain is required or not.

8. Put in your password and confirm the password.

9. Click Apply (not OK).

10. Go back to the General tab of the Properties page.

11. (See Step 1) Click on the Stop button to stop the service.

12. Click on the Start button to restart the service using the new account.

13. Click OK.

The service has now been updated and restarted. Schedule a job in the Kiwi Syslog Server Console and see if this resolved your issue.

Thanks for sharing!

It is a little bit of work but you can also log directly to a SQL database with Kiwi. We had a set of servers we wanted to keep the messages from long term so we set up a SQL database and created an action to write those messages there. It was much easier to search in.

So, some kind of work aroud of all of this would be first install all Windows Update and then install Kiwi Syslog Server, right?

I have problem with some messages, where %MsgText% is wrapped arround double quotes.

It causes problem with external script where %MsgText% is used as a parameter for Windows batch file.

Windows batch files use double quotes for situations where a parameter may contain spaces.

So I need to replace double quotes with simple quotes before everything else.

Is this achievable? If so then how?

This is a very common issue when receiving logs, especially JSON or SNMP traps.

Create a script that replaces the double quotes with single quotes. Then it will write the result back to Fields.Rawmessagetext. This would run as an action before your external script action. This could be an action in the same rule or in a rule that fires before the external script rule.

you could also write the modified message to a custom variable and pass that to your external script. I’d suggest using CHR() replacements and not plain text quotes. Replace double quotes, CHR(34), with single quotes, CHR(39).

I'm having exact error, any fix discovered you can share? I suspect it has something to do with non-standard installation location. I choose drive letter D: instead of C:

Hey braveheart814 and all;

I'm having a similar problem. I want to change the "rows per page" in Kiwi Syslog Web Access and I keep getting prompted to change the password too. All I'm doing is clicking the "save settings" button. I;m never able to change the rows per page, it refuses to make the change without changing the password.