| View previous topic :: View next topic |
| Author |
Message |
humbletech99
Veteran
Joined: 26 May 2005
Posts: 1229
Location: London
|
 Posted: Tue Jan 29, 2008 9:17 am Post subject: Logging - Text File to Syslog |
 |
|
I have some databases which log to a flat text file and I don't think that this version can log to syslog, so I'd like to stream the log file into syslog myself. I figure that keeping something like this running should work:
| Code: | | tail -f /path/to/logfile | logger |
or
| Code: | | logger -f /path/to/logfile |
My question is whether or not there is a better way of doing this?
I think that if this process is restarted, and the logfile is always quite large, that it will read at least some of it again and duplicate that information into syslog which I'd like to avoid.
Is there a better way, some program perhaps that will remember it's position in the logfile and if crashed/restarted will pick up again from that exact point without missing anything or reinserting the same log information again, possibly also handling log truncation or rotation? Does logger -f have an intelligence on this?
_________________
The Human Equation:
value(geeks) > value(mundanes) |
|
| Back to top |
|
 |
bmichaelsen
Veteran
Joined: 17 Nov 2002
Posts: 1277
Location: Hamburg, Germany
|
| Posted: Tue Jan 29, 2008 9:21 am Post subject: |
|
|
| How about setting up a named pipe (fifo) where the database logs to and adding that as a source for syslog-ng? |
|
| Back to top |
|
|
humbletech99
Veteran
Joined: 26 May 2005
Posts: 1229
Location: London
|
| Posted: Tue Jan 29, 2008 9:35 am Post subject: |
|
|
That's a very good idea, the only problem is that it would require shutting down these huge production databases to add a logging destination which is not possible... and what happens if the logger doesn't pick up from the fifo for a while because of some problem? Seems like it could be a big problem with blocking.
I prefer to just follow the current log file and send it in to syslog on this occasion.
_________________
The Human Equation:
value(geeks) > value(mundanes) |
|
| Back to top |
|
|
think4urs11
Bodhisattva
Joined: 25 Jun 2003
Posts: 6659
Location: above the cloud
|
| Posted: Tue Jan 29, 2008 7:38 pm Post subject: |
|
|
Is it needed to have the logs available on time or would it be possible to e.g. rotate the log once per day/hour?
Then you could send the older log to the syslog server with e.g. one hour delay - good enough for some 'what happened between x and y on the DB' analysis.
Depending on your needs logger might not be the best option. To really have the syslog messages 'designed' in the way you want you should use some 'sed-magic' to rearrange the fields, add some proper facility/level, have the correct host name, the process field, etc.
Actually thats how we import log files from our AS400s into a syslog server and iirc we have a similar setup with some windows-based SQL servers.
_________________
Nothing is secure / Security is always a trade-off with usability / Do not assume anything / Trust no-one, nothing / Paranoia is your friend / Think for yourself |
|
| Back to top |
|
|
humbletech99
Veteran
Joined: 26 May 2005
Posts: 1229
Location: London
|
| Posted: Wed Jan 30, 2008 3:32 pm Post subject: |
|
|
logger can apply a program name and priority and facility, that's not a problem.
Really the intelligent log following is the only problem...
It's not an option to go rotating or delaying the logging, I want to have real-time alerts if the db encounters problems (but that are not caught by our active monitoring system because the db is still up)
_________________
The Human Equation:
value(geeks) > value(mundanes) |
|
| Back to top |
|
|
|
Networking & Security
