| View previous topic :: View next topic |
| Author |
Message |
tebrown
n00b
Joined: 02 Oct 2003
Posts: 16
Location: Rochester, NY
|
 Posted: Thu Oct 23, 2003 11:25 pm Post subject: syslog-ng hangs |
 |
|
I have been having a problem with syslog-ng lately. It is similar to the problem people have experienced with metalog where logins block and restarting the logger daemon fixes the problem.
I am running a 2.4.20-gentoo-r7 kernel on an SMP machine.
The /proc/pid/status revealed the processes is sleeping, and the strace doesn't tell me anything useful that I didn't already know, but this is the output:
| Code: |
18:43:58 [4e41c428] read(3, <unfinished ...>
|
So, as near as I can tell, the select wakes up and called the read which then proceded to block. Pretty rude, huh? Unfortunately, I have *no* idea why the select is waking up. I suspect it has to do with the kernel logging, though I have not yet confirmed that fact. I did turn off kernel logging and we'll see if it cures things. If anybody has any insight into things though, I would *greatly* appreciate the input. |
|
| Back to top |
|
 |
tebrown
n00b
Joined: 02 Oct 2003
Posts: 16
Location: Rochester, NY
|
| Posted: Thu Oct 23, 2003 11:51 pm Post subject: Update |
|
|
I think I may have found the problem. I have this as the first 2 (useful) lines of my syslog-ng.conf:
| Code: | source src { unix-stream("/dev/log"); internal(); pipe("/proc/kmsg"); };
source kernsrc { file("/proc/kmsg"); };
|
So, I am guessing I have 2 handlers to /proc/kmsg, which would surely screw things up. |
|
| Back to top |
|
|
fre
n00b
Joined: 19 Apr 2002
Posts: 9
Location: Belgium
|
| Posted: Tue Nov 04, 2003 8:34 am Post subject: |
|
|
Did this fix the problem?
We are having a similar problem on one of the servers where CRON and BIND lock-up.
Then we do a 'syslog-ng stop' which doesn't seem to kill the syslog-ng process so we are forced to do a kill -9.
After the process is killed and restarted, everything seems to work again.
Thanks,
Frederik. |
|
| Back to top |
|
|
tebrown
n00b
Joined: 02 Oct 2003
Posts: 16
Location: Rochester, NY
|
| Posted: Tue Nov 04, 2003 12:23 pm Post subject: |
|
|
It seems to have fixed it. What I did to track it down was:
| Code: |
lsof -p <syslog-pid>
strace -f -v -i -t -p <syslog-pid>
|
That should tell you what file syslog-ng is blocking on, and then you can look at your conf file for possible misconfiguration. |
|
| Back to top |
|
|
fre
n00b
Joined: 19 Apr 2002
Posts: 9
Location: Belgium
|
| Posted: Tue Nov 04, 2003 1:16 pm Post subject: |
|
|
Many thanks! We'll keep an eye on it.
Regards,
Frederik. |
|
| Back to top |
|
|
PT_LAmb
Guru
Joined: 22 Apr 2002
Posts: 332
Location: Lisboa (Lisbon), Portugal
|
| Posted: Tue Jan 27, 2004 10:52 am Post subject: Re: Update |
|
|
| tebrown wrote: | | Code: | source src { unix-stream("/dev/log"); internal(); pipe("/proc/kmsg"); };
source kernsrc { file("/proc/kmsg"); }; |
|
Which should I erase? Does it make any difference?
Which kernel sources are you using? This problem is new to me, the server is running for more that a year now, and this problems started happening a month or so ago. I guess it must have been some sort of software update that triggered this. I recently changed kernel sources from gentoo-sources to gs-sources, so I'm blamming gs for this. Are you too using gs-sources?
Thanks in Advanced
_________________
SETI@Home - http://setiathome.ssl.berkeley.edu/ |
|
| Back to top |
|
|
fre
n00b
Joined: 19 Apr 2002
Posts: 9
Location: Belgium
|
| Posted: Tue Jan 27, 2004 11:03 am Post subject: |
|
|
We changed:
| Code: | source src { unix-stream("/dev/log"); internal(); pipe("/proc/kmsg"); };
|
to:
| Code: | source src { unix-stream("/dev/log" max_connections(500)) ; internal(); };
|
And it works for us ...
Regards,
frederik. |
|
| Back to top |
|
|
PT_LAmb
Guru
Joined: 22 Apr 2002
Posts: 332
Location: Lisboa (Lisbon), Portugal
|
| Posted: Tue Jan 27, 2004 11:15 am Post subject: |
|
|
Ok, I've just learned that none of them should be erased. But then, what should I change?
Later I'll try to confirm that the daemon got locked up, using strace. I have already check that su hangs locked to /dev/log, so I'm pretty sure I'm no the right thread. 
As for the kernel source question, I'm using version 2.4.23_pre8-gss.
Thanks again,
Ricardo Cordeiro
_________________
SETI@Home - http://setiathome.ssl.berkeley.edu/ |
|
| Back to top |
|
|
PT_LAmb
Guru
Joined: 22 Apr 2002
Posts: 332
Location: Lisboa (Lisbon), Portugal
|
| Posted: Tue Jan 27, 2004 11:25 am Post subject: |
|
|
| fre wrote: | We changed:
| Code: | source src { unix-stream("/dev/log"); internal(); pipe("/proc/kmsg"); };
|
to:
| Code: | source src { unix-stream("/dev/log" max_connections(500)) ; internal(); };
|
And it works for us ...
Regards,
frederik. |
Got it. I hope that does the trick.
Shouldn't this be considered a bug? I don't think I changed anything from the default configuration.
Frederik, thanks again.
Ricardo Cordeiro
_________________
SETI@Home - http://setiathome.ssl.berkeley.edu/ |
|
| Back to top |
|
|
|
Networking & Security
