Virtual Mail Guide and too many operations (solved)

Jaxom · Tux's lil' helper Joined: 31 Jan 2003 Posts: 137

I'm having an issue with the virtual mail guide currently, but I'm not sure it's an issue. I set it up and it's working great. except for this one problem. I have what I think is entirely too many smptd and mysqld instances running. I'm not sure if this SHOULD be like that or not.

I left mysql at it's default connections of 100 and today trying to add a user with phpmyadmin I got "too many connections". So when I looked I had about 3 screens full of mysql and smtpd running. I restart and they all came back. I increased my allowed mysql connections and that atleast solved the issue in the short term, but when I do ps -aux now I still have well over a 100 connections and there's only 93 boxes on the machine. Not to mention that some of the connection start times are hours long.

Is this a timeout issue? Is there somewhere I can change that? Or is this normal? The only "time out" I have in the main.cf is this...

kashani · Posted: Thu Jan 15, 2004 8:59 am Post subject:

Not seeing anything like that at all on my system, 30 mail boxes, 45 domains, 100 aliases. Generally you see trivial-rewrite when mail is delievered. I got a few ideas for you to look into.

1. You're getting attacked, spammed, local user has a virus, etc. Best thing to do is start tailing your log file to see what's going on.

2. You've updated mysql recently and things aren't playing nice. Most likely if you've updated from mysql 3.x to 4.x. emerge deep is your friend here.

3. You've got a new firewall or some such that's screwing with the connections.

Out of the three, 1 is the most likely. That tls timeout is just the caching so your average use doesn't have to renegotiate each time. That should actually cause less mysql hits, not more.

I'd do a netstat -n and see where all the postfix connections are coming from. You might want to look at the postfix queue file and see what's in there. I've seen Postfix behave oddly once it starts getting 40k mails in the queue. Use postqueue -p to check that.

kashani
_________________
Will personally fix your server in exchange for motorcycle related shop tools in good shape.

Jaxom · Tux's lil' helper Joined: 31 Jan 2003 Posts: 137

Well, it falls back on getting spammed...dictionary spammed at that. That's what I was afraid of. I do have some mail stuck in outgoing queue that's being rejected by the spammer that sent them. Short of deleting each e-mail individually, is there a command that will dump everything in queue?

I'm sure I could probably stop some of this problem once I manage to figure out how to block, in postfix, anything that's trying to come in to an invalid user.
_________________
Undisputed Heavyweight Champion. If it's undisputed, WHAT'S ALL THE FIGHTING ABOUT?!?! -- George Carlin

kashani · Posted: Fri Jan 16, 2004 12:08 am Post subject:

You can use postsuper -d ALL to delete the entire queue. You might want to flush the queue first to kick out anything that might be real mail before doing it. You could also write a for loop that grep and awk postqueue -p for the addresses and then has postsuper delete them.

kashani
_________________
Will personally fix your server in exchange for motorcycle related shop tools in good shape.

Jaxom · Tux's lil' helper Joined: 31 Jan 2003 Posts: 137

Actually it appears that even though the spammers are hitting it, the mysql connections simply aren't being closed so they are left open until mysql times out the connection.

Which leads me to believe I need to reduce the timeout on mysql (ummm how?), or that there's something kind of out of whack with the install that doesn't tell mysql the connection is finished.
_________________
Undisputed Heavyweight Champion. If it's undisputed, WHAT'S ALL THE FIGHTING ABOUT?!?! -- George Carlin

kashani · Posted: Wed Jan 21, 2004 10:11 pm Post subject:

A bit of speculation here, but generally open processes try not to close mysql connections in the name of persistence. Persistent connections are generally a good thing, until your server starts opening a large number of processes.

Assuming this is somewhat like Apache/php each Postfix process is going to want it's own mysql connection. It doesn't open a single connection to mysql as Postifx is like qmail, lots of little processes. The more Postfix connections, the more mysql connections, and your server starts slowing to a crawl.

I'd try and do a compare of Postfix to mysql processes. If they're reasonable close, then closing mysql connections isn't the problem.

kashani
_________________
Will personally fix your server in exchange for motorcycle related shop tools in good shape.

Jaxom · Tux's lil' helper Joined: 31 Jan 2003 Posts: 137

I have, I end up with approximately 10 mysql connections to each postfix instance. I did set connection_timeout = 20 (20 minutes?) in my.cnf to see if that helps a little bit.

right now what I see are things like this....

kashani · Posted: Thu Jan 22, 2004 1:16 am Post subject:

Have you changed unknown_local_recipient_reject_code = 450 to be 550 yet? 450 is the starting value with 550 being a bit more correct and final. The default Gentoo setting is 450. This may help with rejecting mail from idiots and hopefully cut down on some connections.

kashani
_________________
Will personally fix your server in exchange for motorcycle related shop tools in good shape.

Jaxom · Tux's lil' helper Joined: 31 Jan 2003 Posts: 137

Actually, no I haven't done that. And reading that section AGAIN, I feel like....ummm, a moron maybe? I don't think that's going to resolve the problem with having so many mysql sessions open, but it could help with some of it.

I can't believe I forgot that. I was sitting there looking at that while I was building the system thinking "don't forget to change that" and of course I did.

either rate, I'm sure I'm going to have a large amount of mysql sessions over once this server completely replaces my other 2 thanks to spammers never removing dead accounts from their lists.

Either way, I'll keep an eye on it overnight and see if things improve. I'll let you know for sure in the morning.
_________________
Undisputed Heavyweight Champion. If it's undisputed, WHAT'S ALL THE FIGHTING ABOUT?!?! -- George Carlin

Jaxom · Tux's lil' helper Joined: 31 Jan 2003 Posts: 137

Welp, that was the problem. Instead of having 200+ mysql operations, it's down to a respectable 30. And instead of 45+ open connections (netstat) it's down to 6-12.

So it's safe to say my problem was not having set that to 550 which prompted mail servers to keep trying again and again. Some more so than others.

Thanks alot for your help....again.

_________________
Undisputed Heavyweight Champion. If it's undisputed, WHAT'S ALL THE FIGHTING ABOUT?!?! -- George Carlin

kashani · Posted: Thu Jan 22, 2004 6:14 pm Post subject:

To be honest I didn't think it was going to help you out that much. Good to know it was something that simple. And I'm going through the mail machines around here to make sure I actually got around to doing the same myself on all the systems.

kashani
_________________
Will personally fix your server in exchange for motorcycle related shop tools in good shape.

Jaxom · Tux's lil' helper Joined: 31 Jan 2003 Posts: 137

I honestly didn't expect that to help all that much either, but it took care of the problem here. And thinking about it in retrospect it makes sense, just goes to show sometimes a mole hile does become a mountain

_________________
Undisputed Heavyweight Champion. If it's undisputed, WHAT'S ALL THE FIGHTING ABOUT?!?! -- George Carlin