Problems with login, sh & fork()

[g32]

I have been running into an issue where I suppose the anwser is really quite simple and I am just over looking it, none the less I can't figure it out.

I recently installed gentoo and added a user (called 'g32') under the groups user & wheel. Assuming that the user would operate correctly, I emerged pam and set it up as per the Gentoo Linux Security Guide. Shortly there after I attempted to login using the name g32, and I was presented with this strange error: "-bash: fork: Resource temporarily unavailable", then it drops to a bash prompt, however nearly nothing works (ls, cat, ps, etc all do not work. cd does however work). After confronting this, I attempted to test the extend of this error. I did this by attempting to log in via SSH (namely, OpenSSH) which does work splenedly for root ( .. I know, it's insecure). SSH completely fails to log in the user, citing a fork resource unavailiblity. I also tried to su to the user from root (Note: su does have all the proper permissions, g32 is in wheel, and the binary's flags are correct.) but su also reports: "su: Cannot fork user shell". I assume all of this ties back to fork() in some way. All that auth.log contains are the reports from su that it "cannot execute /bin/bash".

In conclusion (sorry this has been so..verbose), I have researched this topic for a couple of days now and I do not know where else to go. Google suggested su's fork failure responce could mean that I am low on swap, but i'm not, I have few processes running and plenty of both swap and physical memory. I imagine that some setting in PAM or perhaps a GrSecurity setting, has lead to this problem but I don't know which one. I played with /etc/pam.d/sh and /etc/pam.d/other and they have the default settings (I also tried taking all settings off, to no avail), nor do any of these settings seem to be the cause of this problem. I also have requested help in #gentoo on irc.freenode.net, but it has been far to busy for anyone to be able to help me extensively there. Any help with this would be much appricated, and thank you for reading this lengthy post.

-- Jon

[PowerFactor]

Well, I'm pretty sure you are correct in that the fork() issue is the root of all the other problems. This is kind of a shot in the dark, but have you tried rebulilding you kernel without grsecurity. I believe grsecurity has some code to prevent fork-bombing, maybe that that is set too strict somehow on your machine.

[g32]

Thanks for your reply. I just dropped grSecurity out of the kernel but to no avail. I'm not sure I understand the interworkings of PAM but is there a way to disable, limit, or remove it? Maybe that would help, not sure anymore though.

-- Jon

[PowerFactor]

Might want to check your /etc/security/limits.conf. That is apparently where PAM puts resource limits on users. Mine is completely commented.

[g32]

Well. How about that, spent about 6 hours screwing around with this and it was one single config file. I figured it was something I overlooked. Thank you very much

-- Jon