linux memory management

ravalox · Apprentice Joined: 31 Jan 2005 Posts: 217

Okay, I have a question to posit to all us linux users out there about memory management. The kernel does one thing that I can't wrap my mind around. I recently got a copy of neverwinter nights. I feel this runs better on linux than I ever saw it run on windows, however linux has a strange disability. When I'm playing a game, when it consumes past a certain amount of memory, the kernel will kill the process. So I go from game to desktop. I've seen the kernel do this on a number of occassions, gdesklet memory leakage used to consume most of my RAM, so often UT2k4 would be aborted right in front of me. I am positive this is what linux is doing. Now, what I don't get is that I can play neverwinter on windows all day and never have to worry about this. So what's the deal? I don't understand how linux can just do that when a process exceeds some kind of unclear threshold? This hardly seems like an "elegant solution" to memory management problems, when side-by-side windows seems to chug on through.

bollucks · l33t Joined: 27 Oct 2004 Posts: 606

Linux memory management does not do what you say unless your programs have completely exhausted all the ram and swap. More likely the linux versions are more memory hungry and also more unstable since they're usually 3rd class citizen ports of the originals.

luisfelipe · Guru Joined: 09 Apr 2005 Posts: 377

Dude, how much memory/swap you have ? Cuz I've never seen that ?

Perhaps you have something else running that is leaking memory or something.

ravalox · Apprentice Joined: 31 Jan 2005 Posts: 217

I'm running off of 512 megs of RAM, I can't recall the swapspace off the top of my head for that machine. I have another gentoo system that runs off of a gig of RAM and I think has a 512 swap setting that does the same thing to me. I don't have any other processes eating RAM though, I top'ed to look for one.

quickshiftin · Guru Joined: 27 Jul 2004 Posts: 345 Location: Denver, CO

have you tried looking at any system logs at all? maybe your system can provide an explanation.

you said the program gets shutdown once a certain amount of memroy is consumed, how did you arrive at this conclusion; and what amount of memory consumption make the program halt execution?

luisfelipe · Guru Joined: 09 Apr 2005 Posts: 377

Well, you got another problem then. Cuz I played neverwinter for 3 days straight with only 256 ram and
256 swap.
Try closing every single program you have, start X witouth any window manager (or with a really simple one
such as evilwm) and try running nwn to see what happens.

ravalox · Apprentice Joined: 31 Jan 2005 Posts: 217

I am running this after a fresh boot usually with nothing extraneous outside of gnome running. With the increase in memory on my other computer (the one with a gig) the same things happens but it takes MUCH longer for it to. I don't believe this is the client crashing, this has all the symptoms of memory conservation. On my desktop it did the exact same thing with Ut2k4.

luisfelipe · Guru Joined: 09 Apr 2005 Posts: 377

Really weird. Which kernel are you using ? Vanilla, gentoo-sources, or other ?

ravalox · Apprentice Joined: 31 Jan 2005 Posts: 217

the gentoo sources for 2.6.13, but honestly I have an Ubuntu install that does the exact same thing to me.

quickshiftin · Guru Joined: 27 Jul 2004 Posts: 345 Location: Denver, CO

can you post the output of

luisfelipe · Guru Joined: 09 Apr 2005 Posts: 377

Another couple of questions :

1) Is your RAM working ok (memtest and stuff) ?
2) What is your video card and driver version ?

curtis119 · Posted: Tue Jan 31, 2006 7:11 pm Post subject:

This is an extremely odd error your having. If the kernel uses up all the RAM+swap it will do one or all of three things:

Reduce the size of the buffer and page caches
Swap out System V shared memory pages
Swap out and discard the oldest pages

One thing it NEVER does is kill an application outright, if it runs out of physical RAM+swap your system will simply crawl to a halt and require a hard reboot. There are daemons around, like verynice, that will kill a runaway proccess for you and I *think* ubuntu may install one by default (don't quote me on that though) but on a Gentoo box you would have to explicitly install and configure it.

One thing that *would* kill the app is openGL errors. It wouldn't kill it implicitly but if a fatal error occured the app would die. If I were you I would look into the video card (driver) and your openGL implementations.
_________________
Gentoo: it's like wiping your ass with silk.

luisfelipe · Guru Joined: 09 Apr 2005 Posts: 377

Actually, I run Gentoo and I've already seen apps that use all the RAM+swap cause other applications to be killed.
Lots of times actually. I can even reproduce it (it's been happening with me for the last two gcc 4.1 beta ebuilds,
during the installation phase).

martin.k · Posted: Tue Jan 31, 2006 9:03 pm Post subject:

Under emergency memory situations - when Linux runs out of memory (both physical and swap combined) the kernel starts killing processes. It uses an algorithm to work out which process to kill first - it tries to kill offending memory hogs that have been running for a short amount of time first before less used processes that have been running for a long time, which are most likely important system services.
This functionality is known as the out of memory (OOM) killer.

So it's probably your crappy game, app or whatever that causes the above issue. If your game is RAM + swap hog and OOM kills it - that's perfect! If the game is not the cause - then nobody's perfect

_________________
linux-2.6.17 +ck +R4 +lockless +genetic-as +... więcej nie pamiętam

LRU #299256

frenkel · Veteran Joined: 13 May 2003 Posts: 1034 Location: .nl

Are you sure the kernel killed it? I have a lot of friends who experience segmentation faults with neverwinter nights. Then you also drop back to the desktop. Try to start it from a terminal and past the errors on the terminal after the game was killed.

curtis119 · Posted: Wed Feb 01, 2006 9:13 pm Post subject:

ravalox · Apprentice Joined: 31 Jan 2005 Posts: 217

Nwn has only seg faulted on me one time, every other instance of this application behaving unhelpfully it has simply dropped me back to my desktop with an error along the lines of "stopped on line 312". I have learned that gdesklets seems to have a voracious memory leaks, so restarting it periodically helps matters a lot, but it still shouldn't do this.

widan · Posted: Tue Feb 07, 2006 4:07 pm Post subject:

AllenJB · Veteran Joined: 02 Sep 2005 Posts: 1285

An OOM Killer is stupid? What do you really expect to happen when an app consumes all your available memory? I'd rather have the offending app killed as opposed to my system becoming totally useless.

Allen

codergeek42 · Posted: Tue Feb 07, 2006 5:23 pm Post subject:

curtis119 · Posted: Wed Feb 08, 2006 1:56 am Post subject:

Apparently the docs I found on this OOM killer were out of date and highlighted the faults of the BETA code. It seems that it is a good thing after all now that they have the bugs worked out of it. All this time I have been carefully keeping tabs on memory usage on my servers in order to avoid OOM lockups. Not that I will stop doing this since it is still a good idea to keep track of this sort of stuff for utilization tracking but now I won't be so paranoid about it. Thanks for bringing this up. I learned something new today and that's always a Good Thing(tm).
_________________
Gentoo: it's like wiping your ass with silk.