[solved] Can't load more than about 256 MB of memory/process

wbecker · Last edited by wbecker on Wed Jan 20, 2010 10:21 am; edited 1 time in total

It's really bizarre. I have several amd64 systems running but on one particular one, I find that as soon as an application goes over about 256MB ram it either starts using swap or (if I turn swap off) it just crashes.

I have 4GB of memory on this machine, so it oughtn't be a memory issue.

I thought it was just eclipse at the start - so it was a Java issue but I have fiddled with eclipserc to no avail. I then found that plasma and vmware and all sorts of things trigger it.

Is there some funny kernel setting that I could have on which I shouldn't?

I dont use any funny use flags, just:

aCOSwt · Posted: Mon Jan 18, 2010 5:47 pm Post subject:

Could you post the output of

wbecker · Posted: Tue Jan 19, 2010 9:06 am Post subject:

It is this:

aCOSwt · Posted: Tue Jan 19, 2010 9:29 am Post subject:

And the output of :

wbecker · Posted: Tue Jan 19, 2010 9:30 am Post subject:

I havent changed it manually - it is currently on 60

LesCoke · n00b Joined: 02 Jun 2008 Posts: 48 Location: Denton, Tx

Check how you have your kernel memory model setup. Specifically check CONFIG_NOHIGHMEM, CONFIG_HIGMEM4G and CONFIG_HIGHMEM64G. I'm pulling these from an x86 2.6.23 kernel, amd64 and later kernels may have different names. Having 4G of RAM uses the whole 32bit address space leaving no place for the kernel to page VM into, thus these control which PAE model the kernel uses.

But I'm puzzled why the kernel limited the process to only 256M.

Les

wbecker · Posted: Tue Jan 19, 2010 9:41 am Post subject:

That's strange...

If I go make menuconfig, and do a search it appears and tells me that I don't have HIGHMEM selected, but if I go to the place in Processor type and Features, then it is not there... Nor is there any thing to do with HIGHMEM in the .config file.

Has this changed or is it set by default?

aCOSwt · Posted: Tue Jan 19, 2010 9:44 am Post subject:

In the VMWare Server Console, what is the setting for the tweak memory/swap use ?

wbecker · Posted: Tue Jan 19, 2010 9:48 am Post subject:

Oh....

I have reserved 3000MB for VMware. I figured that it would only prevent other processes from using it if it needed it... But it could be that it just prevents anyone else from using that 3Gig.

I have moved it down to 1000MB - I'll see if that helps.

Thanks!

aCOSwt · Posted: Tue Jan 19, 2010 9:49 am Post subject:

LesCoke · n00b Joined: 02 Jun 2008 Posts: 48 Location: Denton, Tx

I checked my older 2.6.19 x86_64 system and found that it was not an option in that kernel; only found FLATMEM and SPARSEMEM. My ulimits are roughly the same and your values look good. I haven't updated this older Sempron64 system since it still has hard drive failures I haven't fixed yet.

Are you certain swap is setup?

I'm currently out of ideas.

Les

wbecker · Posted: Wed Jan 20, 2010 10:19 am Post subject:

I'm pretty sure it was my VMware setting. It's all working now.