| View previous topic :: View next topic |
| Author |
Message |
nwmcsween
n00b
Joined: 25 May 2007
Posts: 41
|
 Posted: Wed Mar 05, 2008 5:56 pm Post subject: Compcache: Use compressed ram as fast swap. |
 |
|
I don't have anything to do with this project other than being a user of it so I thought I would throw it out here for everyone to use.
http://code.google.com/p/compcache/wiki/CompilingAndUsing
It allows you to use a part of compressed ram as a swap device.
_________________
Vanilla kernel without PITA |
|
| Back to top |
|
 |
Sadako
Advocate
Joined: 05 Aug 2004
Posts: 3792
Location: sleeping in the bathtub
|
| Posted: Wed Mar 05, 2008 6:11 pm Post subject: |
|
|
Looks interesting, how well does it work for you, ie does it compress by much and what's the performance impact like?
And just out of interest, what do you mean by "hardened fstab" (in your sig), just using "noexec nodev nosuid" on user writable filesystems?
_________________
"You have to invite me in" |
|
| Back to top |
|
|
nwmcsween
n00b
Joined: 25 May 2007
Posts: 41
|
| Posted: Wed Mar 05, 2008 6:21 pm Post subject: |
|
|
| Hopeless wrote: | Looks interesting, how well does it work for you, ie does it compress by much and what's the performance impact like?
And just out of interest, what do you mean by "hardened fstab" (in your sig), just using "noexec nodev nosuid" on user writable filesystems? |
I haven't run any performance benchmarks but it seems fast I don't exactly know how to test it though since hdparm wouldn't give valid results (maybe time dd if=/dev/random of=/dev/compcache bs=4k count=10000 although thats just write performance and wouldn't really look at read performance). Basically yes except that certain things need different options.
# cat /proc/mounts
| Code: |
/dev/sda3 / jfs rw,relatime 0 0
/dev/sda1 /boot ext2 rw,sync,nosuid,nodev,noexec,noatime 0 0
/dev/mapper/lvm-root /root jfs rw,nosuid,nodev,noatime,relatime 0 0
/dev/mapper/lvm-usr /usr jfs rw,nodev,noatime,relatime 0 0
/dev/mapper/lvm-ulocal /usr/local jfs rw,nosuid,nodev,relatime 0 0
/dev/mapper/lvm-ushare /usr/share reiserfs rw,nosuid,nodev,noatime,relatime 0 0
/dev/mapper/lvm-tmp /tmp ext2 rw,nosuid,nodev,noexec,relatime 0 0
/dev/mapper/lvm-var /var jfs rw,nosuid,nodev,noexec,relatime 0 0
/dev/mapper/lvm-vportage /var/portage reiserfs rw,nosuid,nodev,noatime,relatime 0 0
/dev/mapper/lvm-vtmp /var/tmp ext2 rw,nosuid,nodev,relatime 0 0
/dev/mapper/lvm-opt /opt jfs rw,nosuid,nodev,noatime,relatime 0 0
/dev/mapper/lvm-srv /srv jfs rw,nosuid,nodev,noatime,relatime 0 0
/dev/mapper/lvm-home /home jfs rw,nosuid,nodev,noatime,relatime 0 0
/var/lock /var/lock tmpfs rw,nosuid,nodev,noexec,noatime 0 0
/var/run /var/run tmpfs rw,nosuid,nodev,noexec,noatime 0 0
/proc /proc proc rw,relatime 0 0
udev /dev tmpfs rw,nosuid,relatime 0 0
/sys /sys sysfs rw,relatime 0 0
|
<offtopic>Not fully understanding why mount automatically uses relatime when i specify noatime.. </offtopic>
_________________
Vanilla kernel without PITA |
|
| Back to top |
|
|
Palatis
n00b
Joined: 07 Oct 2006
Posts: 23
Location: Taipei/Taiwan
|
| Posted: Sat Aug 09, 2008 2:02 pm Post subject: |
|
|
I wrote a little program to just occupy memory.
when program allocates more memory but there's no free memory left, the kernel tries to discard clean pages.
if there's no clean pages left, the kernel tries to flush dirty pages so they become clean.
thus, if you allocate memory far beyond your system ram, it swaps.
ex. you have 1GB system memory with 1GB swap, and 512MB in use, and your program tries to allocate 1GB of ram, then at least 512MB must be swapped.
| Code: | // filename: occupy_memory.cpp
// compile: g++ -g -static occupy_memory.cpp -o occupy_memory
// use: occupy_memory <size_in_kbyte>
#include <iostream>
#include <cstdlib>
int main(int argc, char *argv[])
{
if (argc != 2)
{
std::cout << "Usage: " << argv[0] << " <size_in_kbytes>" << std::endl;
return 0;
}
unsigned int bytes_to_occupy = atoi(argv[1]) * 1024;
char * p = new char[bytes_to_occupy];
#pragma omp parallel for
for (unsigned int i = 0;i < bytes_to_occupy;++i)
p[i] = 0xff;
return 0;
} |
with this program, you can `time ./occupy_memory 1048576' and see how long it takes to complete the job with different types of swap. |
|
| Back to top |
|
|
nwmcsween
n00b
Joined: 25 May 2007
Posts: 41
|
| Posted: Mon Aug 11, 2008 7:21 pm Post subject: |
|
|
Alright lets give it a shot:
1. # free -k output
| Code: |
total used free shared buffers cached
Mem: 638812 56832 581980 0 9312 17580
-/+ buffers/cache: 29940 608872
Swap: 191636 0 191636
|
2. # using memeat 700000 WITH compcache swap device
| Code: |
real 0m12.661s
user 0m5.967s
sys 0m6.038s
real 0m12.512s
user 0m6.306s
sys 0m5.257s
real 0m12.988s
user 0m6.232s
sys 0m6.106s
|
3. # using memeat 700000 WITH file based swap device
| Code: |
real 0m16.614s
user 0m6.114s
sys 0m4.000s
real 0m15.381s
user 0m6.354s
sys 0m6.148s
real 0m15.362s
user 0m5.689ss
sysr 0m4.084s
|
Now since there was no background I/O lets test it with some I/O in the background.
4. # find /* -name .keep
4 # memeat 70000 WITH compcache based swap device
(NOTE: after each run echo 1 > /proc/sys/vm/drop_caches was run)
| Code: |
real 0m16.010s
user 0m8.230s
sys 0m4.918s
real 0m16.101s
user 0m8.932s
sys 0m5.140s
real 0m16.534s
user 0m8.303s
sys 0m5.167s
|
5. # find /* -name .keep
5 # memeat 70000 WITH file based swap device
(NOTE: after each run echo 1 > /proc/sys/vm/drop_caches was run)
| Code: |
real 0m20.706s
user 0m8.082s
sys 0m6.164s
real 0m19.833s
user 0m7.912s
sys 0m5.048s
real 0m21.644s
user 0m7.454s
sys 0m5.314s
[/quote]
|
_________________
Vanilla kernel without PITA |
|
| Back to top |
|
|
neuron
Advocate
Joined: 28 May 2002
Posts: 2371
|
| Posted: Tue Aug 12, 2008 8:25 am Post subject: |
|
|
hm.. this might open up some interesting possibilities.
I got 4gb memory in this computer right now, I'm thinking compcache 2gb memory + /var/tmp/portage on tmpfs to lower disk writes, think I'm gonna give that a try.
//edit, trying to put this in the kernel I get this though :/
| Code: |
drivers/block/compcache.c: In function ‘compcache_size_setup’:
drivers/block/compcache.c:465: error: implicit declaration of function ‘strtoul’
make[2]: *** [drivers/block/compcache.o] Error 1
make[1]: *** [drivers/block] Error 2
|
|
|
| Back to top |
|
|
nwmcsween
n00b
Joined: 25 May 2007
Posts: 41
|
| Posted: Wed Aug 13, 2008 5:38 am Post subject: |
|
|
you need to edit it look in compcache.c
_________________
Vanilla kernel without PITA |
|
| Back to top |
|
|
Cyker
Veteran
Joined: 15 Jun 2006
Posts: 1746
|
| Posted: Wed Aug 13, 2008 2:10 pm Post subject: |
|
|
Hnmm... is there something like this for disk?
i.e. compressed disk swap? |
|
| Back to top |
|
|
kernelOfTruth
Watchman
Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)
|
|
| Back to top |
|
|
devsk
Advocate
Joined: 24 Oct 2003
Posts: 2995
Location: Bay Area, CA
|
| Posted: Wed Aug 13, 2008 8:19 pm Post subject: |
|
|
| neuron wrote: | hm.. this might open up some interesting possibilities.
I got 4gb memory in this computer right now, I'm thinking compcache 2gb memory + /var/tmp/portage on tmpfs to lower disk writes, think I'm gonna give that a try.
//edit, trying to put this in the kernel I get this though :/
| Code: |
drivers/block/compcache.c: In function ‘compcache_size_setup’:
drivers/block/compcache.c:465: error: implicit declaration of function ‘strtoul’
make[2]: *** [drivers/block/compcache.o] Error 1
make[1]: *** [drivers/block] Error 2
|
|
do not put it inside kernel. Just untar, make and call the use_compcache.sh script. Easy and no intrusive!
Download latest 0.4 tar from here: http://code.google.com/p/compcache/issues/detail?id=9
I have made some suggestions for further improvements there. Let's see if Nitin responds.
Currently, compcache is using 108MB of actual RAM to produce 360MB of swap on my system. That's a good hit ratio, considering if I asked for 1.5G swap and it used only 512MB RAM for it (with a 3:1 hit ratio), I would have converted my 2GB into 3GB RAM at an additional marginal cost of CPU (and which is not really a concern with a dual core).
I have also asked Nitin to put in some stats on CPU usage, just to understand what kind of cpu usage cost are we looking at. |
|
| Back to top |
|
|
SeaTiger
l33t
Joined: 22 Nov 2007
Posts: 603
Location: Toronto, Ontario, Canada
|
| Posted: Thu Aug 14, 2008 5:38 am Post subject: |
|
|
hmm, interesting project.
But isn't that swap is for times when we run low of physical memory? Seems putting swap in physical memory kind of defeat the purpose of swap. Especially in linux nowadays, linux almost never swap out memory unless really running low on memory. |
|
| Back to top |
|
|
neuron
Advocate
Joined: 28 May 2002
Posts: 2371
|
| Posted: Thu Aug 14, 2008 7:30 pm Post subject: |
|
|
| junksiu wrote: | | But isn't that swap is for times when we run low of physical memory? Seems putting swap in physical memory kind of defeat the purpose of swap. Especially in linux nowadays, linux almost never swap out memory unless really running low on memory. |
Yes, but with high priority it'll also be used as io cache. And having huge amounts of memory allows you to do "crazy" things, like keeping /var/tmp/portage in memory while compiling many things at once.
Simple ebuild for compcache-0.4.ebuild:
| Code: |
# Copyright 1999-2008 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
#inherit linux-mod eutils libtool
inherit linux-mod
DESCRIPTION="Kernel module to allow compressing pages of memory."
HOMEPAGE="http://code.google.com/p/compcache/"
SRC_URI="http://compcache.googlecode.com/files/${P}.tar.gz"
LICENSE="GPL-2"
KEYWORDS="~amd64 ~x86"
#SLOT="0"
pkg_setup() {
CONFIG_CHECK="!BLK_DEV_COMPCACHE"
ERROR_CFG="You already have compcache built into your kernel."
linux-mod_pkg_setup
}
src_compile() {
declare -x ARCH="x86_64"
emake || die "emake failed"
}
src_install() {
einfo "Install"
MODULE_NAMES="compcache(block) lzo1x_compress(block) tlsf(block) lzo1x_decompress(block)"
linux-mod_src_install
dodoc ChangeLog README
}
pkg_postinst() {
linux-mod_pkg_postinst
einfo " - To load compcache:"
einfo "# modprobe compcache && sleep 2 && swapon /dev/ramzswap0 -p 100"
einfo " - To unload compcache:"
einfo "# swapoff /dev/ramzswap0 && rmmod compcache && rmmod tlsf"
einfo "Optionally modprobe can take the following arguments:"
einfo " - compcache_size_kbytes=$SIZE_KB (by default 25% of memory)"
einfo "You might get lzo errors, ignore them."
# todo : don't build lzo modules if they are in the kernel already.
}
|
|
|
| Back to top |
|
|
synss
Apprentice
Joined: 08 Mar 2006
Posts: 282
Location: Dijon > Berlin > Tokyo > Nürnberg > München
|
| Posted: Mon Aug 18, 2008 8:17 am Post subject: |
|
|
How did you get it to compile? I get: | Code: | make -C /lib/modules/2.6.24-gentoo-r7/build M=/usr/local/src/compcache/compcache-read-only modules
make[1]: Entering directory `/lib64/modules/2.6.24-gentoo-r7/build'
make[1]: *** No rule to make target `modules'. Stop.
make[1]: Leaving directory `/lib64/modules/2.6.24-gentoo-r7/build'
make: *** [all] Error 2 |
after checking it out from svn!
_________________
Compress portage tree
Elog viewer
Autodetect swap |
|
| Back to top |
|
|
devsk
Advocate
Joined: 24 Oct 2003
Posts: 2995
Location: Bay Area, CA
|
| Posted: Mon Aug 18, 2008 3:55 pm Post subject: |
|
|
I am using this ebuild:
| Code: |
# Copyright 1999-2008 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
#inherit linux-mod eutils libtool
inherit linux-mod
DESCRIPTION="Kernel module to allow compressing pages of memory."
HOMEPAGE="http://code.google.com/p/compcache/"
SRC_URI="http://compcache.googlecode.com/files/${P}.tar.gz"
LICENSE="GPL-2"
KEYWORDS="~amd64 ~x86"
#SLOT="0"
pkg_setup() {
CONFIG_CHECK="!BLK_DEV_COMPCACHE"
ERROR_CFG="You already have compcache built into your kernel."
linux-mod_pkg_setup
}
src_compile() {
declare -x ARCH="x86_64"
sed -i -e "s:/lib/modules.*build :\$(KERNEL_SOURCE) :g" Makefile
emake KERNEL_SOURCE=/usr/src/linux || die "emake failed"
}
src_install() {
einfo "Install"
MODULE_NAMES="compcache(misc) lzo1x_compress(misc) tlsf(misc) lzo1x_decompress(misc)"
linux-mod_src_install
dodoc ChangeLog README
into /usr/bin
cp ${FILESDIR}/use_compcache.sh ${D}/usr/bin/
}
pkg_postinst() {
linux-mod_pkg_postinst
einfo " - To load compcache:"
einfo "# modprobe compcache && sleep 2 && swapon /dev/ramzswap0 -p 100"
einfo " - To unload compcache:"
einfo "# swapoff /dev/ramzswap0 && rmmod compcache && rmmod tlsf"
einfo "Optionally modprobe can take the following arguments:"
einfo " - compcache_size_kbytes=$SIZE_KB (by default 25% of memory)"
einfo "You might get lzo errors, ignore them."
# todo : don't build lzo modules if they are in the kernel already.
}
|
This will make it compile against the sources in /usr/src/linux and not necessarily the running kernel.
And this use_compcache in /usr/local/portage/sys-kernel/compcache/files (or wherever your overlay is) directory:
| Code: |
#!/bin/bash
# use_compcache (run as *root*)
# - Loads compcache and related modules.
# - Sets up swap device.
#
# Usage: setup_compcache <size in KB>
#
MOD_PARAM=""
SIZE_KB="$1"
SIZE_KB="$1"
LSMOD_BIN=/sbin/lsmod
INSMOD_BIN=/sbin/insmod
MODPROBE_BIN=/sbin/modprobe
SWAPON_BIN=/sbin/swapon
UDEVADM_BIN=/sbin/udevadm
INSMOD()
{
MOD_NAME="$1"
EXIST=`$LSMOD_BIN | grep "$MOD_NAME"`
if [ "$EXIST" != "" ]; then
echo "Module: $MOD_NAME already loaded."
else
$MODPROBE_BIN $MOD_NAME "$MOD_PARAM"
fi
}
# Some distos already have LZO de/compress modules
echo "Loading modules ..."
builtinlzo=`zgrep CONFIG_LZO_COMPRESS=y /proc/config.gz 2>/dev/null`
if [ "$builtinlzo" = "" ]
then
echo "Loading lzo_compress..."
$MODPROBE_BIN -q lzo_compress || INSMOD lzo1x_compress
fi
builtinlzo=`zgrep CONFIG_LZO_DECOMPRESS=y /proc/config.gz 2>/dev/null`
if [ "$builtinlzo" = "" ]
then
echo "Loading lzo_decompress..."
$MODPROBE_BIN -q lzo_decompress || INSMOD lzo1x_decompress
fi
INSMOD tlsf
if [ -z "$SIZE_KB" ]; then
echo "compcache size not given. Using default (25% of RAM)."
else
MOD_PARAM="compcache_size_kbytes=$SIZE_KB"
fi
INSMOD compcache
# /dev/ramzswap0 is not available immediately after insmod returns
# So, let udev complete its work before we do swapon
if [ -f "$UDEVADM_BIN" ]; then
$UDEVADM_BIN settle
else
sleep 2
fi
# Set it as swap device with highest priority
EXIST=`cat /proc/swaps | grep compcache`
if [ "$EXIST" = "" ]; then
echo "Setting up swap device ..."
$SWAPON_BIN /dev/ramzswap0 -p 100
if [ "$?" = "0" ]; then
echo "Done!"
else
echo "Could not add compcache swap device."
fi
else
echo "compcache swap device already active."
fi
exit 0
|
Then, just put "use_compcache 1200000" in my /etc/conf.d/local.start |
|
| Back to top |
|
|
neuron
Advocate
Joined: 28 May 2002
Posts: 2371
|
| Posted: Mon Aug 18, 2008 4:33 pm Post subject: |
|
|
has anyone done any serious performance testing?
I tried without compcache then with compcache and /var/tmp mounted in ram, compcache was slightly slower, I suspect because the kernel had less space to cache things (due to compcache) and activly tried to avoid swapping (saw at the end of the test, compcache was almost never used). Are there any tweakables/kernel patches to have it use swap as agressive as memory? |
|
| Back to top |
|
|
SDNick484
Apprentice
Joined: 05 Dec 2005
Posts: 231
|
| Posted: Mon Aug 18, 2008 6:03 pm Post subject: |
|
|
| neuron wrote: | | Are there any tweakables/kernel patches to have it use swap as agressive as memory? |
Yeah, it's a tuneable called "swappiness" and can be set with a 0-100 value via echo "#" >/proc/sys/vm/swappiness
Setting it to 100 should give you the desired affect, but do a google search on swappiness for details. Personally I use a lower value on my laptop to reduce HDD access. |
|
| Back to top |
|
|
devsk
Advocate
Joined: 24 Oct 2003
Posts: 2995
Location: Bay Area, CA
|
| Posted: Mon Aug 18, 2008 9:46 pm Post subject: |
|
|
| SDNick484 wrote: | | neuron wrote: | | Are there any tweakables/kernel patches to have it use swap as agressive as memory? |
Yeah, it's a tuneable called "swappiness" and can be set with a 0-100 value via echo "#" >/proc/sys/vm/swappiness
Setting it to 100 should give you the desired affect, but do a google search on swappiness for details. Personally I use a lower value on my laptop to reduce HDD access. |
What I have done is left swappiness at default value of 60 and using compcache on my laptop has reduced disk activity during crunch time.
We won't be able to do some actual number comparison unless compcache exports some cpu usage stats.
I had posted my numbers about mythtv compile which used to freeze my system during compile from time to time with IO waits very high and kswapd0 kicking in. The compile time of mythtv reduced a lot (I think from 24 minutes to 17 minutes) and freezes are gone. One of the biggest gain was that during compile, firefox's RSS will fall from 200MB down to 130MB (because of swapping), and it will load it right back (RSS will start climbing real quick because its just reading from RAM and writing to RAM) if I did something in firefox windows. Very little lag even at hightest load times. Awesome effect!
Of course, compiles which don't engage swap are not going to see any gain.
I am not sure about the tmpfs idea. It seems like a stretch and too many layers involved. I never had more than 2GB of RAM to test its effectiveness on larger builds like mythtv or whole of kdebase. Whatever small builds I tried, I didn't see much gain (IIRC, it was about 5%). I think if kernel is caching correctly and your dirty writes are delayed enough, tmpfs doesn't really yield much gain.
Having said that there are a bunch of things you can try with tmpfs (and *only* with tmpfs & compcache).
Set all of them one at a time and test:
1. set swappiness to 100. This will make sure kernel engages swap for buffering and caching.
2. set vfs_cache_pressure to 1 (retain inode cache as much as possible). This will make sure you can get to the library archive you made 2 minutes ago, now required for linking, without doing an IO.
3. set page-cluster to 5 (transfer 128KB chunks at a time during hard fault). Will make your swap transfers more efficient. Remember its all in memory, thanks to compcache.
4. set dirty_writeback_centisecs to 25 seconds i.e. echo 2500 > /proc/sys/vm/dirty_writeback_centisecs . This means pdflush wakes less often to write dirty stuff out.
5. set dirty_background_ratio to 10 (default is 5). This means if with 2GB ram, pdflush will write dirty buffers if they exceed 200MB in size. Of course, they expire at dirty_expire_centisecs, which is 30 seconds.
These should make sure that the compcache is engaged. Please do report back if you find any of these useful. |
|
| Back to top |
|
|
kernelOfTruth
Watchman
Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)
|
| Posted: Mon Aug 18, 2008 10:27 pm Post subject: |
|
|
| Quote: | One of the biggest gain was that during compile, firefox's RSS will fall from 200MB down to 130MB (because of swapping), and it will load it right back (RSS will start climbing real quick because its just reading from RAM and writing to RAM) if I did something in firefox windows. Very little lag even at hightest load times. Awesome effect!
Of course, compiles which don't engage swap are not going to see any gain.
I am not sure about the tmpfs idea. It seems like a stretch and too many layers involved. I never had more than 2GB of RAM to test its effectiveness on larger builds like mythtv or whole of kdebase. Whatever small builds I tried, I didn't see much gain (IIRC, it was about 5%). I think if kernel is caching correctly and your dirty writes are delayed enough, tmpfs doesn't really yield much gain. |
I experienced the same great behavior with firefox + heavy load, 
you're also correct with tmpfs, I've compiled openoffice several times in tmpfs and didn't experience much of a gain (speak savings) in compilation time
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.9.0
https://github.com/kernelOfTruth/pulseaudio-equalizer-ladspa
Hardcore Gentoo Linux user since 2004  |
|
| Back to top |
|
|
neuron
Advocate
Joined: 28 May 2002
Posts: 2371
|
| Posted: Tue Aug 19, 2008 7:40 am Post subject: |
|
|
| devsk wrote: | | I am not sure about the tmpfs idea. It seems like a stretch and too many layers involved. I never had more than 2GB of RAM to test its effectiveness on larger builds like mythtv or whole of kdebase. Whatever small builds I tried, I didn't see much gain (IIRC, it was about 5%). I think if kernel is caching correctly and your dirty writes are delayed enough, tmpfs doesn't really yield much gain. |
Yeah, that's my thinking as well, but I'm still gonna give it a try  , also this computer has 4gb memory, which is why I'm thinking in that direction .
| devsk wrote: | Having said that there are a bunch of things you can try with tmpfs (and *only* with tmpfs & compcache).
Set all of them one at a time and test:
1. set swappiness to 100. This will make sure kernel engages swap for buffering and caching.
2. set vfs_cache_pressure to 1 (retain inode cache as much as possible). This will make sure you can get to the library archive you made 2 minutes ago, now required for linking, without doing an IO.
3. set page-cluster to 5 (transfer 128KB chunks at a time during hard fault). Will make your swap transfers more efficient. Remember its all in memory, thanks to compcache.
4. set dirty_writeback_centisecs to 25 seconds i.e. echo 2500 > /proc/sys/vm/dirty_writeback_centisecs . This means pdflush wakes less often to write dirty stuff out.
5. set dirty_background_ratio to 10 (default is 5). This means if with 2GB ram, pdflush will write dirty buffers if they exceed 200MB in size. Of course, they expire at dirty_expire_centisecs, which is 30 seconds.
These should make sure that the compcache is engaged. Please do report back if you find any of these useful. |
I did run 4 overnight tests before all those settings, even with swapiness at 100 and /var/tmp/portage in tmpfs an emerge @system --jobs 8 didn't touch compcache. Gonna try again with those settings and we'll see if it's more aggressive.
I'd really like to see my system using compcache memory aggressive for diskcache, of course I wont gain much on this system, but I got a few low memory systems as well, and I dont want it agressivly trying to push pages out of ram |
|
| Back to top |
|
|
neuron
Advocate
Joined: 28 May 2002
Posts: 2371
|
| Posted: Tue Aug 19, 2008 9:30 am Post subject: |
|
|
Just tried once to see if I could get it to use compcache, with your more agressive settings, during a gcc compile I noticed this:
| Code: |
#free -m
total used free shared buffers cached
Mem: 3947 3859 88 0 6 2187
-/+ buffers/cache: 1665 2282
Swap: 1973 0 1973
#cat /proc/compcache
DiskSize: 2021180 kB
NumReads: 19
NumWrites: 0
FailedReads: 0
FailedWrites: 0
InvalidIO: 0
GoodCompress: 0 %
NoCompress: 0 %
CurrentPages: 0
CurrentMem: 0 kB
PeakMem: 0 kB
|
Um, 4gb memory, 2gb for compcache, 3.8gb in use and none of it is compcache? Does it only start using the compcache when I'm almost out of memory? (which would be a cool feature, but damn hard for me to test :p ). Also if it was as agressive as memory it'd be putting cache/buffers in swap wouldn't it? |
|
| Back to top |
|
|
devsk
Advocate
Joined: 24 Oct 2003
Posts: 2995
Location: Bay Area, CA
|
| Posted: Tue Aug 19, 2008 3:53 pm Post subject: |
|
|
try mythtv build once, with MAKEOPTS=-j6...
each cc1plus process in that build will ask for like 300-600MB. |
|
| Back to top |
|
|
neuron
Advocate
Joined: 28 May 2002
Posts: 2371
|
| Posted: Tue Aug 19, 2008 4:06 pm Post subject: |
|
|
| yeah, but that'd be testing oom conditions, I want it activly putting buffers and cache in "swap". |
|
| Back to top |
|
|
devsk
Advocate
Joined: 24 Oct 2003
Posts: 2995
Location: Bay Area, CA
|
| Posted: Tue Aug 19, 2008 4:31 pm Post subject: |
|
|
| neuron wrote: | | yeah, but that'd be testing oom conditions, I want it activly putting buffers and cache in "swap". |
it wont do that unless it "runs" out of memory. If there is place for buffers and cache in memory, why would it put in swap? |
|
| Back to top |
|
|
neuron
Advocate
Joined: 28 May 2002
Posts: 2371
|
| Posted: Tue Aug 19, 2008 4:52 pm Post subject: |
|
|
| devsk wrote: | | neuron wrote: | | yeah, but that'd be testing oom conditions, I want it activly putting buffers and cache in "swap". |
it wont do that unless it "runs" out of memory. If there is place for buffers and cache in memory, why would it put in swap? |
Absolutely, the problem is it seems to activly avoid using the swap, which is good in a normal scenario, but bad in a scenario where the swap is almost as fast as the memory. |
|
| Back to top |
|
|
s4e8
Guru
Joined: 29 Jul 2006
Posts: 311
|
| Posted: Thu Aug 21, 2008 1:29 pm Post subject: |
|
|
| The routine compcache_size_setup() is useless. You can use compcache.compcache_size_kbytes=XXXXX syntax for all embeded module_param. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Unsupported Software
(ask intel ! 
)