Possible to control filesystem caching behavior?

[evilben]

Hi all,

What I would like to do is limit the amount of memory used by the kernel for caching certain directories (i.e. where I am working with large files), or alternatively, to reserve a certain amount of memory for caching other directories (like /usr). The goal of this is to make my programs, e.g. firefox, start up quickly whether or not I am doing large copy operations.

Seems like the sort of thing that may be useful for Apache, etc. as well in some settings.

My google searches have been unsuccessful, which may be due to my phrasing. I am still guessing that this functionality is not available.

Thanks

[Kompi]

There is at least the option to tune how strong the kernel leaves memory to cache when more is needed. Change the /proc/sys/vm/vfs_cache_pressure value. You can set this in /etc/sysctl.conf if you want to set a default value.
Here is an explanation what it does: http://www.linuxinsight.com/proc_sys_vm_vfs_cache_pressure.html
But I guess this doesn't help you much to make the cache acting different on specific files/dirs. Probably there are more such kernel option (have a look at /proc/sys/vm) which can help you with that ...
Another way could be to load some files into a ramdisk at boot up. These would be available pretty fast.

[BitJam]

I don't think what you are asking for is possible but this is a Good Thing (TM).

Caching /usr will generally not help one whit for the start up of programs since the caching only helps the 2nd or later time the files are accessed. You always pay the full price the first time.

You would be much better off preloading the key programs when your system boots up. This has many advantages over the simpling caching /usr. First, it only "caches" the programs you are interested in instead of caching every program. Second, this method really does speed up the start up of the programs the first time you want to use them (at the expense of a slightly longer boot). Third, the speed up of having the program already loaded is superior to merely having the libraries and executeables files cached. Fourth, if you run low on memory then these programs will be saved to swapped (if they haven't been used recently) but they would have been dumped from cache way before this time so you win here too. Also, reading in from swap might be marginally faster than cached libraries. It certainly won't be slower. Finally, if your key programs are preloaded then you can use tips such as the previous one to reduce all file caching and thus reduce the chance of your key programs being sent to swap.

KDE has been using this preload trick for years.

[evilben]

Thanks guys. I will read through some of the /proc stuff. vm.swappiness is the only one I've used before (turning it down to 0).

I have 4G of ram, so swapping is fairly undesirable for me. Really, the reason I wanted this was for the 2nd and later starts of programs (e.g., run Firefox, close Firefox, copy large files, start Firefox again). Probably leaving Firefox running would be easiest, but old habits die hard.

[BitJam]

With 4 Gig of RAM, swapping will be unlikely so preloading looks like a very good option for you. There is a Firefox preloader available for Windows and there is also FirefoxQS (quick start) that preloads Firefox for KDE users by putting it into the system tray. I run KDE so I downloaded and complied it. I had no problems and it seems to work. I think I'll keep it even though it's at version 0.1.3 and the documentation is sparse to non-existent.

You would probably be more interested in alltray which is available in Portage. It plays this same trick (putting apps into the system tray) for many different DE's and many different apps. This might be a nearly perfect solution for you.