View previous topic :: View next topic |
Author |
Message |
vh4x0r n00b


Joined: 11 May 2009 Posts: 37
|
Posted: Sun May 17, 2009 4:27 pm Post subject: Filesystem for /usr/portage |
|
|
Hi, I'm installing a new gentoo system on my laptop , and need some help with choosing a filesystem for /usr/portage. Should I choose reiser4 or ext4, or maybe ext2 will be faster? Also can somebody help with the mount options? |
|
Back to top |
|
 |
d2_racing Bodhisattva


Joined: 25 Apr 2005 Posts: 13047 Location: Ste-Foy,Canada
|
Posted: Sun May 17, 2009 4:35 pm Post subject: |
|
|
Ext3 or reiserFs4, ext4 is still experimental. |
|
Back to top |
|
 |
vh4x0r n00b


Joined: 11 May 2009 Posts: 37
|
Posted: Sun May 17, 2009 4:50 pm Post subject: |
|
|
well, which is gonna be faster, ext3 or reiserfs? |
|
Back to top |
|
 |
d2_racing Bodhisattva


Joined: 25 Apr 2005 Posts: 13047 Location: Ste-Foy,Canada
|
Posted: Sun May 17, 2009 5:09 pm Post subject: |
|
|
ReiserFS, because /usr/portage have a lot of small files. |
|
Back to top |
|
 |
yngwin Retired Dev


Joined: 19 Dec 2002 Posts: 4572 Location: Suzhou, China
|
Posted: Sun May 17, 2009 7:04 pm Post subject: |
|
|
I recommend Reiser4. _________________ "Those who deny freedom to others deserve it not for themselves." - Abraham Lincoln
Free Culture | Defective by Design | EFF |
|
Back to top |
|
 |
mandas n00b

Joined: 13 Nov 2006 Posts: 20
|
|
Back to top |
|
 |
d2_racing Bodhisattva


Joined: 25 Apr 2005 Posts: 13047 Location: Ste-Foy,Canada
|
Posted: Mon May 18, 2009 2:42 am Post subject: |
|
|
With small files and I mean a lot, ReiserFS is the best FS out there I think. |
|
Back to top |
|
 |
Ormaaj Guru


Joined: 28 Jan 2008 Posts: 319
|
Posted: Mon May 18, 2009 6:11 am Post subject: |
|
|
d2_racing wrote: | Ext3 or reiserFs4, ext4 is still experimental. | ext4 is considered stable - moreso than reiser4 (and faster). If you look at the benchmarks reiser4 pretty consistently bombs in comparison to everything else.
Heres just one example thats getting kind of old at this point. http://www.phoronix.com/scan.php?page=article&item=ext4_benchmarks&num=2
Also if you are concerned about file fragmentation and performance, isolating /var is a good idea, especially if you use ccache. It also contains a lot of other temporary files and caches, the portage installed package database, the emerge sandbox, and browser caches. That will probably be more critical than /usr/portage. Isolating /usr/portage might speed up syncs, but thats about it. |
|
Back to top |
|
 |
tranquilcool Veteran

Joined: 25 Mar 2005 Posts: 1246
|
Posted: Mon May 18, 2009 8:55 am Post subject: |
|
|
played around with btrfs+compression for some but i have just
gone back to reiser4+compression. no benchmark but i had the
impression performance on btrfs didn't equal reiser4 after a while. _________________ this is a strange strange world. |
|
Back to top |
|
 |
gentoo-dev Apprentice

Joined: 24 Jan 2006 Posts: 172
|
Posted: Mon May 18, 2009 10:49 am Post subject: |
|
|
ext2 with 1K blocks. Simple, clean, fast. |
|
Back to top |
|
 |
djinnZ Advocate


Joined: 02 Nov 2006 Posts: 4831 Location: somewhere in L.O.S.
|
Posted: Mon May 18, 2009 1:06 pm Post subject: |
|
|
squashfs on xfs, reiser 3 is stable but suffer of fragmentation issues for portage (or you can format the portage partition at slowdon). _________________ scita et risus abundant in ore stultorum sed etiam semper severi insani sunt
mala tempora currunt...mater stultorum semper pregna est
Murpy'sLaw:If anything can go wrong, it will - O'Toole's Corollary:Murphy was an optimist 
Last edited by djinnZ on Mon May 25, 2009 1:32 pm; edited 1 time in total |
|
Back to top |
|
 |
yngwin Retired Dev


Joined: 19 Dec 2002 Posts: 4572 Location: Suzhou, China
|
Posted: Tue May 19, 2009 9:27 pm Post subject: |
|
|
Ormaaj wrote: | d2_racing wrote: | Ext3 or reiserFs4, ext4 is still experimental. | ext4 is considered stable - moreso than reiser4 (and faster). If you look at the benchmarks reiser4 pretty consistently bombs in comparison to everything else. |
Says who?
Ext4 considered stable??? That's ridiculous. Reiser4 has been around much, much longer and is a lot more stable. So if you want stable and fast, Reiser4 is the way to go. But you're free to use whatever else.  _________________ "Those who deny freedom to others deserve it not for themselves." - Abraham Lincoln
Free Culture | Defective by Design | EFF |
|
Back to top |
|
 |
yngwin Retired Dev


Joined: 19 Dec 2002 Posts: 4572 Location: Suzhou, China
|
Posted: Tue May 19, 2009 9:29 pm Post subject: |
|
|
djinnZ wrote: | squashfs or xfs |
XFS for portage doesnt make sense. XFS's strength is with big files. _________________ "Those who deny freedom to others deserve it not for themselves." - Abraham Lincoln
Free Culture | Defective by Design | EFF |
|
Back to top |
|
 |
i92guboj Bodhisattva


Joined: 30 Nov 2004 Posts: 10315 Location: Córdoba (Spain)
|
Posted: Tue May 19, 2009 10:08 pm Post subject: |
|
|
For me it'd be ext2, with 1k blocks, yes.
There's nothing that justifies the extra cpu load of a journal on a fs where there's nothing really valuable that you can't get back with a sync. But this much depends on your machine. I don't usually have available last generation hardware, and cpu is a valuable resource for me, so I try to save every cpu cycle that I can.
I really laugh when people seem that worried about the speed of the portage partition. It's a nonsense taking into account that compilations can last hours. Who cares if a search operation takes 4 more seconds? |
|
Back to top |
|
 |
frostschutz Advocate


Joined: 22 Feb 2005 Posts: 2977 Location: Germany
|
Posted: Wed May 20, 2009 10:48 am Post subject: |
|
|
I use ReiserFS for /usr/portage, /usr/src, /var/tmp/ccache, /var/tmp/portage.
It's great for tons of small files, much better than XFS (which is what I use for everything else).
For searching a package I just use eix.
ext2 with small blocks sounds good too, but you have to be careful with extX in general... you run out of inodes, it's over for you, only solution (apart from deleting files) is reformatting this time with more inodes. There's someone every day in the #gentoo channel with exactly this kind of problem. |
|
Back to top |
|
 |
djinnZ Advocate


Joined: 02 Nov 2006 Posts: 4831 Location: somewhere in L.O.S.
|
Posted: Wed May 20, 2009 12:47 pm Post subject: |
|
|
yngwin wrote: | XFS's strength is with big files | I agree but the problem with reiser 3 (an instable filesystem can be reasonable for testing never to work IMHO) and portage is the fragmentation and xfs not suffer about this; is the better choiche if you not use a separate partition and a reasonable alternative if you have a large amount of files in portage/distfiles.
In the computers of mine I use squashfs for portrage over an xfs partition with all the downloaded packages and the local overlay and all work fine.
make.conf: | PORTDIR_OVERLAY="/usr/portage/local"
PORTDIR=/usr/portage/gentoo
PKGDIR=/usr/portage/pkg/packages
DISTDIR=/usr/portage/pkg/distfiles | /usr/portage is on a xfs partition xfs_info /usr/portage: | meta-data=/dev/sdd5 isize=256 agcount=64, agsize=168871 blks
= sectsz=512 attr=1
data = bsize=512 blocks=10807712, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=512 ascii-ci=0
log =internal bsize=512 blocks=2048, version=1
= sectsz=512 sunit=0 blks, lazy-count=0
realtime =none extsz=65536 blocks=0, rtextents=0 | and /usr/portage/gentoo is squashfs. _________________ scita et risus abundant in ore stultorum sed etiam semper severi insani sunt
mala tempora currunt...mater stultorum semper pregna est
Murpy'sLaw:If anything can go wrong, it will - O'Toole's Corollary:Murphy was an optimist  |
|
Back to top |
|
 |
coolsnowmen Veteran


Joined: 30 Jun 2004 Posts: 1479 Location: No.VA
|
Posted: Wed May 20, 2009 6:00 pm Post subject: |
|
|
I have had reiserfs for /usr/portage for about 9 months now. And every couple weeks I get strange I/O errors on a sync. This forces me to unmount it, fsck it, and remount it, then sync again. I don't have any other problems with the other filesystems on the disk (ext4 and ext2).
And as far as the "lots of small files" argument. It doesn't really hold true because /usr/portage doesn't have lots of files per directory. It only has a couple small files, so I don't see any massive speed boost you would get vs. ext2 that would make it worth the problems I have with it.
So, by my experience, I would not recommend it. _________________ emerge: there are no ebuilds to satisfy "moo" |
|
Back to top |
|
 |
DONAHUE Watchman


Joined: 09 Dec 2006 Posts: 7651 Location: Goose Creek SC
|
Posted: Wed May 20, 2009 6:36 pm Post subject: |
|
|
2 reiserfs /usr/portage each has ~139896 items, totalling ~3.6 GB and neither has had an error on a sync in 3 years of life.
mileage varies. |
|
Back to top |
|
 |
frostschutz Advocate


Joined: 22 Feb 2005 Posts: 2977 Location: Germany
|
Posted: Wed May 20, 2009 6:45 pm Post subject: |
|
|
coolsnowmen wrote: | So, by my experience, I would not recommend it. |
I don't trust reiserfs either, that's why I only use it for portage... since it checksums every file, corruption on that partition does not matter.
But so far, no issues. |
|
Back to top |
|
 |
John R. Graham Administrator


Joined: 08 Mar 2005 Posts: 10733 Location: Somewhere over Atlanta, Georgia
|
Posted: Wed May 20, 2009 8:00 pm Post subject: |
|
|
coolsnowmen wrote: | ...And as far as the "lots of small files" argument. It doesn't really hold true because /usr/portage doesn't have lots of files per directory. ... | I believe that a directory with only a couple of files in it is itself a "small file", which is why I believe that the argument does hold true.
For what it's worth, I've been using ReiserFS on /usr/portage on several machines for years now and it's been rock solid for me.
- John _________________ I can confirm that I have received between 0 and 499 National Security Letters. |
|
Back to top |
|
 |
coolsnowmen Veteran


Joined: 30 Jun 2004 Posts: 1479 Location: No.VA
|
Posted: Wed May 20, 2009 8:30 pm Post subject: |
|
|
john_r_graham wrote: | coolsnowmen wrote: | ...And as far as the "lots of small files" argument. It doesn't really hold true because /usr/portage doesn't have lots of files per directory. ... | I believe that a directory with only a couple of files in it is itself a "small file", which is why I believe that the argument does hold true.
|
My mistake, I went back and read the kernel notes on resiserfs, it says:
Quote: | ReiserFS is as fast as ext2, but is very efficient with large directories and small files. |
I remembered it wrong as "large directories of small files" but I guess either satisfy that condition. _________________ emerge: there are no ebuilds to satisfy "moo" |
|
Back to top |
|
 |
pigeon768 l33t

Joined: 02 Jan 2006 Posts: 683
|
Posted: Thu May 21, 2009 10:20 pm Post subject: |
|
|
djinnZ wrote: | yngwin wrote: | XFS's strength is with big files | I agree but the problem with reiser 3 (an instable filesystem can be reasonable for testing never to work IMHO) and portage is the fragmentation and xfs not suffer about this; is the better choiche if you not use a separate partition and a reasonable alternative if you have a large amount of files in portage/distfiles. | There's going to be effectively zero fragmentation in /usr/portage. Ebuilds are generally only a block or two.
xfs is an absolutely miserable filesystem choice for /usr/portage and /var/tmp/portage. xfs is very good at modifying enormous files. If you're running a huge database, put your database on a separate xfs partition - performance will be great. However, there's simply too much overhead involved for it to be useful at parsing through and opening each of the enormous number of files in /usr/portage, or with creating, deleting, or modifying a bunch of files during building.
IMO it's not even very good at running around through /usr/bin.
reiserfs is a great choice for /usr/portage and /var/tmp/portage. Even if you don't like reiserfs because you think it's unstable, (you're wrong) the data in /usr/portage and /var/tmp/portage is trivial to regenerate, and there are hefty speed gains by using it instead of xfs for instance. |
|
Back to top |
|
 |
djinnZ Advocate


Joined: 02 Nov 2006 Posts: 4831 Location: somewhere in L.O.S.
|
Posted: Sat May 23, 2009 10:16 pm Post subject: |
|
|
reiserfs 4 is unstable (reported usable but not stable), this is a fact, reiserfs 3.6 is really stable (more than ext3 in my experience) and the better choiche for /var/tmp IMHO (and I use it, not only xfs on the computers of mine).
You speak about reiserfs but what is the version?
The xfs partition on the pc of mine host directly only the ebuilds written by me, all the files are ever bigger than 10 MB and are only tar archives and squashfs images. All the ebuilds are on squashfs not on xfs.
And the content of /usr/portage/distfiles can be downloaded (but some are really painful as the damned multiple java versions from 1.3 to 1.6 than i need) but is a large amount of data so i will be sure (and never will use an unstable filesystem) and reiser is not the better choiche for this.
In the case you have two separate partitions, one for /usr/portage and one for /usr/portage/distfiles, the fragmentation and the performance degrade will not be a problem.
uh, i have forget to say than i use to sync from mirrors on a separate directory (hosted on reiserfs) in the server and later I update the portage on the other computers, in order to prevent mistakes (and copy a file directly is more useful than update it via rsync in a local lan) and errors (the others pcs are not able to sync). _________________ scita et risus abundant in ore stultorum sed etiam semper severi insani sunt
mala tempora currunt...mater stultorum semper pregna est
Murpy'sLaw:If anything can go wrong, it will - O'Toole's Corollary:Murphy was an optimist  |
|
Back to top |
|
 |
BitJam Advocate

Joined: 12 Aug 2003 Posts: 2513 Location: Silver City, NM
|
Posted: Sun May 24, 2009 9:51 pm Post subject: |
|
|
I've tried several different file systems for /usr/portage over the years. I'm currently using XFS with the smallest blocksize possible. I can typically sync in well under two minutes. If I sync regularly, the time goes down to about one minute. IMO the small block size outweighs almost all other factors because it reduces the disk I/O (due to all the tiny files). My /usr/portage partition is using 346M according to df -h. I suggest using the FS that has the smallest blocksize. But don't use Reiser-3. It does not fragment gracefully and quickly slows down to a crawl.
If syncing starts to slow down then I copy, erase, and repopulate which works to de-frag. I do this maybe one a year or so. |
|
Back to top |
|
 |
frostschutz Advocate


Joined: 22 Feb 2005 Posts: 2977 Location: Germany
|
Posted: Mon May 25, 2009 9:16 am Post subject: |
|
|
BitJam wrote: | If syncing starts to slow down then I copy, erase, and repopulate which works to de-frag. I do this maybe one a year or so. |
If you're using XFS you could just use the defrag tool that comes with XFS... |
|
Back to top |
|
 |
|