What is the Best filesystem for external USB 2.0 Hard Drive

[tapted]

Hey there..

My laptop (a T41) has only a 30GB hard drive (actually 30 million bytes, of which a few million is taken up by IBM's bloody 'restore to defaults' crap, so really it's a bit less than 25GB).

So it gets unfriendly when I start trying to dual boot between WinXP and Gentoo .. and do some DV editing at the same time. So I invested in a USB 2.0 external hard drive enclosure and am, for the most part, happy.

USB 2.0 works fine in Linux and I jazzed it up with a slew of partitions to keep both windows and Gentoo happy.... I even get a sustainable 25 MB/sec, which I'm happy with (heck, that's faster than my laptop hard drive). This one time (not at band camp), when my memory (half a gig) was mostly empty, rsync even reported a princely 45 MB/sec when writing a single, 1.2GB video file to a reiserfs partition. 'corse the sync that followed took a few seconds.

Which is why I'm unhappy when I decided to put a portage tree onto the external hard drive (previously this was an NFS mount). I bzipped an old tree and rsynced the distfiles and it wasn't too bad. But then I started an emerge sync (with a nearby mirror) and it's been chewing up my CPU of all things. top tells my rsync is currently at 72% CPU and has sucked in over 100 minutes of CPU time. And it's still wading through the metadata subtree.

I've noticed that while I'm accessing the drive from multiple threads, there is always a huge performance hit, so it makes me think that rsync's read this, hash this, read that, receive that, compare this and that, retrieve that, write this procedure might be disadvantaged by being on my USB drive in some way...

Thinking about what could be wrong, drew me to the conclusion that perhaps reiserfs is the wrong file system to use for a USB drive. Does anyone out there have any information that says that one of ext2/3, resier, xfs, jfs, whaterverfs is best/unsuited for a partition on a USB 2.0 external hard drive?

Most USB flash drives use vfat (fair enough), but that's not good if you want permissions and (video, eg) files bigger than 2GB (or a partition size bigger than 32GB, for that matter).

Er.. some details, I guess.

My kernel is a

2.6.5-gentoo-r1

and I'm currently running in devfs-only mode (i.e. no udev)

My uxternal HD looks like this:

10GB ntfs primary partition (sda1)
32GB vfat primary partition (sda2)
32GB vfat primary partition (sda3)
------ extended partition (sda4) containing
62MB ext2 logical partition (sda5)
1GB swapfs logical partition (sda6) [dunno what these two were for .. just reflex, I guess]
remainder (~37GB) reiserfs logical partition (sda7)


As soon as this bloody rsync finishes, I'll be whacking on a new 2.6.8 with udev (this is the main thing that got me started on this, acutally, as I didn't have room for a new source tree!)

I also noticed (after doing some forum searches) that I don't have hdparm installed, but I don't know much about it.. are there some tried-and-tested settings I can tell hdparm to inflict on my /dev/sda (or whatever udev decides to call it) to get better performance for USB 2.0?


Sorry for the longish post -- I have time to kill waiting for this bloody rsync to finish.

Moo.

[sklettke]

If you are going to be dual booting (and want to access the external drive in Windows), I'd go with fat32. It is slower but allows easy file access between Linux and Windows. You could also try the new ntfs writing in Linux. For Linux only use, I'd try the new Reiser4 filesystem that was just officially released. http://www.namesys.com

Scott
_________________
Jab.ID: scottk@jabber.org

Kernel: 2.6.11-rc3-nitro0
KDE 3.3.2 with Xorg
MythBox: 2.6.5-gentoo-dev-r2 (LVMed 360GB in XFS; Athlon 2500+)

[xns]

I agree with sklettke with regards to using Reiser4 for linux usage, the fact that its a removable drive, and accidents do happen (Ah crap! Tripped over the damn cable!), Reiser4 is perfectly suited to prevent any filesystem corruptions from accidents like that.

[rush_ad]

but of course Reiser4

[tapted]

Reiser4 it is, then!

Thanks folks.

[tapted]

wow. Those bechmarks look impressive

http://www.namesys.com/benchmarks.html

especially the ones with multiple streams (which appears to be my problem, in particular). Let's hope it carries across to USB2.0!

Moo.

[syadnom]

yes, for linux i would suggest the following.

make a fat32 partition big enough for the following, then use the rest as reiser4

get a copy of bochs install a small linux distro with network support and an FTP server, NFS server, or SAMBA((put this in a disk image in the root of the drive)). this way if you need to run the system on windows, you will have a linux virtual machine that can read the harddrive and export it via NFS/SMB/FTP. so you can just load up the bochs and log into the box to get reiser4 access.

i am doing this right now with a USB2.0 120GB drive and it works great.