choosing file system to a seedbox

simonbcn · n00b Joined: 01 Aug 2011 Posts: 69 Location: Denmark

I going to install Gentoo in a server to use like seedbox (torrent and FTP will be the main services). I've thought in use ext4 to root system and the other disks (2) with LVM2/XFS. The main problem is the fragmentation and the speed in read/write data to/from discs where the files are downloaded. Is XFS the best option?

eccerr0r · Posted: Thu Dec 31, 2015 4:14 pm Post subject:

All filesystems can fragment if you give it a bad allocation order, and bittorrent is about the worst real world example.

If you completely preallocate the files (and not sparse file allocate), fragmentation won't be a problem. Some torrent applications can make an intermediate file and rebuild the file after all chunks are received, this also helps against fragmentation. You can tell this if you see the client writing out the file again and using double disk space temporarily after a download.

Rtorrent by default will fragment your downloads readily - you'll see it at most just do a full file hash check but not rewrite the file. I don't know about Transmission, etc.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

Naib · Posted: Thu Dec 31, 2015 4:20 pm Post subject:

Some bittorrent clients can "preallocate" to mitigate too much corruption
_________________

szatox · Advocate Joined: 27 Aug 2013 Posts: 3430

Well, if you're worried about fragmentation, you can consider triggering "shake" every now and then. That's what cron is for :roll:

simonbcn · n00b Joined: 01 Aug 2011 Posts: 69 Location: Denmark

I'll use Transmission. It's a good client torrent and it has preallocation.

eccerr0r · Posted: Thu Dec 31, 2015 8:31 pm Post subject:

There's an option to fallocate in rtorrent, but you'll need a filesystem that supports fallocate and is Linux specific.

I forgot which bittorrent client I used in the past that downloaded everything into a temporary file, and rewrite the file once all chunks are received. This method has the advantage of quick hash checks of incomplete downloads, no slow file creation on startup, as well as not having to have all disk space available upon startup - just like sparse file allocation. However this does require times again the amount of diskspace free at download completion, and having an extent based filesystem is a must if you're downloading more than one torrent at a time.

Anyway, http://vleu.net/shake/ is indeed a workaround... Make sure you have plenty of extra space and backup. I was planning to write something like that, but I guess I won't need to anymore, alas, I don't fragment my hard disks enough to need it.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

Naib · Posted: Thu Dec 31, 2015 8:52 pm Post subject:

NOTE if the drive is ssd... don't worry
_________________

P.Kosunen · Guru Joined: 21 Nov 2005 Posts: 309 Location: Finland

What kind of hardware is it running on and how fast is network?

I would go with ext4 almost in any case, never had any problems with it.

eccerr0r · Posted: Mon Jan 04, 2016 11:06 pm Post subject:

P.Kosunen, what BT client do you use, and do you preallocate?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

P.Kosunen · Guru Joined: 21 Nov 2005 Posts: 309 Location: Finland

Transmission and yes preallocate.