Strangeness with bittorrent downloads

Tarball

Hi,

I'm currently downloading a couple of bittorrent files. If I do ls -l in the directory where the download is being written is claims the file is 1.8GB in size (this is the size of the complete file) but if I do du -sh ./file it displays the actual amount downloaded (eg 762MB). Can anyone explain why ls reports the size as the complete file size but du reports the actual size.

Also, I just noticed that ls -l reports the total directory size correctly but still reports that the file is 1.8GB

Cheers

Sgaduuw · Posted: Fri Nov 21, 2003 1:55 pm Post subject:

"ls -l" shows you the size that has been allocated by bittorrent for your download
"du -sh" will show you the size that is actually in use on your disk
_________________
my website

Tarball · Posted: Fri Nov 21, 2003 2:10 pm Post subject:

But how could bittorrent allocate 1.8GB without creating a file of that size on the disk?

Also, why would ls -l report total directory size of 700MB but list one of the files as 1.8GB???

Ian · Posted: Fri Nov 21, 2003 2:17 pm Post subject:

I think that bittorrent will create a file of emptyness that is the size of the final file, and as parts are downloaded their added in bit by bit. This makes sure that the download can't be stopped halfway through because of a lack of disk space, and I'm sure it does other funtions as well. Of course, this will play tricks on your mind while the file is downloading, but that's a small price to pay for whatever you're getting

.

Tarball · Posted: Fri Nov 21, 2003 3:13 pm Post subject:

Sorry, still not got my head around this. How would you create a 1.8GB file of 'emptyness'. Even if the file is all zeros surely it is still going to be occupying 1.8GB of disk space???

Kenadin · Posted: Fri Nov 21, 2003 5:23 pm Post subject:

I don't exactly know how the filesystem stuff works, but Bittorrent needs to tell the computer it needs a certain amout of space for the file you are downloading and your computer will allocate that space for the file. This is done because the file is not downloaded sequentually with bittorrent. It downloads different segments of the file from different host at the same time so you could be downloading the end of the file and the beginning at the same time and therefore that space must be already allocated so it can be placed in a correct spot to prevent fragmentation. Dunno if that helps, but thats my understanding of it.

Random Task · Apprentice Joined: 31 Mar 2003 Posts: 171 Location: Canada

The way (I think) it works is that your filesystem sets aside 'blocks' that represent the space that your torrent file(s) will occupy. Then as you actually get the stuff the real information will get 'filled in'.

I noticed this when I had to write a simple ftp client for school. I would always write 'x' blocks and yet the file came out bigger than the original. Then I found out that I wasn't specifying the right 'block' size for the final packet, it wasn't the standard 2000 bytes it was say 1500 bytes (this is only an example).

Hope I haven't confused you too much!
_________________
If you can't kill something with that, it ain't supposed to die...

You're so drunk you're seeing double...
I've got two guns, one for each of ya!
-Val Kilmer as Doc Holliday, Tombstone

Darian · Posted: Fri Nov 21, 2003 11:56 pm Post subject:

Some filesystems can store sparse files more efficently. A sparse file is a file that contains mostly zeros. I know xfs and reiserfs do this. I would supsect that any journaled filesystem could do it because data written to the disk is already being stored up in the journal (where it could be analized for this sort of property) before getting written to the filesystem.