Low priority for hard disk copy

geenux · n00b Joined: 15 Mar 2008 Posts: 40

When I copy some files from a partition to an other, or my hard drive to and external one, all my system is freezing. It works still, but it's very slow because the hard drive is working with the copy. It's really annoying when copying Gigabytes of data, when it takes half an hour or more, and during this time the system is hardly usable.
Is there any way to have a very low priority to file transfers, so that other programs can still work normally ?

NeddySeagoon · Posted: Fri Jul 10, 2009 6:24 pm Post subject:

geenux,

There is no simple answer. It depends what the bottleneck is.
External USB drives cannot use DMA, so writing to USB drives is processor intensive.

You can try to renice your copy process, which may not help if 'dirty' buffers are your problem.
Thats areas of memory occupied by data read from disk and waiting to be written.

Maybe your hard drive could be faster. What does hdparm /dev/.... show for your internal drive?
If your drive is /dev/sd... everything is good and that command will probably fail
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

geenux · n00b Joined: 15 Mar 2008 Posts: 40

NeddySeagoon · Posted: Sat Jul 11, 2009 4:58 pm Post subject:

geenux,

Thats correct for /dev/sda, do you have a hard drive at /dev/hda too ?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

geenux · n00b Joined: 15 Mar 2008 Posts: 40

No, I've only one sata hard drive.

NeddySeagoon · Posted: Sat Jul 11, 2009 7:17 pm Post subject:

geenux,

During or after the errors, are there any SATA of drive related errors in dmesg ?

Please post the output of

geenux · n00b Joined: 15 Mar 2008 Posts: 40

There is no sata errors in dmesg.

NeddySeagoon · Posted: Sun Jul 12, 2009 6:03 pm Post subject:

geenux,

You can rebuild your kernel to provide more agressive preemption than CONFIG_PREEMPT_VOLUNTARY=y
In the Preemption Model (Preemptible Kernel (Low-Latency Desktop)) ---> menu, choose

geenux · n00b Joined: 15 Mar 2008 Posts: 40

I'll try. But my guess is the problem is not coming from the CPU. It's Intel Dual Core 2.2Ghz. I guess the problem comes from disk reading during the copy, as when an application, such as firefox is still in cache, it launches quite rapidly (maybe 3 or 4 fours seconds more than usual), whereas when it is not in the cache, it takes about 20 seconds.
But maybe I'm wrong, i'll rebuild my kernel, according to your proposition.

NeddySeagoon · Posted: Sun Jul 12, 2009 6:43 pm Post subject:

geenux,

Disk latency is certainly a part of the problem as you have to wait while the drive moves the heads to find your program.
A copy from one partition to another involves a lot of head movements.

Reading/Writing to USB mass storage devices is CPU intensive as it cannot use DMA.
The CPU must move all the data.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

geenux · n00b Joined: 15 Mar 2008 Posts: 40

I rebuilt my kernel, and it seems to be exactly the same.
It shall be great if the copy process paused when an other action is required on the disk. So, when an application needs to be started, the copy would stop until it is started. Is it possible to do so ?

Exil · Posted: Tue Jul 14, 2009 12:29 pm Post subject:

nice -n19 command

jcat · Veteran Joined: 26 May 2006 Posts: 1337

Shouldn't that be ionice ?! :wink:

Try scheduling clash 3 for "Beat Effort" or even "Idle".

Alternatively, you could try rsync instead of cp. Use the "--bwlimit" option.

Cheers,
Just