Bad performance on copying/moving/unt(r)aring files

zehnan1

I have a single SATA drive on my amd64 3200+ system (1GB ram). Overall the system is fast and responsive, except when having high disk/cpu activity such as moving/copying files and especially untaring/raring files. If I start rox when copying a large file it takes about 30 seconds to start - normally it takes like 0.5 secs. If a start opera or firefox when unraring, it is a good bet it wont happen until the rar is finished... which could mean up to two minutes. Both browsers start quite fast without disk activity. MP3's occasionally skip on high disk activity, but that's not so common.

hdarm output is fine:

NeddySeagoon · Posted: Sat May 13, 2006 6:29 pm Post subject:

zehnan1,

What preempt options do you have in your kernel ?
Post the output of

zehnan1 · Posted: Sat May 13, 2006 6:34 pm Post subject:

zehnan1 · Posted: Sat May 13, 2006 6:35 pm Post subject:

Oh, I just realized I posted in a network/security forum. I apologize, it should bekernel & hardware.

NeddySeagoon · Posted: Sat May 13, 2006 7:46 pm Post subject:

zehnan1,

Well spotted. - Moved from Networking & Security to Kernel & Hardware.

I'm not familliar with the real time kernel. Can you repeat the tsets with either gentoo-sources or vanilla sources and post your

zehnan1 · Posted: Sun May 14, 2006 9:44 am Post subject:

I experience the same problems with gentoo-sources. Tested with 2.6.15-gentoo-r7 with voluntary preemption.

NeddySeagoon · Posted: Sun May 14, 2006 10:05 am Post subject:

zehnan1,

Turn on

zehnan1 · Posted: Sun May 14, 2006 5:14 pm Post subject:

Thanks for suggestions Neddy, but it didn't help a bit.

Now, I figure this could be scheduler problem. For eyample, I have a task which completes in 5 seconds (like firefox reading its binaries and stuff) when no other task requires hard drive. When another task (like copy) is using hard drive and under asumption the harddisk IO is equally divided between the two tasks, firefox ought to start in twice as much time (plus some overhead), so a bit more that 10 seconds. Unfortunately, it's not like that.

I'm using cfq scheduler, which, I believe, redistributes IO equally among tasks. Am I wrong?

I tried the same case on my freinds gentoo machine. The firefox would start slower, but cca. 2x slower, not 10+x slower as here. The only obvious difference is that I'm using a SATA drive.

Any help would be appreciated. Thanks.

NeddySeagoon · Posted: Sun May 14, 2006 5:30 pm Post subject:

zehnan1,

Its silly question time - run uname -a
Is the date/time shown the date/time that you compiled your kernel ?

I'm using SATA here too. How much slower it gets when HDD I/O is divided between two tasks depends on how far the disk hread has to move between the two files. I would expect the HDD access time to at least double, plus head movement time. which is 'lost' for reading and writing and thats where most of the time goes.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

drvolk · n00b Joined: 05 Jul 2003 Posts: 72

Hi,

I have no SATA but "normal" IDE drives on my Asus A8R-MVP Board, with an AMD64x2 4200+.

And i have the same problems

So the problem seems to be not depending on SATA configs etc.

Please let me know, when you solved your problem. I will do it also also, if i could found a solution.

moesasji · Apprentice Joined: 10 May 2005 Posts: 263

Another me too post as I experience exactly the same problems.
It's driving me nuts as I can't find any solution.
I'm working on a AMD64 X2 4400Hz, 1GB, 1 SATA drive
(but I think my previous P4-system had similar problems, but not to this extend)

My dstat output looks very similar to that posted by zehnan1.
Basically very high IO_wait on any program that uses IO-access.
Programs that are worst: rar, par2repair, K3B when it is checking a checksum.
Opening a simple terminal can take a minute under those conditions, which is normally <1s.
Even typing a piece of text often halts for a second. Definitely not normal on a multi-user system.

PREEMPT kernel-options:

Akkara · Posted: Fri Jan 26, 2007 12:32 pm Post subject:

Could it be the I/O scheduler you're using?

I had found that using the CFQ I/O scheduler works well for me. I also do realtime audio (tho on a gentoo-sources kernel - with 1000 Hz interrupt, preemptible kernel, and preempt big kernel lock).

moesasji · Apprentice Joined: 10 May 2005 Posts: 263

Thanks Akkara for this suggestion.

I've tried wiggling slightly with the SATA-cable during extraction of a rar-file.
It seems to have no effect on the output of dstat or on the responsive-ness of my system.
Anyway, if k3b is calculating the checksum for an iso-file I see reading of typically 60-70MB/s from my HD.
That is about as high as it can get. So I find it difficult to believe that the problem would be in the cable.

Note that if I try to open a simple terminal it only shows up after K3B is finished.
Even though the terminal should be cached in memory and not have to come from HD.

ps) I indeed have cfq set as IO-scheduler, so that is not the solution.
----
~ $ cat /usr/src/linux/.config | grep IOSCHED
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
CONFIG_DEFAULT_IOSCHED="cfq"
----

Janne Pikkarainen · Posted: Sun Jan 28, 2007 10:10 am Post subject:

If everything else (I/O scheduler tuning etc) fails, one might want to try ionice command. If my memory serves me, it's available in unstable/masked sys-apps/util-linux packages.
_________________
Yes, I'm the man. Now it's your turn to decide if I meant "Yes, I'm the male." or "Yes, I am the Unix Manual Page.".

boniek · Guru Joined: 26 Mar 2005 Posts: 373

Ionice is available in sys-process/schedutils which is not hardmasked.
_________________
[HOWTO]New freetype subpixel font rendering for lcd monitors

ashrack · n00b Joined: 26 Nov 2006 Posts: 34

I get the same problem.
Using pata drives.
Ubuntu Edgy with vanilla kernel 2.6.19.2+reiser4+fuse261 and using the DeadLine scheduler Could the scheduler be the problem in my case>?

This is the preempt set:

moesasji · Apprentice Joined: 10 May 2005 Posts: 263

@ashrack: Seeing the problem I don't believe it is related to IO-scheduler.
In my case I could pinpoint this problem to a specific patch introduced into the 2.6.18 kernel.
See this post in the corresponding bugreport.

Basically I could generate a condition that IO is still running, but the whole system stalls for ~10s.
For me this behavior is not present in the 2.6.17 kernels....so running a 2.6.17 kernel seems the best work-around.

Unfortunately response on kernel-bugs seems extremely slow....