| View previous topic :: View next topic |
| Author |
Message |
dekon
Tux's lil' helper
Joined: 21 Aug 2005
Posts: 80
|
 Posted: Sat Apr 15, 2006 1:36 pm Post subject: Slow copying but dma is on :( [SOLVED] |
 |
|
Hello
i have this problem for a long time.
i have one HDD (/dev/hda), when i copy files between directories copy is to slow (on moment is on 13 MB/s after one seconds speed decrease to 8 MB/s). This speed hang on to finish copy  .
In windows copy speed si most high (stable 20 MB/s, sometimes 30 MB/s)
My hardware
| Code: |
00:00.0 Host bridge: nVidia Corporation nForce3 250Gb Host Bridge (rev a1)
00:01.0 ISA bridge: nVidia Corporation nForce3 250Gb LPC Bridge (rev a2)
00:01.1 SMBus: nVidia Corporation nForce 250Gb PCI System Management (rev a1)
00:02.0 USB Controller: nVidia Corporation CK8S USB Controller (rev a1)
00:02.1 USB Controller: nVidia Corporation CK8S USB Controller (rev a1)
00:02.2 USB Controller: nVidia Corporation nForce3 EHCI USB 2.0 Controller (rev a2)
00:08.0 IDE interface: nVidia Corporation CK8S Parallel ATA Controller (v2.5) (rev a2)
00:09.0 IDE interface: nVidia Corporation CK8S Serial ATA Controller (v2.5) (rev a2)
00:0a.0 IDE interface: nVidia Corporation CK8S Serial ATA Controller (v2.5) (rev a2)
00:0b.0 PCI bridge: nVidia Corporation nForce3 250Gb AGP Host to PCI Bridge (rev a2)
00:0e.0 PCI bridge: nVidia Corporation nForce3 250Gb PCI-to-PCI Bridge (rev a2)
00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
01:00.0 VGA compatible controller: nVidia Corporation NV43 [GeForce 6600/GeForce 6600 GT] (rev a2)
02:06.0 FireWire (IEEE 1394): VIA Technologies, Inc. IEEE 1394 Host Controller (rev 80)
02:08.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10)
02:09.0 Multimedia audio controller: C-Media Electronics Inc CM8738 (rev 10)
|
My HDD
| Code: |
gentoo dekon # dmesg|grep hda
ide0: BM-DMA at 0xf000-0xf007, BIOS settings: hda:DMA, hdb:DMA
hda: ST3160021A, ATA DISK drive
hda: max request size: 1024KiB
hda: 312581808 sectors (160041 MB) w/2048KiB Cache, CHS=19457/255/63, UDMA(100)
hda: cache flushes supported
hda: hda1 hda2 < hda5 hda6 >
|
DMA
| Code: |
gentoo dekon # hdparm /dev/hda
/dev/hda:
multcount = 16 (on)
IO_support = 1 (32-bit)
unmaskirq = 1 (on)
using_dma = 1 (on)
keepsettings = 0 (off)
readonly = 0 (off)
readahead = 256 (on)
geometry = 19457/255/63, sectors = 312581808, start = 0
|
hdparm -I /dev/hda
| Code: |
/dev/hda:
ATA device, with non-removable media
Model Number: ST3160021A
Serial Number: 3LJ3A923
Firmware Revision: 8.01
Standards:
Used: ATA/ATAPI-6 T13 1410D revision 2
Supported: 6 5 4 3
Configuration:
Logical max current
cylinders 16383 65535
heads 16 1
sectors/track 63 63
--
CHS current addressable sectors: 4128705
LBA user addressable sectors: 268435455
LBA48 user addressable sectors: 312581808
device size with M = 1024*1024: 152627 MBytes
device size with M = 1000*1000: 160041 MBytes (160 GB)
Capabilities:
LBA, IORDY(can be disabled)
bytes avail on r/w long: 4 Queue depth: 1
Standby timer values: spec'd by Standard, no device specific minimum
R/W multiple sector transfer: Max = 16 Current = 16
Recommended acoustic management value: 128, current value: 0
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=240ns IORDY flow control=120ns
Commands/features:
Enabled Supported:
* READ BUFFER cmd
* WRITE BUFFER cmd
* Host Protected Area feature set
* Look-ahead
* Write cache
* Power Management feature set
Security Mode feature set
SMART feature set
* FLUSH CACHE EXT command
* Mandatory FLUSH CACHE command
* Device Configuration Overlay feature set
* 48-bit Address feature set
SET MAX security extension
* DOWNLOAD MICROCODE cmd
* General Purpose Logging feature set
* SMART self-test
* SMART error logging
Security:
Master password revision code = 65534
supported
not enabled
not locked
not frozen
not expired: security count
not supported: enhanced erase
HW reset results:
CBLID- above Vih
Device num = 0 determined by the jumper
Checksum: correct
|
# hdparm -t -T /dev/hda
| Code: |
/dev/hda:
Timing cached reads: 2220 MB in 2.00 seconds = 1110.17 MB/sec
Timing buffered disk reads: 128 MB in 3.02 seconds = 42.45 MB/sec
|
I use kernel-2.6.15-gentoo-r7, my config is here.
How i must set hdparm or kernel to i can copy more then 8 MB/s ?
_________________
Sorry for my English,i am not good at in this 
Last edited by dekon on Sun Apr 16, 2006 1:21 pm; edited 1 time in total |
|
| Back to top |
|
 |
felicehome
Apprentice
Joined: 30 Sep 2004
Posts: 217
|
| Posted: Sat Apr 15, 2006 2:46 pm Post subject: |
|
|
Hi,
I think that highly depends on the program you use to copy files. If I use konqueror to copy a large file from one directory to another on the same harddrive I get even worse numbers than yours. (about 3MB/s)
But if I use rsync for example | Code: | | rsync -av --progress /source /dest |
.
I get numbers around 30MB/s!
Also read this one for more information about that topic: http://gentoo-wiki.com/TIP_Fast_Copy |
|
| Back to top |
|
|
dekon
Tux's lil' helper
Joined: 21 Aug 2005
Posts: 80
|
| Posted: Sat Apr 15, 2006 3:02 pm Post subject: |
|
|
Hi,
I copy files over mc.
I try some recommendation from gentoo-wiki, thanks for the tip.
_________________
Sorry for my English,i am not good at in this |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54834
Location: 56N 3W
|
| Posted: Sat Apr 15, 2006 3:07 pm Post subject: |
|
|
dekon,
Hmm. I think this a feature. Your copy involves reading data, moving the disk head, then writing data, moving the head ... How fast you can copy a file from one place on a disk to another place on the same disk is related to the number of head movements. That, in turn is related to the size of memory buffer used and the operating systems rules about how old buffers are allowed to be before they are written to disk.
The easiest way round this is to do the copy in two operations, copy the file to a ramdisk, copy it back to the hard drive. This simplistic approach will fail when the file will bot fit on your ramdisk.
You already have a ramdisk, its /dev/shm, so you can write a two pass copy script like this
| Code: | #!/bin/bash
cp %1 /dev/shm/
cp /dev/shm/%1 %2 |
make it executeable and call it
<script_name> source destination. Its only good for single files that fit into your /dev/shm. I've not tested it either,
thats left as an exercise for the reader.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
dekon
Tux's lil' helper
Joined: 21 Aug 2005
Posts: 80
|
| Posted: Sat Apr 15, 2006 5:21 pm Post subject: |
|
|
I download catmeter.
Catmeter is small utility (filter) which reading data from stdin and write this data to stderr and display transfer speed.
My results
| Code: |
dekon@gentoo ~/catmeter-1.0 $ cat /dev/zero | catmeter > /dev/null
396184.000 KBytes/sec (5 sec avg: 391849.812 Kbytes/sec)
|
| Code: |
dekon@gentoo ~/catmeter-1.0 $ cat /dev/zero | catmeter > /dev/null
408374.000 KBytes/sec (5 sec avg: 392151.406 Kbytes/sec)
|
I think that my broblem is in the aplication which i use fo a copying (mc).
Its possible ?
EDIT:
How i set cp to display me transfer speed ?
_________________
Sorry for my English,i am not good at in this |
|
| Back to top |
|
|
dekon
Tux's lil' helper
Joined: 21 Aug 2005
Posts: 80
|
| Posted: Sat Apr 15, 2006 6:51 pm Post subject: |
|
|
I update to sys-apps/coreutils-5.94-r1.
This version included progress-bar-patch.
Copy with cp is slow as mc
| Code: |
dekon@gentoo /home/andree $ cp -gv pak0.pk3.bz2 ~
`pak0.pk3.bz2' -> `/home/dekon/pak0.pk3.bz2'
pak0.pk3.bz2 | 100% | 423 MiB | 10 MiB/s | ETA 00:00.01
|
I play mp3 with beep-media-player, when i copy bmp doesnt play continuous
Please help me solved this problems 
_________________
Sorry for my English,i am not good at in this |
|
| Back to top |
|
|
dekon
Tux's lil' helper
Joined: 21 Aug 2005
Posts: 80
|
| Posted: Sat Apr 15, 2006 7:02 pm Post subject: |
|
|
| NeddySeagoon wrote: | dekon,
You already have a ramdisk, its /dev/shm, so you can write a two pass copy script like this
| Code: | #!/bin/bash
cp %1 /dev/shm/
cp /dev/shm/%1 %2 |
make it executeable and call it
<script_name> source destination. Its only good for single files that fit into your /dev/shm. I've not tested it either,
thats left as an exercise for the reader. |
This doesnt work.
| Code: |
dekon@gentoo ~/catmeter-1.0 $ ./speed /home/andree/pak0.pk3.bz2 /home/dekon
cp: cannot stat `%1': No such file or directory
cp: cannot stat `/dev/shm/%1': No such file or directory
|
df -h
| Code: |
none 250M 0 250M 0% /dev/shm
|
What does it mean, my ramdisk doesnt use ?
_________________
Sorry for my English,i am not good at in this |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54834
Location: 56N 3W
|
| Posted: Sat Apr 15, 2006 7:14 pm Post subject: |
|
|
dekon,
Oops % -> $
| Code: | #!/bin/bash
cp $1 /dev/shm/
cp /dev/shm/$1 $2 |
====== edit ======
I've tested it too.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
dekon
Tux's lil' helper
Joined: 21 Aug 2005
Posts: 80
|
| Posted: Sat Apr 15, 2006 7:28 pm Post subject: |
|
|
Bmp play continous when i used your method.
But this metod is useable for small files ...
Why is copy by "normal way" is so slowly ?
P.S:
My /dev/shm is full how i clean it ?
_________________
Sorry for my English,i am not good at in this |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54834
Location: 56N 3W
|
| Posted: Sun Apr 16, 2006 12:49 am Post subject: |
|
|
dekon,
It can be extended to work with bigger files.
The problem is associated with the size of the disk buffer allocated for the data transfer. A smaller buffer means more disk head movements. No data is transferred while the drive repositions the read/write head.
My small script does a read to RAM followed by a single head movement for the write back to disk.
If the problem is justthe music dropouts, make the music buffer bigger in your player.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
dekon
Tux's lil' helper
Joined: 21 Aug 2005
Posts: 80
|
| Posted: Sun Apr 16, 2006 9:31 am Post subject: |
|
|
| NeddySeagoon wrote: | dekon,
It can be extended to work with bigger files.
The problem is associated with the size of the disk buffer allocated for the data transfer. A smaller buffer means more disk head movements. No data is transferred while the drive repositions the read/write head.
|
I test it, 425 MB desnt big file i think.
Can i adjust disk buffer size ?
_________________
Sorry for my English,i am not good at in this |
|
| Back to top |
|
|
bollucks
l33t
Joined: 27 Oct 2004
Posts: 606
|
| Posted: Sun Apr 16, 2006 10:00 am Post subject: |
|
|
425MB is a big file when it's transferred to the same disk. You can only cache as much as you can fit in ram before it starts writing. Here's a brief kernel lesson in disk writing:
cat /proc/sys/vm/dirty_background_ratio
10
That's the percentage of your ram above which it will start writing the file at (if nothing else is writing at the time). So if you have 1GB of ram, you can probably write up to 100MB quickly, and then larger than that much more slowly as the writes and reads will be competing. Note that this value is not a hard limit and lots of other factors will make it write even earlier, but that's the most you can cache before it starts writing. The upper limit beyond which it will never cache is in /proc/sys/vm/dirty_ratio |
|
| Back to top |
|
|
dekon
Tux's lil' helper
Joined: 21 Aug 2005
Posts: 80
|
| Posted: Sun Apr 16, 2006 1:20 pm Post subject: |
|
|
OK, thanks a lot guys 
_________________
Sorry for my English,i am not good at in this |
|
| Back to top |
|
|
Gentree
Watchman
Joined: 01 Jul 2003
Posts: 5350
Location: France, Old Europe
|
| Posted: Sun Apr 16, 2006 9:26 pm Post subject: |
|
|
also bear in mind that copy performance depends on what part of the disk you read/write to , how full the partition is and what file system you are using and the nature of the files you are copying .
my guess is that windows it at the first partition of your drive , that's generally the fastest part. Obviously dont compare a non-journaling fs like vfat with a journaling one.
_________________
Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86 |
|
| Back to top |
|
|
|
Kernel & Hardware
