mdraid max sync speed

eccerr0r · Posted: Sat Sep 14, 2024 2:52 pm Post subject: mdraid max sync speed

Just wondering if anyone knows anything about this:

It appears the default sync_speed_max is 200000 and it has been this way for ever since at least I'm aware of mdraid - I've been using it for probably almost 2 decades by now.

Sync_speed_max is the maximum disk bandwidth it uses to check/rebuild/repair parity blocks on a RAID. It is in KB/sec.

200,000 KB/sec or 200MB/sec still exceeds the hard drive speeds that I have, some even far, far exceeds. which means it will completely eat out all available disk bandwidth. It's not until about now where SSDs can now exceed 200MB/sec. I suspect some HDDs finally can do it now too.

Has Linux always been designed for SSDs in mind or 2024 hard drives? :D

(My fastest sequential read hard drive is around 180MB/sec now... which is still too slow and thus will be fully consumed by default resync speeds. I just wonder why this 200000 was chosen back in the days...)

Also BTW, this isn't a bad thing as resilver is important as your redundancy is unavailable during this time, at least for RAID1/RAID5. But why 200000 and not simply "whatever bandwidth is available..." as when hard disks reach 300MB/sec there's redundancy left on the table...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

druggo · Posted: Mon Sep 23, 2024 10:04 am Post subject:

The sync speed of an array composed of multiple hard drives.
For example, my 8-disk SSD RAID10, if not limited, can reach 2225960 (2.2GB/s), with each disk's speed approximately 480MB/s.
Therefore, in the era of mechanical HDD, the 200MB/s limit still made sense.

By the way, I found the patch that changes this limit:
https://lore.kernel.org/lkml/Pine.LNX.4.33.0112191607110.7629-200000@localhost.localdomain/T/
_________________
HighWayToHell(blog)
https://blog.druggo.org/

eccerr0r · Posted: Mon Sep 23, 2024 1:58 pm Post subject:

Well, your SSD RAID clearly can exceed the 200MB/sec limit in the kernel, do you actually bump up the number? Have you had a failure yet?

I'm just saying for my mechanical drives, even back in 2001, 200MB/sec may well be 400MB/sec or even infinite. IIRC in 2001, hard disks were in the 10GB? range, and disk head rates were a few MB/sec. I don't think people had 100 disk RAIDs back then... that would tax that 200MB/sec number?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

druggo · Posted: Mon Sep 23, 2024 2:32 pm Post subject:

yes, to release the potential ( echo 20000000 > sync_speed_max ), a new raid wait for failure

back in 2001, I had a 40GB IBM glass-based disk support ATA100 , so max speed is 100MB/s .
_________________
HighWayToHell(blog)
https://blog.druggo.org/

eccerr0r · Posted: Mon Sep 23, 2024 2:44 pm Post subject:

There was no way those 40GB disks reached 100MB/sec and I think it's more like ATA33. Head rates are at most 20MB/sec or so.

I think it wasn't until 60GB disks where some had ATA66/ATA100. The fastest head rate I've seen on 120GB ATA100s were like 60MB/sec or so.

The fastest ATA100/133 disk I've seen was around 70MB/sec and this was more like 2008 ish... In any case this is still well lower than 200MB/sec.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

druggo · Posted: Mon Sep 23, 2024 3:38 pm Post subject:

You are right, a 40GB disk cannot achieve 100MB/s, as the ATA100 was the interface specification.

According to the article https://www.anandtech.com/show/591 , written in 2000:
40GB IBM deskstar can reach 32MB/s, then a six disk raid may syncing up to 192MB/s

_________________
HighWayToHell(blog)
https://blog.druggo.org/

eccerr0r · Posted: Mon Sep 23, 2024 4:45 pm Post subject:

However this sync speed is not 192MB/sec it's still 32MB/sec. This 200MB/sec number appears to be bottlenecking the writes, not reads.

When assembling arrays for the first time which actually does a "recovery," despite the 200MB/sec setting, my test 5-disk RAID5 was clearly reading four disks at 100MB/sec -- 400MB/sec read -- and writing the fifth at 100MB/sec. So it's not attainable for me now, and definitely not attainable then.

BTW I suspect back then they were for certain not targeting ATA, UDMA or not -- it was U160 or maybe U320 SCSI most likely. But even those disks didn't have head rates this high, and U160 bus itself would be bottlenecked at 160MB/sec anyway (and U320 a bit better but you'd still need to read disks, which eat a huge amount of bandwidth). So what the heck with this 200MB/sec number?!
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?