raid1 read errors

[jpsollie]

Hi everyone,

I have a question about raid1:
In recent artices, we see everyone encourages RAID6 over raid5 because of the impact of uncorrected read/write errors. In RAID6, this can be corrected, in RAID5, this cannot.
So, can you eliminate this error with RAID1 as well? Or do I need 3 drives for it?

Thx for the advice
_________________
The power of Gentoo optimization (not overclocked): [img]https://www.passmark.com/baselines/V10/images/503714802842.png[/img]

[eccerr0r]

On a 3-disk (minimum non degenerate) RAID5 (66% disk space efficiency), and two disks fails, you're SOL.
Of course in a 2-disk RAID 1 (50% disk efficiency), if two disks fail, you're SOL.

The whole worry is having two disks fail during rebuild. So yes, if you're worried about two disks failing during rebuild of RAID1 then you would need a 3-disk RAID1 (33% disk efficiency).

Minimum non-degenerate RAID6 requires 4 disks (50% disk efficiency) so pick your poison. If you can tolerate the speed penalty of RAID6 it's safer than 4-disk RAID1+0 which has the same disk efficiency.

Technically all these minimum cases, the chances of having a second disk fail is not that great - it's there but not horrible. The problem comes in if you have a large, say 10 disk RAID where the chance of a second of the remaining disks failing goes up significantly. The reason for a 10 disk RAID5 is mainly for disk efficiency, where now it's 90% disk space efficient but MTBF goes down by about an order of magnitude...) 10 spindle RAID5 does give you a little bit of iops and burst read improvement however.

In any case you still should backup to another medium and not pray that more than one disk will fail...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[jpsollie]

Hi eccerr0r,

Thank you for your response,

I was not really looking for a case where the disk goes bad, but mainly an undetected read/write error occurs. Let's say in raid1 hd0 writes value 0 and hd1 writes value 1, or in raid5 hd0 writes 0, hd1 writes 0 whereas it should have written 1, and hd2 writes 1 (XOR 0 1 -> 1), same error in raid6
In raid5, the raid will detect the error because data verification will see the XOR is mismatched, but will be unable to correct it. In raid6, this could be corrected.
What happens in case of raid1?
_________________
The power of Gentoo optimization (not overclocked): [img]https://www.passmark.com/baselines/V10/images/503714802842.png[/img]

[eccerr0r]

I think people are too worried about hard drive surface induced corruption. If you're worried about other sources of corruption, you need to invest in better hardware that supports end to end protection (parity or ECC). Hard drive media itself is protected from corruption by ECC. If the hard drive media errs, it will be detected by the drive and this error will be flagged to the hardware and then to the OS, and the OS will deal with it (read error). No problem here. I suppose the emphasis is that a disk surface error showing up undetected to the computer is exceedingly rare.

However what you're worried about is if there's memory corruption and the computer write bad data to the disk. Most of the time the error is when writing bad data due to memory corruption or data is corrupted from the SATA controller (or whatever connection you're using) and back through memory to the CPU. The subsequent read, bad data will come in, and there's no way to tell whether the data was correct. You're screwed here. None of the RAID systems will help you here.

Yes, even RAID-5 and RAID-6, because neither RAID system actually checks whether the data is correct, even though it can detect - this is done for speed because reading the parity sector wastes disk bandwidth and time (to compute). Same for RAID1, but actually could read both disks and simply compare, but you still can't tell whether which is the correct data when they differ.

RAID6 can figure out what the correct data was however... but it wasn't the intent of RAID6. It only checks all disks if it can't figure out the data reading the minimum (a 4 disk raid 6, it will only use data from 2 disks, but of course with striping, it will distribute among the 4 disks.)
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[ulenrich]

[eccerr0r]

The problem I had with disks a long while ago:

As a backup I copy files from disk to disk frequently. However I found that with big files, the files would fail diff and of course md5sum. Could not figure out where the source of the problem was until I ran memtest86 and found bad RAM was the reason why I would seemingly write files to disk that when read back, failed checksum, but did not give a bad sector error. I've had both bad disk controllers and bad RAM be sources of apparent disk corruption to date (verified by swapping the RAM and/or disk to another machine and verifying the disk behaves correctly on another machine or disk controller), for modern disks I have yet to see a medium source error that wasn't a computer problem. Garbage to the disk, garbage from the disk.

"I have yet to see an undetected error." (Except when it later fails md5sum...)

Modern disks write forward error correction bits when writing to disk, so when reading back it can detect and correct errant blocks upon read. I'm not entirely sure how many disks actually verify writes these days because write speeds tend to mirror read speeds, but one would expect if it's a verify write it would take twice as long to write as it would need to take two revolutions (three, if you count the seek ID pass, and even more with shingling). But FEC will fix errors or at least detect them upon reads (and the hard disk should try to rewrite/reallocate the track then), so it will be very rare that you see undetectable corruption sourced from the media...

BTW single bit errors in JPGs or other compressed files will cause quite a bit of corruption...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[ulenrich]

Argggh, yeah - pictures are compressed!
(compression bites failure tolerance)
Edit: error tolerance

[erm67]

use BTRFS raid1c2-3-4 (or ZFS), and move all error handling to the fs level: for a NAS buy only drives with SCT ERC timeout support (some drives branded as NAS drives doesn't) and set it to a very small level so that control is returned ASAP to the kernel. Only the filesystem knows which copy is correct and the hardware error recovery is most of the times useless; sometimes the ERC timeout of the drive is higher than the controller timeout or higher than the kernel timeout. On my NAS I use 7 seconds ERC timeout for my disks as recommended by the smartmontools devs. (and btrfs raid1c3 with 4 disks so I can lose 2 disks without loosing data 62.5% space efficency, with more drives the space eficency is even better)
look for smartctl-timeouts_v1.xx.zip for a set of udev rules for RAID arrays.
https://raid.wiki.kernel.org/index.php/Timeout_Mismatch
_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net