| View previous topic :: View next topic |
| Author |
Message |
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20067
|
 Posted: Sat Nov 16, 2019 4:30 am Post subject: RAID > LVM > fs alignment and layout (ab)use? |
 |
|
The drive is 3.7TiB @ 5400RPM and will be used for backups, distfiles and general storage (no OS).
Partition alignment seems simple enough, but I've not found much about RAID, LVM and file systems working together or against one another. LVM options that seemed possibly relevant were dataalignment, physicalextentsize, stripesize. mkfs.ext4 has block size which seems to work with physical alignment. Also stride and stripe_width which mention RAID. But with LVM between the fs and RAID, that seems like a potential issue.
My thought is to use multiple RAID partitions primarily to reduce scrub times, but also as a possible hedge against partial disk failure. At 310.5G each, 12 partitions seem like the best compromise.
Single Disk Layout:
RAID 0: sdb1, sdb2, sdb3 (= md123)
RAID 0: sdb4, sdb5, sdb6 (= md456)
"LVM mirror": md123, md456 (= vg123456) 931.5G available to a file system.
The main problem I see would be migrating the mirrors to the second drive. It's been a while and I don't recall the steps. Implications of mirroring across drives seems to make expansion more of a challenge.
The only other relevant methods I've seen were fixed size OS partition mirrors (/, /usr, /var) and usage of whole disks for large pools. Maybe I'd be better off with the latter and just go for a straight mirror.
Any recommendations?
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
 |
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3178
|
| Posted: Sat Nov 16, 2019 2:26 pm Post subject: |
|
|
| Quote: | Single Disk Layout:
RAID 0: sdb1, sdb2, sdb3 (= md123)
RAID 0: sdb4, sdb5, sdb6 (= md456)
|
Why would you do something like that? Striping across a single device looks like a downgrade from using those partitions in JBOD or using the underlying device directly. You're losing (already poor) performance and increasing the splash effect from local failures.
Second thing, it's a spinning drive, so alignment is not really that important, they have small block sizes anyway and finding the correct location takes way longer than reading a few extra bytes.
Now, AFAIR LVM uses 2MB headers everywhere and 4MB extents by default, so if your partitions are aligned, your FS on top of LVM should also be aligned.
MDRAID with metadata 0.9 stores its superblock at the end of device, so it does not affect location of whatever you put on top of it in any way (handy for booting from RAID 1 - bootloader does not have to speak RAIDish to work with this setup)
Not sure about metadata 1.2, superblock is in front of the usable space, but I don't remember it's size. I think it's 1 MB, correct me if I'm wrong.
If you want to extend it easily, you may consider skipping MDRAID completely and go for LVM RAID instead, though in this case you have to provide those parameters every time you create a new LV (I mean inform lvm that you want something else than "linear" target). I think it's a bit slower and less mature (using dmraid instead of mdraid), but it would let you add a bunch of new partitions to the volume group and then pvmove some data to "rebalance" the load (and make it more fault-resistant). Just make sure you know how to activate your volume group in a degraded state, AFAIK unlike MDRAID, degraded LVM will not come online until you explicitly demand it.
Quite frankly I wouldn't bother designing a complex partitioning scheme here. Either get a second drive and make a proper RAID with actual redundancy built in, or just don't make a RAID in the first place.(or create a raid1 with an empty slot, so you can simply plug in a second drive later and synchronize it without shutting the whole thing down for a few hours) and accept the fact, that if your drive fails, your backups are gone.
2 copies on a single drive is a false hope anyway. |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20067
|
| Posted: Sat Nov 16, 2019 9:30 pm Post subject: |
|
|
As I mentioned, it would only be temporary until I have the second drive and could move that half. But it is a bit complicated. I liked it less and less as I was writing the post.
I couldn't find anything on alignment, so I'm glad you don't think it will be a problem either. Sometimes layers on layers can wreak havoc.
For simplicity, I'm leaning toward a single partition (the way I normally use it), but the "partial disk failure" method seemed worth exploring.
3.7TiB seems to eliminate version 0.9 metadata, and that seems to be the preferred version for some reason. | man madm wrote: | | This format limits arrays to 28 component devices and limits component devices of levels 1 and greater to 2 terabytes. |
I may just make it an interim OS drive until I get another disk.
Thanks for your input.
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54397
Location: 56N 3W
|
| Posted: Sun Nov 17, 2019 12:07 am Post subject: |
|
|
pjp,
With version 0.9 metadata you can use kernel raid auto assemble.
All other metadata versions require an initrd if root is on raid.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20067
|
| Posted: Sun Nov 17, 2019 4:37 am Post subject: |
|
|
Thanks. Seems initrd is getting more difficult to avoid
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Other Things Gentoo
