initramfs fails lvm2 root device on md-based raid disk

g.tomassoni · n00b Joined: 13 Aug 2005 Posts: 51

Dears,

I used genkernel --lvm2 --gensplash initrd to build the initramfs-genkernel-x86-2.6.12-gentoo-r8 file for my kernel-genkernel-x86-2.6.12-gentoo-r8.

My root fs is a logical volume of a vg whose pv is a md-based raid1 of two disk partitions (/dev/sda6 and /dev/sdb6).

I also have the /boot fs on a md-based raid1 of two disk partitions (/dev/sda1 and /dev/sdb1).

The above-mentioned partitions are marked as Linux raid autodetect and autodetection recognized /dev/md0 as composed by /dev/sd[ab]1 and /dev/md1 as composed by /dev/sd[ab]6.

Everything works fine as long as I use a classical initrd file to start the os. However, using a initramfs-style file doesn't work.

The reason I found is that when bootup is controlled by a initramfs file, the kernel init code doesn't invoke the prepare_namespace() function (see line#679 of init/main.c) which is in charge of eventually invoking the autostart_arrays() function in drivers/md/md.c.

This leads to the fact that lvm is unable to discover any md-based vg through vgscan, thereby not activating it. In turn, this leads to the fact that the root device specified in the kernel commandline is not found and the kernel itself asks me for a suitable device.

Assuming correct the behaviour of the linux kernel to not automatically start any md-based raid, the problem seems rooted in the /etc/initrd.scripts' setup_md_device() function, which seems designed to automatically start a raid iff the associated device (/dev/md*) is specified as the root device in the linux commandline. Since my root is a lvm lv, this is not the case and setup_md_device() skips md discovery as a whole.

I also found that issuing the following commands from a shell obtained in the same context of initramfs:

johndo · n00b Joined: 22 Nov 2003 Posts: 32

The symptoms you describe match what I am going through.
The only diff

Is there anyway to force genernel to use initrd then?

edit: the only differences are that Im doing lvm2 on raid5

g.tomassoni · n00b Joined: 13 Aug 2005 Posts: 51

Hi johndo,

I placed a bug for it, and someone at Gentoo kindly replied to test boot setting which didn't worked before some tuning.

You have to identify the /dev/md? devices which is the pv containing you root's lv, then you have to add the setting:

johndo · n00b Joined: 22 Nov 2003 Posts: 32

I added lvmraid=/dev/md0 to myand my system's functionalty was restored.

my real_root device is actually /dev/hda3. I only use my md0 device for /home, /opt, /tmp, /usr, and /var.
This is so that I will have a semi-usable system in case the raid ever fails, like it did here (:

on a side note: one of the messages I got was that real_root was detected on a lvmraid array.
This is probably just a cosmetic error and doesn't actually affect anything.

g.tomassoni · n00b Joined: 13 Aug 2005 Posts: 51

johndo,

this is weird: I guessed that the lvmraid= and real_root= setting were needed only when the root was on a lvm-over-raid device...

Given your configuration, you should be capable to start your system by simply specifing root=/dev/hda3 and omitting any lvmraid= setting...

johndo · n00b Joined: 22 Nov 2003 Posts: 32

A lot of the bootup programs depend on files that are on the raid array, so I do need it for bootup
I may have been able to start the system and mount the array after it booted by logging in as root and running the mdstart commands, but things like apache and Xorg won't work until I do.

You also need real_root when root=/dev/ram0, which genkernel specifies by default, iirc.

Very recently (same day I got lvm on raid working again) I had a problem with the volume group on the array: it doesn't exist anymore. /dev/md0 exists, but I can't get vgscan to find my ''raid5" volume group on it. My old kernel build can not find it either. I don't know if this is realted or not, but I don't know how I lost it or how to get it back.

taskara · Posted: Sat Oct 01, 2005 11:56 pm Post subject:

I'm trying to get my Gentoo system built on an lvm group on top of raid5, but it won't go.

I am using a genkernel --lvm2 created initrd but when it searches for raid volumes it find nothing! (I did also have --dmraid but I think that's for sudo hardware onboard raid controllers)
However, if I boot to the kernel directly without an initrd, it DOES find the raid devices (but then fails, naturally, because it can't find lvm volumes).

I'm wondering whether this issue is related?

Essentially I have:
sda5, b5, c5, d5 all on a linux software raid 5 array (/dev/md5)
I then have an lvm group called "system" under that, and then volumes "slash" and "home" under that.

The machine is at the office, so I may head down there and try adding "lvmraid=/dev/md5" to grub's kernel line.

Any thoughts?

Thanks,
-c
_________________
Kororaa install method - have Gentoo up and running quickly and easily, fully automated with an installer!

taskara · Posted: Sun Oct 02, 2005 2:58 am Post subject:

lvmraid option got me booting - but it is strange! The initrd still says that it can't find any software raid devices, but now it CAN find the lvm volumes and it boots all the way. Once I log in I see that I have all my raid devices started, and all the right things are mounted.

There is an issue however, that after it finds the lvm group and start booting, I get "rm - can't remove /sys/blah*" spewing across the screen for about 2 minutes, then it goes and boots just fine.

Has anyone heard of these issues?

I assume lvmraid option is new as I had this working without it previously. Perhaps Gentoo devs have changed the way all this works?

Cheers,
-c
_________________
Kororaa install method - have Gentoo up and running quickly and easily, fully automated with an installer!

Wilke · Posted: Tue Jan 24, 2006 9:57 am Post subject:

Hope it's ok if I kick an old topic to the top, but I seem to have the same or at least a very similar problem.

My setup is the same as mentioned by the topicstarter. I did run genkernel with all the right flags as described in this topic (--lvm2, set the real_root and lvmraid kernel parameters etc.). However, the kernel cannot mount my LVM-on-RAID1 root partition while booting from the ramfs.

It works fine when I boot from a different root partition (non-LVM/RAID) and then the LVM-on-RAID partitions will also be mounted just fine by the normal Gentoo init-scripts. But when I try to mount the LVM-on-RAID root during boot, the problem is that the RAID is not detected at all, and hence the LVM volumes cannot be found. Even the manual solution described in the topicstart does not work for me: when I run 'mdstart /dev/md0' and then 'cat /proc/mdstat' it will just say there are no RAID devices at all.

Maybe it has to do with genkernel - should I instruct genkernel to run menuconfig and manually change all the RAID and LVM-options to builtin rather than modules? I don't really see how it would help because the initramfs loads the modules just fine. But still it cannot find the RAID. It seems as if it needs the /etc/mdadm.conf in order to detect the RAID volumes or something....(mdadm and /etc/mdadm.conf are not available in the initramfs - as they are both on the root partition itself).

P.S. Yes, the partitions are marked 'fd' - Linux raid autodetect and all that

overkll · Posted: Tue Jan 31, 2006 4:10 am Post subject:

Wilke,

Came across this post while researching LVM. Read the whole thing and I think you're the only one who thought of modules vs built in.

If you build the raid driver into the kernel, all md devices will be found and configured automagically, thanks to the raid info stored on the disk (superblock). /etc/raidtab or /etc/mdadm.conf are not needed. Don't believe it? Just pop a fedora core install cd into your system and boot up. It doesn't have your mdadm.conf file yet it will create the devices correctly. To bad gentoo cd's don't do the same. :oops:

I don't have any real experience yet with LVM2, but I don't see why you shouldn't build that driver into the kernel as well. Personally, I build everything into the kernel (using menuconfig) unless its only available as an external kernel module ie nvidia, ivtv, etc. I don't need an initrd or initramfs. I just boot the kernel directly with grub.

Does lvm *NEED* and initramfs?

Isn't all the necessary LVM info to start up LVM devices already written on the disk? If so I would think that the built-in kernel driver will start it up like a champ.

Wilke · Posted: Sat Feb 04, 2006 7:53 pm Post subject:

Just to confirm, compling LVM + RAID support into the kernel solved the problem - even if you still use it in combination with initramfs (to automatically load any other modules).

I still use genkernel, just make sure you add '--menuconfig' as a parameter and build everything related to LVM/RAID into the kernel.

Command I used to create the kernel: genkernel --lvm --menuconfig all.

So now I wonder, is there a technical reason why the RAID system will not autodetect RAID-partitions when it is loaded as a module from initramfs? Or is this just a "implementation detail" or "bug" in the current kernel? Could this be fixed in the kernel or is there a very good reason why it's like this?

g.tomassoni · n00b Joined: 13 Aug 2005 Posts: 51