[SOLVED]Kernel fails to mount RAID1 root dir at boot

fearofcarpet · Last edited by fearofcarpet on Mon Oct 02, 2006 12:15 pm; edited 1 time in total

I am re-installing Gentoo after a major hardware upgrade and I decided to do the whole RAID thing this time around... I followed this guide: http://www.gentoo.org/doc/en/gentoo-x86+raid+lvm2-quickinstall.xml exactly, except I used the Jackass 2006.1 tarball and ck-sources (2.6.17) instead of the Gentoo stage 3 tarball and gentoo-sources. It is an x86 (32bit) build on an Athlon FX-55 on an nForce4 board with the onboard Nvidia RAID disabled.

Everything works just great with the following drive layout on 2 SATA drives, /dev/sda and /dev/sdb (I set "Linux RAID autodetect" in fdisk too):

/dev/md1 -> /boot (RAID1)
/dev/md2 -> / (RAID1)
/dev/md3 -> /dev/vg (RAID0 with /usr /home /var /opt as LVMs)
/dev/sda5 and /dev/sdb5 as swap

All partitions are ext except /home which is reiser and everything works flawlessly in chroot from the liveCD. When I try to boot it directly however, the kernel tells me a few things (which I have to catch a glimpse of as they fly by before the kernel panics) when it loads 'md'.

First, that the UUIDs differ between /dev/sda1 and sdb1 and /dev/sda2 and sdb2, then a message that /dev/md2 has already been initialized and thus can't be initialized by 'md' (I don't have the exact message because it is only on-screen for about 3 seconds). It then attempts to mount /dev/md2 as reiserfs (it is ext3) which of course fails and the kernel panics because it can't load the root partition. If I recompile the kernel with reiser as a module the kernel says it failed to mount /dev/md2 after trying ext3 ext2 and vfat (which are the filesystems that are compiled in) and then panics.

After all the garbage about /dev/md2 it says "adding /dev/sda1 and /dev/sdb1 to /dev/md1" (or whatever the exact message is) and tells me it had no problem constructing the RAID1 array between /dev/sda1 and /dev/sdb1 at /dev/md1 which, other than being smaller, is identical to /dev/md2 (both ext filesystems in RAID1, both created the same way). There is no mention of /dev/md3 at all, however.

Anyway, I've been trying to get this thing working for 2 days and am about ready to just give up and mount /dev/sda2 as /, but I'm not sure about a couple of things... First, why can't the kernel load /dev/md2 which mounts fine from the liveCD? Why on Earth does it think that it is a reiser partition when the LiveCD has no problem detecting and mounting it as ext?

Second, and perhaps more importantly at this point, how to I remove the two partitions from the RAID array? As it stands if I just reformat the individual partitions and point the kernel at /dev/sda2 with GRUB (instead of /dev/md2) it refuses to mount because 'md' still thinks /dev/sda2 and /dev/sdb2 need to be in a RAID1 array (and you can't mount individual partitions after they've been stuck to /dev/md*).

Third, and least importantly, does anyone know what the error about /dev/md2 already having been created is all about? And is this related to the inability of the kernel to mount it correctly? And why doesn't the kernel say boo about /dev/md3?

I would appreciate any advice... This is my first attempt at software RAID in Linux and I'm really impressed with the performance while installing/compiling under chroot so I'd very much like to get the stupid thing working. I'm pretty sure I'm not a complete idiot and I run Gentoo on two other desktops (one even uses hardware RAID0) and my work laptop, but clearly I'm missing something really obivous and silly.

R!tman · Posted: Thu Sep 07, 2006 8:45 pm Post subject:

Please post 'cat /proc/mdstat' when you use the livecd(if you don't want to type it use ">" to save to a file on a disk, usb-stick, ...), the relevant part of your kernel config regarding raid stuff and file systems and your grub.conf file.

EDIT: ...and /etc/fstab

fearofcarpet · Posted: Thu Sep 07, 2006 9:18 pm Post subject:

R!tman · Posted: Thu Sep 07, 2006 9:43 pm Post subject:

This looks mostly ok to me, except for the "superblock is not persistent" in your mdadm details for md2 (good thinking, forgot about that!).

Try to assemble md2 using the "--update" option which should update the superblock. I guess "--update=summaries" should do it. Check with "mdadm --detail /dev/md2" again. If this doesn't work you can also try "--update=resync". This can take longer as the raid may have to resync ;-)

, you can watch using

fearofcarpet · Posted: Fri Sep 08, 2006 1:02 am Post subject:

R!tman · Posted: Fri Sep 08, 2006 5:58 am Post subject:

fearofcarpet · Posted: Fri Sep 08, 2006 10:28 am Post subject:

The really confounding thing is that if I take sda2 and sdb2 out of the RAID1 array and just boot to /dev/sda2 I get the same error - even when using a backup of the perfectly functional install I had before I decided to monkey with it... I'm think it may be time to just start over from scratch, but if I take the time to rebuild a fresh installation and get the same error I may very well break down into tears.

I will re-check all the kernel parameters and re-emerge everything udev related when I get home... I'll be sure to post the results, hopefully from a happy Gentoo/RAID install

R!tman · Posted: Fri Sep 08, 2006 11:01 am Post subject:

fearofcarpet · Posted: Fri Sep 08, 2006 11:33 am Post subject:

Yeah, I'm pretty sure my problem is covered in there because this installation was in fact installed from "old media". I bet I'm missing /dev/console and /dev/null... Though the solution is a bit cryptic (presumably the mount --bind / /test and mknod stuff has to be done from the livecd). At any rate I have to wait until after work to play with it (there is nothing more distracting than having to go to work with a sick computer at home!).

I always mount /proc and /dev before chrooting too

R!tman · Posted: Fri Sep 08, 2006 5:41 pm Post subject:

fearofcarpet · Posted: Fri Sep 08, 2006 11:30 pm Post subject:

Yup, that was the problem... I needed to mknod /dev/null and /dev/console. So satisfying to see those init scripts fly by

Thanks for the help!

R!tman · Posted: Sat Sep 09, 2006 5:36 am Post subject: