| View previous topic :: View next topic |
| Author |
Message |
tnt
Veteran
Joined: 27 Feb 2004
Posts: 1227
|
 Posted: Fri Feb 25, 2011 1:39 pm Post subject: grub setup on software RAID1 |
 |
|
kernel 2.6.37-gentoo
sys-boot/grub-0.97-r10
I have boot partition set up on 2 members RAID1:
| Code: | md0 : active raid1 sdh2[1] sda2[0]
488192 blocks [2/2] [UU]
bitmap: 1/60 pages [4KB], 4KB chunk
|
| Code: | Filesystem Size Used Avail Use% Mounted on
/dev/md0 462M 9.7M 429M 3% /boot
|
| Code: | GNU GRUB version 0.97 (640K lower / 9216K upper memory)
[ Minimal BASH-like line editing is supported. For the first word, TAB
lists possible command completions. Anywhere else TAB lists the possible
completions of a device/filename. ]
grub> find /boot/grub/stage1
(hd0,1)
(hd7,1)
|
when I try to install grub on first member of my RAID1, it installs without a problem:
| Code: | grub> root (hd0,1)
Filesystem type is ext2fs, partition type 0x83
grub> setup (hd0)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/e2fs_stage1_5" exists... yes
Running "embed /boot/grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
Running "embed /boot/grub/e2fs_stage1_5 (hd0,1)"... failed (this is not fatal)
Running "install /boot/grub/stage1 (hd0) /boot/grub/stage2 p /boot/grub/menu.lst "... succeeded
Done.
|
but, when I try to install it the same way on the second RAID1 member, it fails with error 16:
| Code: | grub> root (hd7,1)
Filesystem type is ext2fs, partition type 0x83
grub> setup (hd7)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/e2fs_stage1_5" exists... yes
Running "embed /boot/grub/e2fs_stage1_5 (hd7)"... failed (this is not fatal)
Running "embed /boot/grub/e2fs_stage1_5 (hd7,1)"... failed (this is not fatal)
Running "install /boot/grub/stage1 (hd7) /boot/grub/stage2 p /boot/grub/menu.lst "... failed
Error 16: Inconsistent filesystem structure
|
I've tryed to install grub on the second RAID1 member, but point it to the root partition on the first RAID1 member. no luck:
| Code: | grub> root (hd0,1)
Filesystem type is ext2fs, partition type 0x83
grub> setup (hd7)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/e2fs_stage1_5" exists... yes
Running "embed /boot/grub/e2fs_stage1_5 (hd7)"... failed (this is not fatal)
Running "embed /boot/grub/e2fs_stage1_5 (hd0,1)"... failed (this is not fatal)
Running "install /boot/grub/stage1 d (hd7) /boot/grub/stage2 p /boot/grub/menu.lst "... failed
Error 16: Inconsistent filesystem structure
|
any ideas how to install grub on both RAID1 members?
_________________
gentoo user |
|
| Back to top |
|
 |
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54830
Location: 56N 3W
|
| Posted: Fri Feb 25, 2011 3:05 pm Post subject: |
|
|
tnt,
You are doing it all correctly.
| info grub: | 16 : Inconsistent filesystem structure
This error is returned by the filesystem code to denote an internal
error caused by the sanity checks of the filesystem structure on
disk not matching what it expects. This is usually caused by a
corrupt filesystem or bugs in the code handling it in GRUB. |
Are you sure you boot raid1 is running ?
What does /proc/mdstat say about it?
Save what you need out of /boot reformat /boot and put the files back.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
tnt
Veteran
Joined: 27 Feb 2004
Posts: 1227
|
| Posted: Fri Feb 25, 2011 4:03 pm Post subject: |
|
|
I have the same problem on other 2 boxes, also.
about /boot filesystem/RAID health, I can mount and use:
- md0 as RAID1 (made from sda2 and sdh2) device with ext2 on it
- sda2 as separate device with ext2 on it
- sdh2 as separate device with ext2 on it
| Code: | root@gentoo /tmp # mount /boot
root@gentoo /tmp # df -h /boot
Filesystem Size Used Avail Use% Mounted on
/dev/md0 462M 9.7M 429M 3% /boot
root@gentoo /tmp # umount /boot
root@gentoo /tmp # mdadm -S /dev/md0
mdadm: stopped /dev/md0
root@gentoo /tmp # mount -t ext2 -o ro /dev/sda2 /boot
root@gentoo /tmp # ls -lah /boot/grub/
total 446K
drwxr-xr-x 2 root root 1.0K Nov 19 15:18 .
drwxr-xr-x 4 root root 1.0K Feb 23 14:23 ..
-rw-r--r-- 1 root root 197 Dec 18 2009 default
-rw-r--r-- 1 root root 135 Dec 18 2009 device.map
-rw-r--r-- 1 root root 8.6K Aug 2 2010 e2fs_stage1_5
-rw-r--r-- 1 root root 7.7K Aug 2 2010 fat_stage1_5
-rw-r--r-- 1 root root 7.0K Aug 2 2010 ffs_stage1_5
-rw-r--r-- 1 root root 1.1K Nov 19 15:18 grub.conf
-rw-r--r-- 1 root root 7.0K Aug 2 2010 iso9660_stage1_5
-rw-r--r-- 1 root root 8.5K Aug 2 2010 jfs_stage1_5
lrwxrwxrwx 1 root root 9 Apr 8 2010 menu.lst -> grub.conf
-rw-r--r-- 1 root root 7.2K Aug 2 2010 minix_stage1_5
-rw-r--r-- 1 root root 9.6K Aug 2 2010 reiserfs_stage1_5
-rw-r--r-- 1 root root 34K Aug 2 2010 splash.xpm.gz
-rw-r--r-- 1 root root 512 Aug 2 2010 stage1
-rw-r--r-- 1 root root 106K Aug 2 2010 stage2
-rw-r--r-- 1 root root 106K Dec 18 2009 stage2.old
-rw-r--r-- 1 root root 106K Aug 2 2010 stage2_eltorito
-rw-r--r-- 1 root root 7.2K Aug 2 2010 ufs2_stage1_5
-rw-r--r-- 1 root root 6.6K Aug 2 2010 vstafs_stage1_5
-rw-r--r-- 1 root root 9.4K Aug 2 2010 xfs_stage1_5
root@gentoo /tmp # umount /boot
root@gentoo /tmp # mount -t ext2 -o ro /dev/sdh2 /boot
root@gentoo /tmp # ls -lah /boot/grub/
total 446K
drwxr-xr-x 2 root root 1.0K Nov 19 15:18 .
drwxr-xr-x 4 root root 1.0K Feb 23 14:23 ..
-rw-r--r-- 1 root root 197 Dec 18 2009 default
-rw-r--r-- 1 root root 135 Dec 18 2009 device.map
-rw-r--r-- 1 root root 8.6K Aug 2 2010 e2fs_stage1_5
-rw-r--r-- 1 root root 7.7K Aug 2 2010 fat_stage1_5
-rw-r--r-- 1 root root 7.0K Aug 2 2010 ffs_stage1_5
-rw-r--r-- 1 root root 1.1K Nov 19 15:18 grub.conf
-rw-r--r-- 1 root root 7.0K Aug 2 2010 iso9660_stage1_5
-rw-r--r-- 1 root root 8.5K Aug 2 2010 jfs_stage1_5
lrwxrwxrwx 1 root root 9 Apr 8 2010 menu.lst -> grub.conf
-rw-r--r-- 1 root root 7.2K Aug 2 2010 minix_stage1_5
-rw-r--r-- 1 root root 9.6K Aug 2 2010 reiserfs_stage1_5
-rw-r--r-- 1 root root 34K Aug 2 2010 splash.xpm.gz
-rw-r--r-- 1 root root 512 Aug 2 2010 stage1
-rw-r--r-- 1 root root 106K Aug 2 2010 stage2
-rw-r--r-- 1 root root 106K Dec 18 2009 stage2.old
-rw-r--r-- 1 root root 106K Aug 2 2010 stage2_eltorito
-rw-r--r-- 1 root root 7.2K Aug 2 2010 ufs2_stage1_5
-rw-r--r-- 1 root root 6.6K Aug 2 2010 vstafs_stage1_5
-rw-r--r-- 1 root root 9.4K Aug 2 2010 xfs_stage1_5
root@gentoo /tmp # umount /boot
root@gentoo /tmp # mdadm --assemble /dev/md0 /dev/sda2 /dev/sdh2
mdadm: /dev/md0 has been started with 2 drives.
root@gentoo /tmp # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid6] [raid5] [raid4]
md0 : active raid1 sda2[0] sdh2[1]
488192 blocks [2/2] [UU]
bitmap: 0/60 pages [0KB], 4KB chunk
unused devices: <none>
root@gentoo /tmp #
|
so, it seems that both partitions (sda2 and sda7) contain fully accessible filesystem
and, then again, as I showed in previous post, I cannot install grub on hd7 (sdh) even if I use "root hd(0,1)", and installation to hd0 (sda) is possible with the same root. 
_________________
gentoo user |
|
| Back to top |
|
|
gentoo_ram
Guru
Joined: 25 Oct 2007
Posts: 513
Location: San Diego, California USA
|
| Posted: Fri Feb 25, 2011 6:56 pm Post subject: |
|
|
That's a lot of hard drives. The only thing I can think is that there's some funkiness going on with the drive mapping between what Linux sees and what GRUB thinks the BIOS will see. Of course, the 'find' command output doesn't back up my theory, but it's the only thing I can think.
I have my /boot on a RAID 1 as well. I did the same thing you did...
root (hd0,0)
setup (hd0)
root (hd1,0)
setup (hd1)
Worked fine for me. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54830
Location: 56N 3W
|
| Posted: Fri Feb 25, 2011 7:27 pm Post subject: |
|
|
tnt,
What is in /boot/grub/device.map ?
Does it correspond with your view of the system ?
| Code: | grub> root (hd0,1)
Filesystem type is ext2fs, partition type 0x83
grub> setup (hd7) |
would not produce the desired result, even if it worked. The grub stage1 and stage 1.5, if its used are changed during the install so that the parts can find one another.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
tnt
Veteran
Joined: 27 Feb 2004
Posts: 1227
|
| Posted: Tue Mar 01, 2011 11:48 pm Post subject: |
|
|
device.map looks like:
| Code: | (fd0) /dev/fd0
(hd0) /dev/sda
(hd1) /dev/sdb
(hd2) /dev/sdc
(hd3) /dev/sdd
(hd4) /dev/sde
(hd5) /dev/sdf
(hd6) /dev/sdg
(hd7) /dev/sdh |
I understand that installing grub on sdh (hd7) MBR and pointing it to the partition on some other hard disk would not produce the desired result - I just tried that to see if there's something wrong with the partition on sdh (hd7,1).
this way, it seems the problem is not partition-related. probably it's MBR-related...
_________________
gentoo user |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54830
Location: 56N 3W
|
| Posted: Wed Mar 02, 2011 6:39 pm Post subject: |
|
|
tnt,
What is attached to?
Is it a bootable controller?
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
tnt
Veteran
Joined: 27 Feb 2004
Posts: 1227
|
| Posted: Wed Mar 02, 2011 9:41 pm Post subject: |
|
|
it should be.
it is secondary jmicron controler and that hard drive is listed in boot order in BIOS.
it is on the second place in boot order (if frist one which is sda fails).
_________________
gentoo user |
|
| Back to top |
|
|
jormartr
Apprentice
Joined: 02 Jan 2008
Posts: 174
|
|
| Back to top |
|
|
tnt
Veteran
Joined: 27 Feb 2004
Posts: 1227
|
| Posted: Wed Mar 02, 2011 11:32 pm Post subject: |
|
|
thx a lot, but:
| Quote: | grub> device (hd0) /dev/sda
grub> root (hd0,1)
Filesystem type is ext2fs, partition type 0x83
grub> setup (hd0)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/e2fs_stage1_5" exists... yes
Running "embed /boot/grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
Running "embed /boot/grub/e2fs_stage1_5 (hd0,1)"... failed (this is not fatal)
Running "install /boot/grub/stage1 (hd0) /boot/grub/stage2 p /boot/grub/menu.lst "... succeeded
Done.
grub> device (hd0) /dev/sdh
grub> root (hd0,1)
Filesystem type is ext2fs, partition type 0x83
grub> setup (hd0)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/e2fs_stage1_5" exists... yes
Running "embed /boot/grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
Running "embed /boot/grub/e2fs_stage1_5 (hd0,1)"... failed (this is not fatal)
Running "install /boot/grub/stage1 (hd0) /boot/grub/stage2 p /boot/grub/menu.lst "... failed
Error 16: Inconsistent filesystem structure
grub>
|
_________________
gentoo user |
|
| Back to top |
|
|
tnt
Veteran
Joined: 27 Feb 2004
Posts: 1227
|
| Posted: Wed Mar 02, 2011 11:36 pm Post subject: |
|
|
ok, this seems to do the trick:
1. stop /dev/md0 (RAID1 boot partition made from /dev/sda2 and /dev/sdh2)
| Code: | mdadm -S /dev/md0
mdadm: stopped /dev/md0 |
2. mount (read-only) /dev/sdh2 as /boot
| Code: | | mount -o ro -t ext2 /dev/sdh2 /boot |
3. setup grub:
| Code: | grub> find /boot/grub/stage1
(hd0,1)
(hd7,1)
grub> root (hd7,1)
Filesystem type is ext2fs, partition type 0x83
grub> setup (hd7)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/e2fs_stage1_5" exists... yes
Running "embed /boot/grub/e2fs_stage1_5 (hd7)"... failed (this is not fatal)
Running "embed /boot/grub/e2fs_stage1_5 (hd7,1)"... failed (this is not fatal)
Running "install /boot/grub/stage1 (hd7) /boot/grub/stage2 p /boot/grub/menu.lst "... succeeded
Done.
grub> |
I realy don't understand why everything else failed and this succeded, but that's the way it is...
_________________
gentoo user |
|
| Back to top |
|
|
jormartr
Apprentice
Joined: 02 Jan 2008
Posts: 174
|
| Posted: Thu Mar 03, 2011 11:42 am Post subject: |
|
|
Just for checking that there is no inconsistency between the two members on the raid, execute
| Code: |
echo check > /sys/block/md0/md/sync_action
|
then, check /proc/mdstat, and when you see that the check has finished, check the values with
| Code: |
cat /sys/block/md0/md/mismatch_cnt
|
that will show you, the number of sectors on the raid1, that differ on members. That obviously should not happen, but sometimes it happens. If the value is 0, it's ok. |
|
| Back to top |
|
|
tnt
Veteran
Joined: 27 Feb 2004
Posts: 1227
|
| Posted: Thu Mar 03, 2011 12:15 pm Post subject: |
|
|
thx a lot
| Code: | echo check > /sys/block/md0/md/sync_action
cat /sys/block/md0/md/mismatch_cnt
0
|
and, just in case, what to do (how to sync) array if there's non-zero value ?
_________________
gentoo user |
|
| Back to top |
|
|
AngelKnight
Tux's lil' helper
Joined: 14 Jan 2003
Posts: 127
|
| Posted: Thu Mar 03, 2011 4:48 pm Post subject: Grub disks != linux disks |
|
|
There's a note in the grub notes somewhere (man page?) to the effect that when you supply (hdX) disk specifiers in grub, you have to be mindful of what the *BIOS* will see, *NOT* what grub-running-on-linux will see.
Disclaimer: This set of steps works out for me and is based on sound principles, but I haven't seen your precise computer. (This *should* work, but I don't guarantee it won't break your system and require a livecd, misfile your taxes, set your garbage on fire or impregnate your puppy...)
For your setup, what I might recommend is:
1) run grub like this:
| Code: | # grub --device-map=/dev/null
(grub) device (hd0) /dev/sda
(grub) root (hd0,1)
(grub) setup (hd0)
(grub) quit (quit or exit? I never remember...) |
2) then, run grub like this:
| Code: | # grub --device-map=/dev/null
(grub) device (hd0) /dev/sdh
(grub) root (hd0,1)
(grub) setup (hd0)
(grub) quit |
The reason is that whichever disk the BIOS picks, it's likely to be BIOS disk 0x80, which is what grub calls "hd0" under whichever circumstances grub actually loads at the time. |
|
| Back to top |
|
|
|
Kernel & Hardware
