Broken RAID-1, can't find answers [SOLVED]

[Groll]

I set up one of my home PCs with Gentoo 2004.1 a couple of months ago (and loving it after running Fedora at work), and about six weeks ago added two 40GB IDE drives in RAID-1, using mdadm and the Software RAID HOW-TO. It was configured with persistent superblocks and autostarted fine on system boot.

This had been running well for around four-five weeks, then suddenly threw a fit. I hadn't used the PC for around 3 days, but when I restarted it, I suddenly found that md0 hadn't started. The PC had been shut down properly after the last use.

The two discs are on a separate controller card with a CMD649 chip, drivers compiled into the kernel (2.6.7). The two drives, hde & hdg, have single partitions on, which occupy the whole disc. MD is also compiled into the kernel, with 0 & 1 personalities included (I intend adding a RAID-0 later for doing some video work).

To cut a long story short, the log shows the following info on the md startup:

Sep 15 11:32:18 kea md: Autodetecting RAID arrays.
Sep 15 11:32:18 kea md: invalid raid superblock magic on hde1
Sep 15 11:32:18 kea md: hde1 has invalid sb, not importing!
Sep 15 11:32:18 kea md: could not import hde1!
Sep 15 11:32:18 kea md: invalid raid superblock magic on hdg1
Sep 15 11:32:18 kea md: hdg1 has invalid sb, not importing!
Sep 15 11:32:18 kea md: could not import hdg1!
Sep 15 11:32:18 kea md: autorun ...
Sep 15 11:32:18 kea md: ... autorun DONE.

then later ...

Sep 15 12:15:13 kea md: bug in file drivers/md/md.c, line 1513
Sep 15 12:15:13 kea
Sep 15 12:15:13 kea md: **********************************
Sep 15 12:15:13 kea md: * <COMPLETE RAID STATE PRINTOUT> *
Sep 15 12:15:13 kea md: **********************************
Sep 15 12:15:13 kea md0:
Sep 15 12:15:13 kea md: **********************************
Sep 15 12:15:13 kea
Sep 15 12:15:29 kea md: md0 stopped.
Sep 15 12:15:31 kea md: bug in file drivers/md/md.c, line 1513
Sep 15 12:15:31 kea
Sep 15 12:15:31 kea md: **********************************
Sep 15 12:15:31 kea md: * <COMPLETE RAID STATE PRINTOUT> *
Sep 15 12:15:31 kea md: **********************************
Sep 15 12:15:31 kea md0:
Sep 15 12:15:31 kea md: **********************************
Sep 15 12:15:31 kea
Sep 15 12:17:52 kea md: md0 stopped.

[Let me know if more from /var/log/messages would help].

The examine option of mdadm shows what appears to me to be good superblocks:

/dev/hde:
Magic : a92b4efc
Version : 00.90.00
UUID : 40291234:fb1f4ff1:f81698b7:c3424fdc
Creation Time : Thu Aug 5 12:42:55 2004
Raid Level : raid1
Device Size : 39082560 (37.27 GiB 40.02 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0

Update Time : Tue Sep 14 13:31:29 2004
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : 247b69d8 - correct
Events : 0.43906


Number Major Minor RaidDevice State
this 0 33 0 0 active sync /dev/ide/host2/bus0/target0/lun0/disc

0 0 33 0 0 active sync /dev/ide/host2/bus0/target0/lun0/disc
1 1 34 0 1 active sync /dev/ide/host2/bus1/target0/lun0/disc

and:

/dev/hdg:
Magic : a92b4efc
Version : 00.90.00
UUID : 40291234:fb1f4ff1:f81698b7:c3424fdc
Creation Time : Thu Aug 5 12:42:55 2004
Raid Level : raid1
Device Size : 39082560 (37.27 GiB 40.02 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0

Update Time : Tue Sep 14 13:31:29 2004
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : 247b69db - correct
Events : 0.43906


Number Major Minor RaidDevice State
this 1 34 0 1 active sync /dev/ide/host2/bus1/target0/lun0/disc

0 0 33 0 0 active sync /dev/ide/host2/bus0/target0/lun0/disc
1 1 34 0 1 active sync /dev/ide/host2/bus1/target0/lun0/disc

If I tried to do an assemble with mdadm, it barfs with:

mdadm: /dev/hde1 has no superblock - assembly aborted.

That seems to conflict with what --examine shows, though I'm no guru.

So, the question is, is there a way to repair the superblocks to the satisfaction of all? Any other way to recover the raid intact?

There seems to be little information to help in this situation after scouring both this forum and the Internet at large. Maybe I just haven't used the right combination of keywords for my searches.

Alternatively, how can I break the mirror to retrieve the data? I've seen this mentioned in passing, but with no details.

It would be good to recover the raid intact. I have a backup of the important stuff, but there is a lot of other stuff it would take significant time to recover.

Is the apparent "md: bug in file drivers/md/md.c, line 1513" reported in the syslog real, or a red herring?

All suggestions & help very gratefully received!

Cheers,
Steve.

[j@ckrabbit]

I recall reading in the Orielly "Managing Raid on Linux" a mdadm command that wipes only the superblock clean. Then it mentioned that without the superblock you could still start the array with an init script. I don't have the book with me though. I could check it out this evening (or you might consider hitting a bookstore, its a very informative book).

[hungarian]

mdadm --zero-superblock /dev/hda1

Tell me, have you made sure that your partition typed are type "fd" ?
_________________
Sophisticated, cultivated, technically superior,
Gentoo Linux delivers the most rewarding computing experience.
www.gentoo.org
BSc(Hons) CompSci / BSc. CompSci / RHCE

[Groll]

Thanks for the input guys. I've zeroed the superblocks on both drives, but cannot mount them either as individual drives or as an array. The system just complains about invalid superblocks. I've read through the man pages but can't really find any clues as to what to try next.

I can afford to experiment, since I've dd'd the drives onto spare discs that I was about to set up as raid 0 for video work, but haven't been able to work out what to try next.

The disc types were set to fd. It had all been working nicely for a several weeks ...

Cheers,
Steve.

[Groll]

I recreated the array with mdadm, and luckily the parition was still intact. All data recovered.

[mh]

thanks for following up your original question with your solution. it helped.