| View previous topic :: View next topic |
| Author |
Message |
alexandero
n00b
Joined: 14 Jan 2004
Posts: 19
Location: Vienna, Austria
|
 Posted: Tue Sep 07, 2004 2:30 pm Post subject: Input/Output error using RAID5 |
 |
|
The short story: I accidentially formatted one of my 3 RAID5 devices / is mounted on (running reiser) with ext2. Now I'm having Input/Output errors when trying to access some files. Immediately marking the disk as failed and resyncing onto a spare disk didn't solve the problem.
The long story: I ran into some problems when trying to convert /boot to RAID1, so I decided to backup /boot, reformat all boot partitions on the 4 disks I'm using and revert to basic ext2. Unfortunately instead of typing "mke2fs /dev/sda1", "mke2fs /dev/sdb1",... I typed "mke2fs /dev/sda2", which is one of my RAID5 disks (/dev/md0). Don't tell me - the moment it happened one could hear me screaming in a pretty large radius.
When trying to access certain directories (e.g. /var/virusmails, owned by amavis), the following appears in /var/log/syslog:
| Code: |
Sep 7 16:09:15 mail kernel: ReiserFS: warning: is_tree_node: node level 0 does not match to the expected one 1
Sep 7 16:09:15 mail kernel: ReiserFS: md0: warning: vs-5150: search_by_key: invalid format found in block 787153. Fsck?
|
and also
| Code: |
Sep 7 14:07:44 mail kernel: ReiserFS: md0: warning: zam-7001: io error in reiserfs_find_entry
|
Also even having marked /dev/sda2 as failed, removed it from raid, reformatted it with reiser, added it back as spare and marked sdd2 as failed (to have sda-sdc using RAID and sdd as spare), I'm having continuing problems.
It seems as if /dev/md0 is basically now a mixed reiser *and* ext2 system possible??), and forcing a fsck at reboot (shutdown -r -F) doesnt help. I get an error message about fsck.ext2 not found (I have installed the file tools though), and fsck.reiser thinks everything is ok as the disk was properly unmounted.
I then tried to boot from a LiveCD to at least have a shell from which I can try some fsck - but unfortunately I have no access to /dev/md0 when booting from the CD (no SCSI RAID support?).
/proc/mdstat and mdadm --detail both don't show any errors:
| Code: |
[root@mail pts/0] mdadm --detail /dev/md0
/dev/md0:
Version : 00.90.01
Creation Time : Sun Sep 5 17:52:13 2004
Raid Level : raid5
Array Size : 17717120 (16.90 GiB 18.14 GB)
Device Size : 8858560 (8.45 GiB 9.07 GB)
Raid Devices : 3
Total Devices : 4
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Tue Sep 7 16:28:08 2004
State : clean
Active Devices : 3
Working Devices : 4
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 32K
Number Major Minor RaidDevice State
0 8 18 0 active sync /dev/sdb2
1 8 34 1 active sync /dev/sdc2
2 8 2 2 active sync /dev/sda2
3 8 50 -1 spare /dev/sdd2
UUID : 15ee71d0:ccdf01b7:70140e44:51b34960
Events : 0.98200
|
So basically what I'm trying to find out is
a) is there a RAID5 tool to check data integrity?
b) is it possible that a RAID system is using ext2 *and* reiser, or is it possible I already had these file errors before moving my root to raid?
c) how can I rewrite my reiser-superblocks / how do I mount my root as read only?
d) what can I do that things don't get worse, and how do I fix the problems I already have?
Thanks for any hints and help in advance.
EDIT: some additional information: I'm using dev-sources 2.6.8.1, and:
| Code: |
[root@mail pts/0] fdisk -l /dev/sda
Disk /dev/sda: 9105 MB, 9105018880 bytes
64 heads, 32 sectors/track, 8683 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 32 32752 83 Linux
/dev/sda2 33 8683 8858624 fd Linux raid autodetect
[root@mail pts/0] fdisk -l /dev/sdb
Disk /dev/sdb: 9105 MB, 9105018880 bytes
64 heads, 32 sectors/track, 8683 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 * 1 32 32752 83 Linux
/dev/sdb2 33 8683 8858624 fd Linux raid autodetect
[root@mail pts/0] fdisk -l /dev/sdc
Disk /dev/sdc: 9127 MB, 9127781376 bytes
64 heads, 32 sectors/track, 8704 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/sdc1 * 1 32 32752 83 Linux
/dev/sdc2 33 8683 8858624 fd Linux raid autodetect
[root@mail pts/0] fdisk -l /dev/sdd
Disk /dev/sdd: 9105 MB, 9105018880 bytes
64 heads, 32 sectors/track, 8683 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/sdd1 * 1 32 32752 83 Linux
/dev/sdd2 33 8683 8858624 fd Linux raid autodetect
|
| Code: |
[root@mail pts/0] cat /etc/raidtab
raiddev /dev/md0
raid-level 5
nr-raid-disks 3
nr-spare-disks 1
persistent-superblock 1
parity-algorithm left-symmetric
chunk-size 32
device /dev/sdb2
raid-disk 0
device /dev/sdc2
raid-disk 1
device /dev/sda2
raid-disk 2
device /dev/sdd2
spare-disk 0
|
| Code: |
[root@mail pts/1] cat /etc/fstab
/dev/sda1 /boot ext2 noauto,noatime 1 1
/dev/md0 / reiserfs defaults 1 1
/swapfile swap swap deafuls 0 0
/dev/cdroms/cdrom0 /mnt/cdrom iso9660 noauto,ro 0 0
none /proc proc defaults 0 0
none /dev/shm tmpfs defaults 0 0
none /sys sysfs defaults 0 0
|
|
|
| Back to top |
|
 |
Idler921
n00b
Joined: 28 Aug 2004
Posts: 51
Location: Hong Kong
|
| Posted: Tue Sep 07, 2004 5:09 pm Post subject: |
|
|
a) is there a RAID5 tool to check data integrity?
No! use fsck.reiserfs instead.
b) is it possible that a RAID system is using ext2 *and* reiser, or is it possible I already had these file errors before moving my root to raid?
your should remove the ext2 formatted device from raid immediately using this command:
mdadm --remove /dev/md0 /dev/sda2
c) how can I rewrite my reiser-superblocks / how do I mount my root as read only?
fsck.reiserfs --fix-fixable
d) what can I do that things don't get worse, and how do I fix the problems I already have?
pray to god
If gentoo live cd doesn't help, use knoppix may be a good idea. |
|
| Back to top |
|
|
alexandero
n00b
Joined: 14 Jan 2004
Posts: 19
Location: Vienna, Austria
|
| Posted: Tue Sep 07, 2004 5:28 pm Post subject: |
|
|
Thanks for the fast reply!
| Quote: |
No! use fsck.reiserfs instead.
|
Unfortunately I have no idea how to mount the root partition as read only to be able to use fsck on it. I didnt try knoppix yet, but when booting from the LiveCD, I have no access to my raid.
| Quote: | your should remove the ext2 formatted device from raid immediately using this command:
mdadm --remove /dev/md0 /dev/sda2
|
did that as stated above, reformatted and added it back. Didn't help though. Meanwhile more and more files became not accessible, and finally after a reboot the system crashed while booting. Now I deleted my raid and am restoring my backup from 2 days ago.
I'd still appreciate to know how to run fsck on root in case I encounter such problems again (not that I plan for format a drive again, but one never knows ;) |
|
| Back to top |
|
|
Idler921
n00b
Joined: 28 Aug 2004
Posts: 51
Location: Hong Kong
|
| Posted: Tue Sep 07, 2004 5:39 pm Post subject: |
|
|
to do fsck, it is no need to mount it as read only.
if gentoo live-cd has mdadm, try this:
mdadm --assemble /dev/md0 /dev/sd[abcd]2 |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Kernel & Hardware
