View previous topic :: View next topic |
Author |
Message |
gladbach n00b
Joined: 15 Jun 2002 Posts: 50
|
Posted: Mon Dec 01, 2008 9:03 pm Post subject: mdadm unmark drive as spare? |
|
|
I have a 4 drive raid 5 array using mdadm. sda1 sdb1 sdc1 sdd1
sdb1 is actually a failed drive, and needs to be replaced. However, somewhere along the way, sdc1 was marked as spare, and I can not seem to get it to go back to active. sdc1 seems completely ok, other than it is marked as spare, and md0 won't run because there are 2 active, 1 spare drives.
is there a way to correct this, or am I screwed?
Code: | mdadm --examine /dev/sd[a-d]1
/dev/sda1:
Magic : a92b4efc
Version : 00.90.00
UUID : b597dc3c:20d08964:3f9e0da9:f22ba86d
Creation Time : Thu Apr 24 11:33:36 2008
Raid Level : raid5
Used Dev Size : 244195840 (232.88 GiB 250.06 GB)
Array Size : 732587520 (698.65 GiB 750.17 GB)
Raid Devices : 4
Total Devices : 3
Preferred Minor : 0
Update Time : Mon Dec 1 13:50:29 2008
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 1
Spare Devices : 0
Checksum : 5132d69a - correct
Events : 0.5156
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 0 8 1 0 active sync /dev/sda1
0 0 8 1 0 active sync /dev/sda1
1 1 0 0 1 active sync
2 2 0 0 2 faulty removed
3 3 8 49 3 active sync /dev/sdd1
/dev/sdb1:
Magic : a92b4efc
Version : 00.90.00
UUID : b597dc3c:20d08964:3f9e0da9:f22ba86d
Creation Time : Thu Apr 24 11:33:36 2008
Raid Level : raid5
Used Dev Size : 244195840 (232.88 GiB 250.06 GB)
Array Size : 732587520 (698.65 GiB 750.17 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 0
Update Time : Fri Nov 7 21:49:10 2008
State : active
Active Devices : 3
Working Devices : 4
Failed Devices : 0
Spare Devices : 1
Checksum : 51138ee4 - correct
Events : 0.5156
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 1 8 17 1 active sync /dev/sdb1
0 0 8 1 0 active sync /dev/sda1
1 1 8 17 1 active sync /dev/sdb1
2 2 0 0 2 faulty removed
3 3 8 49 3 active sync /dev/sdd1
4 4 8 33 4 spare /dev/sdc1
/dev/sdc1:
Magic : a92b4efc
Version : 00.90.00
UUID : b597dc3c:20d08964:3f9e0da9:f22ba86d
Creation Time : Thu Apr 24 11:33:36 2008
Raid Level : raid5
Used Dev Size : 244195840 (232.88 GiB 250.06 GB)
Array Size : 732587520 (698.65 GiB 750.17 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 0
Update Time : Mon Dec 1 13:48:30 2008
State : clean
Active Devices : 2
Working Devices : 3
Failed Devices : 1
Spare Devices : 1
Checksum : 5132d67a - correct
Events : 0.5154
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 4 8 33 4 spare /dev/sdc1
0 0 8 1 0 active sync /dev/sda1
1 1 0 0 1 faulty removed
2 2 0 0 2 faulty removed
3 3 8 49 3 active sync /dev/sdd1
4 4 8 33 4 spare /dev/sdc1
/dev/sdd1:
Magic : a92b4efc
Version : 00.90.00
UUID : b597dc3c:20d08964:3f9e0da9:f22ba86d
Creation Time : Thu Apr 24 11:33:36 2008
Raid Level : raid5
Used Dev Size : 244195840 (232.88 GiB 250.06 GB)
Array Size : 732587520 (698.65 GiB 750.17 GB)
Raid Devices : 4
Total Devices : 3
Preferred Minor : 0
Update Time : Mon Dec 1 13:50:29 2008
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 1
Spare Devices : 0
Checksum : 5132d6d3 - correct
Events : 0.5156
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 3 8 49 3 active sync /dev/sdd1
0 0 8 1 0 active sync /dev/sda1
1 1 0 0 1 faulty removed
2 2 0 0 2 faulty removed
3 3 8 49 3 active sync /dev/sdd1
|
|
|
Back to top |
|
|
gladbach n00b
Joined: 15 Jun 2002 Posts: 50
|
Posted: Thu Dec 04, 2008 1:52 pm Post subject: |
|
|
no ideas? |
|
Back to top |
|
|
gladbach n00b
Joined: 15 Jun 2002 Posts: 50
|
Posted: Sat Dec 13, 2008 8:25 pm Post subject: |
|
|
I resolved the issue on my own. Once I had a replacement for the physical bad drive, I was able to force the rebuild (previously would error out) using the command
Code: | mdadm --create /dev/md0 --assume-clean --level=5 --chunk=128 --raid-devices=4 /dev/sda1 missing /dev/sdc1 /dev/sdd1 |
and then re-add sdb1 for a full rebuild. fsck found a lot of errors upon reboot, however, after running through it, I have yet to find any actual dataloss. |
|
Back to top |
|
|
fryfrog n00b
Joined: 09 May 2002 Posts: 45
|
Posted: Sun Oct 10, 2010 10:49 am Post subject: |
|
|
I just wanted to say thanks for posting this.
I just used this to save my friends 5x1.5T raid5 array.
During a motherboard issue and upgrade, his array failed out a disk and then soon after another disk. At some point during his troubleshooting, the disk that was still "good" was re-added but as a spare. So the array had 3/5 disks and a spare. The 5th disk looked legitimately toasted, the raid superblock was missing and the partition information was odd. It was *probably* fixable, but I doubt it would have been easy.
Using the 3 good disks, I figured out their original positions from 'mdadm -E /dev/sdX' and ended up with something like 'mdadm --create /dev/md0 --assume-clean --level=5 --chunk=64 --raid-devices=5 missing /dev/sdb1 missing /dev/sdd1 /dev/sdf1' which obviously wouldn't work. I randomly put the disk that was "good" (but believed itself to be a spare) in the first "missing" spot, used 'mdadm -o /dev/md0' to mark it read only, but attempting to mount it did not work. I then re-did the command with the "spare" disk in the second missing spot, 'mdadm -o /dev/md0' to make it read only again and mount believed there was an xfs filesystem there (but wouldn't mount it due to a read only block device)!
Amazing!
The superblock of the xfs file system was missing or damaged, so it wouldn't mount to play back the log. A run of 'xfs_repair -L /dev/md0' nuked the log but recovered the file system. The only victim was a single jpg ending up in lost+found.
I hope this helps someone else some day too.
I think the most important things to note are the raid level, chunk and number of devices. Hopefully you know most or all of this, but 'mdadm -E /dev/sd??' should be able to help you figure things out. It will also tell you the position the disk was in the last time it was in the array and *confusingly* also the device it was. During my first attempts, one of the disks said it was in a certain position and called /dev/sde1, so I used that device in that position... but of course they had been re-arranged. I was glad when I realized this.
Note: I used obviously sequential /dev/sdX1 devices in the example, but in real life they were all mixed up because really, who cares where drives are plugged in ;) |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|