Replacing a failed disk in a software mirror - Revision history

Ppapa: /* Solaris */

2018-09-19T04:24:21Z

‎Solaris

Ppapa: /* Solaris */

2018-09-19T04:23:44Z

‎Solaris

Ppapa: /* CentOS/RedHat */

2011-01-13T05:17:31Z

‎CentOS/RedHat

Ppapa: /* CentOS/RedHat */

2011-01-13T05:14:41Z

‎CentOS/RedHat

Ppapa: /* CentOS/RedHat */

2011-01-13T05:03:39Z

‎CentOS/RedHat

Ppapa: /* CentOS/RedHat Linux */

2011-01-13T03:19:05Z

‎CentOS/RedHat Linux

Ppapa at 03:15, 13 January 2011

2011-01-13T03:15:53Z

Ppapa at 02:36, 26 November 2010

2010-11-26T02:36:20Z

Ppapa at 00:55, 15 November 2010

2010-11-15T00:55:08Z

Ppapa at 00:54, 15 November 2010

2010-11-15T00:54:40Z

@@ Line 76: / Line 76: @@
 This means that the slice in question has been set as the dump device, instead of the mirror slice.  You can confirm it with:
-   # dumpadm
+   # '''dumpadm'''
          Dump content: kernel pages
           Dump device: /dev/dsk/c0t1d0s1 (dedicated)
@@ Line 85: / Line 85: @@
 You fix it with:
-   # dumpadm -d  /dev/md/dsk/d10
+   # '''dumpadm -d  /dev/md/dsk/d10'''
          Dump content: kernel pages
           Dump device: /dev/md/dsk/d10 (swap)

@@ Line 65: / Line 65: @@
    usb0/1                         unknown      empty        unconfigured ok
    usb0/2                         unknown      empty        unconfigured ok
 . Physically replace the failed drive
@@ Line 129: / Line 157: @@
 then it is using the wrong version of 'dd'.  Change  your PATH to put /usr/bin/dd first in your PATH.
 == CentOS/RedHat ==

← Older revision		Revision as of 05:17, 13 January 2011
Line 300:		Line 300:
	Done.		Done.
	grub> '''quit'''		grub> '''quit'''
		+
		+
		+	Now you should be mirrored and good to go :-)

@@ Line 249: / Line 249: @@
    mdadm --add /dev/md4 /dev/sda2
    mdadm --add /dev/md5 /dev/sda5
 . Now install grub on the master boot record (MBR) of the new disk

← Older revision		Revision as of 05:03, 13 January 2011
Line 132:		Line 132:

	== CentOS/RedHat ==		== CentOS/RedHat ==
		+
		+	So you have a failed disk in your software raid. When you look at /proc/mdstat it looks something like:
		+
		+	# cat /proc/mdstat
		+	Personalities : [raid1]
		+	md1 : active raid1 sdb1[1] sda1[2](F)
		+	104320 blocks [2/1] [_U]
		+
		+	md4 : active raid1 sdb2[1] sda2[2](F)
		+	6289344 blocks [2/1] [_U]
		+
		+	md5 : active raid1 sdb5[1] sda5[2](F)
		+	4192832 blocks [2/1] [_U]
		+
		+	md2 : active raid1 sdb6[1] sda6[0]
		+	2096384 blocks [2/2] [UU]
		+
		+	md3 : active raid1 sdb7[1] sda7[2](F)
		+	16570944 blocks [2/1] [_U]
		+
		+	md0 : active raid1 sdb3[1] sda3[2](F)
		+	6289344 blocks [2/1] [_U]
		+
		+	unused devices: <none>
		+
		+	This indicates that /dev/sda has failed.
		+
		+	To replace it, do the following:
		+
		+	1. Use mdstat to fail and remove each of the slices associated with the failed disk
		+
		+	mdadm --manage /dev/md1 --fail /dev/sda1
		+	mdadm --manage /dev/md1 --remove /dev/sda1
		+
		+	/proc/mdstat should look something like this:
		+
		+	# cat /proc/mdstat
		+	Personalities : [raid1]
		+	md1 : active raid1 sdb1[1]
		+	104320 blocks [2/1] [_U]
		+
		+	md4 : active raid1 sdb2[1] sda2[2](F)
		+	6289344 blocks [2/1] [_U]
		+
		+	md5 : active raid1 sdb5[1] sda5[2](F)
		+	4192832 blocks [2/1] [_U]
		+
		+	md2 : active raid1 sdb6[1] sda6[0]
		+	2096384 blocks [2/2] [UU]
		+
		+	md3 : active raid1 sdb7[1] sda7[2](F)
		+	16570944 blocks [2/1] [_U]
		+
		+	md0 : active raid1 sdb3[1] sda3[2](F)
		+	6289344 blocks [2/1] [_U]
		+
		+	unused devices: <none>
		+
		+	Repeat for each slice of the failed disk.
		+
		+	/proc/mdstat should now look like this:
		+
		+	# cat /proc/mdstat
		+	Personalities : [raid1]
		+	md1 : active raid1 sdb1[1]
		+	104320 blocks [2/1] [_U]
		+
		+	md4 : active raid1 sdb2[1]
		+	6289344 blocks [2/1] [_U]
		+
		+	md5 : active raid1 sdb5[1]
		+	4192832 blocks [2/1] [_U]
		+
		+	md2 : active raid1 sdb6[1]
		+	2096384 blocks [2/1] [_U]
		+
		+	md3 : active raid1 sdb7[1]
		+	16570944 blocks [2/1] [_U]
		+
		+	md0 : active raid1 sdb3[1]
		+	6289344 blocks [2/1] [_U]
		+
		+	unused devices: <none>
		+
		+	2. Shutdown the server and replace the failed disk
		+
		+	shutdown -h now
		+
		+	3. Startup the server and boot off the good disk.
		+
		+	4. Copy the partition table from the good disk to the replacement disk
		+
		+	sfdisk -d /dev/sdb \| sfdisk /dev/sda
		+
		+	5. To make sure that there are no remains from previous RAID installations on /dev/sda, we run the following commands:
		+
		+	mdadm --zero-superblock /dev/sda1
		+	mdadm --zero-superblock /dev/sda2
		+	mdadm --zero-superblock /dev/sda3
		+	mdadm --zero-superblock /dev/sda5
		+	mdadm --zero-superblock /dev/sda6
		+	mdadm --zero-superblock /dev/sda7
		+
		+	If there was nothing to zero, you'll get this response:
		+
		+	mdadm: Unrecognised md component device - /dev/sda1
		+
		+	All good!
		+
		+	6. Add the new disks partitions to the metadevices
		+
		+	mdadm --add /dev/md0 /dev/sda3
		+	mdadm --add /dev/md1 /dev/sda1
		+	mdadm --add /dev/md2 /dev/sda6
		+	mdadm --add /dev/md3 /dev/sda7
		+	mdadm --add /dev/md4 /dev/sda2
		+	mdadm --add /dev/md5 /dev/sda5
		+
		+	7. Now install grub on the master boot record (MBR) of the new disk
		+
		+	grub
		+
		+	Probing devices to guess BIOS drives. This may take a long time.
		+
		+
		+	GNU GRUB version 0.97 (640K lower / 3072K upper memory)
		+
		+	[ Minimal BASH-like line editing is supported. For the first word, TAB
		+	lists possible command completions. Anywhere else TAB lists the possible
		+	completions of a device/filename.]
		+	grub> '''root (hd0,0)'''
		+	root (hd0,0)
		+	Filesystem type is ext2fs, partition type 0x83
		+	grub> '''setup (hd0)'''
		+	setup (hd0)
		+	Checking if "/boot/grub/stage1" exists... no
		+	Checking if "/grub/stage1" exists... yes
		+	Checking if "/grub/stage2" exists... yes
		+	Checking if "/grub/e2fs_stage1_5" exists... yes
		+	Running "embed /grub/e2fs_stage1_5 (hd0)"... 15 sectors are embedded.
		+	succeeded
		+	Running "install /grub/stage1 (hd0) (hd0)1+15 p (hd0,0)/grub/stage2 /grub/grub.conf"... succeeded
		+	Done.
		+	grub> '''quit'''

← Older revision		Revision as of 03:19, 13 January 2011
Line 131:		Line 131:


−	== CentOS/RedHat ~~Linux~~ ==	+	== CentOS/RedHat ==

← Older revision		Revision as of 03:15, 13 January 2011
Line 1:		Line 1:
		+
		+	== Solaris ==
		+
		+
	So here's the scenario, you have a server with disk mirrored with Solaris Disksuite, c0t0d0 and c0t1d0. The disk c0t1d0 has failed and you want to replace it without shutting down the box. We have the following metadevices and sub-mirrors:		So here's the scenario, you have a server with disk mirrored with Solaris Disksuite, c0t0d0 and c0t1d0. The disk c0t1d0 has failed and you want to replace it without shutting down the box. We have the following metadevices and sub-mirrors:

Line 125:		Line 129:

	then it is using the wrong version of 'dd'. Change your PATH to put /usr/bin/dd first in your PATH.		then it is using the wrong version of 'dd'. Change your PATH to put /usr/bin/dd first in your PATH.
		+
		+
		+	== CentOS/RedHat Linux ==

@@ Line 8: / Line 8: @@
 * d50: d51 and  d52
-This procedure had been tested and works on a SunFire V125.  You may have to alter this slightly depending on the hardware you are running.  Consult the User Guide for you server on how best to replace a drive on a running system.
+This procedure had been tested and works on a SunFire V125 running Solaris 10.  You may have to alter this slightly depending on the hardware you are running.  Consult the User Guide for you server on how best to replace a drive on a running system.
 . Delete the meta databases stored on the failed disk, stored in this case on slice 7 of the disk

← Older revision		Revision as of 00:55, 15 November 2010
Line 124:		Line 124:
	dd: unrecognized operand `oseek'=`1' Try `dd --help' for more information.		dd: unrecognized operand `oseek'=`1' Try `dd --help' for more information.

−	then it is ~~suing~~ the wrong version of 'dd'. Change your PATH to put /usr/bin/dd first in your PATH.	+	then it is using the wrong version of 'dd'. Change your PATH to put /usr/bin/dd first in your PATH.

← Older revision		Revision as of 00:54, 15 November 2010
Line 119:		Line 119:

	installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c0t1d0s0		installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c0t1d0s0
		+
		+	'''NOTE:''' If you get this error with the installboot command
		+
		+	dd: unrecognized operand `oseek'=`1' Try `dd --help' for more information.
		+
		+	then it is suing the wrong version of 'dd'. Change your PATH to put /usr/bin/dd first in your PATH.