Managing Software RAID Arrays - Revision history

Digimer: /* Managing Partitions */

2011-04-07T12:43:47Z

Managing Partitions

← Older revision		Revision as of 12:43, 7 April 2011
Line 584:		Line 584:
	</source>		</source>

	The error above is because the disk we altered is used by the root ~~of the operating~~ system. This is why we will need to reboot before we can proceed, but not until all of the disks on the server have been altered.		The error above is because the disk we altered is used by the root file system. This is why we will need to reboot before we can proceed, but not until all of the disks on the server have been altered.

	So go back and repeat these steps for <span class="code">/dev/sdb</span>, <span class="code">/dev/sdc</span> and <span class="code">/dev/sdd</span>.		So go back and repeat these steps for <span class="code">/dev/sdb</span>, <span class="code">/dev/sdc</span> and <span class="code">/dev/sdd</span>.

Digimer: /* Ensuring that /dev/md3 is no longer in use */

2011-04-07T12:42:23Z

Ensuring that /dev/md3 is no longer in use

← Older revision		Revision as of 12:42, 7 April 2011
Line 78:		Line 78:

	We'll look for for local file systems by using <span class="code">df</span> and checking <span class="code">/etc/fstab</span>.		We'll look for for local file systems by using <span class="code">df</span> and checking <span class="code">/etc/fstab</span>.

			'''Note''': If you have ''any'' reason to suspect trouble with your system, you may want to not use <span class="code">df</span> as it can hang when a mount breaks. Instead, run <span class="code">cat /proc/mounts</span>. The output is a little more cryptic, but it should always return you to the terminal.

	<source lang="bash">		<source lang="bash">

Digimer: /* Creating the new RAID Arrays */

2011-04-07T03:39:53Z

Creating the new RAID Arrays

← Older revision		Revision as of 03:39, 7 April 2011
Line 597:		Line 597:

	<source lang="bash">		<source lang="bash">
	mdadm --create /dev/md3 --homehost=localhost.localdomain --raid-devices=4 --level=5 /dev/sd[abcd]5		mdadm --create /dev/md3 --homehost=localhost.localdomain --raid-devices=4 --level=5 --spare-devices=0 /dev/sd[abcd]5
	</source>		</source>
	<source lang="text">		<source lang="text">
Line 631:		Line 631:

	<source lang="bash">		<source lang="bash">
	mdadm --create /dev/md4 --homehost=localhost.localdomain --raid-devices=4 --level=5 /dev/sd[abcd]6		mdadm --create /dev/md4 --homehost=localhost.localdomain --raid-devices=4 --level=5 --spare-devices=0 /dev/sd[abcd]6
	</source>		</source>
	<source lang="text">		<source lang="text">
Line 638:		Line 638:

	<source lang="bash">		<source lang="bash">
	mdadm --create /dev/md5 --homehost=localhost.localdomain --raid-devices=4 --level=5 /dev/sd[abcd]7		mdadm --create /dev/md5 --homehost=localhost.localdomain --raid-devices=4 --level=5 --spare-devices=0 /dev/sd[abcd]7
	</source>		</source>
	<source lang="text">		<source lang="text">
Line 645:		Line 645:

	<source lang="bash">		<source lang="bash">
	mdadm --create /dev/md6 --homehost=localhost.localdomain --raid-devices=4 --level=5 /dev/sd[abcd]8		mdadm --create /dev/md6 --homehost=localhost.localdomain --raid-devices=4 --level=5 --spare-devices=0 /dev/sd[abcd]8
	</source>		</source>
	<source lang="text">		<source lang="text">
Line 685:		Line 685:
	</source>		</source>

	Almost done. We just need to add the new arrays to <span class="code">/etc/mdadm.conf</span> so that they're assembled on boot. Note that you can proceed now. There is no need to wait for the arrays to finish synchronizing.		Almost done. We just need to add the new arrays to <span class="code">/etc/mdadm.conf</span> so that they're assembled on boot. Note that you can proceed now. There is no need to wait for the arrays to finish synchronizing.

			'''Note''': Until the array has finished synchronizing, the unedited output from the next command will include <span class="code">spares=1</span>, even though there are no spares in the array. You can manually delete these arguments or wait for all arrays to finish synchronizing, if you prefer.

			To append the four new arrays to the <span class="code">/etc/mdadm.conf</span> file, you can use the following command.

			''Before'':

			<source lang="bash">
			cat /etc/mdadm.conf
			</source>
			<source lang="text">
			# mdadm.conf written out by anaconda
			DEVICE partitions
			MAILADDR root
			ARRAY /dev/md2 level=raid5 num-devices=4 uuid=7af5fde9:646394dd:d46d09a3:eb495b50
			ARRAY /dev/md0 level=raid1 num-devices=4 uuid=2280ed9e:24f99bf5:4cb4f32c:f3b58eb4
			ARRAY /dev/md1 level=raid5 num-devices=4 uuid=5ae2c898:5837f4a0:a3f0a617:955802c1
			</source>

			''After'':

	~~<span class="code"></span>~~
	<source lang="bash">		<source lang="bash">
			mdadm --detail --scan \| grep -e md3 -e md4 -e md5 -e md6 \| sed -e "s/spares=1 //" >> /etc/mdadm.conf
			cat /etc/mdadm.conf
	</source>		</source>
	<source lang="text">		<source lang="text">
			# mdadm.conf written out by anaconda
			DEVICE partitions
			MAILADDR root
			ARRAY /dev/md2 level=raid5 num-devices=4 uuid=7af5fde9:646394dd:d46d09a3:eb495b50
			ARRAY /dev/md0 level=raid1 num-devices=4 uuid=2280ed9e:24f99bf5:4cb4f32c:f3b58eb4
			ARRAY /dev/md1 level=raid5 num-devices=4 uuid=5ae2c898:5837f4a0:a3f0a617:955802c1
			ARRAY /dev/md3 level=raid5 num-devices=4 metadata=0.90 UUID=a9010440:446a912e:bfe78010:bc810f04
			ARRAY /dev/md4 level=raid5 num-devices=4 metadata=0.90 UUID=7c8aff84:31b8e4c2:bfe78010:bc810f04
			ARRAY /dev/md5 level=raid5 num-devices=4 metadata=0.90 UUID=de81e923:12d84547:bfe78010:bc810f04
			ARRAY /dev/md6 level=raid5 num-devices=4 metadata=0.90 UUID=63c9bfd4:70cb8535:bfe78010:bc810f04
	</source>		</source>

			And, we're done!

	{{footer}}		{{footer}}

Digimer: /* Managing Partitions */

2011-04-07T02:46:11Z

Managing Partitions

← Older revision		Revision as of 02:46, 7 April 2011
Line 587:		Line 587:

	Once done editing all of the disks, reboot before proceeding.		Once done editing all of the disks, reboot before proceeding.

			== Creating the new RAID Arrays ==

			The final steps are:
			* Create the four new arrays using <span class="code">mdadm</span>.
			* Add the new arrays to <span class="code">/etc/mdadm.conf</span>.

			The command to actually create the new arrays is pretty straight forward. Let's look at the first one to create the new <span class="code">/dev/md3</span>.

			<source lang="bash">
			mdadm --create /dev/md3 --homehost=localhost.localdomain --raid-devices=4 --level=5 /dev/sd[abcd]5
			</source>
			<source lang="text">
			mdadm: array /dev/md3 started.
			</source>

			The <span class="code">--homehost=</span> switch is used to tell <span class="code">mdadm</span> to automatically assemble this array should it move to another host. This is because, by default, the actual host name is used to tag the array. Without this, should you move this array to another system, it would not auto assemble as the current host name would not match the new server's host name. However, you may wish to omit this if you prefer the default behaviour.

			One you run the above command, you should be able to <span class="code">cat</span> the <span class="code">/proc/mdstat</span> file and see the new array assembling.

			<source lang="bash">
			cat /proc/mdstat
			</source>
			<source lang="text">
			Personalities : [raid6] [raid5] [raid4] [raid1]
			md3 : active raid5 sdd5[4] sdc5[2] sdb5[1] sda5[0]
			58628928 blocks level 5, 64k chunk, algorithm 2 [4/3] [UUU_]
			[==============>......] recovery = 73.6% (14388164/19542976) finish=1.8min speed=45201K/sec

			md0 : active raid1 sdd1[3] sdc1[2] sdb1[1] sda1[0]
			264960 blocks [4/4] [UUUU]

			md1 : active raid5 sdd3[3] sdc3[2] sdb3[1] sda3[0]
			6289152 blocks level 5, 256k chunk, algorithm 2 [4/4] [UUUU]

			md2 : active raid5 sdd2[3] sdc2[2] sdb2[1] sda2[0]
			62917632 blocks level 5, 256k chunk, algorithm 2 [4/4] [UUUU]

			unused devices: <none>
			</source>

			You don't need to worry about thrashing the disks by creating all of the arrays now. The arrays will wait for the previous one to finish syncing before beginning to sync the next array.

			<source lang="bash">
			mdadm --create /dev/md4 --homehost=localhost.localdomain --raid-devices=4 --level=5 /dev/sd[abcd]6
			</source>
			<source lang="text">
			mdadm: array /dev/md4 started.
			</source>

			<source lang="bash">
			mdadm --create /dev/md5 --homehost=localhost.localdomain --raid-devices=4 --level=5 /dev/sd[abcd]7
			</source>
			<source lang="text">
			mdadm: array /dev/md5 started.
			</source>

			<source lang="bash">
			mdadm --create /dev/md6 --homehost=localhost.localdomain --raid-devices=4 --level=5 /dev/sd[abcd]8
			</source>
			<source lang="text">
			mdadm: array /dev/md6 started.
			</source>

			Now if we look again at <span class="code">/proc/mdstat</span>, we'll see the <span class="code">resync=DELAYED</span> on the new arrays pending synchronization.

			<source lang="bash">
			cat /proc/mdstat
			</source>
			<source lang="text">
			Personalities : [raid6] [raid5] [raid4] [raid1]
			md6 : active raid5 sdd8[4] sdc8[2] sdb8[1] sda8[0]
			750516288 blocks level 5, 64k chunk, algorithm 2 [4/3] [UUU_]
			resync=DELAYED

			md5 : active raid5 sdd7[4] sdc7[2] sdb7[1] sda7[0]
			293001216 blocks level 5, 64k chunk, algorithm 2 [4/3] [UUU_]
			resync=DELAYED

			md4 : active raid5 sdd6[4] sdc6[2] sdb6[1] sda6[0]
			293001216 blocks level 5, 64k chunk, algorithm 2 [4/3] [UUU_]
			[>....................] recovery = 4.0% (3970688/97667072) finish=34.5min speed=45208K/sec

			md3 : active raid5 sdd5[3] sdc5[2] sdb5[1] sda5[0]
			58628928 blocks level 5, 64k chunk, algorithm 2 [4/4] [UUUU]

			md0 : active raid1 sdd1[3] sdc1[2] sdb1[1] sda1[0]
			264960 blocks [4/4] [UUUU]

			md1 : active raid5 sdd3[3] sdc3[2] sdb3[1] sda3[0]
			6289152 blocks level 5, 256k chunk, algorithm 2 [4/4] [UUUU]

			md2 : active raid5 sdd2[3] sdc2[2] sdb2[1] sda2[0]
			62917632 blocks level 5, 256k chunk, algorithm 2 [4/4] [UUUU]

			unused devices: <none>
			</source>

			Almost done. We just need to add the new arrays to <span class="code">/etc/mdadm.conf</span> so that they're assembled on boot. Note that you can proceed now. There is no need to wait for the arrays to finish synchronizing.

Digimer at 20:50, 6 April 2011

2011-04-06T20:50:01Z

Show changes

Digimer: Created page with '{{howto_header}} This quickie covers: * Tearing down an existing software RAID 5 array. * Deleting it's underlying primary partition on each array member. * Creating a new s…'

2011-04-06T15:29:14Z

Created page with '{{howto_header}} This quickie covers: * Tearing down an existing software RAID 5 array. * Deleting it's underlying primary partition on each array member. * Creating a new s…'

Páàjì titun

{{howto_header}}

This quickie covers:
* Tearing down an existing software [[RAID]] 5 array.
* Deleting it's underlying primary partition on each array member.
* Creating a new set of four extended partitions.
* Creating four new RAID 5 arrays out of the new extended partitions.
* Ensuring that /etc/mdtab and /etc/fstab are updated.

= Warnings And Assumptions =

This tutorial covers fundamentally changing the storage on a server. There is a very high, very real chance that all data can be lost.

'''DO NOT FOLLOW THIS TUTORIAL ON A LIVE MACHINE'''! Use this to practice on a test machine only.

It is assumed that you have a fundamental understanding of and comfort with the Linux command line, specifically, the bash terminal.

= Let's Begin =

So then, let's get to work.

== Viewing The Current Configuration ==

We want to look at three things:
* The current RAID devices.
* What is using the RAID device we plan to delete.
* The current partition layout.

=== Current RAID Configuration ===

Checking the current RAID devices involves see what devices are online and what are configured. The first is checked by looking at the /proc/mdstat file, and the later by looking in the /etc/mdstat file.

<source lang="bash">
cat /proc/mdstat
</source>
<source lang="text">
Personalities : [raid6] [raid5] [raid4] [raid1]
md0 : active raid1 sdd1[3] sdc1[2] sdb1[1] sda1[0]
264960 blocks [4/4] [UUUU]

md1 : active raid5 sdd3[3] sdc3[2] sdb3[1] sda3[0]
6289152 blocks level 5, 256k chunk, algorithm 2 [4/4] [UUUU]

md3 : active raid5 sdd4[3] sdc4[2] sdb4[1] sda4[0]
1395148608 blocks level 5, 64k chunk, algorithm 2 [4/4] [UUUU]

md2 : active raid5 sdd2[3] sdc2[2] sdb2[1] sda2[0]
62917632 blocks level 5, 256k chunk, algorithm 2 [4/4] [UUUU]

unused devices: <none>
</source>

<source lang="bash">
cat /etc/mdadm.conf
</source>
<source lang="text">
DEVICE partitions
MAILADDR root
ARRAY /dev/md2 level=raid5 num-devices=4 uuid=7af5fde9:646394dd:d46d09a3:eb495b50
ARRAY /dev/md0 level=raid1 num-devices=4 uuid=2280ed9e:24f99bf5:4cb4f32c:f3b58eb4
ARRAY /dev/md1 level=raid5 num-devices=4 uuid=5ae2c898:5837f4a0:a3f0a617:955802c1
ARRAY /dev/md3 level=raid5 num-devices=4 metadata=0.90 spares=1 UUID=a2636590:fcb1e82a:3f1d7145:41a20e6d
</source>

So we see that four devices are configured and operating. We will tear down /dev/md3.

=== Ensuring that /dev/md3 is no longer in use ===

We need to ensure that /dev/md3 is no longer in use is by checking what is mounted and confirming that no other program uses it.

There are *many* applications that might use the raw storage space;
* A normal file systems.
* [[DRBD]]
* [[LVM]]
* others.

We'll look for for local file systems by using df and checking /etc/fstab.

<source lang="bash">
df -h
</source>
<source lang="text">
Filesystem Size Used Avail Use% Mounted on
/dev/md2 57G 2.5G 51G 5% /
/dev/md0 251M 37M 201M 16% /boot
tmpfs 7.7G 0 7.7G 0% /dev/shm
none 7.7G 40K 7.7G 1% /var/lib/xenstored
</source>

It's not mounted.

<source lang="bash">
cat /etc/fstab
</source>
<source lang="text">
/dev/md2 / ext4 defaults 1 1
/dev/md0 /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
/dev/md1 swap swap defaults 0 0
</source>

There is no corresponding entry.

Now let's see if it's part of a [[DRBD]] resource. This involves checking /proc/drbd, if it exists, and then checking /etc/drbd.conf. If you're not using DRBD, neither of these files will exist and you can move on.

'''Note''': If you have a DRBD resource using /dev/md3, make sure that the resource is not in use by [[LVM]] '''before''' destroying the DRBD resource. If you don't remove LVM first, then you might have trouble later as the LVM signature may persist.

First, look in /etc/drbd.conf and see if any of the configured resources use /dev/md3. If they do, make sure you tear down the matching resource on the other node.

<source lang="bash">
cat /etc/drbd.conf
</source>
<source lang="text">
#
# please have a a look at the example configuration file in
# /usr/share/doc/drbd83/drbd.conf
#

global {
usage-count yes;
}

common {
protocol C;
syncer {
rate 15M;
}

disk {
fencing resource-and-stonith;
}

handlers {
outdate-peer "/sbin/obliterate";
}

net {
allow-two-primaries;
after-sb-0pri discard-zero-changes;
after-sb-1pri discard-secondary;
after-sb-2pri disconnect;
}

startup {
become-primary-on both;
}
}

resource r0 {
device /dev/drbd0;
meta-disk internal;

on an-node01.alteeve.com {
address 192.168.2.71:7789;
disk /dev/md3;
}

on an-node02.alteeve.com {
address 192.168.2.72:7789;
disk /dev/md3;
}
}
</source>

Here we see that /dev/md3 is in fact in use by DRBD and it's block device name is /dev/drbd0. Given this, we'll want to go back and look at df and /etc/fstab again to make sure that /dev/drbd0 wasn't listed. It wasn't, so we're ok to proceed.

We won't actually destroy this resource yet, we'll come back to it once we know that LVM is not using /dev/md3 or /dev/drbd0.

Now let's look to make sure it's not an [[LVM]] physical volume.

<source lang="bash">
</source>
<source lang="text">
</source>


<source lang="bash">
</source>
<source lang="text">
</source>

{{footer}}