How to set up software RAID 1 on a running system ( Incl UEFI boot mechanism )
How to create a RAID1 array with an active Ubuntu 13.04 till 16.10 hard drive.
Prereq
Server without a RAID array on a single hard drive. Server with the UEFI boot mechanism.
Note Create a backup of your hard disk first
Requirements
sudo apt-get install mdadm rsync initramfs-tools
If you havent already installed postfix, set the option ‘no configuration’ for the postfix installation dialog.
First attach your new hard drive (best to have one with the same size or bigger) and then execute the following commands to copy the partitions of /dev/sda to the second drive (in this case /dev/sdb): Copy partition tables
sudo sfdisk -d /dev/sda > partition.txt sudo sfdisk --force /dev/sdb < partition.txt
Note
If you have a GPT partition table you need to install gdisk and copy the partition table with the following commands:
Copy partiton tables using GPT
sudo apt-get install gdisk sudo sgdisk -R=/dev/sdb /dev/sda sudo sgdisk -G /dev/sdb
The first command here copies the partition table from /dev/sda to /dev/sdb (don’t mix it up). And the second one randomizes the GUID of the new partitions on /dev/sdb.
Next check if the partition tables are identical between both hard drives. Execute the following commands and compare the output of fdisk for each hard drive. Also note the specific partition names of your swap (/dev/sdb3 in this case) and the main boot partition (/dev/sdb2 in this case).
Check partition tables
sudo fdisk /dev/sda Command (m for help): p Command (m for help): q sudo fdisk /dev/sdb Command (m for help): p Command (m for help): q
Note If you have a GPT partition table you need to compare the partitions with these commands: Check partition tables
sudo gdisk -l /dev/sda sudo gdisk -l /dev/sdb
Now we need to create the a new degraded RAID array of the newly partitioned hard drive. Execute the following commands and double check the correct partition names for the swap and the main partition. Create degraded RAID array
sudo mdadm --create /dev/md0 --level=1 --raid-devices=2 missing /dev/sdb2 sudo mdadm --create /dev/md1 --level=1 --raid-devices=2 missing /dev/sdb3
Note
If you have a working postfix installation, then you should receive a warning email from mdadm saying that you have a degraded RAID array. So you don’t need to check that the mails arrive when you have a problem with the mdadm RAID hard drives.
After that we need to format the newly created degraded RAID partitions with the following commands: Format the RAID array partitions
sudo mkfs.ext4 /dev/md0 sudo mkswap /dev/md1
Next we need to specify the devices in the mdadm.conf file. Edit the file Edit the mdadm.conf file
sudo vi /etc/mdadm/mdadm.conf
and add the following property to the file: Add property DEVICE to mdadm.conf
DEVICE /dev/sda* /dev/sdb*
The last thing to do before installing GRUB2 on that drive is to add the RAID arrays to the end of the mdadm.conf files. To do this execute the following commands: Show the RAID arrays
sudo mdadm --detail --scan
Now copy the output and paste it at the end of the mdadm.conf file.
After that reconfigure the mdadm package with the following command: Reconfigure mdadm package
sudo dpkg-reconfigure mdadm
In the configure dialog of mdadm answer always yes and add your email address (or the user account of the configured postfix user) for important messages from mdadm. (e.g. disk failures etc.)
After the creation of the new degraded RAID array we need to install GRUB2 on each hard drive. To do this we just reconfigure the grub-pc package with the following command: Reconfigure grub-pc package
sudo dpkg-reconfigure grub-pc You need to select both hard drives as installation targets, /dev/sda and /dev/sdb and not /dev/md0. '''Note'' If the package grub-pc is not installed (because you are using grub-efi) then you need to do the following: Copy the EFI boot partition of sda to sdb by executing the following command: (Be carful and select the correct partitions!) <pre> sudo dd if=/dev/sda1 of=/dev/sdb1
Now we need to copy the existing data to the new hard drive with the following commands:
Copy existing data to new drive
sudo mkdir /tmp/mntroot sudo mount /dev/md0 /tmp/mntroot sudo rsync -auHxv --exclude=/proc/* --exclude=/sys/* --exclude=/tmp/* /* /tmp/mntroot/
After that we need to edit the /tmp/mntroot/etc/fstab as root and change the UUID’s of the data partition and the swap partitions. If you use the UUID notation for your partitions you can get them using the following commands:
Get UUID's of RAID partitions
sudo blkid /dev/md0 /dev/md0: UUID="2f1cff5c-8a14-48df-a926-f69b6e2d6bee" TYPE="ext4" sudo blkid /dev/md1 /dev/md1: UUID="c59ef916-c073-4a27-ae94-bc98b2668a73" TYPE="swap"
If you have the old notation you can use /dev/md0 for the data partition and /dev/md1 for the swap partition.
We also need to unmount the before mounted folder and delete it. Do this by executing the following commands: Unmount and delete mount folder
sudo umount /tmp/mntroot sudo rm -rf /tmp/mntroot
Before the reboot we need to check if the grub menu is shown at boot. Open up the /etc/default/grub file as root and change the following values: Edit Grub config
GRUB_HIDDEN_TIMEOUT=10 GRUB_HIDDEN_TIMEOUT_QUIET=false GRUB_CMDLINE_LINUX_DEFAULT=""
The last value is not needed but it’s good to see whats happening at the start up. After that we need to update GRUB with the following command: Update Grub
sudo update-grub
Now we reboot the machine (press ESC if there is a countdown) and in the Grub menu you can edit the start up properties by pressing e. Add/adapt the following properties: Add/adapt start up properties
insmod mdraid1x set root='(md/0)' linux /boot/vmlinuz-3.8.0-23-generic root=/dev/md0 ro
If you are using the UUID partition format set the last parameter to: root=UUID={UUID-of-md0}
After you have finished editing the properties press F10 to boot.
After the successfull boot, test everything on the new hard drive and have a look at the output of the mount command. This should show the following: Mount output
/dev/md0 on / type ext4 (rw,noatime,errors=remount-ro)
Here you see that Ubuntu is using the new degraded RAID array which is using the /dev/sdb hard drive.
Now we are going to wipe the original hard drive by adding it to the RAID array. Be sure to have a backup of your hard disk somehow. If not do it now. Lets add the /dev/sda drive to the degraded RAID array with the following commands: Add /dev/sda to RAID array
sudo mdadm /dev/md0 -a /dev/sda2 sudo mdadm /dev/md1 -a /dev/sda3
You can watch the progress of the synchronization of the drives with the following command: See the progress of the RAID synchronization <code. sudo watch -n1 cat /proc/mdstat
Wait till the synchronization is finished and then you can quit it with CTRL-C.
The next step is to test if everything worked with the synchronization to the original hard drive /dev/sda. To do that we just boot of the original hard drive. First we need to update grub with the following command: (This is just for the MBR boot option) Update Grub on both hard drives
sudo grub-install /dev/sda sudo grub-install /dev/sdb
If you have the EFI boot option you just need to redo the Grub config with the following command: Update Grub for the EFI boot option
sudo grub-mkconfig -o /boot/grub/grub.cfg
This generates a new grub.cfg file in which the new RAID hard drive is the root from which Ubuntu now boots.
After a successful reboot (if not read the last two notes on how to fix Grub issues) without changing the startup properties in the Grub menu the mount command should print the following: Output of mount command
/dev/md0 on / type ext4 (rw,errors=remount-ro)
We are now finished with the RAID1 array creation. If you want to test the RAID1 array and simulate a hard disk failure you can read on.
For that, turn off the machine with the following command: Shutdown the machine
sudo shutdown -h now
And then we physically disconnect the new hard drive (/dev/sdb) and start up the system normally without changing options in the grub menu. After a successful boot, shut the machine down and physically attach the new hard drive again.
Note If you are using the UEFI boot option and you can’t boot you need to go to your BIOS and change the UEFI boot partition (the name is by default ubuntu) to be the first to boot. Maybe you have multiple entries with the name ubuntu, then you need to figure out which one works. 🙂 If this doesn’t help follow the instructions in the next note.
Note If you get a error with the following message: “error: file ‘/boot/grub/i386-pc/normal.mod’ not found.” or any other grub error and you end up with a ‘grub rescue>’ prompt you need to re install grub on the /dev/sda hard disk. To do this start from a Ubuntu live CD and use BootRepair (I did that for my EFI boot on the server and it worked like a charm) to fix Grub or try to enter the following commands in the command line: (Just for the MBR boot option. This one worked on virtual machine with the MBR boot) Reinstall Grub on /dev/sda
sudo apt-get install mdadm sudo mdadm -A -R /dev/md9 /dev/sda1 sudo mount /dev/md9 /mnt sudo grub-install /dev/sda --root-directory=/mnt sudo umount /mnt sudo mdadm -S /dev/md9 sudo halt
Now start up the machine. After the boot you should be able to see the following output when executing the mount command: Output of mount
/dev/md0 on / type ext4 (rw,errors=remount-ro)
Ok we are now sure that the original hard drive is working as expected in a degraded RAID state. (without the new hard drive /dev/sdb) Now shutdown the machine again and physically attach the new hard drive /dev/sdb again. Start the machine.
If the machine booted up normally you need to execute the following command to see that the RAID array is incomplete and that the second drive isn’t part of it: See status of RAID array
cat /proc/mdstat
To add the new drive to the RAID array and start a synchronization execute the following command: Add the new hard drive to the RAID array
sudo mdadm /dev/md0 -a /dev/sdb1
You can again watch the status of the synchronization with the following command: Watch the synchronization status
sudo watch -n1 cat /proc/mdstat
After the synchronization is complete you need to shut down the machine and physically remove the original hard drive /dev/sda and try booting with the new drive /dev/sdb. After a successful boot and a check that everything works as expected, shut down the machine again and boot up with both hard drives connected.
After that we need to add the original hard drive back to the RAID array by executing the following command: Add /dev/sda to the RAID array
sudo mdadm /dev/md0 -a /dev/sda1
You can again watch the status of the synchronization with the following command: Watch the synchronization status
sudo watch -n1 cat /proc/mdstat
Ok now we are done. We now have a fully tested RAID1 array.
If you have problems or want to remove the RAID array or just one hard drive follow these steps:
For that we need to set the desired hard drive as faulty. To do this execute the following command with the correct hard drive name: Set /dev/sda to faulty
sudo mdadm /dev/md0 --set-faulty /dev/sda1
After that we need to remove the hard drive from the RAID array with the following command: Remove /dev/sda from the array
sudo mdadm /dev/md0 -r /dev/sda1
You can do this for each hard drive you want to remove from the RAID array. To completely remove the RAID array and mdadm you need to stop the RAID array and remove the super blocks. But do a backup before executing the commands and you can just do them within a booted live CD environment. Choose the correct RAID array name and hard drive name. Stop mdadm and emove super block
sudo mdadm --stop /dev/md0 sudo mdadm --zero-superblock /dev/sda1
