SysAdmin

This document describes how to install XenServer 6.0.2 on a node without hardware raid.

Install Software

Install XenServer 6.0.2 on /dev/sda and do NOT configure any local storage (it is easier to do that afterwards). /dev/sda should containt three partitions, please verify with the following command:

[codesyntax lang="bash"]

sgdisk -p /dev/sda

[/codesyntax]

The first partition is used for XenServer installation, the second one is used for backups during XenServer upgrades.

1. Now we are going to use /dev/sdb as the mirror disk. Clear the partition table.
[codesyntax lang="bash"]

sgdisk --zap-all /dev/sdb

[/codesyntax]

2. Install a GPT table on /dev/sdb
[codesyntax lang="bash"]

sgdisk --mbrtogpt --clear /dev/sdb

[/codesyntax]

3. Create partitions on /dev/sdb. Please note that the following commands are dependent on your installation. Copy the start and the last sectors from the /dev/sda (output of sgdisk -p /dev/sda)
[codesyntax lang="bash"]

sgdisk --new=1:34:8388641 /dev/sdb
sgdisk --typecode=1:fd00 /dev/sdb
sgdisk --attributes=1:set:2 /dev/sdb
sgdisk --new=2:8388642:16777249 /dev/sdb
sgdisk --typecode=2:fd00 /dev/sdb
sgdisk --new=3:16777250:3907029134 /dev/sdb
sgdisk --typecode=3:fd00 /dev/sdb

[/codesyntax]

4. Create RAID devices
[codesyntax lang="bash"]

mknod /dev/md0 b 9 0
mknod /dev/md1 b 9 1
mknod /dev/md2 b 9 2
mdadm --create /dev/md0 --level=1 --raid-devices=2 missing /dev/sdb1
mdadm --create /dev/md1 --level=1 --raid-devices=2 missing /dev/sdb2
mdadm --create /dev/md2 --level=1 --raid-devices=2 missing /dev/sdb3

[/codesyntax]

5. Create bitmaps for each RAID device. Bitmaps slightly impact throughput but significantly reduce the rebuilt time when the array fails.
[codesyntax lang="bash"]

mdadm --grow /dev/md0 -b internal
mdadm --grow /dev/md1 -b internal
mdadm --grow /dev/md2 -b internal

[/codesyntax]

6. Format the root disk and mount it at /mnt
[codesyntax lang="bash"]

mkfs.ext3 /dev/md0
mount /dev/md0 /mnt

[/codesyntax]

7. Copy the root filesystem to the RAID array (please be patient this step may take a while).
[codesyntax lang="bash"]

cp -vxpR / /mnt

[/codesyntax]

8. Change the root filesystem in /mnt/etc/fstab to /dev/md0.
[codesyntax lang="bash"]

sed -r -i 's,LABEL=root-\w+ ,/dev/md0 ,g' /mnt/etc/fstab

[/codesyntax]

9. Install the bootloader on the second hard disk.
[codesyntax lang="bash"]

mount --bind /dev /mnt/dev
mount -t sysfs none /mnt/sys
mount -t proc none /mnt/proc
chroot /mnt /sbin/extlinux --install /boot
dd if=/mnt/usr/share/syslinux/gptmbr.bin of=/dev/sdb

[/codesyntax]

10. Make a new initrd image which contains a driver for the new root filesystem on the software RAID array.
[codesyntax lang="bash"]

chroot /mnt
mkinitrd -v -f --theme=/usr/share/splash --without-multipath /boot/initrd-`uname -r`.img `uname -r`
exit

[/codesyntax]

11. edit /mnt/boot/extlinux.conf and replace every mention of the old root filesystem (root=LABEL=xxx) with root=/dev/md0.
[codesyntax lang="bash"]

sed -r -i 's,root=LABEL=root-\w+ ,root=/dev/md0 ,g' /mnt/boot/extlinux.conf
sed -r -i 's,root=LABEL=root-\w+ ,root=/dev/md0 ,g' /boot/extlinux.conf

[/codesyntax]

12. Unmount the new root and reboot. Important: Remember to use the boot menu of your BIOS to boot from the second hard disk this time!
[codesyntax lang="bash"]

umount /mnt/proc
umount /mnt/sys
umount /mnt/dev
umount /mnt
reboot

[/codesyntax]

13. XenServer is up again, include /dev/sda in the array
[codesyntax lang="bash"]

sgdisk --typecode=1:fd00 /dev/sda
sgdisk --typecode=2:fd00 /dev/sda
sgdisk --typecode=3:fd00 /dev/sda
mdadm -a /dev/md0 /dev/sda1
mdadm -a /dev/md1 /dev/sda2
mdadm -a /dev/md2 /dev/sda3

[/codesyntax]

14. The array needs to complete its initial build/synchronisation. That will take a while.
[codesyntax lang="bash"]

watch --interval=1 cat /proc/mdstat

[/codesyntax]

15. Add /dev/md2 as a local SR to XenServer.
[codesyntax lang="bash"]

xe sr-create content-type=user device-config:device=/dev/md2 name-label="Local Storage" shared=false type=lvm

[/codesyntax]

type=ext is required if you turned on thin provisioning in the installer. Otherwise use type=lvm

Final notes:

* The second partition is used by XenServer for backups, which is why its the same size as the first partition. If you put the install CD in and boot it, an option shows up for "restore XenServer 6.0 from backup partition"

* I have created bitmaps for each raid as well. In the event of the host going down dirty, the raids can require a synch. Simply doing this is enough to add a bitmap for changed pages.

doing cat /proc/mdstat will now say something like

cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sda1[1] sdb1[0]
4193216 blocks [2/2] [UU]
bitmap: 128/128 pages [512KB], 16KB chunk

md1 : active raid1 sda2[1] sdb2[0]
4193216 blocks [2/2] [UU]
bitmap: 0/128 pages [0KB], 16KB chunk

md2 : active raid1 sda3[1] sdb3[0]
968372864 blocks [2/2] [UU]
bitmap: 0/231 pages [0KB], 2048KB chunk

* If you are installing on server which can not boot from the second disk, you must physically swap the two drives to make the machine boot off sdb and use /dev/md0 as root

* If you are going to setup a Xen 6 installation over network (via PXE) and the installation process hangs right after "Freeing unused kernel memory: 280k freed", you pass to the kernel (/tftpboot/pxelinux.cfg/main.menu) the following parameter xencons as follows:

append xenserver6/xen.gz dom0_mem=752M com1=9600,8n1 console=com1,tty --- xenserver6/vmlinuz console=tty0 console=ttyS0,9600n8 xencons=ttyS0,9600n8 answerfile=http://netboot.vendio.com/xenserver6/answers.xml install --- xenserver6/install.img

* To speed up the raid build process the following command can be used (default value is 1000):
[codesyntax lang="bash"]

echo 100000 > /proc/sys/dev/raid/speed_limit_min

[/codesyntax]

* TIP: You can use the attached script to automate the steps 1 to 9.

Good luck

The script: xen6.sh.zip

Source: http://blog.codeaddict.org/?p=5