Tag Archives: ubuntu

How to add debug symbols for MariaDB Debian/Ubuntu packages

Publicat la 14 October 2015 No comments

I don't know about other distributions, but I know that the debug symbols were stripped from Debian/Ubuntu packages. If there are some crashes reported I won't be able to fully analyze them. The only way to fix this problem is to build the packages again.

[codesyntax lang="bash"]

git clone https://github.com/MariaDB/server.git
cd server/
git branch -a
git checkout 10.0

apt-get install libdistro-info-perl
apt-get install fakeroot
apt-get install libreadline-gplv2-dev libpam0g-dev dpatch libjemalloc-dev
apt-get install libboost-all-dev libjudy-dev libjudydebian1
apt-get install build-essential dpkg-dev devscripts hardening-wrapper
apt-get build-dep mysql-server

patch -p1 < /path/to/patch.txt

./debian/autobake-deb.sh

[/codesyntax]

Note: Here is the patch.txt

How to disable dnsmasq on ubuntu based distribution

Publicat la 9 December 2014 No comments

dnsmasq is a lightweight DNS, TFTP, PXE, router advertisement and DHCP server. It is intended to provide coupled DNS and DHCP service to a LAN.
From time to time dnsmasq decided to resolve some hosts over a VPN tunnel to their external IP address instead the internal one. This was quite annoying... After digging a little bit I found that the root cause of all my VPN heartache was the dnsmasq daemon controlling my DNS. And, related, network-manager. Ok, so how can we disable dnsmasq?!

[codesyntax lang="bash"]

vim /etc/NetworkManager/NetworkManager.conf
:%s/dns=dnsmasq/#dns=dnsmasq/g
:wq

[/codesyntax]

Ta-da! No more problems! We're all set!

How to enable syntax highlighting in less

Publicat la 3 December 2013 No comments

I wanted to have syntax highlighting for a php file I was debugging. Source-highlight given a source file, produces a document with syntax highlighting.

These are the output formats already supported:

  • HTML
  • XHTML
  • LATEX
  • MediaWiki (new)
  • ODF (new)
  • TEXINFO
  • ANSI color escape sequences (you can use this feature with less)
  • DocBook

These are the input languages (or input formats) already supported (in alphabetical order):

  • Ada
  • Asm
  • Applescript
  • Awk
  • Autoconf files
  • Bat
  • Bib
  • Bison
  • C/C++
  • C#
  • CakePhp templates
  • Clipper
  • Cobol
  • Configuration files (generic)
  • Caml
  • Changelog
  • Css
  • D
  • Diff
  • Emacs Lisp
  • Erlang
  • errors (compiler output)
  • Flex
  • Fortran
  • GLSL
  • Haskell
  • Haskell literate programming
  • Haxe
  • Html
  • ini files
  • IsLisp (new)
  • Java
  • Javalog
  • Javascript
  • KDE desktop files
  • Latex
  • Ldap files
  • Lilypond (new)
  • Lisp
  • Logtalk
  • Log files
  • lsm files (Linux Software Map)
  • Lua
  • Makefile
  • Manifest
  • M4
  • ML
  • Opa
  • Oz
  • Pascal
  • Perl
  • pkg-config files
  • PHP
  • Po
  • Postscript
  • Prolog
  • Properties files
  • Protobuf (Google's Protocol Buffers)
  • Python
  • R statistics programming language (new)
  • RPM Spec files
  • Ruby
  • Scala
  • Scheme
  • Shell
  • S-Lang
  • Sql
  • T/Foswiki TML markup
  • Tcl
  • Texinfo
  • UPC (unified parallel C)
  • Vala
  • VBscript
  • XML
  • XOrg conf files

Exactly what I needed!

[codesyntax lang="bash"]

apt-get install source-highlight
echo "" >> ~/.bashrc
echo "export LESSOPEN=\"| /usr/share/source-highlight/src-hilite-lesspipe.sh %s\"" >> ~/.bashrc
echo "export LESS=' -R '" >> ~/.bashrc

[/codesyntax]

Note: Be sure this line isn't present in your .bashrc because it will interfere with source-highlight:
[codesyntax lang="bash"]

 [ -x /usr/bin/lesspipe ] && eval "$(lesspipe)"

[/codesyntax]

References:

Migrate from SVN to GIT - step by step tutorial

Publicat la 16 July 2013 2 comentarii

Lately I faced a new challenge. To fully migrate our SVN environment to GIT. After reading a little bit on internet I come up with this plan:

  1. Deploy a GIT server
  2. Create a new bare repository
  3. Make a SVN Clone on a different machine
  4. Push it on GIT server

Assumptions:

SVN Repository: https://192.168.0.5/svn/documentation

GIT Server: 192.168.0.22

Client: A Microsoft Windows machine with some IP from 192.168.0.1/24

Prerequisite packages for our client: TortoiseGit  and msysgit. For this tutorial I used: TortoiseGit-1.8.4.0-64bit.msi and msysGit-fullinstall-1.8.3-preview20130601.exe

Add C:\msysgit\msysgit\cmd to the %PATH% (I am not going to cover this here, but you can check this website: http://www.computerhope.com/issues/ch000549.htm)

1. Deploy GIT/GITOLINE/GITWEB

Server side - I assume we have a fresh debian/ubuntu installation

1.1. Install required software

[codesyntax lang="bash"]

apt-get install vim git

[/codesyntax]

1.2. Create user git with disabled password and login shell bash

[codesyntax lang="bash"]
adduser \
 --system \
 --shell /bin/bash \
 --gecos 'git version control' \
 --group \
 --disabled-password \
 --home /home/git \
 git

[/codesyntax]

1.3.  Now login with user git

[codesyntax lang="bash"]

su -l git

[/codesyntax]

1.4.  Install Gitolite

[codesyntax lang="bash"]

cd /home/git; git clone git://github.com/sitaramc/gitolite
mkdir $HOME/bin
gitolite/install -ln

[/codesyntax]

Client side

1.5. Generate RSA key (Create RSA key, after hiting the command just press enter when it ask for passphrase.). This key will be transferred to git server.

[codesyntax lang="bash"]

C:\Users\joe>C:\msysgit\msysgit\bin\ssh-keygen -t rsa -C "Git-Admin"
Generating public/private rsa key pair.
Enter file in which to save the key (//.ssh/id_rsa): c:\users\joe\.ssh\id_rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in c:\users\joe\.ssh\id_rsa.
Your public key has been saved in c:\users\joe\.ssh\id_rsa.pub.
The key fingerprint is:
aa:38:be:f9:3c:d0:87:fe:de:5a:a9:71:1d:d3:38:01 Git-Admin

[/codesyntax]

1.6. From client scp the key into git server.

[codesyntax lang="bash"]

C:\msysgit\msysgit\bin\scp C:\Users\joe\.ssh\id_rsa.pub root@192.168.0.22:~

[/codesyntax]

Server side

1.7. Now setup the Git Admin. Please note that we must be logged in with git user. To check this use whoami command

[codesyntax lang="bash"]

whoami
sudo mv /root/id_rsa.pub /home/git/Git-Admin.pub; sudo chown git:git /home/git/Git-Admin.pub
bin/gitolite setup -pk Git-Admin.pub

[/codesyntax]

1.8. Git user should be in the same group as your Apache user

[codesyntax lang="bash"]

sudo usermod -a -G www-data git
sudo chmod ug+rx /home/git/repositories

[/codesyntax]

Client side

1.9. Clone gitolite-admin repository. Now gitolite-admin directory will be present after cloneing from git server. The gitolite.conf is mainly use for User and group ACL for git server. Now from your system you can easily manage the user group ACL. But for this you have to do git push.

[codesyntax lang="bash"]

git clone git@192.168.0.22:gitolite-admin.git
cat gitolite-admin/conf/gitolite.conf

[/codesyntax]

repo gitolite-admin

    RW+     =   Git-Admin
 
repo testing
    RW+     =   @all

Server side

1.10. Edit the /home/git/.gitolite.rc and change the UMASK to 0002 and GIT_CONFIG_KEYS to '.*'

[codesyntax lang="bash"]

vim /home/git/.gitolite.rc
:%s/0077/0002/
:%s/'',/'.*',/
:wq

[/codesyntax]

1.11. Install gitweb

[codesyntax lang="bash"]

sudo apt-get install gitweb

[/codesyntax]

1.12. Configure gitweb. Edit /usr/lib/cgi-bin/gitweb.cgi and change the value of $projectroot as "/home/git/repositories" and $projects_list as "/home/git/projects.list"

[codesyntax lang="bash"]

sudo vim /usr/lib/cgi-bin/gitweb.cgi
:%s/our $projectroot = "\/pub\/git";/our $projectroot = "\/home\/git\/repositories\/";/
:%s/our $projects_list = "";/our $projects_list = "\/home\/git\/projects.list";/
:wq

[/codesyntax]

[codesyntax lang="bash"]

sudo vim /etc/gitweb.conf
:%s/$projectroot = "\/var\/cache\/git";/$projectroot = "\/home\/git\/repositories\/";/
:%s/$projects_list = $projectroot;/$projects_list = "\/home\/git\/projects.list";/
:wq

[/codesyntax]

Note: if the $projects_list is commented, uncomment it (remove the # from the begining of the line)

2. Migrate from SVN to GIT

Server side

2.1. Create a new git bare repository

[codesyntax lang="bash"]

cd /home/git/repositories/
git init --bare documentation.git

[/codesyntax]

Client side

2.2. Getting the author information.

[codesyntax lang="bash"]

svn log --xml | grep -P "^<author" | sort -u | perl -pe 's/<author>(.*?)<\/author>/$1 = /' > users.txt

[/codesyntax]

That gives you the log output in XML format - you can look for the authors, create a unique list, and then strip out the XML. (Obviously this only works on a machine with grep, sort, and perl installed.) Then, redirect that output into your users.txt file so you can add the equivalent Git user data next to each entry. The users.txt file should look like:

(no author) = Alex <alex@example.com>
bear = Bear <kong@example.com>
monika = Monika <cristina@example.com>
igor = Igor <igor@example.com>
joe = Joe <joe@example.com>

You must execute this command on SVN repository folder. If you are using Windows, then the previous command will not work, so I suggest the following approach.

* Generate the file (svn_users.txt), change it as you like and trasfer it to a linux box (I use GIT server for this)

* Connect to GIT server and process the file there using:

[codesyntax lang="bash"]

cat svn_users.txt | grep -P "^<author" | sort -u | perl -pe 's/<author>(.*?)<\/author>/$1 = /' > users.txt
rm svn_users.txt

[/codesyntax]

* Transfer the file users.txt to Windows machine.

2.3. Add the new repo that we want to migrate to gitolite-admin\conf\gitolite.conf. The file should look something like this:

repo gitolite-admin
    RW+     =   Git-Admin
 
repo testing
    RW+     =   @all
 
repo documentation
    RW+     =   @all

At this point all users will have RW access to our git repository.

2.4. Add new users to GIT. For each user you will need to have to a pair of public/private keys. Copy public keys to keydir folder in gitolite-admin clone, add and push the changes.

2.5. Import SVN repository. This is going to take a while, so go and grab a snack or something.

[codesyntax lang="bash"]

git svn clone https://192.168.0.5/svn/documentation --authors-file=users.txt --no-metadata

[/codesyntax]

2.6. The last thing to do is add the Git server as a remote and push to it.

[codesyntax lang="bash"]

git remote add origin git@192.168.0.22:documentation.git
git push origin --all
git push origin --tags

[/codesyntax]

3. Delete the imported SVN repository on client side and re-import it from GIT server

Server side

4. Securing Gitweb with htpasswd. At this point the Gitweb web page is accessible to all, which of course is not a good idea. So, if you would like to have a set of web pages that are protected, requiring a username/password to gain access, you will have to use htpasswd in apache and add users and passwords.

4.1. Configure htpasswd access in Gitweb

[codesyntax lang="bash"]

sudo vim /etc/apache2/conf.d/gitweb

[/codesyntax]

Alias /gitweb /usr/share/gitweb
 
<Directory /usr/share/gitweb>
    Options FollowSymLinks +ExecCGI
    AddHandler cgi-script .cgi
        AllowOverride None
    Order allow,deny
    Allow from all
    AuthType Basic
    AuthName "Git Access"
    Require valid-user
    AuthUserFile /etc/apache2/gitweb-htpasswd
</Directory>

4.2. Create htpasswd username and password.

[codesyntax lang="bash"]
sudo touch /etc/apache2/gitweb-htpasswd
sudo htpasswd -m /etc/apache2/gitweb-htpasswd user1
sudo htpasswd -m /etc/apache2/gitweb-htpasswd user2
sudo htpasswd -m /etc/apache2/gitweb-htpasswd user3

[/codesyntax]

4.3. Restart the apache server

[codesyntax lang="bash"]

sudo /etc/init.d/apache2 restart

[/codesyntax]

Note: You can use TortoiseGIT for doing all git operations on Windows Machine, of course. I was lazy and I didn't take screenshots with steps that you should do with TortoiseGIT :)

Links:

How to install the same debian packages on another system

Publicat la 21 February 2013 No comments

The approach is simple: generate a list of installed packages on Debian-based systems and install this list of packages on the new box. This approach is useful when you want to install the same packages on another fresh OS install for example.

1. To export the list of installed packages, proceed as follows:
[codesyntax lang="bash"]

dpkg --get-selections | grep -v deinstall > LIST_FILE

[/codesyntax]

The file LIST_FILE is small so it can be emailed to yourself by using:
[codesyntax lang="bash"]

dpkg --get-selections | grep -v deinstall > LIST_FILE && cat LIST_FILE | mailx -s "`hostname -f`: Package list" "user@test.org"

[/codesyntax]

2. Once you’ve got your server up and running with a fresh base install
[codesyntax lang="bash"]

apt-get update
apt-get dist-upgrade

[/codesyntax]

3. Move your LIST_FILE file into your home directory and run the following commands to recover the previous generated list:
[codesyntax lang="bash"]

dpkg --clear-selections
dpkg --set-selections < LIST_FILE

[/codesyntax]

4. Install the packages
[codesyntax lang="bash"]

aptitude install

[/codesyntax]

Paravirtulization with Citrix XenServer 5.5 and Ubuntu 9.10

Publicat la 25 January 2013 1 comentariu

Few days ago I had a task to P2V an old Ubuntu 9.10 machine. The P2V process was very easy and went smooth. The challenge was how to get this VM paravirtualized. After reading on the net how others have done this and what problems they had, I managed to finish my task pretty quick. Anyway... I hope maybe this post helps someone - definitely will help me if I will have to do this task again.

This post describes with simple step-by-step instructions how to install Ubuntu 9.10 VM as a paravirtualized virtual machine on a Citrix XenServer 5.5.

Creating Our Guest Ubuntu VM
Our first step is to get an Ubuntu VM installed as a typical HVM. You can find many different options on the web about partitioning and recommended partition sizes. A default installation of Ubuntu 9.10 will install on two partitions:

  • a root (/) partition, which includes the boot system (/boot), and
  • a swap partition.

For this article I installed Ubuntu with default partition options.

Configuring XenServer

First login on XenServer console.

Get UUID for the new created VM.

[codesyntax lang="bash"]

xe vm-list name-label="ubuntu-vm" params=uuid --minimal

[/codesyntax]

ed788e42-aabd-f78e-180a-5e46ec8b2465

Find the VM’s hard drive, known as a virtual block device (VBD):

[codesyntax lang="bash"]

xe vm-disk-list uuid=ed788e42-aabd-f78e-180a-5e46ec8b2465

[/codesyntax]

Disk 0 VBD:
uuid ( RO)             : ceb500b7-b154-2251-2fcd-5de05da50368
    vm-name-label ( RO): ubuntu10.04
       userdevice ( RW): 0

Mark the VBD as bootable:

[codesyntax lang="bash"]

xe vbd-param-set uuid=ceb500b7-b154-2251-2fcd-5de05da50368 bootable=true

[/codesyntax]

We don't want our VM to run as HVM:

[codesyntax lang="bash"]

xe vm-param-set uuid=ed788e42-aabd-f78e-180a-5e46ec8b2465 HVM-boot-policy=
xe vm-param-set uuid=ed788e42-aabd-f78e-180a-5e46ec8b2465 PV-bootloader=pygrub

[/codesyntax]

pygrub can’t handle grub2, so we have to manually set these parameters for paravirtualization:

[codesyntax lang="bash"]

xe vm-param-set uuid=ed788e42-aabd-f78e-180a-5e46ec8b2465 PV-bootloader-args="--kernel=/boot/vmlinuz-2.6.31-14-generic --ramdisk=/boot/initrd.img-2.6.31-14-generic"
xe vm-param-set uuid=ed788e42-aabd-f78e-180a-5e46ec8b2465 PV-args="root=UUID=706a70b4-09ee-4682-8f08-c8eb79ddd410 ro quiet"

[/codesyntax]

Notes:

  • 706a70b4-09ee-4682-8f08-c8eb79ddd410 - UUID for the bootable partition. You can find it in grub configuration file or using blkid command
  • if you have a separate partition for boot then you will have to change kernel and ramdisk parameters to --kernel=/vmlinuz-2.6.31-14-generic --ramdisk=/initrd.img-2.6.31-14-generic

Close and restart your XenCenter client (it appears to be a bit buggy and doesn’t let you type into the new console until it’s restarted), and boot up your VM (which will now start in PV mode).

Install XenServer tools

Attach the XenServer tools ISO image (xs-tools.iso) and mount the CD on your VM.

[codesyntax lang="bash"]

mount /dev/cdrom /mnt

[/codesyntax]

Install XenServer tools

[codesyntax lang="bash"]

dpkg -i /mnt/Linux/xe-guest-utilities_5.5.0-466_amd64.deb

[/codesyntax]

During the install, you would have likely noticed a couple errors, specifically:

update-rc.d: warning: xe-linux-distribution start runlevel arguments (S) do not match LSB Default-Start values (2 3 4 5)
...
[: 31: configure: unexpected operator

The package was build for Debian, not for Ubuntu so we don't have to worry about error message. We need to adjust the default start/kill runlevels.

[codesyntax lang="bash"]

update-rc.d -f xe-linux-distribution remove
update-rc.d xe-linux-distribution defaults

[/codesyntax]

Since now we are paravirtualized, XenServer will want to use HVC0, not the traditionally TTY.

[codesyntax lang="bash"]

sed -e "s/tty1/hvc0/ig" /etc/init/tty1.conf | sudo bash -c 'cat > /etc/init/hvc0.conf'

[/codesyntax]

Accessing GUI on ubuntu paravirtualized VM

If you try and start the GUI on a paravirtualized Ubuntu VM in XenServer, you’ll get the following error:

Primary device is not PCI
(EE) open /dev/fb0: No such file or directory
(EE) No devices detected

In a paravirtualized world there is no such thing as a physical console (nor is there a physical CPU, physical memory etc). Hence for completely paravirtualized OSes (with a paravirtualized kernel like Xen) there’s no GUI console.

In other words, use VNC for now:

Install VNC
[codesyntax lang="bash"]

apt-get install vnc4server

[/codesyntax]

Set the VNC resolution (whatever resolution you want to see on your desktop machine you’ll be using the VNC client on
[codesyntax lang="bash"]

vncserver -geometry 1280x1024 -depth 24

[/codesyntax]

Create a password and VNC server should create some configuration files and start up.

Now we need to edit one of the configuration files

[codesyntax lang="bash"]

vncserver -kill :1

[/codesyntax]

[codesyntax lang="bash"]

vim ~/.vnc/xstartup

[/codesyntax]

#!/bin/sh
# Uncomment the following two lines for normal desktop:
unset SESSION_MANAGER
exec sh /etc/X11/xinit/xinitrc

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &
x-terminal-emulator -geometry 1280x1024+10+10 -ls -title “$VNCDESKTOP Desktop” &
x-window-manager &
[codesyntax lang="bash"]

:wq

[/codesyntax]

Start up the VNC server again
[codesyntax lang="bash"]

vncserver -geometry 1280x1024 -depth 24

[/codesyntax]

Troubleshooting

Can't type at login prompt.

I ran into one instance where I could see the login prompt but nothing I typed appeared.  First, make sure you click inside the console window.  If that does not resolve the issue, close and reopen XenCenter.  This fixed the issue when I encountered it.

PV is not working and I need to get the VM back up.

[codesyntax lang="bash"]

xe vm-param-set uuid= HVM-boot-policy="BIOS order"

[/codesyntax]

To return to PV mode, clear the HVM-boot-policy parameter.

[codesyntax lang="bash"]

xe vm-param-set uuid= HVM-boot-policy=

[/codesyntax]

Please note that HVM-boot-policy parameter IS case sensitive.

Links:
http://www.aikidokatech.com/?p=30
http://blog.403labs.com/post/1546501840/paravirtulization-with-citrix-xenserver-5-5-and-ubuntu
http://sysadmin.circularvale.com/server-config/accessing-a-gui-on-ubuntu-xenserver-vm/

How to migrate a single disk Linux System to software RAID1

Publicat la 23 October 2012 19 comentarii

This guide explains how to set up software RAID1 on an already running Linux (Ubuntu 12.10) system. The GRUB2 bootloader will be configured in such a way that the system will still be able to boot if one of the hard drives fails (no matter which one).

Preliminary Note

In this tutorial I am using an Ubuntu 12.10 system with two disks, /dev/sda and /dev/sdb which are identical in size.
/dev/sdb is currently unused, and /dev/sda has the following partition:

/dev/sda1: / partition, ext4;
/dev/sda5: swap

After completing this guide I will have the following situation:
/dev/md0: / partition, ext4;
/dev/md1: swap

The current situation:

[codesyntax lang="bash"]

df -h

[/codesyntax]

root@ubuntu:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        19G  969M   17G   6% /
udev            494M  4.0K  494M   1% /dev
tmpfs           201M  272K  201M   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            502M     0  502M   0% /run/shm
none            100M     0  100M   0% /run/user
root@ubuntu:~#

[codesyntax lang="bash"]

fdisk -l

[/codesyntax]

root@ubuntu:~# fdisk -l

Disk /dev/sda: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders, total 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00059a4b

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048    39845887    19921920   83  Linux
/dev/sda2        39847934    41940991     1046529    5  Extended
/dev/sda5        39847936    41940991     1046528   82  Linux swap / Solaris

Disk /dev/sdb: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders, total 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdb doesn't contain a valid partition table
root@ubuntu:~#

Installing mdadm

First of all install md tools:

[codesyntax lang="bash"]

aptitude install initramfs-tools mdadm

[/codesyntax]

In order to avoid reboot, let's load few kernel modules:

[codesyntax lang="bash"]

modprobe linear
modprobe multipath
modprobe raid0
modprobe raid1
modprobe raid5
modprobe raid6
modprobe raid10

[/codesyntax]

Now:

[codesyntax lang="bash"]

cat /proc/mdstat

[/codesyntax]

root@ubuntu:~# cat /proc/mdstat

Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
unused devices:
root@ubuntu:~#

Preparing the second disk

To create a software RAID1 on a running system, we have to prepare the second disk added to the system (in this case /dev/sdb) for RAID1, then copy the contents from the first disk (/dev/sda) to it, and finally add the first disk to the RAID1 array.

Let's copy the partition table from /dev/sda to /dev/sdb so that the both disks have the exactly same layout:

[codesyntax lang="bash"]

sfdisk -d /dev/sda | sfdisk --force /dev/sdb

[/codesyntax]

root@ubuntu:~# sfdisk -d /dev/sda | sfdisk --force /dev/sdb
Checking that no-one is using this disk right now ...
Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
OK

Disk /dev/sdb: 2610 cylinders, 255 heads, 63 sectors/track

sfdisk: ERROR: sector 0 does not have an msdos signature
/dev/sdb: unrecognized partition table type
Old situation:
No partitions found
New situation:
Units = sectors of 512 bytes, counting from 0

Device Boot    Start       End   #sectors  Id  System
/dev/sdb1   *      2048  39845887   39843840  83  Linux
/dev/sdb2      39847934  41940991    2093058   5  Extended
/dev/sdb3             0         -          0   0  Empty
/dev/sdb4             0         -          0   0  Empty
/dev/sdb5      39847936  41940991    2093056  82  Linux swap / Solaris
Warning: partition 1 does not end at a cylinder boundary
Warning: partition 2 does not start at a cylinder boundary
Warning: partition 2 does not end at a cylinder boundary
Warning: partition 5 does not end at a cylinder boundary
Successfully wrote the new partition table

Re-reading the partition table ...

If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
(See fdisk(8).)
root@ubuntu:~#

And the output of the command:

[codesyntax lang="bash"]

fdisk -l

[/codesyntax]

root@ubuntu:~# fdisk -l

Disk /dev/sda: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders, total 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00059a4b

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048    39845887    19921920   83  Linux
/dev/sda2        39847934    41940991     1046529    5  Extended
/dev/sda5        39847936    41940991     1046528   82  Linux swap / Solaris

Disk /dev/sdb: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders, total 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *        2048    39845887    19921920   83  Linux
/dev/sdb2        39847934    41940991     1046529    5  Extended
/dev/sdb5        39847936    41940991     1046528   82  Linux swap / Solaris
root@ubuntu:~#

Change the partitions type on /dev/sdb to Linux raid autodetect:

[codesyntax lang="bash"]

sfdisk --change-id /dev/sdb 1 fd
sfdisk --change-id /dev/sdb 5 fd

[/codesyntax]

root@ubuntu:~# sfdisk --change-id /dev/sdb 1 fd
Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
Done

root@ubuntu:~# sfdisk --change-id /dev/sdb 5 fd
Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
Done

root@ubuntu:~#

To make sure that there are no remains from previous RAID installations on /dev/sdb, we run the following commands:

[codesyntax lang="bash"]

mdadm --zero-superblock /dev/sdb1
mdadm --zero-superblock /dev/sdb5

[/codesyntax]

If you receive the following error messages then there are no remains from previous RAID installations, which is nothing to worry about:

root@ubuntu:~# mdadm --zero-superblock /dev/sdb1
mdadm: Unrecognised md component device - /dev/sdb1
root@ubuntu:~# mdadm --zero-superblock /dev/sdb5
mdadm: Unrecognised md component device - /dev/sdb5
root@ubuntu:~#

Creating RAID arrays

Now use mdadm to create the raid arrays. We mark the first drive (sda) as "missing" so it doesn't wipe out our existing data:

[codesyntax lang="bash"]

mdadm --create /dev/md0 --level=1 --raid-disks=2 missing /dev/sdb1
mdadm --create /dev/md1 --level=1 --raid-disks=2 missing /dev/sdb5

[/codesyntax]

root@ubuntu:~# mdadm --create /dev/md0 --level=1 --raid-disks=2 missing /dev/sdb1
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
root@ubuntu:~# mdadm --create /dev/md1 --level=1 --raid-disks=2 missing /dev/sdb5
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md1 started.
root@ubuntu:~#

[codesyntax lang="bash"]

cat /proc/mdstat

[/codesyntax]

root@ubuntu:~# cat /proc/mdstat 
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md1 : active raid1 sdb5[1]
      1045952 blocks super 1.2 [2/1] [_U]
      
md0 : active raid1 sdb1[1]
      19905408 blocks super 1.2 [2/1] [_U]
      
unused devices: <none>
root@ubuntu:~# 

The output above means that we have two degraded arrays ([_U] or [U_] means that an array is degraded while [UU] means that the array is ok).

Create the filesystems on RAID arrays (ext4 on /dev/md0 and swap on /dev/md1)

[codesyntax lang="bash"]

mkfs.ext4 /dev/md0
mkswap /dev/md1

[/codesyntax]

root@ubuntu:~# mkfs.ext4 /dev/md0
mke2fs 1.42.5 (29-Jul-2012)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
1245184 inodes, 4976352 blocks
248817 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
152 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
    4096000

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done   

root@ubuntu:~# mkswap /dev/md1
mkswap: /dev/md1: warning: don't erase bootbits sectors
        on whole disk. Use -f to force.
Setting up swapspace version 1, size = 1045948 KiB
no label, UUID=728f7cfe-bd95-43e5-906d-c8a70023d081
root@ubuntu:~# 

Adjust mdadm configuration file which doesn't contain any information about RAID arrays yet:

[codesyntax lang="bash"]

cp /etc/mdadm/mdadm.conf /etc/mdadm/mdadm.conf_orig
mdadm --examine --scan >> /etc/mdadm/mdadm.conf

[/codesyntax]

Display the content of /etc/mdadm/mdadm.conf:

[codesyntax lang="bash"]

cat /etc/mdadm/mdadm.conf

[/codesyntax]

root@ubuntu:~# cat /etc/mdadm/mdadm.conf
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default (built-in), scan all partitions (/proc/partitions) and all
# containers for MD superblocks. alternatively, specify devices to scan, using
# wildcards if desired.
#DEVICE partitions containers

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST

# instruct the monitoring daemon where to send mail alerts
MAILADDR root

# definitions of existing MD arrays

# This file was auto-generated on Tue, 23 Oct 2012 04:36:40 -0700
# by mkconf $Id$
root@ubuntu:~#

Adjusting The System To RAID1

Let's mount /dev/md0:

[codesyntax lang="bash"]

mkdir /mnt/md0
mount /dev/md0 /mnt/md0

[/codesyntax]

[codesyntax lang="bash"]

mount

[/codesyntax]

root@ubuntu:~# mount
/dev/sda1 on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
none on /run/user type tmpfs (rw,noexec,nosuid,nodev,size=104857600,mode=0755)
/dev/md0 on /mnt/md0 type ext4 (rw)
root@ubuntu:~# 

Change the UID values in /etc/fstab with the UUID values returned by blkid:

[codesyntax lang="bash"]

blkid /dev/md0 /dev/md1

[/codesyntax]

root@ubuntu:~# blkid /dev/md0 /dev/md1
/dev/md0: UUID="4a49251b-e357-40a4-b13f-13b041c55a9d" TYPE="ext4"
/dev/md1: UUID="728f7cfe-bd95-43e5-906d-c8a70023d081" TYPE="swap"
root@ubuntu:~#

After changing the UUID values the /etc/fstab should look as follows:

[codesyntax lang="bash"]

cat /etc/fstab

[/codesyntax]

root@ubuntu:~# cat /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# # / was on /dev/sda1 during installation
UUID=4a49251b-e357-40a4-b13f-13b041c55a9d / ext4 errors=remount-ro 0 1
# swap was on /dev/sda5 during installation
UUID=728f7cfe-bd95-43e5-906d-c8a70023d081 none swap sw 0 0
/dev/fd0 /media/floppy0 auto rw,user,noauto,exec,utf8 0 0
root@ubuntu:~#

Next replace /dev/sda1 with /dev/md0 in /etc/mtab:

[codesyntax lang="bash"]

sed -e "s/dev\/sda1/dev\/md0/" -i /etc/mtab

[/codesyntax]

[codesyntax lang="bash"]

cat /etc/mtab

[/codesyntax]

root@ubuntu:~# cat /etc/mtab
/dev/md0 / ext4 rw,errors=remount-ro 0 0
proc /proc proc rw,noexec,nosuid,nodev 0 0
sysfs /sys sysfs rw,noexec,nosuid,nodev 0 0
none /sys/fs/fuse/connections fusectl rw 0 0
none /sys/kernel/debug debugfs rw 0 0
none /sys/kernel/security securityfs rw 0 0
udev /dev devtmpfs rw,mode=0755 0 0
devpts /dev/pts devpts rw,noexec,nosuid,gid=5,mode=0620 0 0
tmpfs /run tmpfs rw,noexec,nosuid,size=10%,mode=0755 0 0
none /run/lock tmpfs rw,noexec,nosuid,nodev,size=5242880 0 0
none /run/shm tmpfs rw,nosuid,nodev 0 0
none /run/user tmpfs rw,noexec,nosuid,nodev,size=104857600,mode=0755 0 0
/dev/md0 /mnt/md0 ext4 rw 0 0
root@ubuntu:~#

Setup the GRUB2 boot loader.

Create the file /etc/grub.d/09_swraid1_setup as follows:

[codesyntax lang="bash"]

cp /etc/grub.d/40_custom /etc/grub.d/09_swraid1_setup
vim /etc/grub.d/09_swraid1_setup

[/codesyntax]

#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
menuentry 'Ubuntu, with Linux 3.5.0-17-generic' --class ubuntu --class gnu-linux --class gnu --class os {
    recordfail
    insmod mdraid1x
    insmod ext2
    set root='(md/0)'
    linux   /boot/vmlinuz-3.5.0-17-generic root=/dev/md0 ro   quiet
    initrd  /boot/initrd.img-3.5.0-17-generic
}

Make sure you use the correct kernel version in the menuentry (in the linux and initrd lines).

[codesyntax lang="bash"]

uname -r

[/codesyntax]

root@ubuntu:~# uname -r
3.5.0-17-generic
root@ubuntu:~#

Update grub configuration and adjust our ramdisk to the new situation:

[codesyntax lang="bash"]

update-grub
update-initramfs -u

[/codesyntax]

root@ubuntu:~# update-grub
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-3.5.0-17-generic
Found initrd image: /boot/initrd.img-3.5.0-17-generic
Found memtest86+ image: /boot/memtest86+.bin
done
root@ubuntu:~# update-initramfs -u
update-initramfs: Generating /boot/initrd.img-3.5.0-17-generic
W: mdadm: /etc/mdadm/mdadm.conf defines no arrays.
root@ubuntu:~#

Copy files to the new disk

Copy the files from the first disk (/dev/sda) to the second one (/dev/sdb)

[codesyntax lang="bash"]

cp -dpRx / /mnt/md0

[/codesyntax]

Preparing GRUB2 (Part 1)

Install GRUB2 boot loader on both disks (/dev/sda and /dev/sdb):

[codesyntax lang="bash"]

grub-install /dev/sda
grub-install /dev/sdb

[/codesyntax]

root@ubuntu:~# grub-install /dev/sda
Installation finished. No error reported.
root@ubuntu:~# grub-install /dev/sdb
Installation finished. No error reported.

Now we reboot the system and hope that it boots ok from our RAID arrays:

[codesyntax lang="bash"]

reboot

[/codesyntax]

Preparing /dev/sda

If everything went well, you should now find /dev/md0 in the output of:

[codesyntax lang="bash"]

df -h

[/codesyntax]

root@ubuntu:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/md0         19G  985M   17G   6% /
udev            494M  4.0K  494M   1% /dev
tmpfs           201M  304K  201M   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            502M     0  502M   0% /run/shm
none            100M     0  100M   0% /run/user
root@ubuntu:~#

The output of:

[codesyntax lang="bash"]

cat /proc/mdstat

[/codesyntax]

root@ubuntu:~# cat /proc/mdstat 
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md1 : active raid1 sdb5[1]
      1045952 blocks super 1.2 [2/1] [_U]
      
md0 : active raid1 sdb1[1]
      19905408 blocks super 1.2 [2/1] [_U]
      
unused devices: <none>
root@ubuntu:~#

Change the partitions type on /dev/sda to Linux raid autodetect:

[codesyntax lang="bash"]

sfdisk --change-id /dev/sda 1 fd
sfdisk --change-id /dev/sda 5 fd

[/codesyntax]

root@ubuntu:~# sfdisk --change-id /dev/sda 1 fd
Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
Done

root@ubuntu:~# sfdisk --change-id /dev/sda 5 fd
Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
Done

root@ubuntu:~#

[codesyntax lang="bash"]

fdisk -l

[/codesyntax]

root@ubuntu:~# fdisk -l

Disk /dev/sda: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders, total 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00059a4b

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048    39845887    19921920   fd  Linux raid autodetect
/dev/sda2        39847934    41940991     1046529    5  Extended
/dev/sda5        39847936    41940991     1046528   fd  Linux raid autodetect

Disk /dev/sdb: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders, total 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *        2048    39845887    19921920   fd  Linux raid autodetect
/dev/sdb2        39847934    41940991     1046529    5  Extended
/dev/sdb5        39847936    41940991     1046528   fd  Linux raid autodetect

Disk /dev/md0: 20.4 GB, 20383137792 bytes
2 heads, 4 sectors/track, 4976352 cylinders, total 39810816 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/md0 doesn't contain a valid partition table

Disk /dev/md1: 1071 MB, 1071054848 bytes
2 heads, 4 sectors/track, 261488 cylinders, total 2091904 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/md1 doesn't contain a valid partition table
root@ubuntu:~#

Now we can add /dev/sda1 and /dev/sda5 to the respective RAID arrays:

[codesyntax lang="bash"]

mdadm --add /dev/md0 /dev/sda1
mdadm --add /dev/md1 /dev/sda5

[/codesyntax]

root@ubuntu:~# mdadm --add /dev/md0 /dev/sda1
mdadm: added /dev/sda1
root@ubuntu:~# mdadm --add /dev/md1 /dev/sda5
mdadm: added /dev/sda5
root@ubuntu:~#

Take a look at:

[codesyntax lang="bash"]

cat /proc/mdstat

[/codesyntax]

root@ubuntu:~# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 
md1 : active raid1 sda5[2] sdb5[1]
      1045952 blocks super 1.2 [2/1] [_U]
          resync=DELAYED
      
md0 : active raid1 sda1[2] sdb1[1]
      19905408 blocks super 1.2 [2/1] [_U]
      [=======>.............]  recovery = 36.4% (7247872/19905408) finish=1.0min speed=205882K/sec
      
unused devices: <none>
root@ubuntu:~#

Then adjust /etc/mdadm/mdadm.conf to the new situation:

[codesyntax lang="bash"]

cp /etc/mdadm/mdadm.conf_orig /etc/mdadm/mdadm.conf
mdadm --examine --scan >> /etc/mdadm/mdadm.conf

[/codesyntax]

Display the content of /etc/mdadm/mdadm.conf:

[codesyntax lang="bash"]

cat /etc/mdadm/mdadm.conf

[/codesyntax]

root@ubuntu:~# cat /etc/mdadm/mdadm.conf
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default (built-in), scan all partitions (/proc/partitions) and all
# containers for MD superblocks. alternatively, specify devices to scan, using
# wildcards if desired.
#DEVICE partitions containers

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST

# instruct the monitoring daemon where to send mail alerts
MAILADDR root

# definitions of existing MD arrays

# This file was auto-generated on Tue, 23 Oct 2012 04:36:40 -0700
# by mkconf $Id$
ARRAY /dev/md/0 metadata=1.2 UUID=89e5afc0:2d741a2c:7d0f40f0:a1457396 name=ubuntu:0
ARRAY /dev/md/1 metadata=1.2 UUID=ce9163fc:4e168956:5c9050ad:68f15735 name=ubuntu:1
root@ubuntu:~#

Preparing GRUB2 (Part 2)

Now it's safe to delete /etc/grub.d/09_swraid1_setup

[codesyntax lang="bash"]

rm -f /etc/grub.d/09_swraid1_setup

[/codesyntax]

Update our GRUB2 bootloader configuration and install it again on both disks (/dev/sda and /dev/sdb)

[codesyntax lang="bash"]

update-grub
update-initramfs -u
grub-install /dev/sda
grub-install /dev/sdb

[/codesyntax]

Reboot the machine

[codesyntax lang="bash"]

reboot

[/codesyntax]