Migrate from SVN to GIT – step by step tutorial

Lately I faced a new challenge. To fully migrate our SVN environment to GIT. After reading a little bit on internet I come up with this plan:

  1. Deploy a GIT server
  2. Create a new bare repository
  3. Make a SVN Clone on a different machine
  4. Push it on GIT server

Assumptions:

SVN Repository: https://192.168.0.5/svn/documentation

GIT Server: 192.168.0.22

Client: A Microsoft Windows machine with some IP from 192.168.0.1/24

Prerequisite packages for our client: TortoiseGit  and msysgit. For this tutorial I used: TortoiseGit-1.8.4.0-64bit.msi and msysGit-fullinstall-1.8.3-preview20130601.exe

Add C:\msysgit\msysgit\cmd to the %PATH% (I am not going to cover this here, but you can check this website: http://www.computerhope.com/issues/ch000549.htm)

1. Deploy GIT/GITOLINE/GITWEB

Server side - I assume we have a fresh debian/ubuntu installation

1.1. Install required software

apt-get install vim git

1.2. Create user git with disabled password and login shell bash

adduser \
 --system \
 --shell /bin/bash \
 --gecos 'git version control' \
 --group \
 --disabled-password \
 --home /home/git \
 git

1.3.  Now login with user git

su -l git

1.4.  Install Gitolite

cd /home/git; git clone git://github.com/sitaramc/gitolite
mkdir $HOME/bin
gitolite/install -ln

Client side

1.5. Generate RSA key (Create RSA key, after hiting the command just press enter when it ask for passphrase.). This key will be transferred to git server.

C:\Users\joe>C:\msysgit\msysgit\bin\ssh-keygen -t rsa -C "Git-Admin"
Generating public/private rsa key pair.
Enter file in which to save the key (//.ssh/id_rsa): c:\users\joe\.ssh\id_rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in c:\users\joe\.ssh\id_rsa.
Your public key has been saved in c:\users\joe\.ssh\id_rsa.pub.
The key fingerprint is:
aa:38:be:f9:3c:d0:87:fe:de:5a:a9:71:1d:d3:38:01 Git-Admin

1.6. From client scp the key into git server.

C:\msysgit\msysgit\bin\scp C:\Users\joe\.ssh\id_rsa.pub root@192.168.0.22:~

Server side

1.7. Now setup the Git Admin. Please note that we must be logged in with git user. To check this use whoami command

whoami
sudo mv /root/id_rsa.pub /home/git/Git-Admin.pub; sudo chown git:git /home/git/Git-Admin.pub
bin/gitolite setup -pk Git-Admin.pub

1.8. Git user should be in the same group as your Apache user

sudo usermod -a -G www-data git
sudo chmod ug+rx /home/git/repositories

Client side

1.9. Clone gitolite-admin repository. Now gitolite-admin directory will be present after cloneing from git server. The gitolite.conf is mainly use for User and group ACL for git server. Now from your system you can easily manage the user group ACL. But for this you have to do git push.

git clone git@192.168.0.22:gitolite-admin.git
cat gitolite-admin/conf/gitolite.conf

repo gitolite-admin

    RW+     =   Git-Admin
 
repo testing
    RW+     =   @all

Server side

1.10. Edit the /home/git/.gitolite.rc and change the UMASK to 0002 and GIT_CONFIG_KEYS to '.*'

vim /home/git/.gitolite.rc
:%s/0077/0002/
:%s/'',/'.*',/
:wq

1.11. Install gitweb

sudo apt-get install gitweb

1.12. Configure gitweb. Edit /usr/lib/cgi-bin/gitweb.cgi and change the value of $projectroot as "/home/git/repositories" and $projects_list as "/home/git/projects.list"

sudo vim /usr/lib/cgi-bin/gitweb.cgi
:%s/our $projectroot = "\/pub\/git";/our $projectroot = "\/home\/git\/repositories\/";/
:%s/our $projects_list = "";/our $projects_list = "\/home\/git\/projects.list";/
:wq
sudo vim /etc/gitweb.conf
:%s/$projectroot = "\/var\/cache\/git";/$projectroot = "\/home\/git\/repositories\/";/
:%s/$projects_list = $projectroot;/$projects_list = "\/home\/git\/projects.list";/
:wq

Note: if the $projects_list is commented, uncomment it (remove the # from the begining of the line)

2. Migrate from SVN to GIT

Server side

2.1. Create a new git bare repository

cd /home/git/repositories/
git init --bare documentation.git

Client side

2.2. Getting the author information.

svn log --xml | grep -P "^<author" | sort -u | perl -pe 's/<author>(.*?)<\/author>/$1 = /' > users.txt

That gives you the log output in XML format - you can look for the authors, create a unique list, and then strip out the XML. (Obviously this only works on a machine with grep, sort, and perl installed.) Then, redirect that output into your users.txt file so you can add the equivalent Git user data next to each entry. The users.txt file should look like:

(no author) = Alex <alex@example.com>
bear = Bear <kong@example.com>
monika = Monika <cristina@example.com>
igor = Igor <igor@example.com>
joe = Joe <joe@example.com>

You must execute this command on SVN repository folder. If you are using Windows, then the previous command will not work, so I suggest the following approach.

* Generate the file (svn_users.txt), change it as you like and trasfer it to a linux box (I use GIT server for this)

* Connect to GIT server and process the file there using:

cat svn_users.txt | grep -P "^<author" | sort -u | perl -pe 's/<author>(.*?)<\/author>/$1 = /' > users.txt
rm svn_users.txt
* Transfer the file users.txt to Windows machine.

2.3. Add the new repo that we want to migrate to gitolite-admin\conf\gitolite.conf. The file should look something like this:

repo gitolite-admin
    RW+     =   Git-Admin
 
repo testing
    RW+     =   @all
 
repo documentation
    RW+     =   @all

At this point all users will have RW access to our git repository.

2.4. Add new users to GIT. For each user you will need to have to a pair of public/private keys. Copy public keys to keydir folder in gitolite-admin clone, add and push the changes.

2.5. Import SVN repository. This is going to take a while, so go and grab a snack or something.

git svn clone https://192.168.0.5/svn/documentation --authors-file=users.txt --no-metadata

2.6. The last thing to do is add the Git server as a remote and push to it.

git remote add origin git@192.168.0.22:documentation.git
git push origin --all
git push origin --tags

3. Delete the imported SVN repository on client side and re-import it from GIT server

Server side

4. Securing Gitweb with htpasswd. At this point the Gitweb web page is accessible to all, which of course is not a good idea. So, if you would like to have a set of web pages that are protected, requiring a username/password to gain access, you will have to use htpasswd in apache and add users and passwords.

4.1. Configure htpasswd access in Gitweb

sudo vim /etc/apache2/conf.d/gitweb

Alias /gitweb /usr/share/gitweb
 
<Directory /usr/share/gitweb>
    Options FollowSymLinks +ExecCGI
    AddHandler cgi-script .cgi
        AllowOverride None
    Order allow,deny
    Allow from all
    AuthType Basic
    AuthName "Git Access"
    Require valid-user
    AuthUserFile /etc/apache2/gitweb-htpasswd
</Directory>

4.2. Create htpasswd username and password.

sudo touch /etc/apache2/gitweb-htpasswd
sudo htpasswd -m /etc/apache2/gitweb-htpasswd user1
sudo htpasswd -m /etc/apache2/gitweb-htpasswd user2
sudo htpasswd -m /etc/apache2/gitweb-htpasswd user3

4.3. Restart the apache server

sudo /etc/init.d/apache2 restart

Note: You can use TortoiseGIT for doing all git operations on Windows Machine, of course. I was lazy and I didn't take screenshots with steps that you should do with TortoiseGIT :)

Links:

2 thoughts on “Migrate from SVN to GIT – step by step tutorial”

  1. If you want to add description to your repositories all you have to is:

    1. Open the .gitolite.rc file in git HOME directory and edit post_compile section.

    POST_COMPILE => [
            'post-compile/ssh-authkeys',
            'post-compile/update-git-configs',
            'post-compile/update-gitweb-access-list',
            'post-compile/update-git-daemon-access-list',
            'post-compile/update-description-file',
        ],

    2. Then add description for repo in gitolite.conf

    repo    testing
    RW+     =   @all
    desc = "GIT testing repository"

    3.Then commit and push your changes.

  2. To add a customized theme (from http://kogakure.github.com/gitweb-theme/):

    sudo mv /usr/share/gitweb/static/gitweb.js /usr/share/gitweb/static/gitweb.js.orig
    sudo mv /usr/share/gitweb/static/gitweb.css /usr/share/gitweb/static/gitweb.css.orig
    cd /tmp
    git clone git://github.com/kogakure/gitweb-theme.git
    cd gitweb-theme
    sudo cp gitweb.css gitweb.js /usr/share/gitweb/static/

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.