SysAdmin

Lately I faced a new challenge. To fully migrate our SVN environment to GIT. After reading a little bit on internet I come up with this plan:

1. Deploy a GIT server
2. Create a new bare repository
3. Make a SVN Clone on a different machine
4. Push it on GIT server

Assumptions:

SVN Repository: https://192.168.0.5/svn/documentation

GIT Server: 192.168.0.22

Client: A Microsoft Windows machine with some IP from 192.168.0.1/24

Prerequisite packages for our client: TortoiseGit  and msysgit. For this tutorial I used: TortoiseGit-1.8.4.0-64bit.msi and msysGit-fullinstall-1.8.3-preview20130601.exe

Add C:\msysgit\msysgit\cmd to the %PATH% (I am not going to cover this here, but you can check this website: http://www.computerhope.com/issues/ch000549.htm)

1. Deploy GIT/GITOLINE/GITWEB

Server side - I assume we have a fresh debian/ubuntu installation

1.1. Install required software

apt-get install vim git

1.2. Create user git with disabled password and login shell bash

adduser \
 --system \
 --shell /bin/bash \
 --gecos 'git version control' \
 --group \
 --disabled-password \
 --home /home/git \
 git

[/codesyntax]

su -l git

[codesyntax lang="bash"]

cd /home/git; git clone git://github.com/sitaramc/gitolite
mkdir $HOME/bin
gitolite/install -ln

[/codesyntax]

Client side

1.5. Generate RSA key (Create RSA key, after hiting the command just press enter when it ask for passphrase.). This key will be transferred to git server.

[codesyntax lang="bash"]

C:\Users\joe>C:\msysgit\msysgit\bin\ssh-keygen -t rsa -C "Git-Admin"
Generating public/private rsa key pair.
Enter file in which to save the key (//.ssh/id_rsa): c:\users\joe\.ssh\id_rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in c:\users\joe\.ssh\id_rsa.
Your public key has been saved in c:\users\joe\.ssh\id_rsa.pub.
The key fingerprint is:
aa:38:be:f9:3c:d0:87:fe:de:5a:a9:71:1d:d3:38:01 Git-Admin

[/codesyntax]

1.6. From client scp the key into git server.

[codesyntax lang="bash"]

C:\msysgit\msysgit\bin\scp C:\Users\joe\.ssh\id_rsa.pub root@192.168.0.22:~

[/codesyntax]

Server side

1.7. Now setup the Git Admin. Please note that we must be logged in with git user. To check this use whoami command

[codesyntax lang="bash"]

whoami
sudo mv /root/id_rsa.pub /home/git/Git-Admin.pub; sudo chown git:git /home/git/Git-Admin.pub
bin/gitolite setup -pk Git-Admin.pub

[/codesyntax]

1.8. Git user should be in the same group as your Apache user

[codesyntax lang="bash"]

sudo usermod -a -G www-data git
sudo chmod ug+rx /home/git/repositories

[/codesyntax]

Client side

1.9. Clone gitolite-admin repository. Now gitolite-admin directory will be present after cloneing from git server. The gitolite.conf is mainly use for User and group ACL for git server. Now from your system you can easily manage the user group ACL. But for this you have to do git push.

[codesyntax lang="bash"]

git clone git@192.168.0.22:gitolite-admin.git
cat gitolite-admin/conf/gitolite.conf

[/codesyntax]

repo gitolite-admin

Server side

1.10. Edit the /home/git/.gitolite.rc and change the UMASK to 0002 and GIT_CONFIG_KEYS to '.*'

[codesyntax lang="bash"]

vim /home/git/.gitolite.rc
:%s/0077/0002/
:%s/'',/'.*',/
:wq

[/codesyntax]

1.11. Install gitweb

[codesyntax lang="bash"]

sudo apt-get install gitweb

[/codesyntax]

1.12. Configure gitweb. Edit /usr/lib/cgi-bin/gitweb.cgi and change the value of $projectroot as "/home/git/repositories" and $projects_list as "/home/git/projects.list"

[codesyntax lang="bash"]

sudo vim /usr/lib/cgi-bin/gitweb.cgi
:%s/our $projectroot = "\/pub\/git";/our $projectroot = "\/home\/git\/repositories\/";/
:%s/our $projects_list = "";/our $projects_list = "\/home\/git\/projects.list";/
:wq

[/codesyntax]

[codesyntax lang="bash"]

sudo vim /etc/gitweb.conf
:%s/$projectroot = "\/var\/cache\/git";/$projectroot = "\/home\/git\/repositories\/";/
:%s/$projects_list = $projectroot;/$projects_list = "\/home\/git\/projects.list";/
:wq

[/codesyntax]

Note: if the $projects_list is commented, uncomment it (remove the # from the begining of the line)

2. Migrate from SVN to GIT

Server side

2.1. Create a new git bare repository

[codesyntax lang="bash"]

cd /home/git/repositories/
git init --bare documentation.git

[/codesyntax]

Client side

2.2. Getting the author information.

[codesyntax lang="bash"]

svn log --xml | grep -P "^<author" | sort -u | perl -pe 's/<author>(.*?)<\/author>/$1 = /' > users.txt

[/codesyntax]

That gives you the log output in XML format - you can look for the authors, create a unique list, and then strip out the XML. (Obviously this only works on a machine with grep, sort, and perl installed.) Then, redirect that output into your users.txt file so you can add the equivalent Git user data next to each entry. The users.txt file should look like:

(no author) = Alex <alex@example.com>
bear = Bear <kong@example.com>
monika = Monika <cristina@example.com>
igor = Igor <igor@example.com>
joe = Joe <joe@example.com>

You must execute this command on SVN repository folder. If you are using Windows, then the previous command will not work, so I suggest the following approach.

* Generate the file (svn_users.txt), change it as you like and trasfer it to a linux box (I use GIT server for this)

* Connect to GIT server and process the file there using:

[codesyntax lang="bash"]

cat svn_users.txt | grep -P "^<author" | sort -u | perl -pe 's/<author>(.*?)<\/author>/$1 = /' > users.txt
rm svn_users.txt

[/codesyntax]

2.3. Add the new repo that we want to migrate to gitolite-admin\conf\gitolite.conf. The file should look something like this:

At this point all users will have RW access to our git repository.

2.4. Add new users to GIT. For each user you will need to have to a pair of public/private keys. Copy public keys to keydir folder in gitolite-admin clone, add and push the changes.

2.5. Import SVN repository. This is going to take a while, so go and grab a snack or something.

[codesyntax lang="bash"]

git svn clone https://192.168.0.5/svn/documentation --authors-file=users.txt --no-metadata

[/codesyntax]

2.6. The last thing to do is add the Git server as a remote and push to it.

[codesyntax lang="bash"]

git remote add origin git@192.168.0.22:documentation.git
git push origin --all
git push origin --tags

[/codesyntax]

3. Delete the imported SVN repository on client side and re-import it from GIT server

Server side

4. Securing Gitweb with htpasswd. At this point the Gitweb web page is accessible to all, which of course is not a good idea. So, if you would like to have a set of web pages that are protected, requiring a username/password to gain access, you will have to use htpasswd in apache and add users and passwords.

4.1. Configure htpasswd access in Gitweb

sudo vim /etc/apache2/conf.d/gitweb

sudo touch /etc/apache2/gitweb-htpasswd
sudo htpasswd -m /etc/apache2/gitweb-htpasswd user1
sudo htpasswd -m /etc/apache2/gitweb-htpasswd user2
sudo htpasswd -m /etc/apache2/gitweb-htpasswd user3

[/codesyntax]

sudo /etc/init.d/apache2 restart

Note: You can use TortoiseGIT for doing all git operations on Windows Machine, of course. I was lazy and I didn't take screenshots with steps that you should do with TortoiseGIT :)

Links:

• http://git-scm.com/book/en/Git-and-Other-Systems-Migrating-to-Git
• http://sharadchhetri.com/2013/05/31/how-to-create-own-git-server-with-gitolite-and-gitweb-in-ubuntu/
• http://www.twibble.org/2012/11/moving-from-subversion-to-git-with-gitolite/
• http://gitolite.com/gitolite/users.html