Create an SVN (Subversion) Server on Linux (Debian / Ubuntu – Redhat / CentOS)

I use SVN a lot – for my personal knowledgebase that I maintain for myself, for my development projects, server configurations, and for my documents (oddly enough). I think it’s a little more thorough than just using a piece of backup software in that there is revision control – and that is the most important thing to me with some of these documents and development bits. For some reason I prefer SVN over git – I think it’s a little more mature and has had bit more time to develop – no doubt that git is a good platform, I just tend to prefer SVN.

In any event, I’ve found softwares for Windows that will run an SVN server, but I think it requires a lot less work to set up on Linux and it runs far more efficiently. Anyway, I’ll detail about how to get that set up and running here.

Let’s start with prerequisites (as usual): You’ll need a box (either virtual or not) with as much space as you think you’ll need. I would suggest at least 8 gigs if it’s a VM – and depending on the server that you use (if you’re using a full out CentOS 6.3 install with GUI, it might be prudent to double that to 16GB) – just make sure it’s practical. It does not need to be a dedicated server – at least not for this instance – we will be using a relatively obscure port dedicated to SVN (3690).

And as usual, you’ll need some form of root.

Alright – here we go.

First you’ll need to install really only one thing – and that is subversion.

Debian / Ubuntu

$ sudo apt-get install subversion

Redhat / CentOS – as root:

# yum install subversion

Once subversion is installed, now we need to create the actual repositories. I generally tend to make a directory solely for this purpose – usually in /var.

# mkdir /var/svn

Once we’ve create the main directory that will house our repositories, we need to go ahead and create the actual repositories. To do that, we issue this command:

# cd /var/svn
# svnadmin create repo_name

After we’ve created the repository, we need to make it accessible. I’m usually not too concerned about this being ultra private – generally I make these local only – as in available to the LAN only – so normally I just will create a password for the repository.

All the configuration files are held within the repository now – so lets say you did this:

# svnadmin create /var/svn/foo

Within the /var/svn/blah folder, there are now going to be a folder we are concerned with, and that is a folder called “conf”.

In the conf folder (which should be at /var/svn/foo/conf). In that folder are two files we are concerned with – one is called “passwd” and the other is “svnserve.conf”.

Let’s first edit “passwd”. The “passwd” file looks like this:

### This file is an example password file for svnserve.
### Its format is similar to that of svnserve.conf. As shown in the
### example below it contains one section labelled [users].
### The name and password for each user follow, one account per line.
 
[users]
# harry = harryssecret
# sally = sallyssecret

As you can see, a username and a password commented out. If you want to create a username and password for yourself and for other users, add some in here.

[users]
foo = bar
luke = skywalker
frodo = baggins
# harry = harryssecret
# sally = sallyssecret

Now we have some users set up.

Next we need to tell the SVN server that we want it to authenticate users, we need to edit “svnserve.conf”.

There are a few lines in here that are concerned with:

One is “# anon access = read”.
Another is “# auth-access = write”.
Another is “# password-db = passwd”.

The first, anon access = read means that anyone can read the repository, regardless of authentication. Meaning anyone can “read” your repository. They can’t edit it, but they can read it. In some cases, this is a big no no. Especially if you have sensitive config files that contain database passwords. You can change it to this:

anon access = none

This will make it so that they have to have a password to even read the repository.

The second is “auth access”. This means authorized access. I tend to keep this on “write”. If you have a username and password, you are probably actively working on the project.

And last, uncomment “password-db = passwd”. This just means that it will use the “passwd” file we previously set up to authenticate users.

Alright – now the repository is set up, the users are up and running. Make sure port 3960 is open.

Last – we need to edit / create a few files to make a service out of this. On CentOS / Redhat – this is already done, but on Ubuntu and Debian, this is not done.

For Debian / Ubuntu, I wrote a little start up script that can be stuck in init.d.

Do this:

# sudo nano /etc/init.d/svnserve

And paste this in (changing your SVN root directory of course):

#!/bin/sh
#
# start/stop subversion daemon.
#
# Let's set some variables first
# If we want more than one root svn path, we can set it here
#
SVN_ROOT_PATH="/var/svn/foo"
SVN_USER="root"
SVNSERVE="/usr/bin/svnserve"
#
# Let's make sure SVN is actually installed
#
test -f $SVNSERVE || 'echo "svnserve not found at $SVNSERVE"; exit 0'
#
# If we want more than one root path, we can set it here
#
OPTIONS="-d -r $SVN_ROOT_PATH"
#OPTIONS="-d -r $SVN_ROOT_PATH2"
#
# Alright - let's start the daemon
#
case "$1" in

start)
        echo "Starting subversion daemon: svnserve."
        start-stop-daemon -S -o -q -u $SVN_USER -c $SVN_USER -v -x $SVNSERVE -- $OPTIONS

;;

stop) 
        echo "Stopping subversion daemon: svnserve."
        start-stop-daemon -K -q -o -x $SVNSERVE

;;

reload) 

;;

force-reload)
        $0 restart
        
;;


restart)
        $0 stop
        $0 start

;;

*)
	echo "Usage: /etc/init.d/svnserve {start|stop|reload|restart}"
	exit 1
	
;;

esac

exit

Save and close. Now the SVN daemon starts on boot.

If we want to connect to your SVN repo, there are some handy utilities – For Windows you can use TortoiseSVN or Eclipse, for Linux you can use subversion on the command line (I’ll write another post here in the near future) or Eclipse as well.

Set service to start on boot in Linux

In Linux, sometimes after updating, services will no longer start on boot – or their init level goes back to 0. This just means the service will no longer start at boot (or on login, or anything else other than if you manually ran service whatever start)

To fix this, we can just check the config level and then add it to runlevels 2, 3, 4, 5 – which means it will start on boot. 5 just gives it access to X, so if it’s an app with a GUI, it will HAVE to be on level 5 too.

If you want to read more about runlevels, read here.

$ chkconfig --list sshd

In CentOS it will usually come back with something like this:

sshd            0:off   1:off   2:on    3:on    4:on    5:on    6:off

So you see ssh is set to start on boot. What if it returns back something like this?

sshd            0:off   1:off   2:off    3:off    4:off    5:off    6:off

Well, then we need to fix it so it starts on boot. That can be done like this (as root):

$ chkconfig --level 2345 sshd on

That will restore it or put it at all those run levels – so if you reboot, it will start on boot.

How to set up a Local CentOS Repository

If you have a lot of CentOS servers in high bandwidth environment and you have some spare room on your servers (you’ll want about 80 gigs, to be on the safe side but it really depends on how many distros and editions you are running). On the example I’ll be running you through here, I’m going to create a repository for CentOS 5.8 and 6.3. CentOS 5.8 receives end of life updates in 2014, so you’ve got two years or so before it hits that – the company I work for currently has far more 5.8 boxes than 6.3, but I am in the process of converting them over.

In any event, here is what you need:

A spare server – either virtual or physical with at least a gig of RAM, and at the very least 40 gigs of HDD space (if you are only serving 1 release). Two releases as of this time is about 39 gigs. You’ll definitely need root on your server as well. You’ll definitely want a good internet connection (gigabit is probably preferable – because on your first sync it will download the entire repository).

Let’s get started shall we?

First off, I’m going to assume you have an up to date CentOS 6.3 (or 5.8, both will function the same) box ready to go. That is where we will start. Fire up a terminal and become root and then do this:

$ yum install httpd

Apache is how we will be serving our updates to our other boxes. Once Apache is installed, you should make sure it’s working by visiting http://localhost – it should come up with the default Apache / CentOS site.

Next you need to create some directories in your web directory (/var/www/html). It depends on what you all want from your repository as well. You can do everything in the main repo – which is CentOS-Plus, Extras, Main (OS), and Updates. The repository will not work without main and updates. I’m going to do all four.

# mkdir /var/www/html/centos && cd /var/www/html/centos
# mkdir -pv 6/centosplus 6/os 6/updates 6/extras 6/contrib 6.3/centosplus 6.3/os 6.3/updates 6.3/extras 5/centosplus 6.3/contrib 5/os 5/updates 5/extras 5/centosplus 5/contrib 5.8/centosplus 5.8/os 5.8/updates 5.8/extras 5.8/contrib

Alright, now your directories are in place. Now comes syncing the repository. If you are on a reasonable internet connection, it should take anywhere from a half hour to a couple hours. If you are on a 1.5Mbs line… you best wait until tomorrow before you even bother looking.

I’ll share a script I wrote that syncs it all. It’s somewhat variable and you can change it as you need. If you want the log to work as well, where it is in my script, you also need to do this:

$ mkdir /var/log/repo
touch /var/log/repo/sync.log

We will add this to our log rotation as well here at the end.

Here is the script that I wrote. I use Gigenet because they are the closest mirror to me and they also happen to have gigabit.

#!/bin/sh
#
# Set some environment variables first
#
RSYNC="/usr/bin/rsync"
MIRROR="rsync://mirrors.gigenet.com/centos"
EXCLUDE="--exclude=debug"
REPO="/var/www/html/centos"
LOG="/var/log/repo/sync.log"
#
set $(date)
# 
echo "Syncing Updates to Local Repo on $(date)" >> $LOG
#
$RSYNC -avrt $MIRROR/6.3/updates/x86_64 $EXCLUDE $REPO/6.3/updates/ >> $LOG
$RSYNC -avrt $MIRROR/6/updates/x86_64 $EXCLUDE $REPO/6/updates/ >> $LOG
$RSYNC -avrt $MIRROR/5.8/updates/x86_64 $EXCLUDE $REPO/5.8/updates/ >> $LOG
$RSYNC -avrt $MIRROR/5/updates/x86_64 $EXCLUDE $REPO/5/updates/ >> $LOG
#
echo "Updates Processed" >> $LOG
echo "  " >> $LOG
echo "Syncing Base System" >> $LOG
#
$RSYNC -avrt $MIRROR/6.3/os/x86_64 $EXCLUDE $REPO/6.3/os/ >> $LOG
$RSYNC -avrt $MIRROR/6/os/x86_64 $EXCLUDE $REPO/6/os/ >> $LOG
$RSYNC -avrt $MIRROR/5.8/os/x86_64 $EXCLUDE $REPO/5.8/os/ >> $LOG
$RSYNC -avrt $MIRROR/5/os/x86_64 $EXCLUDE $REPO/5/os/ >> $LOG
#
echo "Base System Processed" >> $LOG
echo "  " >> $LOG
echo "Syncing Extras" >> $LOG
#
$RSYNC -avrt $MIRROR/6.3/extras/x86_64 $EXCLUDE $REPO/6.3/extras/ >> $LOG
$RSYNC -avrt $MIRROR/6/extras/x86_64 $EXCLUDE $REPO/6/extras/ >> $LOG
$RSYNC -avrt $MIRROR/5.8/extras/x86_64 $EXCLUDE $REPO/5.8/extras/ >> $LOG
$RSYNC -avrt $MIRROR/5/extras/x86_64 $EXCLUDE $REPO/5/extras/ >> $LOG
#
echo "Syncing Extras Complete" >> $LOG
echo "  " >> $LOG
echo "Syncing CentOS Plus" >> $LOG
#
$RSYNC -avrt $MIRROR/6.3/centosplus/x86_64 $EXCLUDE $REPO/6.3/centosplus/ >> $LOG
$RSYNC -avrt $MIRROR/6/centosplus/x86_64 $EXCLUDE $REPO/6/centosplus/ >> $LOG
$RSYNC -avrt $MIRROR/5.8/centosplus/x86_64 $EXCLUDE $REPO/5.8/centosplus/ >> $LOG
$RSYNC -avrt $MIRROR/5/centosplus/x86_64 $EXCLUDE $REPO/5/centosplus/ >> $LOG
#
echo "Extras Processed" >> $LOG
echo "  " >> $LOG
echo "Processing Contrib" >> $LOG
#
$RSYNC -avrt $MIRROR/6.3/contrib/x86_64 $EXCLUDE $REPO/6.3/contrib/ >> $LOG
$RSYNC -avrt $MIRROR/6/contrib/x86_64 $EXCLUDE $REPO/6/contrib/ >> $LOG
$RSYNC -avrt $MIRROR/5.8/contrib/x86_64 $EXCLUDE $REPO/5.8/contrib/ >> $LOG
$RSYNC -avrt $MIRROR/5/contrib/x86_64 $EXCLUDE $REPO/5/contrib/ >> $LOG
#
echo "Contrib Processed" >> $LOG
echo "  " >> $LOG
echo "Finshished Syncing at $(date)" >> $LOG

If you want to use a different mirror, you can find more at CentOS’s Mirror List. Just be sure it has RSYNC capabilities, or this script will fail.

Create this script as a document called something like “syncrepos.sh” and save it to your Desktop for now.

$ chmod +x syncrepos.sh

And now for the initial sync.

$ ./syncrepo.sh

While we are waiting, it might be a good idea to add your log to the log rotation – these logs can get slightly large because I log everything. If you don’t care about logs, just delete the “>> $LOG” after every line. If not, do this:

$ nano /etc/logrotate.d/repo

And enter this in to nano:

/var/log/repo/*log {
    missingok
    weekly
    rotate 5
    compress
    postrotate
    endscript
}

Once that is done, save it with CTRL + X and call it something like “repo” or “repo-sync”.

Next, make sure your server either has a FQDN or a static IP. For this example, I’ll use the example of 10.0.0.25 as the static IP. Now, to test it, we can stay on this same box and edit your /etc/yum.repos.d/CentOS-Base.repo

$ nano /etc/yum.repos.d/CentOS-Base.repo

Comment out (by putting a “#” sign in front of it) the line that says:

mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os

And uncomment (remove the “#” sign) the line says:

baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/gpgcheck=1

Change baseurl to this:

baseurl=http://10.0.0.25/centos/$releasever/os/$basearch/gpgcheck=1

You need to do this to ALL of the repositories you want – that means if you want updates, do the same to the updates line, if you want Extras, do the same to Extras, if you want CentOS-Plus, do the same to that as well, and if you want Contrib, do the same as well. And make sure if you want them enabled that “Enabled = 1”

Hit CTRL + X in nano to save your changes and then run:

$ yum update

If all goes well, you can now do this to all of your other servers. Since this a direct sync of a legit mirror of CentOS, GPG checks will not be a problem.

It may be a good idea to add that script to cron so it runs daily – so any new updates that are pushed out are pulled by rsync once a day or so.

And that’s that.

Updated Kernel for CentOS

I use a lot of CentOS lately (one of my favorite distros is a CentOS derivative called Stella) for both server applications and practical desktops. I guess truth be told I don’t mind Gnome 3 that much. But it certainly isn’t my favorite. I like Mate quite a bit, but I guess for the time being, I’m just going to hold on to those rock solid distros that run Gnome 2.

Anyway, my feelings aside, a lot of companies use CentOS as servers. My issue has been that their kernel is still at 2.6. I don’t have a problem with this usually, but in the case of a modern desktop or server, sometimes drivers are missing that could be provided by a newer kernel. Cue this thread.

I use a repo called ElRepo from ElRepo.org – it provides up to date kernels from kernel.org (the latest stable tree).

Just follow the instructions on that page and you’ll be all good.

Unix / Linux Directory Structure

A lot of times people who are new with Linux / Unix ask all sorts of questions about the file system structure / directory structure. If you used a Windows Based computer for your whole life, moving to Linux can  be a little intimidating because it is radically different. So I figured I will start with the basics and go with how the file system is actually structured. This is a little image from LinuxConfig.org and it should be enough to help you understand what / where binaries / files go and why.

 

Linux Directory Structure / File Tree

Using Libreoffice to Batch Convert Documents to PDF

Believe it or not, LibreOffice actually has a really handy tool built in to it for
converting files to a different file types (including .PDF, which is awesome).

The tool itself is in /usr/bin and it’s called lowriter.

The command to convert documents through Linux Terminal is

$ lowriter --convert-to "File Type" *."FileType"

We could do this for example:

$ lowriter --convert-to pdf *.doc

and that would convert any .doc files in a directory to .pdf.

To move them, you can do an -outdir at the end to put them in a different directory.

$ lowriter --convert-to pdf *.doc -outdir /directory/to/path

If you have only one or two documents you want converted it might just be easier to open them in Libreoffice and then “Save As” though. I’ve found this particular method works best for batch though and really is the easiest (and cheapest) way to convert documents to different formats.

How to install an .rpm from the terminal

Installing a .rpm package from the terminal is a little different than installing a similar package as .deb in a different distro.

There are a few different commands, all of which start with “rpm”. This command will install a package:

$ sudo rpm -ivh packagename.rpm

This will “upgrade” a package:

$ sudo rpm -Uvh packagename.rpm

And if you want to find out what dependencies an .rpm has, you can run this against your .rpm:

$ sudo rpm -qpR packagename.rpm

It’s fairly easy and not complicated, but it’s still a question I see / hear a lot.

How to install a .deb from the terminal

I tend to use the terminal more than most people I think. When I was very new at Linux, I hated the terminal – I thought it was really dumb to have such an advanced operating system, like Linux, use something little more than a command line. I’ve since revised my view on that and I think it would absolutely stupid not to have one and I spend far more time in a Linux terminal than I do with the GUI.

What’s nice about the terminal is that it doesn’t change much, unlike the GUIs. Anyway, there are a few ways to install .deb packages from the terminal. One that will resolve dependencies and one that won’t.

Dependencies, in Linux, are other packages required by the package you are trying to install. Similarly, in Windows, if you were trying to install a game, a dependency might be DirectX, or if you have an app written in .net, you’d need the .net framework.

In any event, the first way (and probably the more common way) is dpkg. To do this in the terminal you simply cd to where the directory is that houses your .deb file and run this:

$ sudo dpkg -i yourpackage.deb

The only problem with dpkg is that it doesn’t offer any dependency resolution. So if your package fails to install because of dependency issues, you’ll have to run this:

$ sudo apt-get -f install

Which will resolve any unmet dependencies (if those dependencies are in your repositories) as well as install the package you were trying to install. For kernel packages and other other packages that require no dependencies, this is the way to go.

For packages that require dependencies, I tend to use gdebi. It’s basically the same syntax, except like this:

$ sudo gdebi yourpackage.deb

Fairly straight forward. If your dependencies are not in the repository, you’ll have to manually start finding the packages and installing them as they need to be installed and it’s basically going to be a chain of installs.

Cron Daemon Email Spam

I had a Cron Job on one of our servers here just start spamming my email the week. It was basically a cron I set up every half hour to rotate some fast moving apache logs. Anyway, long story short, this cron just started emailing me every 30 minutes every time the job ran.

The easiest thing to do for a situation like this, is just to add to add > /dev/null 2>&1 to the end of your file in crontab (cron.d, cron.weekly, whatever).

5 0 * * * root /scripts/backup_test > /dev/null 2>&1

Of course, the better thing to do would be to find any issues in your email as it usually prints the output of the job. But in some cases, I don’t care. I set this script up to email me when it finishes successfully and if it doesn’t run, I get another email.