How to set up a Local CentOS Repository

If you have a lot of CentOS servers in high bandwidth environment and you have some spare room on your servers (you’ll want about 80 gigs, to be on the safe side but it really depends on how many distros and editions you are running). On the example I’ll be running you through here, I’m going to create a repository for CentOS 5.8 and 6.3. CentOS 5.8 receives end of life updates in 2014, so you’ve got two years or so before it hits that – the company I work for currently has far more 5.8 boxes than 6.3, but I am in the process of converting them over.

In any event, here is what you need:

A spare server – either virtual or physical with at least a gig of RAM, and at the very least 40 gigs of HDD space (if you are only serving 1 release). Two releases as of this time is about 39 gigs. You’ll definitely need root on your server as well. You’ll definitely want a good internet connection (gigabit is probably preferable – because on your first sync it will download the entire repository).

Let’s get started shall we?

First off, I’m going to assume you have an up to date CentOS 6.3 (or 5.8, both will function the same) box ready to go. That is where we will start. Fire up a terminal and become root and then do this:

$ yum install httpd

Apache is how we will be serving our updates to our other boxes. Once Apache is installed, you should make sure it’s working by visiting http://localhost – it should come up with the default Apache / CentOS site.

Next you need to create some directories in your web directory (/var/www/html). It depends on what you all want from your repository as well. You can do everything in the main repo – which is CentOS-Plus, Extras, Main (OS), and Updates. The repository will not work without main and updates. I’m going to do all four.

# mkdir /var/www/html/centos && cd /var/www/html/centos
# mkdir -pv 6/centosplus 6/os 6/updates 6/extras 6/contrib 6.3/centosplus 6.3/os 6.3/updates 6.3/extras 5/centosplus 6.3/contrib 5/os 5/updates 5/extras 5/centosplus 5/contrib 5.8/centosplus 5.8/os 5.8/updates 5.8/extras 5.8/contrib

Alright, now your directories are in place. Now comes syncing the repository. If you are on a reasonable internet connection, it should take anywhere from a half hour to a couple hours. If you are on a 1.5Mbs line… you best wait until tomorrow before you even bother looking.

I’ll share a script I wrote that syncs it all. It’s somewhat variable and you can change it as you need. If you want the log to work as well, where it is in my script, you also need to do this:

$ mkdir /var/log/repo
touch /var/log/repo/sync.log

We will add this to our log rotation as well here at the end.

Here is the script that I wrote. I use Gigenet because they are the closest mirror to me and they also happen to have gigabit.

#!/bin/sh
#
# Set some environment variables first
#
RSYNC="/usr/bin/rsync"
MIRROR="rsync://mirrors.gigenet.com/centos"
EXCLUDE="--exclude=debug"
REPO="/var/www/html/centos"
LOG="/var/log/repo/sync.log"
#
set $(date)
# 
echo "Syncing Updates to Local Repo on $(date)" >> $LOG
#
$RSYNC -avrt $MIRROR/6.3/updates/x86_64 $EXCLUDE $REPO/6.3/updates/ >> $LOG
$RSYNC -avrt $MIRROR/6/updates/x86_64 $EXCLUDE $REPO/6/updates/ >> $LOG
$RSYNC -avrt $MIRROR/5.8/updates/x86_64 $EXCLUDE $REPO/5.8/updates/ >> $LOG
$RSYNC -avrt $MIRROR/5/updates/x86_64 $EXCLUDE $REPO/5/updates/ >> $LOG
#
echo "Updates Processed" >> $LOG
echo "  " >> $LOG
echo "Syncing Base System" >> $LOG
#
$RSYNC -avrt $MIRROR/6.3/os/x86_64 $EXCLUDE $REPO/6.3/os/ >> $LOG
$RSYNC -avrt $MIRROR/6/os/x86_64 $EXCLUDE $REPO/6/os/ >> $LOG
$RSYNC -avrt $MIRROR/5.8/os/x86_64 $EXCLUDE $REPO/5.8/os/ >> $LOG
$RSYNC -avrt $MIRROR/5/os/x86_64 $EXCLUDE $REPO/5/os/ >> $LOG
#
echo "Base System Processed" >> $LOG
echo "  " >> $LOG
echo "Syncing Extras" >> $LOG
#
$RSYNC -avrt $MIRROR/6.3/extras/x86_64 $EXCLUDE $REPO/6.3/extras/ >> $LOG
$RSYNC -avrt $MIRROR/6/extras/x86_64 $EXCLUDE $REPO/6/extras/ >> $LOG
$RSYNC -avrt $MIRROR/5.8/extras/x86_64 $EXCLUDE $REPO/5.8/extras/ >> $LOG
$RSYNC -avrt $MIRROR/5/extras/x86_64 $EXCLUDE $REPO/5/extras/ >> $LOG
#
echo "Syncing Extras Complete" >> $LOG
echo "  " >> $LOG
echo "Syncing CentOS Plus" >> $LOG
#
$RSYNC -avrt $MIRROR/6.3/centosplus/x86_64 $EXCLUDE $REPO/6.3/centosplus/ >> $LOG
$RSYNC -avrt $MIRROR/6/centosplus/x86_64 $EXCLUDE $REPO/6/centosplus/ >> $LOG
$RSYNC -avrt $MIRROR/5.8/centosplus/x86_64 $EXCLUDE $REPO/5.8/centosplus/ >> $LOG
$RSYNC -avrt $MIRROR/5/centosplus/x86_64 $EXCLUDE $REPO/5/centosplus/ >> $LOG
#
echo "Extras Processed" >> $LOG
echo "  " >> $LOG
echo "Processing Contrib" >> $LOG
#
$RSYNC -avrt $MIRROR/6.3/contrib/x86_64 $EXCLUDE $REPO/6.3/contrib/ >> $LOG
$RSYNC -avrt $MIRROR/6/contrib/x86_64 $EXCLUDE $REPO/6/contrib/ >> $LOG
$RSYNC -avrt $MIRROR/5.8/contrib/x86_64 $EXCLUDE $REPO/5.8/contrib/ >> $LOG
$RSYNC -avrt $MIRROR/5/contrib/x86_64 $EXCLUDE $REPO/5/contrib/ >> $LOG
#
echo "Contrib Processed" >> $LOG
echo "  " >> $LOG
echo "Finshished Syncing at $(date)" >> $LOG

If you want to use a different mirror, you can find more at CentOS’s Mirror List. Just be sure it has RSYNC capabilities, or this script will fail.

Create this script as a document called something like “syncrepos.sh” and save it to your Desktop for now.

$ chmod +x syncrepos.sh

And now for the initial sync.

$ ./syncrepo.sh

While we are waiting, it might be a good idea to add your log to the log rotation – these logs can get slightly large because I log everything. If you don’t care about logs, just delete the “>> $LOG” after every line. If not, do this:

$ nano /etc/logrotate.d/repo

And enter this in to nano:

/var/log/repo/*log {
    missingok
    weekly
    rotate 5
    compress
    postrotate
    endscript
}

Once that is done, save it with CTRL + X and call it something like “repo” or “repo-sync”.

Next, make sure your server either has a FQDN or a static IP. For this example, I’ll use the example of 10.0.0.25 as the static IP. Now, to test it, we can stay on this same box and edit your /etc/yum.repos.d/CentOS-Base.repo

$ nano /etc/yum.repos.d/CentOS-Base.repo

Comment out (by putting a “#” sign in front of it) the line that says:

mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os

And uncomment (remove the “#” sign) the line says:

baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/gpgcheck=1

Change baseurl to this:

baseurl=http://10.0.0.25/centos/$releasever/os/$basearch/gpgcheck=1

You need to do this to ALL of the repositories you want – that means if you want updates, do the same to the updates line, if you want Extras, do the same to Extras, if you want CentOS-Plus, do the same to that as well, and if you want Contrib, do the same as well. And make sure if you want them enabled that “Enabled = 1”

Hit CTRL + X in nano to save your changes and then run:

$ yum update

If all goes well, you can now do this to all of your other servers. Since this a direct sync of a legit mirror of CentOS, GPG checks will not be a problem.

It may be a good idea to add that script to cron so it runs daily – so any new updates that are pushed out are pulled by rsync once a day or so.

And that’s that.