Back up Hudson job history in Subversion

The popular continuous build tool Hudson has a few options for those looking to backup their setups. Most people will suggest one of 2 plugins, either the SCM Sync which pushes config changes into Subversion with a log message, or the Backup Plugin which can be configured to wrap up your main config, build job configs, histories and artefacts. I wanted something that blended these 2- a way of pushing all the config and history for my jobs into Subversion.

I began to look into the source to contribute my own plugin and came across a Team Lazer Beez blog post where they have used a described a Hudson job which runs commands in the shell. It adds newly created jobs to Subversion, removes deleted ones and checks in any other changes to your config files. There were a couple of issues I had with it as it stood- mainly that it couldn’t cope with job names containing spaces, so I edited it a bit.

# Add any new conf files, jobs, users, and content.
svn add --parents *.xml jobs/* users/* userContent/* fingerprints/*

# Add the names of plugins so that we know what plugins we have.
ls -1 plugins > plugins.list
svn add -q plugins.list

# Ignore things in the root we don't care about.
echo -e "war\nlog\n*.log\n*.tmp\n*.old\n*.bak\n*.jar\n*.json" > myignores
svn propset svn:ignore -F myignores . && rm myignores

# Ignore things in jobs/* we don't care about.
echo -e "builds\nlast*\nnext*\n*.txt\n*.log\nworkspace*\ncobertura\njavadoc\nhtmlreports\nncover\ndoclinks" > myignores
svn propset svn:ignore -F myignores jobs/* && rm myignores

# Remove anything from SVN that no longer exists in Hudson.
svn st |grep "\!" |awk '{printf "\""; for (i=2;i<=NF;i++) {printf "%s%s",sep, $i;sep=" "}; printf "\"\n"}' | xargs -r svn rm

# And finally, check in of course, showing status before and after for logging.
svn st && svn ci --non-interactive -m "automated commit of Hudson configuration" --username user --password xxxxxxxxx && svn st

It is working great as a job set to run ‘@midnight’, or with a manual triggering of it after a particularly rigorous config editing.

Hopefully this will prove useful to someone else.

Advertisements

Ubuntu/Debian Apache: Hide your server details on error pages

By default, a freshly installed Apache on Debian-based distros print the server name and apache version in the footer of every error pages.

We gain nothing from displaying this information; in fact we are only making it easier for someone to find a known exploit in our version, so we will turn this off.

To turn this off for all VirtualHosts we’ll set it in the main config file, usually found at /etc/apache2/apache2.conf. At the bottom of this file, add:

ServerSignature Off

If you are still using the default config (/etc/apache2/sites-available/default) you will need to remove the line from there. Also check any other config files you might have inherited from anything else running behind apache.

Updated:

Closer inspection shows that the default install has a file in conf.d called security. This is well worth reading as it has a basic secure config ready to be dropped in, including hiding the server details.

Updated even later:

A more complete list of good ways to secure your apache, check out this great post by Anson Cheung.

Var Args in Bash: Sub-arrays using array expansion modifiers

On a recent adventure into Bash scripting, I recently wanted a function which took 2 parameters (host and port) and could be modified by any number of optional parameters.

There’s a lot of ways to do this with putting arrays or shifts, but there is also a reasonably elegant way to do it. In this example I’m using cURL, which requires hostname and port number but in some cases I may want to append other options.

function varArgExample(){
    local host=$1
    local port=$2
    curl http://$host:$port "${@:3}"
}

The magic is the “${@:3}” variable expansion at the end. It uses the well know ${@} variable, the array of all arguments passed, but it adds the little known option to say only use the array from element 3 onwards.

If only 2 varables are passed in, “${@:3}” expands out to “” (nothing, just an empty string) without any problems, acting much like varArgs in Java 6.

Should you want to stop at a certain element in the array you can also add a second option. For example,

echo "${@:2:5}"

would expand out to the values of $2, $3 and $4. Obviously, you can also start using integer variables to control the expansion and start getting into some really complicated behaviours too.

start=3; end=6
echo "${@:$start:$end}"

MeeGo: My new netbook OS

My trusty Acer Aspire One has yet again  become the test bed for a new Linux distribution I want to try and this time it’s the turn of “MeeGo”, a project especially made for netbooks and smartphones that is getting a fair bit of buzz around itself.

MeeGo is focused on online content and despite some installation issues (a few notes on that below) it is a nice experience. A lot of online functionality, such as IM, twitter and email, is built into tabs on  the main UI and definitely makes me use my netbook more as it was intended- as a quick, lightly powered way of viewing online content, instead of the full Ubuntu install I’ve previously tried to use for coding and reading.

Whilst the online integration is very slick, MeeGo is currently a little light on the services it does this with. All of the main IM services are available (MeeGo uses Empathy for IM) and email is taken for granted, MeeGo only really supports Twitter and Last.fm in any especially meaningful way. I’m sure this will expand, though it may be a long wait with MeeGo subscribing to the model of putting out new versions every 6 months.

One thing that seems mostly finished already is the UI design. MeeGo certainly has a strong identity: it is colourful and cartoony but not over-the-top. I really like the design- playful but functional and relatively minimal. The main UI is based on a customisable row of icons which open tabs into the integration functions, such as a slick settings tab, application menu and a number of tabs for using online services.

The “main” (well, first) tab is the MyZone homepage which aggregates online events from Twitter and Last.fm into a wall of what you’re friends are up to and displays this along with your calendar appointments, tasks, email notifications and quck launch icons for your favourite apps. However, straying off the main tabs into software you may have installed (like Skype), or even the default media client, see you dropped off the crisp and slick design and into a plain, grey application window. This could (and probably will) be solved with some nicer looking frames but for now it just looks a bit ugly, especially in contrast to how nice everything else looks.

All in all, I really quite like MeeGo and I won’t be rushing back to Ubuntu netbook remix any time soon; not because Ubuntu NBR is bad (far from it!) but more because MeeGo is a bit faster and really provides the features best suited to the device in a useful way. It’s worth trying if you have a slightly neglected netbook and I’m sure the next couple of versions will see it really come into its own, especially as more and more device manufacturers (including Nokia, so far behind in the device OS stakes that its not even funny) are announcing MeeGo as the OS for their next waves of devices.

Installation Notes

If I’ve (hopefully) inspired you to give MeeGo a try, here are a couple of things I got caught up on:

  1. The MeeGo netbook binary image comes as a .img file, as opposed to .iso which must be copied byte-wise to a USB stick. I usually use the awesome unetbootin to make bootable USB sticks but it is not capable of doing this with an img file…despite not explicitly warning you of this and even appearing to succeed. To make a working bootable usb stick from the img file, use either dd on Linux or Win32diskimager on Windows.
  2. There is a known issue with setting DNS servers in MeeGo on some hardware. This meant I could connect to WiFi networks but the OS had no idea where to go for DNS stuff, so it constantly timed out when trying to view pages and sign in to accounts. A fix is incomming, but in the mean time you can use the terminal to add the DNS server address to /etc/resolv.conf manually. There is a useful thread on the MeeGo forums here.
  3. A must for any mobile device I wield is the ability to use Skype. Again, the MeeGo forum has a quick tutorial for anyone not used to installing software via Yum.

Quick Tip: Lines of Code in sub-folders

Just a quick one- I searched around for a bit longer than I wanted to find out how to find out the number of lines in certain file types in a folder.

The solution was:

find -type f \( -iname "*.java" -o -iname "*.gradle" \) -exec wc -l {} \; | awk '{lines += $1 ; files += 1 ; print  }; END { print "Lines total is: ", lines ," in ", files ," files"}'

As you can see, its a find for all files with names matching the *.java and *.gradle regexs, then a little awk magic to neaten up the out put and display totals at the end.