This feed contains pages in the "ubuntu" category.

Since I enabled comments in this blog, I finally needed to configure a split DNS for my network.

There are various reasons why one needs a split DNS and as it's usually pointed out, the reasons are usually non-technical. In my case the reasons are technical: I have a NAT in my local network that allows me to host this website locally. What causes problems is that the domain name points to the external IP address and that doesn't work from the inside. So split DNS it is.

There are various ways of building a split DNS, one can use the views feature in bind9 or you can set up 2 separate DNS servers that provide different information (and redirect your local resolver to use the internal server). The latter is more secure if the internal zone is sensitive.

I decided to use a hybrid solution. I already knew that PowerDNS Recursor was capable of serving authoritative zones (think pre-cached) so I decided to leverage on that. Setting this up turned out to be simpler than I expected.

First I made a copy of the existing zone and edited it to fit my needs. I changed the IP address of to point to the IP address on the local network. I also adjusted some other entries that pointed to the local network.

Next I modified bind to listen on the external IP address. This can be accomplished by adding a listen-on {; }; to the options in the configuration. I also disabled the resolver by adding recursion no;, this forces the bind to work as authoritative only.

Then I installed the PowerDNS Recursor (pdns-recursor package in debian) and configured it to listen on the internal address only (local-address= and added the pre-cached zone to the configuration with

Now, after restarting both daemons, I had a working split DNS with minimal configuration. I was also able to change the external DNS to authoritative only mode, which is a good idea in any case.

Posted Mon Mar 23 22:38:37 2009 Tags: ubuntu

For some time I've suffered from the infamous clocksource problem with all Linux hosts that aren't running the Citrix provided kernels. I'm bit old fashioned and I want to run Debian provided kernels instead the Citrix ones, mostly because the Debian kernel receives security updates.

During the fight with my own server last night, it finally dawned to me.

The clocksource problem appears after you suspend a Linux host and the kernel in the virtual machine starts spewing this:

Mar  5 09:24:17 co kernel: [461562.007153] clocksource/0: Time went backwards: ret=f03d318c7db9 delta=-200458290723043 shadow=f03d1d566f4a offset=143675d9

I've been trying to figure out what is different with Citrix and Debian kernels, because the problem doesn't occur with the Citrix provided kernel.

The final hint to solving this problem came from Debian wiki. The same issue is mentioned there, but the workaround is not something I like. I perfer making sure that the host server has the correct time and the virtual machine just follows that time.

But the real clue was the clocksource line. It turns out that the Citrix kernel uses jiffies as the clocksource per default, while Debian uses the xen clocksource. It would make sense that the xen clocksource is more accurate since it's native to the hypervisor.

So by just running this on the domU fixes the problem:

echo "jiffies"> /sys/devices/system/clocksource/clocksource0/current_clocksource

There is no need to decouple the clock from the host, which is exactly what I needed. To make this change permanent, you need to add clocksource=jiffies to the bootparameters of your domU kernel.

You can do this by modifying grub configuration and adding clocksource=jiffies to the kopt line and running update-grub. Or you can use XenCenter and modify the virtual machine parameters and clocksource=jiffies to boot parameters.

It's also worth noting that this problem does apply to plain vanilla Debian installations as well, so reading that whole wiki page is a good idea.

Posted Thu Mar 5 17:26:28 2009 Tags: ubuntu

I've been bitten by grub upgrades and installations on Debian family domU servers. Apparently there are others out there who have been bitten too.

The bug itself is caused by a missing device entry, probably because of udev. Anyway, grub-probe tries to discover the root device so that update-grub can properly generate a menu.lst. In certain scenarios the root device itself doesn't exist. Here is an example from a configuration generated with xen-tools:

Hydrogen:/etc/xen# grep phy Neon.cfg 
disk    = [ 'phy:Local1/Neon-disk,sda1,w', 'phy:Local1/Neon-swap,sda2,w' ]

While this is a valid configuration, the device sda doesn't exists within the virtual machine. As a workaround the above blog entry suggests manually adding the sda device and the device entry in

This solution does work, but it will fail with the next upgrade. The proper solution is to adjust the Xen configuration so that the root device is created. And since Xen uses different naming scheme for devices we can upgrade to that too. So the above example becomes:

Hydrogen:/etc/xen# grep phy Neon.cfg 
disk    = [ 'phy:Local1/Neon-disk,xvda,w', 'phy:Local1/Neon-swap,xvdb,w' ]

You also need to adjust the existing grub configuration and fstab within the domU. It's a bit more work and requires an additional reboot, but it gives you a peace of mind that the next upgrade will work without a hitch.

Posted Tue Feb 17 07:57:53 2009 Tags: ubuntu

As an obligatory note, Debian Lenny was released earlier today. Which means that sysadmins all over the world are starting to upgrade their servers.

There is an oddly little known tool that each and every sysadmin should install on at least one server they maintain, called apt-listchanges. It lists changes made to packages since the currently installed version. Sure that information will be overwhelming on major upgrades, but what is useful even on major upgrades is the capability to parse News files in the same way.

News files contain important information about the package in question. For example a maintainer could list known upgrade problems there, like is done in the lighttpd package. Or list changes in package specific default behaviour, like is done in Vim package.

Sure, you will notice these in time, but it's nice to get a heads up before a problem bites you.

Posted Sun Feb 15 22:35:57 2009 Tags: ubuntu

Since I keep ending up in situations where I need to clean up postfix queue from mails sent by a single host and always forget the command, I'm posting it here. Maybe someone else will find it useful as well.

To begin with, you need to determine the IP address of the culprit you want to eliminate. How you do this, is up to you. Grepping logs or examining the files in the queue both work. But for some reason there doesn't appear to be a good tool to get statistics on the sending IP addresses, only the origin and destination domains.

Once you have determined the IP address which you want to purge, you can use the following spell. You might have to repeat the same line for active and incoming queues as well, but usually deferred is the queue I have the most mails.

grep -lrE '[^0-9][^0-9]' /var/spool/postfix/deferred | xargs -r -n1 basename | postsuper -d -

It's important that the IP address has escaped dots, because dots can account for any character. In the worst case it will end up matching a lot of wrong IP addresses. Another important bits are the '[^0-9]' groups in both ends of the pattern. Those make the IP address only match that particular IP address. Without that extra limitation would match anything that has 1 as the last number in the first octet and 1 as the first number of the last octet. For example: would be a valid match.

The other important bit, yet oddly unknown, is the postsuper command. Postsuper modifies the queue and -d flag makes it delete files in the queue by QueueID. For some reason I keep on seeing all sorts of find -exec rm {} spells all over, which isn't really that nice for the daemon itself.

So here it is, one more tidbit I've been meaning to write up for quite some time now. Enjoy!

Posted Sun Jan 25 11:23:36 2009 Tags: ubuntu

"You never call, you never write. I hardly know you anymore."

Yes, I've been meaning to write up on several things. For some time now, I've been a happy VIM user and a while back I ran in to a blog post where someone mentioned a new feature they found in VIM which got me to explore the vim-scripts package.

There are a lot of scripts out there that extend VIM far beyond what it can do by default. And it's quite powerful even without the scripts. One of the neat little scripts I decided to install by default was surround, it allows one to easily replace surrounding parenthesis, tags or quotation marks.

There are a lot of scripts in the vim-scripts package, but it's not always clear how to enable the scripts. Thats where vim-addon-manager comes to play, it provides a vim-addon command that allows you to easily enable or disable the scripts.

I'm still trying to grasp the full potential of all the new commands available, but it certainly appears that I'll be having even more fun writing stuff. It's kind of odd, at first when you start to use vi-like editors, you struggle. But in the end it's just such a convenient way of editing files that it really does grow on you.

Posted Sat Jan 24 19:50:34 2009 Tags: ubuntu

Some people consider the gnome usability guidelines a nuisance and some consider certain applications way too simplistic. While it is really hard to get the usability right, it's well worth it.

We need to keep in mind that as computer oriented people we tend to see things differently. Things that are simple to us aren't really that simple to the "normal people". But one of the simple things we can do the insure that the software we write serve the people it's designed for is to remove all not needed pop-ups and questions.

A good way to detect these would be to ask yourself why would a user choose anything else but the most logical option. It's kind of hard to explain, so lets pick an example:

Firefox is updated, the first question it usually asks after upgrade is about incompatible extensions. The user is presented with choices, check for updates or cancel. Now, we are dealing with internet browser so the user should be connected to the internet so there is no problem with checking for the updates, we can rule out that scenario. The other scenario that I can come up with is that a developer doesn't want to update some certain extension.

So at least I can't come up with any reasonable scenario why someone would want to select anything else but to upgrade the main option of upgrading the extensions. Why not just leave out the option and instead do the upgrade automatically. If you wish to be transparent, you can show the user that you are doing the upgrade. Or you can do like some applications do and just do it without bothering the user with the options.

I know I sound like a Google fanboy, but Google generally gets this right. Their applications skip out all upgrade related notices and just do the upgrade. Regular user doesn't want to upgrade because the user has been scared with incompatibility notices and upgrade checklists for so long. Just going ahead with the upgrade in complete silence keeps their software up to date as well.

Another example would be from few years back: The Ubuntu installation. Back in the day Debian was working on Debian Installer, which is also used as the main installer in Ubuntu alternative installation media. Debian Installer is capable of doing most things silently, but with Debian it by default asks a lot of questions. It doesn't matter, since most people who install Debian can be categorized as developers. But in my opinion, the thing that made Ubuntu a success was that it doesn't ask the questions that can be answered without asking the user.

So, back to usability. There are basically 2 camps, the "normal" users and the developers. Developers want and need to see a lot of the backend behaviour, just to debug problems. Currently a lot of the open source software is focused towards developers while they are gaining grounds on the "normal" population as well. We should start focusing on the users for a change.

Posted Sun Jan 4 13:48:56 2009 Tags: ubuntu

Recently I fixed my OpenVPN tunnel and figured out what was wrong with my bridge setup. Here is the complete setup that I have.


I wanted to have as much of the configuration on the server as possible so that I could easily add more clients and wouldn't have a need to update client configuration when ever the server preferences change.

Here is my OpenVPN server configuration:

mode server
dev tap0
ifconfig-pool-persist /var/run/openvpn-ip.txt
keepalive 10 60
push "route"
push "dhcp-option DNS"
ca /root/easy-rsa-2.0/keys/ca.crt
key /root/easy-rsa-2.0/keys/
cert /root/easy-rsa-2.0/keys/
dh /root/easy-rsa-2.0/keys/dh1024.pem
up /etc/openvpn/

and the up script:

# Bind the tunnel interface to the bridge


ifup $1
brctl addif $BRIDGE $1

There is nothing really special about that configuration. The server is in TLS mode configures a bridge. The keys are generated with easy-rsa by following a openvpn howto entry. The up script just binds the tap0 device to the network bridge after bringing up the device.

Next I created the interface configuration by adding the following to /etc/network/interfaces:

iface br0 inet static  
 bridge-ports eth0

The trick here is to create a single bridge with just the eth0 device. We use the up script for openvpn to add the tunnel device to the bridge. Otherwise the bridge would never contain the proper devices.


As for the client, you simply set the client to use the CA-certificate and Host key created with easy-rsa, set the hostname and tunnel type. Tunnel type is assumed to be a tun-device instead of tap, so in my case I needed to change it too.

There is no need to tell the client anything else. Everything else will be negotiated through the tunnel. And since I use NetworkManager to set up my tunnels I didn't have a need to drop in to a shell even once at the client.

Posted Thu Aug 21 01:45:59 2008 Tags: ubuntu

After my post yesterday about locales and my problems, I got some comments.

Bryan, Simon and Wouter all pointed out what eventually figured out too but didn't mention in the post (I actually had to re-read the post to see what I wrote) that LC_ALL overrides all of the settings. Even though I figured it out eventually, with these comments it finally me that the purpose is actually to temporarily override the locale.

Another thing that didn't occur to me while figuring out the correct locale for my system was that I'm thinking about it all wrong. Since I've already gotten used to broken locales I kept thinking that I want en_US locale with some Finnish settings, but in reality I wanted fi_FI locale with English language. I have to admit that I was pretty sceptical about the solution but decided to try it out. And to my surprise, it actually worked!

So the final solution is this: LANG="fi_FI.UTF-8"

It is a pretty clean solution and I'm happy to live with it. And as Simon pointed out, administrator changes should be preserved through upgrades. If not, one should file a bug for it.

There we go, another problem solved. Thanks for the comments and suggestions.

Posted Wed Aug 6 00:01:27 2008 Tags: ubuntu

Linux has this great thing called locales. You can basically control everything through a few system variables. The system is so flexible that you could in fact have English language with Swedish dates and Chinese error messages.

The problem with the current system is that it's controlled mostly with one variable. Just about everyone I know (including me) set the system locale to en_US because they don't want to use their local language as the system language. This in turn causes new problems, mostly due to the metric system used in the sane countries.

I finally got around to debug how to get the system to speak my language. The key here is the file /etc/default/locale which on Debian based systems contain the locale settings. Usually there is just one line in that file.


Since I live in Finland and want my computer to speak English I can add the following lines:


This replaces the settings for numerals (decimal separator and such), paper (yes, the rest of the world uses A4), name, address, telephone and measurement (YAY, metric system!). This way I have a nice English speaking system with Finnish settings.

I tried setting LC_ALL to fi_FI.UTF-8 but that causes gnome to speak Finnish to me, even though the LANGUAGE and LANG settings are set the English. LC_TIME is something I'd like to use, but I find the Finnish abbreviations for weekdays and months to be confusing. LC_MESSAGES causes gnome to talk partially Finnish, the general locale is English as it's supposed to be, but for example gnome-panel changes the menu entries to Finnish.

I wish that it was easier to set these settings. I also doubt that various tools know how to respect these settings and will overwrite that file with the default setting. That is why I'm writing this entry so that I remember how to fix it when that happens.

In the end the system is flexible but it's built in a funny way. The actual variable built by combining 2 or 3 values: language, country and possibly encoding. So to make thing easy for me I would have to declare en_FI.UTF-8 locale and start translating applications to that locale. I don't want to, so I'm sticking to this "temporary" solution.

Posted Mon Aug 4 18:54:49 2008 Tags: ubuntu