• Proxy support
• Add more upload arguments: safety, privacy, public/friends/private, search_hidden
• Cache the users full name, username and NSID (jcrosby)
• Fix UTF-8 encoding problems
• Verify our cached token before using it

Grab a tarball here or the Bazaar tree here.

• Chain the metadata lookups (Bastien Nocera)
• Finish the libmusicbrainz3 metadata fetcher (BN)
• Add a GVFS metadata fetcher as fallback (BN)
• Make libcdio option, as it breaks the GPL+Exception license (BN)
• Export ASIN, Discogs, Wikipedia in the internal metadata (BN)
• Lots of other cleanups to the metadata code (BN)
• Remove copy of the id3mux plugin, assume it exists now (BN)
• Remove Encoding field from desktop file (Pacho Ramos)
• Add Audio to desktop categories (Patryk Zawadzki)
• Correctly parse CDDA URLs (Matthew Martin)
• Don't free the option context

GNOME Keyring will store passwords in a pretty secure manner, so somehow I need to fetch the password from there. A quick look at the OfflineIMAP manual revealed that I can write Python functions which return the password, so I should be abe to hook into the keyring from OfflineIMAP. This should be fairly simple:

import gobject, gnomekeyring

# The keyring needs to know the application name
if gobject.get_application_name() is None:
  gobject.set_application_name("offlineimap")

def keyring(user, host):
  keys = gnomekeyring.find_network_password_sync(user=user, server=host, protocol="imap")
  # First one will do nicely thanks
  return keys[0]["password"]
...
remotepasseval = keyring("rburton", "imapmail.intel.com")

After writing a small tool to add the key to the keyring, to my surprise this worked first time. I bounced with glee, but ten minutes later I had error messages from OfflineIMAP running from cron in my inbox...

GNOME Keyring uses an environment variable to find the daemon, which isn't set in a cron environment. GNOME Keyring will fall back to using DBus to find the daemon, but the DBus session bus environment variable isn't set. DBus will fall back to reading the session bus address from the X root window, but DISPLAY isn't set so that doesn't work either... EPIC FAIL.

But, I thought, I upgraded to Network Manager 0.7 last week which bought in ConsoleKit. If I ask ConsoleKit for my sessions I should be able to find a session with has an X connection, then I can set DISPLAY appropriately and then the chain described above will work, and I'll have my password. Shockingly, this worked first time too:

import dbus, os
if not os.getenv("DISPLAY"):
  # Get the ConsoleKit manager
  bus = dbus.SystemBus()
  manager_obj = bus.get_object('org.freedesktop.ConsoleKit', '/org/freedesktop/ConsoleKit/Manager')
  manager = dbus.Interface(manager_obj, 'org.freedesktop.ConsoleKit.Manager')
  
  # For each of my sessions..
  for ssid in manager.GetSessionsForUnixUser(os.getuid()):
    obj = bus.get_object('org.freedesktop.ConsoleKit', ssid)
    session = dbus.Interface(obj, 'org.freedesktop.ConsoleKit.Session')
    # Get the X11 display name
    dpy = session.GetX11Display()
    if dpy:
      # If we have a display, set the environment variable
      os.putenv("DISPLAY", dpy);
      break

(man, I really with python-dbus had a better syntax for getting objects with a specific interface)

So there you go, integrating OfflineIMAP with the GNOME Keyring via ConsoleKit and DBus. Surprisingly this was pretty easy to do, thanks to DBus and the magic provided by ConsoleKit it is 100% hack free.

"^(?<task>.+) (?:by|due|on)? (?<month>\\w+) (?<day>\\d{1,2})(?:st|nd|rd|th)?$"

Apparently the average translator doesn't think that learning PCRE-style regular expressions, and reading the source that uses this string to understand how it is to be used, is appropriate. [note: this is sarcasm]

Maybe I should have added some translator comments to clarify exactly what I meant by this. These monster strings (all in koto-date-parser.c) are GRegex regular expressions which are used to parse the user's input to try and extract meaningful date information. To translate these strings you'll need to have a basic understanding of regular expressions: if you don't then skip them and hopefully someone who does will finish the translation. If you know regular expressions then translating these strings is easy, honest.

The golden rule is to never translate the words which look like this: (?<foo>. These are markers which identify portions of the input (such as task or month) and need to remain in English, although they can be moved around if required. The rest of the strings are translatable. I'll give an example using the French translation by Stéphane Raimbault. First, the string in English and a worked example:

"^(?<task>.+) (?:by|due|on)? (?<day>\\d{1,2})(?:st|nd|rd|th)? (?<month>\\w+)$"

First, we have a sequence of any characters identified as task, which magically expands to be as many as possible. This is optionally followed by one of the words "by", "due" or "on". This is followed by one or two digits identified as day followed by "st", "nd", "rd" or "th". Finally a sequence of characters which is identified as month. If the user had entered "pay bills on 2nd june" then task would be "pay bills", day would be "2", and month would be "june". Tasks can then turn "june" into a month number through other translations, and it now knows what date the user entered. In French, this translates as follows:

"^(?<task>.+) (?:pour|prévu|pour le)? (?<day>\\d{1,2})(?:er|e)? (?<month>\\w+)$"

See, I said it was easy! All I need now is a legion of translators who understand regular expressions enough to correctly translate the new Tasks... [this, again, is sarcasm] Luckily, plans are afoot to move the Tasks source to the GNOME Subversion server, so the full fury of the GNOME translation team can attack this.

NP: Trailer Park, Beth Orton

• Magic date parser when adding tasks
• Support libunique in the GTK+ port (Jonny Lamb)
• Support OWL window menus in the GTK+ port
• Save and restore the edit window size
• Add a clickable note icon (#548)
• Ellipsize the undo/redo menu items (#741)
• Make sure the date popup doesn't go off the screen (#752)

The most interesting change in this release is the magic date parser, which first landed back in March. This lets you use Google Calendar style descriptive tasks such as "release tasks today", "do shopping next tuesday" or "pay bills on 2nd". There are many patterns that are matched but I need two things from any users of Tasks.

1. Translations. At the moment there are only English and French translations for the strings, which are critical for the parser to work. Translators, please update the translations!
2. Feedback. The parser handles all of the natural language expressions that I thought would be useful. There are probably plenty more which are not handled, so if you find one which isn't handled (or is handled incorrectly) then please file a bug.

Oh, and one last thing. The OpenMoko and Maemo ports have likely bitrotted. New functionality has been added to the platform abstraction and I don't think those ports were updated. If someone actively uses Tasks on either Maemo or OpenMoko and is willing to test builds before release, please contact me.

• Don't crash when exiting
• Don't distribute full GFDL with docs
• Correctly parse CDDA URLs (#550131, thanks Matthew Martin)

• Stop playback when the disc is re-read (Matthew Martin)
• Only eject the disc if tracks were ripped (MM)
• Don't try and move the non-existant temp file when skipping (MM)
• Free the option context (Pierre Benz)
• Don't block until n-c-b quits when copying discs
• Fix playback track switching (MM)

• Fix play+pause+play (#523182, thanks Matthew Martin)
• Add %ay, album year (#522909, Juan F. Giménez Silva)

In other late GUADEC news I finally reviewed the rest of my GUADEC photos and uploaded them to Flickr. I'll try and not take a month to upload next time, honest!

Also Johan Kristell posted to the list for the first time with an implementation of the Digital Security Camera specifications, both server and client. GUPnP Network Camera currently only supports still images, but as it is based on GStreamer video can't be far away.

OPTIMISATION FAIL.

NP: Roseland NYC Live, Portishead

As of version 0.1 it is barely functional but it does let you update or login to a cowbuilder. It requires that the cowbuilders are named /var/cache/pbuilder/*.cow and doesn't support "traditional" pbuilder rootstraps yet, but that is planned. Anyway, cowbuilders are the future.

If anyone else thinks this could be useful there is a tarball and a Bazaar repository. I must also thank the wonderful Ulisse Perusin for the rocking icon he created.

NP: Cosmos, Murcof

The tarball is here, and packages for Debian are being worked on next.

Useful, but not exposed anywhere. Until now...

This is a small Epiphany extension which adds all presentation URLs it finds to the Nearby Sites menu, just like the URLs discovered using Avahi. It needs a bit more work as it doesn't yet handle being unloaded or devices disappearing, but it is certainly usable now.

If anyone else wants to have a go with it, the source can be fetched using Bazaar from here. Watch out for the currently hard-coded paths...

NP: Aerial, 2562

• Port to GIO (Michael Terry)
• Update URL handling for New GIO World Order (Bastien Nocera)
• Fix display problems with the cluebar (Pekka Vuorela)
• Add audio preview when overwriting (Luca Cavalli)
• Use GtkVolmeButton instead of BaconVolume (MT)
• Fix crash when no profile is selected (Matthew Martin)
• Add []<> to the special character list (MM)
• Make the year and disc entries a11y (Patrick Wade)
• Fix error handling in CD playback (Tim-Philipp Müller)
• Require intltool 0.40

I really need some heavy testing on the GIO rewrite, so please try and extract tracks to as many different targets as possible. Although I expect confirmation that using an unmounted remote location currently fails, it should be possible to use this to write to Samba, OBEX-FTP, and so on.

The tarball is here, and packages for Debian are being built now.

In other news postr.dev has seen a lot of development and is looking pretty damn neat at the moment.

char *ip;
GetExternalIPAddress (proxy, &ip, &error);

Or this for non-blocking calls:

static void
external_ip_cb (GUPnPServiceProxy *proxy, char * ip,
                GError *error, gpointer userdata)
{
  // ...
}
...
  GetExternalIPAddress_async (proxy, external_ip_cb, NULL);

I've ported my test applications to use the bindings, which are available in this Bazaar repository. It appears to work quite well, I just need to test it against all of the official service descriptions and add a few small features.

gupnp_service_proxy_send_action (proxy,
                                   "AddPortMapping", &error,
                                   /* In arguments */
                                   "NewRemoteHost", G_TYPE_STRING, "",
                                   "NewExternalPort", G_TYPE_UINT, external_port,
                                   "NewProtocol", G_TYPE_STRING, "TCP",
                                   "NewInternalPort", G_TYPE_UINT, internal_port,
                                   "NewInternalClient", G_TYPE_STRING, internal_host,
                                   "NewEnabled", G_TYPE_BOOLEAN, TRUE,
                                   "NewPortMappingDescription", G_TYPE_STRING, desc,
                                   "NewLeaseDuration", G_TYPE_UINT, 0,
                                   NULL,
                                   /* Out arguments */
                                   NULL);

Now, that really is quite tiresome. It basically means that you need to have the service reference to hand when coding, because you need to know the name and type of each argument. Luckily for DBus part of dbus-glib is a binding tool which can create type-safe wrappers so that making method calls is much easier. Wouldn't it be nice if there was something similar for GUPnP, which generated inline functions with prototypes like this:

static inline gboolean
AddPortMapping (GUPnPServiceProxy *proxy,
                char * in_NewRemoteHost,
                unsigned int in_NewExternalPort,
                char * in_NewProtocol,
                unsigned int in_NewInternalPort,
                char * in_NewInternalClient,
                gboolean in_NewEnabled,
                char * in_NewPortMappingDescription,
                unsigned int in_NewLeaseDuration,
                GError **error);

Well, now there is. I've put the initial code here but will be moving this into GUPnP itself shortly. The next task is to add asynchronous wrappers just as in dbus-glib, but that shouldn't be too hard.

The prerequisites are Anjuta, the Anjuta Poky SDK plugin, and QEMU. These are all available for installation from our Debian repository for Debian/Ubuntu users, everyone else will have to build from source, sorry! You'll also need a Poky ARM SDK and QEMU ARM images from the Poky web site. The SDK is a tarball which contains a cross compiler with base libraries (glibc, GTK+, and so on) and should be extracted onto your machine (it extracts the SDK into /usr/local/poky). The QEMU image consists of a kernel and a ext3 file system which will boot Poky inside QEMU.

To start I fetched a checkout of Tasks and loaded up Anjuta. I don't have an existing Anjuta project for Tasks, so I used File → New → Project From Existing Sources to create a project using the checkout. At this point I could do native development using Build → Run Configure and Build → Build Project to configure and compile the source, but we want to cross-compile.

To activate cross compiling go to Edit → Preferences → General → Installed Plugins and enable the Poky SDK plugin. This will add a new page Poky SDK to the preferences dialog. We're using an external toolchain so set the SDK root to /usr/local/poky/eabi-glibc/arm and the toolchain triplet to arm-poky-linux-gnueabi. We're also using QEMU instead of a real device so set the paths to the kernel and root filesystem (remembering to uncompress the filesystem). We're now done configuring, so the preferences dialog can be closed. However notice that if you switch from using a SDK to building with a full Poky tree you can use the cross-compiler it produces directly, and you can also use an external device instead of QEMU: the only requirement is that you can SSH into it.

Now to do the build. Use Build → Run Configure to configure Tasks, passing any extra options you want. Note that if you want to debug your build in the future you'll need to enter CFLAGS=-g here to disable optimisation (autoconf sets -O2 -g by default, which isn't useful for debugging). The configure script is then ran with the right environment and options for cross compiling, and with any luck will successfully configure. Then hit Build → Build Project and watch the cross-compiler do its thing. When that has worked, you can prove to yourself that the right thing has happened.

$ file tasks
  tasks: ELF 32-bit LSB executable, ARM, version 1 (SYSV), for GNU/Linux 2.6.14, dynamically linked (uses shared libs), not stripped

We have an ARM binary, ready for deployment. Start the virtual machine with Tools → Start QEMU (this may ask for your root password to configure networking) and once it has booted you can install the project into the VM with Tools → Deploy. This will run make install to a temporary directory and then rsync it to the VM. Now you can either interact with the VM directly (if the application installed a new desktop file, then it should appear on the desktop), or use Tools → Run Remote to execute a binary directly: entering tasks will execute the freshly installed Tasks. Neat, huh?

For the final trick there is even GDB integration. Tools → Debug Remote will let you specify a local binary (to extract debug symbols from, say src/gtk/tasks) and a remote binary to run, and then start a GDB on the VM and connect to it. The binary will be initially running but paused at the entrypoint, so you can add breakpoints and then continue execution.

Hopefully this post has been a good overview of the integration available between Poky and Anjuta. In the future I hope to see Nemiver integrated into Anjuta, and gdbserver support in Nemiver, which would be a killer combination for Poky integration.

NP: One On Twoism, Various

$ ./gypsy-fireeagle.py 00:0B:0D:88:A4:A3
got 51.861145 0.156275
Updated FireEagle

The first line is me running my script (this one is 64 lines, but it is half whitespace), telling it where my GPS is. The second line is the current position that my rather cheap and nasty GPS determined. The third line tells me that Fire Eagle has been updated with those coordinates.

Suffice to say I'm very impressed with Yahoo's geocoding software. My GPS never settles to an accurate reading and will happily jitter around a 20 metre wide circle for hours, but the location Fire Eagle is reporting me at is two doors away. I'm not exaggerating: it says number 9 on my street when it should be number 5. That is some incredibly accurate mapping they have.

$ python geohack.py 
Using position 51.872330 0.161950
Got WOEID 12775
Got town Bishop's Stortford

Now to write a GeoClue provider which will fill in the locality information from the position. Long-term grand plans involve integrating all of this geo magic into Postr, somehow.

NP: Third, Portishead

Update: I now have an account!

For the last few days I've been learning more about UPnP and testing it with the few devices I have around the house. One of these is a cheap ADSL router, which apparently has the lamest UPnP stack on in existence. It does however support the WAN IP Connection interface, so you can use UPnP to get the external IP address and manipulate the port mapping. I'll skip over the horrific security violations this involves, because it's a useful demonstration that the majority of people will be able to test.

Today we'll start simple and get our external IP address using GUPnP. The first thing to be done is to create a Control Point, which in the UPnP model handles discovery of resources, be them devices or services (a device can have multiple services). When creating a control point you can specify the URN of the resource you want to target. In this case we want all services providing WANIPConnection so we'd use urn:schemas-upnp-org:service:WANIPConnection:1. If you want to browse for all services then use ssdp:all (SSDP being the Simple Service Discovery Protocol).

static GMainLoop *main_loop;

int
main (int argc, char **argv)
{
  GError *error = NULL;
  GUPnPContext *context;
  GUPnPControlPoint *cp;
  
  /* libsoup requires threading, so we have to initialise it */
  g_thread_init (NULL);
  g_type_init ();

  /* Default GLib context, default host IP, default port */
  context = gupnp_context_new (NULL, NULL, 0, &error);
  if (error) g_error (error->message);

  /* Create a control point targeting WAN IP Connection services */
  cp = gupnp_control_point_new
    (context, "urn:schemas-upnp-org:service:WANIPConnection:1");
  /* The service-proxy-available signal is emitted when any services which match
     our target are found */
  g_signal_connect (cp,
		    "service-proxy-available",
		    G_CALLBACK (service_proxy_available_cb),
		    NULL);
  
  /* Tell the control point to start searching */
  gssdp_resource_browser_set_active (GSSDP_RESOURCE_BROWSER (cp), TRUE);

  /* Enter the main loop */
  main_loop = g_main_loop_new (NULL, FALSE);
  g_main_loop_run (main_loop);

  /* Clean up */
  g_main_loop_unref (main_loop);
  g_object_unref (cp);
  g_object_unref (context);
  
  return 0;
}

static void
service_proxy_available_cb (GUPnPControlPoint *cp,
                            GUPnPServiceProxy *proxy)
{
  /* ... */
}

Now we have an application which searches for the service we specified and calls service_proxy_available_cb for each one it found. Now, to get the external IP address we need to invoke the GetExternalIPAddress action. This action takes no in arguments, and has a single out argument called "NewExternalIPAddress". Yes, the naming scheme is stupid. GUPnP has a set of methods to invoke actions -- which will be very familiar to anyone who has used dbus-glib -- where you pass a NULL-terminated varargs list of (name, type, value) tuples for the in arguments, then a NULL-terminated varargs list of (name, value, return location) tuples for the out arguments. A simple implementation would be as follows.

static void
service_proxy_available_cb (GUPnPControlPoint *cp,
                            GUPnPServiceProxy *proxy)
{
  GError *error = NULL;
  char *ip = NULL;
  
  gupnp_service_proxy_send_action (proxy,
				   /* Action name and error location */
				   "GetExternalIPAddress", &error,
				   /* IN args */
				   NULL,
				   /* OUT args */
				   "NewExternalIPAddress",
				   G_TYPE_STRING, &ip,
				   NULL);
  
  if (error == NULL) {
    g_print ("External IP address is %s\n", ip);
    g_free (ip);
  } else {
    g_printerr ("Error: %s\n", error->message);
    g_error_free (error);
  }
  g_main_loop_quit (main_loop);
}

Note that _send_action blocks until the service has replied. If you need to make non-blocking calls then use gupnp_service_proxy_begin_action which takes a callback.

So, that is searching for services and invoking actions in GUPnP. Next time I'll cover subscribing to state variables, and routers which can't count.

NP: Folk But Not Folk, Various

To download it you can either grab the tarball or fetch the bzr tree.

Maybe it's time to get a dedicated build machine, my poor laptop gets quite toasty when building Poky. Then again it seems happy enough, so maybe I should just use an external keyboard to avoid boiling my hands.

NP: Oneric, Boxcutter

• Update the status bar after uploading
• Add a Switch User menu item
• Add Add/Remove buttons to the image pane
• Install the Nautilus extension to the new extension path
• Don't select groups when the name is clicked
• Don't display errors when posting to moderated groups
• Show a warning on exit if there are images to upload (thanks Germán Póo-Caamaño)

The tarball is here, and packages for Debian have been uploaded.

• Add Send To Group options
• Add Privacy and Safety options
• Use a multi-line entry for the Description field
• Show the user's name in the status bar
• Fix the resizing of the preview

The tarball is here, and packages for Debian have been uploaded.

mobile-2.24

This changes GConf and EDS to use the DBus ports, and provides meta-mobile-platform which builds the complete platform.

pimlico

This builds Contacts, Dates and Tasks, providing meta-pimlico.

matchbox

This builds Matchbox Panel, Matchbox Desktop, Matchbox Keyboard and Matchbox Window Manager, providing meta-matchbox.

Also, Poky is building images nightly with the complete platform in, which will let you build and test software in a PDA-style environment with QEMU, running on x86 or a number of ARM-based devices (such as Nokia N800, Sharp Zaurus or OpenMoko).

Last week at the Collaboration Summit in Austin (which I couldn't attend for personal reasons) there was a day-long GNOME Mobile meeting, as a result of which there is now a long list of packages which need to be considered for addition to the Platform (such as HAL, Gypsy and Geoclue), and a few changes (such as replacing gnome-vfs with gvfs). I hope to review the proposals fairly shortly, so that we can hopefully make an initial GNOME Mobile 2.24 platform release alongside the Desktop release in September.

In other news, this is exactly what the Internet is for.

NP: Blue Moon Station, Solar Fields

I need SMTP AUTH and TLS, which means nbsmtp, masqmail, and nullmailer are out. I want a local queue for when I'm not online which means esmtp, ssmtp, msmtp, and nullmailer are out (I'm not convinced that msmtp's queue scripts are reliable enough). Surely there must be a simple SMTP relay which will reliably manage a queue if the mail cannot be sent! If not, does anyone know of a good guide to configuring Postfix or Exim to do this?

Because I'm fairly lame there are not matching Debian packages yet, but I'll get around to that tomorrow. In other news, a very nice man called Ian Lawrence wrote a buzzword-compliant tutorial where he uses Gypsy to talk to a Bluetooth GPS, tests it with my Gypsy Status 10-minute hack, and then uses Django to redirect the user to the relevant geohash.org page.

NP: Remembranza, Murcof