Thursday, December 1, 2011

Spicy Vinagre

One of the more eventful version bumps during the GNOME 3.2 update was with Vinagre, the Remote Desktop Viewer, which supports a number of protocols via plugins.  These plugins used to be managed by libpeas, a GNOME framework which makes it easier to develop and manage plugins.  However, in the case of Vinagre, it was a bit of an overkill, as the only plugins that were needed were those that shipped with Vinagre itself.

So, during the 3.2 development cycle, the devs decided to replace libpeas with their own VinagreStaticExtension class. In theory, a sensible move, but they made one major mistake: they never added any code to actually load the plugins!

Now that got your attention, didn't it?

Of course, if you try vinagre 3.2 on Linux, you may find that it works perfectly (but we'll get back to that).  But only because they used the most unportable, Linux/ELF-centric hack that I've ever seen (and trust me, I've seen my fair share of them).  It took me a while to figure it out at first, but here is what they did:
  1. the plugin init functions are marked __attribute__((constructor)), a GCC extension which causes the indicated function to be called automatically;
  2. the plugins rely on symbols in the main binary, of course, but furthermore...
  3. the executable also relied on symbols in the vnc plugin, and to top it off...
  4. the executable is linked against the plugins!! (If you don't think you misread that, then try again.)
In short, the only reason that this worked is that, by linking the plugins into the executable regardless of symbol dependency, the ELF runtime loader would load the plugin as a runtime dependency, at which time the plugin init function would be called due to being marked as a ctor.

To be fair, technique #1 is also used by LADSPA plugins, and #2 isn't uncommon and can be made to work even on PE platforms. However, #3, while theoretically possible, is impractical with plugins due to a lack of rpath in PE linkage, and relying on #4 wouldn't work at all because with PE only those DLLs whose symbols are required are hard-coded as runtime dependencies. And here's the kicker: even on ELF platforms, this doesn't work if linked with -Wl,--as-needed.

Fortunately, the fix is fairly easy: make the plugins static instead, link them into the executable as before, and actually call the ctors in main(). So not only is vinagre 3.2 working and available with the rest of GNOME 3.2 in Ports, but while I was at it, I added SPICE protocol support as well.

No comments: