IOChannels and Spawning

For the most part, we like to avoid wrapping as much of glib as possible, instead preferring the .Net framework capabilities. We hide as many GList and GHashtable parameters as possible, for example.

I've received several comments in the past that the .Net APIs for IO and process spawning didn't really match up to the glib API. Taking this feedback, I spent the last couple days wrapping, testing, and documenting a new glib-sharp binding for g_spawn_* and GIOChannel.

Here's an example usage from the docs:

using GLib;
using System;

public class SpawnTest  {

        public static void Main (string[] args)
        {
                new SpawnTest ();
        }

        MainLoop main_loop;
        IOChannel channel;

        public SpawnTest ()
        {
                main_loop = new MainLoop ();

                try {
                        Process proc;
                        int stdin = Process.IgnorePipe;
                        int stdout = Process.RequestPipe;
                        int stderr = Process.IgnorePipe;
                        GLib.Process.SpawnAsyncWithPipes (null, new string[] {"pwd"}, null,
                                                          SpawnFlags.SearchPath, null, 
                                                          out proc, ref stdin, ref stdout, 
                                                          ref stderr);
                        channel = new IOChannel (stdout);
                        channel.AddWatch (0, IOCondition.In | IOCondition.Hup, new IOFunc (ReadStdout));
                } catch (Exception e) {
                        Console.WriteLine ("Exception in Spawn: " + e);
                }

                main_loop.Run ();
        }

        bool ReadStdout (IOChannel source, IOCondition condition)
        {
                if ((condition & IOCondition.In) == IOCondition.In) {
                        string txt;
                        if (source.ReadToEnd (out txt) == IOStatus.Normal)
                                Console.WriteLine ("[SpawnTest output] " + txt);
                }
                if ((condition & IOCondition.Hup) == IOCondition.Hup) {
                        source.Dispose ();
                        main_loop.Quit ();
                        return true;
                }
                return true;
        }
}

I'd appreciate it if anyone who has been missing this functionality in Gtk# would try it out and let me know any issues you find, or places the API can be improved.

Exert 0.1

I've spent a little spare time hacking on exert recently and have reached the point where it has all the features I use from my online running log. I decided to switch over to using it for my log, so that means I need to maintain database compat from now on. I figured I might as well release it in case anyone else was interested in playing with it.

There's a source tarball available, and I used the opensuse build service to provide packages for 10.3 and fedora. Enjoy.

Moko Bindings

One of the comments on my last blog entry suggested a libmokoui binding. Since it was a nice small target, I went ahead and whipped up a binding for it this afternoon. It took me longer to write the tutorial I added to the wiki than it did to produce the binding.

Granted this was a very simple binding, since it is small and the source code was written perfectly for the GAPI parser's consumption. It also has no auto* magic. The tarball contains generated sources which could be copied into your project and compiled and installed as a private assembly. There is really no point to GAC magic for an assembly with no guarantees.

Bind this...

We are looking to add to the already impressive list of libraries accessible from mono. If you have any special requests of the latest and greatest libraries which you like to access from the sexy managed language of your choice, please let us know your needs so we can prioritize our efforts.

You can make your wishes known by commenting on this blog entry, mailing gtk-sharp-list@lists.ximian.com, filing a bug report, or writing on a cocktail napkin and mailing it to us, if you can figure out where we live.

GInterface Implementation

Finally "finished" the framework for GInterface registration in Gtk# this week. I wrote up a small tutorial and API description. It points to the small test application I wrote to demonstrate TreeModel implementation, which is the most common reason people have requested GInterface registration for the past few years.

Here's a screenshot of the sample which uses reflection to display the assemblies, types, and members in the application's AppDomain.

Reflection: a Cautionary Tale

Back in the day, I needed a mechanism to execute a method for a class for each new subclass of that type. Class constructors were not sufficient, because they are only run once per type, not once for every subclass of that type.

I decided to annotate the init method with an attribute and use reflection to lookup the method. When we register a new GType, we scan the class hierarchy for private static methods with a GLib.ClassInitializerAttribute and invoke any that exist. We use a similar mechanism to hook overridden virtual methods into the GObject class vtables for signal default handlers using the GLib.DefaultSignalHandlerAttribute.

This strategy works, but it turns out to be pretty suboptimal, especially from a memory usage standpoint. In order to find the methods with [ClassInitializer] defined, we had to load all the private static methods in the type hierarchy, and then iterate over them to look for the attribute. This caused a pretty substantial memory spike for types like Gtk.Widget which have a ton of static method delegate implementations for signal marshaling.

The root of the error was in not recognizing that the Class init problem is a one-to-one relationship of class to method, where the signal delegate problem I borrowed from is a one-to-many scenario. There is a [DefaultSignalHandler] method for each signal defined by a class.

To improve the mechanism, I added a GLib.TypeInitializerAttribute which takes a type and method name argument. With this attribute applied to a type declaration, we can lookup the specific methods by name and avoid loading all the static methods in the entire class hierarchy. This feature is now committed to trunk and the branch svn.

Since we have released the GLib.ClassInitializerAttribute as public API in a stable release, I couldn't just remove the reflection for it, even though it is unlikely that anyone out there really discovered the feature and is using it. In order to make it possible for subclass authors to avoid the reflection step, I also added the GLib.IgnoreClassInitializersAttribute which can be applied to any assemblies which contain GLib.Object subclass declarations. Svn trunk and branch add this attribute to all the gtk-sharp and gnome-sharp assemblies, so that all the types defined by the packages avoid this reflection overhead. The internal usage of [ClassInitializer] has been ported to [TypeInitializer] as well.

Thanks to Lluis for identifying the issue and Paolo for his usual insightful feedback in alternative approaches to the functionality. Lluis also helped refine the implementation of the [ClassInitializer] back-compat mechanism. My understanding is that the recent changes reduced MonoDevelop startup memory usage by about a MB.

Change change change

After living with 4 German Shepherds for over 10 years, the past year has held a lot of painful if not unexpected changes for Sherry and I. We lost OD last fall at ten and a half. This summer was particularly bad, losing our 11 year old JR just two weeks after 13 year old Maggie.

Sherry, Qman, and I have sort of been moping around the house for the last month. It's amazing how empty a house can feel.

That all changed Wednesday night.



It's equally amazing how quickly a 3 month old puppy can fill up a house. Her name is Beebee. We haven't had a puppy in the house in over 11 years. The frantic dashes to the back yard during housebreaking. The brief periods of high-energy romping separated by hours of food-induced comas.

Qman has not been all that interested in playing her little puppy games, but has been very tolerant of her. At times, it seems like he's just laying there rolling his eyes at her. She has a pretty amazing personality. Extremely friendly and confident, and the shepherd drive to please is already evident. We are really looking forward to watching her develop and anticipating the fun we'll have earning obedience, agility, and tracking titles.

Oops, time to take her out back again!

Hack Week

This week was hack week at Novell. At the beginning of the week, it looked like I was going to be spending most of my time supporting new Gtk# users and fixing bugs they were tripping over. Things settled down a bit on Tuesday though, and I got to put some time into Exert for the remainder of the week.

Exert is a small Gtk# app that uses sqlite as a database to manage a fitness log. There are tons of running logs on the web and PC versions as well. While I plan to use exert as my own fitness log and potentially explore some new usability improvements, my main goal with exert is to provide an example of a small-scale database front-end application using Gtk#.

Prior to this week, I already had developed the database backend and created a couple of simple editor widgets to manage simple text only log entries. Additionally, I created a monthly calendar view to serve as the primary user interface for the application. There's a screenshot of a fairly stale version of the calendar view up on the sourceforge project page.

This week, I added a few new features:

• Rendering of BodyWeight entries directly on the CalendarView
• A cardio entry editor for time/distance workouts with rest/avg/peak Heartrate logging.
• A running entry editor that builds on cardio entry but also tracks shoe usage.
• A Settings infrastructure based on XmlSerialization.

Here's a shot of the new RunningEditor editing an entry.


So far, I haven't added any summary reports for running, but I will eventually add the usual weekly/monthly/annual summaries, as mile as accumulated mileage for Shoes.

Upcoming features include a specialized weightlifting entry editor to make it quick and easy to add and clone existing workouts. That will probably have to wait until the next hack week, though.

Spring cleaning

After a brief hiatus helping out with the winforms bug count, I've spent the last few weeks shaking out bugs in Gtk#. The win32 instability issues with trunk seem to have been smoothed out. For the last week, I chased a bunch of ref management and object dispose/destroy issues down in MD and Stetic. Things seem pretty stable now with trunk, but as always, please report any trouble you run into.

I'd like to encourage anyone who has an interest in 2.10 to try out current svn trunk with your applications. It is possible you may see issues since we are getting closer to accurate object finalization now. Many of the issues I worked through in Stetic, for example, were related to using objects after they were already disposed, or marked for finalization by the GC through the circular ref handling capabilities of the GC. If you are doing any object destruction in Dispose overrides, for example, that can cause trouble. If you are invoking base.Dispose in a GLib.Object subclass, it should always be called after you do the additional work. When objects are leaking stuff like this doesn't bite you, but it just might now.

We would like to pull together a release in the near future, and I think things are stable enough at this point for more people to kick the tires. I've revised the Gtk# Plan wiki page to describe the recent changes in the tree and the planned feature additions.

And on a completely unrelated and personal note, I ran my first 5K race over the weekend and survived to hack another day.

Exceptional Delegates

I committed some rework to the GObject signal system in Gtk# today. The signal system relies heavily on native to managed callback delegates for all the unmanaged widget signals we bind. Paolo and Lluis pointed out a gap in our previous implementation, where we neglected to take into account the interaction of this code with managed exception handling.

When an exception is thrown in a managed event handler, we need to ensure that the stack is not unrolled across native-to-managed boundaries in the call stack. When that happens, you tend to get unpredictable behavior with stack corruption and segfaults. Today's fixes add try/catch blocks to the signal and virtual method delegate marshallers to ensure the stack only unrolls to the boundary.

The catch blocks raise static events on a new GLib.ExceptionManager class. If you want to "catch" events happening in managed signal handlers, you can attach to the UnhandledException event. The UnhandledExceptionArgs passed to your delegate expose an IsTerminating property (from its System.UnhandledExceptionEventArgs base class) to indicate if the exception is recoverable and an additional ExitApplication property to request program termination if desired after the event propogation is complete.

Lluis, always the bearer of interesting insight, pointed out we need similar code for Idle and Timeout handlers, which made me realize that all the Callback parameters like foreach method callbacks and DestroyNotify handlers are also vulnerable to the problem.

I'll be fixing this bug for a while, but feel free to try out the new signal stuff on trunk in the meantime and report any warts you run across.

Dogfooding MD

Good progress lately on Gtk#. Got the toggle ref updates into svn, in addition to the Gtk.Object destruction rework. I did a lot of testing on the patch in MonoDevelop, Banshee, F-spot, and some toy apps of mine to feel more confident about such a fundamental change to ref management in the binding.

Having gone through the exercise of building bleeding edge versions of all those apps to test them against bleeding edge Gtk#, I decided it would be good to continue dogfooding like this regularly. I plan to pull the trunk sources of MD, Banshee, and F-spot once a week and run them against Gtk# trunk from now on.

Dogfooding Banshee and F-spot is a simple thing, since I use both applications regularly. MonoDevelop has been something I've only goofed around with every now and then, though. To create a nice dogfooding opportunity I decided to convert my little fitness log application to a MonoDevelop project so I can hack on it in MonoDevelop for an hour or two each week. I wrote a wiki page on my experience of converting the subversion-based autotools project to an MD project. It required surprisingly little manual effort thanks to the nice toys Lluis and folks have been building into MD lately.

Leaky Documents

Making slow but steady progress on the documentation and auditing for the 2.10 API. Hopefully I'll finish up the Printing API next week. I've been refining some of the tools we use to add boilerplate-ish documentation for things like default signal handler virtual methods, and some of the semi-internal API the binding has to expose for binding purposes.

Since writing documentation doesn't make me feel sexy, I've been mixing in some bugfixing and new development along the way. Today I spent some time hacking toggle ref support into GLib.Object. Toggle refs are a new ref type added in glib-2.8. They are a special ref type which provides notification when transitioning from sole-ownership to shared-ownership of an object.

It's important to account for this with managed subclasses because we have to hold a strong reference to the managed object as long as native-owned refs exist. If Gtk# is the sole owner of a native object, we can allow the managed reference handling to drive garbage collection. However, when any native code takes a native ref on a managed subclass, we need to artificially maintain a managed ref to the object until the native code releases its ref.

Since we maintain a GObject* keyed hashtable of outstanding GLib.Object instances, we can use the toggle ref notification to switch between weak and strong references in the hash. While we have shared ownership, the hash will contain a normal object ref, avoiding garbage collection. When we transition to sole ownership, we'll hold a WeakReference to the object in the hash.

My first pass at toggle ref implementation seems to work well for plain GLib.Object subclasses. Now I just have to make it work within the context of GtkObject destruction and so on. I won't be committing anything until I've done a considerable amount of testing, since ref management changes have historically been a source of pain.

Good enough

This morning, I committed the ilContrast tool I blogged about on Sunday. It is in the mono-tools product in svn trunk. The tool can compare two individual assemblies which you can provide either by cmdline argument or interactively using the FileChooser.

Some potential features remain for bored hackers and artists to implement:

• It needs a super-sexy icon. I just copied MonoIcon.png.
• It could probably use some persistence mechanism to recall the last "project".
• Multiple assembly comparisons. The mono-api-info tool looks like it can do it, just needs hooking up.
• Maybe a tabbed view for displaying multiple comparisons.
  
• The About dialog could use translation to 67 different languages.
  
• It has a hard dependency on Gecko#. A fallback to GtkHtml like monodoc provides might be useful.

After finishing the standalone tool, it occurred to me that this might have made a nice MonoDevelop plugin instead. But the tool is handy as is, and will probably work its way into my Gtk# hacking.

Update:

In case anyone was actually considering the GtkHtml fallback hack, don't bother. As Miguel pointed out to me, the javascript requirement makes that a non-starter. A more likely hack in that area would be to remove the Gecko# dep by directly rendering the mono-api-diff output with Gtk# itself, perhaps using a TreeView.

ilContrast

After helping Miguel with a corcompare run on IRC Friday afternoon, we decided it would be nice to have a GUI tool that would hook it all up for you. Running the three cmdline tools in the current manner is not the most user-friendly experience.

I threw a few hours at the problem this weekend and have a little Gecko# frontend sort of working. It has a few warts, but it's at least to the point of rendering recognizable information. Look for it to hit the mono-tools module some time this week, hopefully.

Hang on

Delegate persistence is an interesting challenge in a pinvoke binding like Gtk#. Hang on to delegates too long and you tie up the delegate instance memory and keep objects alive which are providing the delegate's invocation target. Release the delegate too soon and you get crashes when the native code tries to invoke it posthumously.

Most of the callback parameters in Gtk+ are destroy notified, taking away the guesswork of when to release the delegate. We have supported notified delegate release for a while now.

Of the non-notified callbacks, most only need to persist for the duration of the method to which they are passed. Foreach methods are a common user of this style of delegate scope. I refer to this as "call" scope. You may have noticed warnings for method parameters defaulted to call scope. The generator treats all callback parameters as call scope if the method signature doesn't contain destroy notification and a scope attribute isn't explicitly provided via metadata.

The challenge appears when a callback parameter doesn't fit into either of these paradigms. There are callbacks which persist until removed by another method. Some of these luckily also provide destroy notification, but not all of them. There is also another class of callbacks that I will call "async" since they are typically provided to allow non-blocking implementations of functionality. The task is queued up by the method call and control returns before the task is actually completed.

Often these async methods take a delegate parameter that is invoked at the completion of the operation to notify the caller the operation is finished. I ran across an example of this method type while documenting the new Gtk printing API from 2.10. Since I was bored of writing documentation, I decided to take a crack at implementing async scope callbacks in the generator.

If you have any async callback delegates in your bindings, you can now add the scope="async" attribute to the parameter with metadata to automatically handle the persistence for this new scope.

Back in the Saddle

After a lengthier than expected hiatus, I've been turned loose again to dedicate most of my time to Gtk#. I'm looking forward to adding some new features and improving the memory management of the existing bindings.

I've completed an initial triage of outstanding bugs, along with some work bringing the documentation up to speed with current monodoc updater. I use the monodoc update as a first check for API stability, in addition to the CorCompare-based audit mechanism in svn, so it's important to me to have the docs updating with as little noise as possible.

My initial focus will be working on the higher priority bugs, while also documenting and auditing the remainder of the new types added in the Gtk# 2.10 release. While I document the new types, I'll be auditing the API for correctness and making any adjustments for members the automated code generator bound incorrectly. Hopefully within a couple of weeks I'll be able to move on to new features like GInterface Registration.