The Free Software Foundation objected to UDI for various reasons. Mostly, I think, it was because they were afraid that people who are not them would choose to use drivers that were non-free. As I’ve written about before here, there are people who think that forcing people to use free software is somehow freedom—and I will not go into it in any great depth here, because I have done that in the past. Suffice it to say that forcing anything is not freedom; it cannot be freedom. So, the Free Software Foundation, I think, was really afraid that they would have to do more work to be able to stick to their own requirement of using 100% free software on their own computer systems. (And hey, Roy, if you’re reading—I’m not saying that the FSF is wrong, and I’m not putting myself in a position opposite of that of the FSF. I suspect you think so anyway, but hey, I just figured I would point that out.)

Even if the free software operating systems did not adopt the UDI specification, why didn’t proprietary operating systems? This is perhaps the most puzzling thing to me. It seems that in this event, none of the operating systems—free or proprietary—did what would have made sense. After all, even if only Apple and Microsoft adopted a common device driver specification, that would save a lot of time, effort, and improve user experience all the way around. Apple users would be able to use all the hardware that Microsoft users could use—and the inverse would also be true. The amount of time that device driver authors would have to spend writing and debugging driver code would go way down—free software driver authors would be able to write a driver once, for example, and all systems (including free software systems that chose to support the specification) would benefit.

I could see an objection of a driver specification that was binary-only. However, UDI was not—it mandated an ABI so that drivers that are built for a particular platform were binary-compatible with operating systems on the same platform, but it also mandated an API, so that drivers would be source compatible to any operating system that implemented the specification, on any platform. That by itself would seem to me to be positive motivation to hardware manufacturers to release the source code to drivers so that they can support operating systems that are on platforms that do not exist yet, or have not been considered (or have been considered to be nonviable or unsupported platforms).

So, I have to wonder why a common device driver specification was never implemented in various operating systems. It would seem to be a common sense thing, especially given that there are so many operating systems. It would make the coexistence of operating systems a lot easier, and it would promote choice. It might encourage bits of proprietary code on free software operating systems, but it would also enable people to drop the excuse that “free operating system x does not support device y”, and would as a result potentially increase the number of free software programs and operating systems in use, even if there is a minor cost in terms of certain drivers. And those drivers could always be replaced—a common driver specification would make it easier to understand the structure of drivers generally, and make it easier for lawful, clean-room reverse engineering to be done on those drivers.

Imagine, for example, if drivers for graphics cards, TV tuner cards, video and audio encoding/decoding cards, modems, storage chipsets, motherboard chipsets, USB chipsets, IEEE-1394 chipsets, graphics tablet devices, touch screens, debugging interfaces, network devices, and so forth were all written to a common specification, it would reduce the amount of code which needed testing. It would increase user choice in both hardware and operating systems—something which I still hold is quite likely the most valuable freedom we have. It would increase reliability, since the users of Windows, OS X, Linux, the various BSD systems, and other, not-so-mainstream operating systems would be able to run the same driver code and collectively supply debugging information and perform testing in a multitude of environments. It would increase security, because then common code that is well-known could be used on all platforms and not just the one it was written for. It would do for device drivers what POSIX has done for user-mode application software. I do not believe that I could be convinced that this would be anything other than a good thing.

Also, it could bring back old operating systems.  Imagine what life could be like, for example, if OS/2 had a “UDI driver” written for it, and it could then take advantage of newer drivers never intended for it. Or any other very old operating system which is no longer supported and could still be useful, for any of a number of reasons…

Many projects that are all about being open and transparent provide mechanisms for their users to get in touch with the developers.  Launchpad has become an excellent place to do these things; Launchpad’s Answers and Bugs components are truly excellent ways to get in touch with people that manage software there.  The system is so good that I’d advocate that every project try to move to it.  Launchpad does have its issues, though they are often quickly fixed (and can be fixed even quicker now that it is free software).  But between Bazaar (a truly amazing and flexible distributed version control system) for code hosting and the other components of Launchpad such as the Answers, Bugs, and Translations components, Launchpad provides virtually everything that a project needs to communicate effectively with its users aside from the project’s Web site.  And users seem to naturally be able to reach out using it.

In the nearly one year that I’ve been working with AllTray, I’ve talked—well, written to—several people who have asked questions and acted in their own ways to work to improve the software.  Development on AllTray has been made much easier just because people are willing to speak up.  Everything from the random “thank you” to “hey, are you going to do this?” or “I’d like to see feature x in the new version” or whatever.  It’s truly great.  I can’t say that every project has users quite like AllTray’s, because AllTray is something of a niche application, but AllTray’s users are a great model for other projects.  There has been virtually no negativity from users and everything has been constructive.

To anyone who manages a project that doesn’t use Launchpad, I’d strongly encourage them to do so.  Its users are great, and as far as management of the overall project goes, it’s very easy for project people to communicate back and manage communications in a single spot.  It’s also great that users can ask a question and that the project can do things like go, “oh, hey, this is really a bug,” or vice versa with bugs→questions.  And if you don’t use Launchpad because you don’t want to use Bazaar (I hear that users of git are pretty dedicated to it; I can understand that, but it’s just not for me—it’s too complex, and I like that Bazaar just stays out of my way), consider using LP for everything but code hosting.  Or consider using Launchpad’s code hosting to mirror your project from git, it does this quite nicely.  Or, find some way to do what Launchpad does wherever you do project management.  It works really well, and it’s a big help.

And to AllTray’s users. A big Thank You.  You guys and gals are great.  Keep communicating!

One of the very few times I have uttered this phrase, but it works now:  Thanks, Microsoft.

Vala supports an object-oriented means of programming. It performs “assisted” memory management, and handles a lot of things which are tedious to do in C directly for you. This is in contrast to a runtime system like the JVM or the CLR, where the entire program requires this runtime engine and a (usually pretty large) standard library. These things do full memory management by way of garbage collection, which is not a horrible idea, but that, plus the extra layers of software can slow things down. (Not that they have to: an entire operating system written in Java or CLI would have zero cost of having child processes which are Java or CLI processes; instead, native executables would require emulation and a performance hit. However, this is tangential…)

Getting back on track, Vala is targeted at the GObject type system. This means that programs that are written in Vala do require some libraries at run-time, namely the GLib and GLib-Object libraries. These libraries are part of the LSB spec, and are installed on most reasonable and modern systems, with possible exception for systems configured to be servers. Effectively, though, the cost is nothing. This brings me to the point I originally wanted to write about: using libraries in Vala.

In order to use a library in Vala, there must be a .vapi file for it. This VAPI file is essentially a description of the library; it provides Vala with information it uses to then call the entrypoints in the file. The VAPI file is also very powerful: You can write one to describe a regular C library that doesn’t use GObject, and it provides assistance to be able to “objectify” that library if you want. Creating this bindings is a manual process, but it isn’t terribly hard—though Vala documentation is, at the moment, lacking a bit in terms of consumer-oriented documentation. As is the case with free software, when user documentation doesn’t exist, we can always refer to the source code for additional detail and clarification if necessary. (And this isn’t terribly hard to do with the Vala compiler, it’s written in Vala!)

In the VAPI file, you can define one or more namespaces, and then enumerations, constants, classes, methods, etc., within those namespaces.

The Example

For this example, we’ll use the libuuid library, which comes with e2fsprogs. On Ubuntu, ensure that you have the development headers installed by executing sudo apt-get install uuid-dev prior to getting started—you’ll need those headers installed in order to be able to build software using the bindings. Then, create yourself a working directory; ~/uuid-vala is what I’ll be using in this example. We’ll go through a few different iterations of this binding, progessively adding functionality and usefulness to it. I’ll take you through the steps that I went through to generate this VAPI file, sans a few emails to the Vala emailing list.

A good starting point for creating a binding in Vala is API documentation; in the case of libuuid, there are the man pages uuid, uuid_clear, uuid_compare, uuid_copy, uuid_generate, uuid_is_null, uuid_parse, uuid_time, and uuid_unparse. From here, we can get the function prototypes that we need—the library’s entrypoints. What we don’t get from them is the definitions of the data types and some of the other data. For that, we refer to the header file—stated in the man pages as uuid/uuid.h. The header gives us the rest of the information that we know; we need the definitions for uuid_t, UUID_VARIANT_* and UUID_TYPE_DCE_*. Also, because we don’t have a complete set of class libraries/bindings wrapping basic POSIX functionality, we need to import time_t and struct timeval.

One thing to consider here before we actually write the VAPI file is which types are opaque for the purposes of the bindings, and which ones are transparent. Again, if we had a complete set of class libraries/bindings, this wouldn’t be a problem—we’d just use the existing types. For now, we’ll assume that the uuid_t type is opaque, and that we need to use the time_t and struct timeval types, but we’ll not be wrapping them in this first go-around.

Without futher ado, here’s the first iteration of the VAPI file, with commentary and explanation following:

/*
 * libuuid.vapi - Vala bindings for the e2fsprogs library libuuid
 * Copyright (c) 2009 Michael B. Trausch <mike@trausch.us>
 * License: GNU LGPL v3 as published by the Free Software Foundation
 *
 * This binding is a (mostly) strict binding to the function-oriented
 * nature of this C library.
 */

[CCode(cheader_filename = "uuid/uuid.h",
	   lower_case_cprefix = "uuid_", cprefix = "")]
namespace LibUUID {
	[CCode(cprefix = "UUID_VARIANT_")]
	public enum Variant {
		NCS,
		DCE,
		MICROSOFT,
		OTHER
	}

	[CCode(cprefix = "UUID_TYPE_DCE_")]
	public enum Type {
		TIME = 1,
		RANDOM = 4
	}

	[CCode(cname = "time_t")]
	public struct Time : int{}

	[CCode(cname = "struct timeval")]
	public struct TimeValue {
		[CCode(cname = "tv_sec")]
		public int seconds;

		[CCode(cname = "tv_usec")]
		public int microseconds;
	}

	[SimpleType]
	[CCode(cname = "uuid_t")]
	public struct UUID : char {}

	public void clear(UUID uu);
	public int compare(UUID uu1, UUID uu2);
	public void copy(UUID destination, UUID source);
	public void generate(UUID output);
	public void generate_random(UUID output);
	public void generate_time(UUID output);
	public int is_null(UUID uu);
	public int parse(string uuid_input, UUID uu);

	[CCode(cname = "uuid_time")]
	public Time get_time(UUID uu, out TimeValue time_ret);
	public void unparse(UUID uu,
						[CCode(array_length = false,
							   array_nullterminated = false)]
						char[37] uuid_asciiz);
	public void unparse_upper(UUID uu,
							  [CCode(array_length = false,
									 array_nullterminated = false)]
							  char[37] uuid_asciiz);
	public void unparse_lower(UUID uu,
							  [CCode(array_length = false,
									 array_nullterminated = false)]
							  char[37] uuid_asciiz);

	// Not documented in the man page, but exposed in the headers.
	[CCode(cname = "uuid_type")]
	public int get_type(UUID uu);

	[CCode(cname = "uuid_variant")]
	public int get_variant(UUID uu);

	// Part of the C library, but provided to make life easier on an
	// application programmer.
	[CCode(cname = "ctime", cheader_filename = "time.h")]
	public weak string get_unixtime_to_string(ref LibUUID.Time seconds);
}

As I mentioned earlier, Vala is a language that is similar in many ways to the C# language. If you’ve worked with C# code, you’ll recognize the [CCode …] immediately prior to the namespace LibUUID declaration as an attribute. The attribute looks similar to a function call—and while I haven’t gotten terribly deep into Vala, I have the feeling that Vala doesn’t treat attributes as function calls. After all, this isn’t the CLR that we’re talking about, where attributes are really classes, and instantiations of them are special ways of instantiating them and attaching them to classes. Anyway, this particular attribute contains three parameters:

• cheader_filename = "uuid/uuid.h" – This attribute informs Vala that the bindings come from the specified header file. This attribute can be applied to (at least) namespaces and methods.
• lower_case_prefix = "uuid_" – This attribute informs Vala that the methods are prefixed with uuid_ in C code. Looking down a bit, this means that public void clear(UUID uu); is really void uuid_clear(uuid_t uu) when called from C.
• cprefix = "" – This states that we don’t want to use a C prefix for anything else. Between this and the previous parameter, this means that if we have to bind something that doesn’t fit the naming conventions that we’ve already laid out, we’ll have to manually define them when we define the item in question.

The next thing after the namespace declaration is the the enum declaration for Variant. This enum maps the series of constants in the header file which define the UUID_VARIANT type. The same goes for Type, for UUID_TYPE. The cprefix parameter to the attached CCode attribute says that these enums are mapped to constants defined in the header file, and it will then generate C code which references those constants (which will then be “expanded” by the preprocessor when the C compiler gets to the source code).

After these, we find some type definitions that make the binding a little bit easier to use. Specifically, for time_t, struct timeval, and uuid_t. These definitions provide some help with type-safety in Vala, and are terribly useful.

Now, we are down to the methods. The first 8 methods are regular, in that they follow all of the previously declared rules. As an example, a call to LibUUID.clear(uu) (where uu is a UUID data type) will expand to uuid_clear(uu) in C code. Since clear returns void, Vala doesn’t have to generate any code to handle the return value. In the case of compare, it will of course assign the return value to a variable in C and support you working with it later. The first non-regular function here is get_time, which has the real name uuid_time, not uuid_get_time. I renamed it to make more sense in the context of writing new code in Vala, though this was an optional choice.

Next, we see the unparse family of functions. These take a binary UUID and expand them to a string such that you can use the UUID in your user interface or when passing code to a database server or other external program. This was challenging, and it is the most fragile part of the binding. In other words, what I did here works, but it’s really not good practice to do generally. This is an example of a fairly easy-to-implement binding. I wouldn’t recommend distribution of the binding in production code, because you can still shoot yourself in the foot with it—the caller is tasked with the burden of creating a char buffer long enough to hold the return value, and if they don’t, it’s pretty likely that glibc will kill the program at run time with the message “buffer overrun detected” and a very verbose backtrace and memory mapping dump.

The unparse family takes a UUID as input, and a char buffer as output. Because the library assumes that it will be passed a 37 byte buffer, it doesn’t have any parameters to specify the length and will go ahead and blindly overwrite whatever you may have in the buffer for the first 37 bytes (36 bytes hold the UUID, and the last byte holds the trailing NULL byte, as is common in C when dealing with strings). We have the array_nullterminated parameter set to false because this is really an output parameter, though we’ve not told Vala that. Why? Because the UUID library deals with char[] buffers and char* strings, and when you pass those in C, you’re passing them by reference anyway. You only have to explicitly pass the address of them if you’re using statically-allocated storage (in C, that is). And even things that appear static (uuid_t type) aren’t because of the underlying type being a pointer to a char buffer. Confused yet? Ahh, the reasons I try to stay away from C most days…

Anyway, more of the same for the other unparse functions.

Now, we have those variant and type enumerations, but nothing in the man pages talked about using them; that’s fine, of course, since we got them from the header. There were two other functions in the header, which we have mapped as get_type and get_variant, which can be used to get information on the UUID being worked with.

Finally, we have an import from time.h. The reason that this is here, instead of leaving it up to the application programmer to provide it, is because of the way Vala seems to treat external references. It seems to want to create a prototype for them if you declare them in a .vala file, but it doesn’t seem to do that when declared in a .vapi file. Since it’s helpful to not have a fatal compiler error choking on conflicting prototypes, we’ve provided an interface to ctime() directly in this VAPI implementation.

So, is this a good binding or not? Well, let’s take a look at a sample program that uses the binding, and judge from that:

/*
 * uuid-fo.vala - Test for libuuid.vapi bindings for libuuid.
 * Copyright (c) 2009 Michael B. Trausch <mike@trausch.us>
 * License: GNU GPL v3 as published by the Free Software Foundation
 */

public class Program {
	public static int main(string[] args) {
		LibUUID.UUID uuid_time = {};
		LibUUID.UUID uuid_rand = {};

		LibUUID.generate_time(uuid_time);
		LibUUID.generate_random(uuid_rand);

		print_uuid_info(uuid_time);
		print_uuid_info(uuid_rand);

		return(0);
	}

	public static void print_uuid_info(LibUUID.UUID uu) {
		stdout.printf("UUID:\t\t%s\n", helper_uuid_to_string(uu));
		stdout.printf("  Type:\t\t%s\n", helper_uuidtype_to_string(uu));
		stdout.printf("  Variant:\t%s\n",
					  helper_uuidvariant_to_string(uu));

		if(LibUUID.get_type(uu) == LibUUID.Type.TIME) {
			stdout.printf("  Time:\t\t%s\n", helper_uuidtime_to_string(uu));
		}

		stdout.printf("\n");
	}

	public static string helper_uuidtype_to_string(LibUUID.UUID uu) {
		int uuidtype = LibUUID.get_type(uu);

		switch(uuidtype) {
		case LibUUID.Type.TIME:
			return("Time");
		case LibUUID.Type.RANDOM:
			return("Random");
		default:
			return("Unknown, invalid, or not a UUID");
		}
	}

	public static string helper_uuidvariant_to_string(LibUUID.UUID uu) {
		int uuidvariant = LibUUID.get_variant(uu);

		switch(uuidvariant) {
		case LibUUID.Variant.NCS:
			return("NCS");
		case LibUUID.Variant.DCE:
			return("DCE");
		case LibUUID.Variant.MICROSOFT:
			return("Microsoft");
		case LibUUID.Variant.OTHER:
			return("Other");
		default:
			return("Unknown, invalid, or not a UUID");
		}
	}

	public static string helper_uuidtime_to_string(LibUUID.UUID uu) {
		LibUUID.Time uuidTime;
		LibUUID.TimeValue uuidTimeValue;
		uuidTime = LibUUID.get_time(uu, out uuidTimeValue);

		int seconds, microseconds;
		seconds = uuidTimeValue.seconds;
		microseconds = uuidTimeValue.microseconds;
		LibUUID.Time secs = (LibUUID.Time)seconds;

		string uuid_timestamp = LibUUID.get_unixtime_to_string(ref secs);
		GLib.StringBuilder sb = new GLib.StringBuilder();

		sb.append_printf("%s", uuid_timestamp);
		sb.truncate(24);

		sb.append_printf(" (and %d microseconds)", microseconds);

		return(sb.str);
	}

	public static string helper_uuid_to_string(LibUUID.UUID uu) {
		char[37] uu_chars = new char[37];
		LibUUID.unparse(uu, uu_chars);

		GLib.StringBuilder sb = new GLib.StringBuilder();

		foreach(char c in uu_chars) {
			if(c == '\0') break;
			sb.append_printf("%c", c);
		}

		return(sb.str);
	}
}

This program’s output:

Wednesday, 2009-Feb-18 at 14:25:19 - mbt@zest - Linux v2.6.29-rc5
Ubuntu Intrepid:[1-83/10081-0]:uuid-vala> valac -o uuid uuid-fo.vala --pkg libuuid --vapidir . -X -luuid

Wednesday, 2009-Feb-18 at 14:25:59 - mbt@zest - Linux v2.6.29-rc5
Ubuntu Intrepid:[1-84/10082-0]:uuid-vala> ./uuid
UUID:		fbf2a006-fdf1-11dd-859f-5ae5840a05cf
  Type:		Time
  Variant:	DCE
  Time:		Wed Feb 18 14:26:04 2009 (and 691047 microseconds)

UUID:		ae610032-d4bf-42c7-93ff-465dd2e712bb
  Type:		Random
  Variant:	DCE

This program is a bit large for such a simple task, don’t you think? Take a look at the helper functions that are required to make the bindings for libuuid usable. We can certainly do better than this—those helper functions are likely to be needed by just about anyone that works with the libuuid library in Vala. Nearly everyone is going to want to get the UUID as a string, and while the API provides a means of reaching that goal, it doesn’t do it for us, and there is no reason why it shouldn’t. After all, the way it is “just works” in existing C code, and future Vala code should “just work,” too.

The hard part is that the way that the UUID library works is a bit unusual. When you bind something with Vala’s VAPI support, you can create a class out of it, if it gives you a pointer type of some sort. But this doesn’t, and attempting to do so will get the C compiler rather unhappy with you. The answer in this case is to create the VAPI binding, and then create a class that wraps the binding. The reason is that the UUID library doesn’t act very object-like at all, requiring that each UUID that you use be essentially statically allocated; no functions return a new UUID, and there is no way to free them that is consistent with the API. You can certainly dynamically allocate them and work with them, but the UUID library doesn’t seem to think that way.

So, here is a much simplified binding, which we will use as a basis for a wrapper:

/*
 * libuuid.vapi - Vala bindings for the e2fsprogs library libuuid
 * Copyright (c) 2009 Michael B. Trausch <mike@trausch.us>
 * License: GNU LGPL v3 as published by the Free Software Foundation
 *
 * This is a bare simple binding which will be wrapped by a
 * newly-created Vala object.  It's only slightly different than the
 * first version, in that it makes no attempt to make life easier on
 * the programmer using this binding.
 */

[CCode(cheader_filename = "uuid/uuid.h",
	   lower_case_cprefix = "uuid_", cprefix = "UUID_")]
namespace Native.LibUUID {
	// Constants
	private const int VARIANT_NCS;
	private const int VARIANT_DCE;
	private const int VARIANT_MICROSOFT;
	private const int VARIANT_OTHER;
	private const int TYPE_DCE_TIME;
	private const int TYPE_DCE_RANDOM;

	[CCode(cname = "time_t")]
	public struct time_t : int{}

	[CCode(cname = "struct timeval")]
	public struct timeval {
		public int tv_sec;
		public int tv_usec;
	}

	[CCode(cname = "uuid_t")]
	public struct UUID : char {}

	public void clear(UUID uu);
	public void copy(UUID dest, UUID src);
	public void generate(UUID outp);
	public void generate_random(UUID outp);
	public void generate_time(UUID outp);
	public void unparse(UUID uu,
						[CCode(array_length = false,
							   array_nullterminated = false)]
						char[37] outp);
	public void unparse_upper(UUID uu,
							  [CCode(array_length = false,
									 array_nullterminated = false)]
							  char[37] outp);
	public void unparse_lower(UUID uu,
							  [CCode(array_length = false,
									 array_nullterminated = false)]
							  char[37] outp);

	public int compare(UUID uu1, UUID uu2);
	public int is_null(UUID uu);
	public int parse(string uu_in, UUID outp);
	public int type(UUID uu);
	public int variant(UUID uu);

	public time_t time(UUID uu, out timeval TimeInfo);

	[CCode(cname = "ctime_r", cheader_filename = "time.h")]
	public weak string unixtime_to_string(ref time_t secs,
										  [CCode(array_length = false,
												 array_nullterminated = false)]
										  char[] ret);
}

This is (mostly) the same, with a few different choices made in representing information from C. Now, the thing to do is create a wrapper around this; this wrapper will be either compiled into a Vala application or it could be compiled as a shared library, being useful generally to other GLib/GObject programs or other Vala programs.

/*
 * libwrapuuid.vala - Wrapper for the simple libuuid bindings.
 * Copyright (c) 2009 Michael B. Trausch &ltmike@trausch.us>
 * License: GNU LGPL v3 as published by the Free Software Foundation
 *
 * This is a wrapper around libuuid.vapi which presents an object-oriented
 * interface to libuuid.
 */

namespace Trausch.Vala.UUID {
	public enum Variant {
		NCS = Native.LibUUID.VARIANT_NCS,
		DCE = Native.LibUUID.VARIANT_DCE,
		Microsoft = Native.LibUUID.VARIANT_MICROSOFT,
		Other = Native.LibUUID.VARIANT_OTHER
	}

	public enum Type {
		Time = Native.LibUUID.TYPE_DCE_TIME,
		Random = Native.LibUUID.TYPE_DCE_RANDOM
	}

	public errordomain UUIDError {
		MALFORMED_STRING_INPUT,
		NOT_SET,
		WRONG_TYPE,
		FAILED
	}

	public class UUID : GLib.Object {
		private Native.LibUUID.UUID uuid;
		private Type defaultType;

		construct {
			this.defaultType = Type.Time;
			this.generate();
		}

		public static UUID create_from_string(string input) throws UUIDError {
			UUID newUUID = new Trausch.Vala.UUID.UUID();
			newUUID.UUID = input;

			return(newUUID);
		}

		public string? UUID {
			owned get {
				if((bool)Native.LibUUID.is_null(this.uuid)) {
					return(null);
				}

				return(this.to_string());
			}

			set {
				if(value == null) {
					Native.LibUUID.clear(this.uuid);
				}

				if(Native.LibUUID.parse(value, this.uuid) < 0) {
					throw new
						UUIDError.MALFORMED_STRING_INPUT("Malformed string");
				}
			}
		}

		public Type UUIDType {
			get {
				if((bool)Native.LibUUID.is_null(this.uuid))
					throw new UUIDError.NOT_SET("No UUID is currently stored");

				Type uuidType = (Type)Native.LibUUID.type(this.uuid);
				return(uuidType);
			}

			set {
				Type current_type = this.UUIDType;
				Type new_type = value;

				if(current_type != new_type) {
					this.defaultType = new_type;
					this.generate();
				}
			}

			default = Type.Time;
		}

		public Variant UUIDVariant {
			get {
				if((bool)Native.LibUUID.is_null(this.uuid))
					throw new UUIDError.NOT_SET("No UUID is currently stored");

				Variant uuidVariant =
					(Variant)Native.LibUUID.variant(this.uuid);
				return(uuidVariant);
			}
		}

		public void generate() {
			switch(this.defaultType) {
			case Type.Random:
				Native.LibUUID.generate_random(uuid);
				break;
			case Type.Time:
				Native.LibUUID.generate_time(uuid);
				break;
			default:
				Native.LibUUID.generate(uuid);
				break;
			}
		}

		public int compare(UUID that) {
			return(Native.LibUUID.compare(this.uuid, that.uuid));
		}

		public bool equals(UUID that) {
			bool retval = false;
			int compare_result = Native.LibUUID.compare(this.uuid, that.uuid);
			if(compare_result == 0) retval = true;

			return(retval);
		}

		public UUID duplicate() {
			Trausch.Vala.UUID.UUID newUUID = new Trausch.Vala.UUID.UUID();
			newUUID.UUID = this.to_string();

			return(newUUID);
		}

		public string to_string() {
			char[] chars = new char[37];
			Native.LibUUID.unparse(this.uuid, chars);
			GLib.StringBuilder sb = new GLib.StringBuilder();

			foreach(char c in chars) {
				if(c == '\0') break;

				sb.append_printf("%c", c);
			}

			string retval = sb.str;
			return(retval);
		}

		public void from_string(string input) throws UUIDError {
			this.UUID = input;
		}

		private Native.LibUUID.timeval get_raw_time() throws UUIDError {
			if(this.UUIDType != Type.Time)
				throw new UUIDError.WRONG_TYPE("Invalid op: not a time UUID");

			if((bool)Native.LibUUID.is_null(this.uuid))
				throw new UUIDError.NOT_SET("No UUID is currently stored");

			Native.LibUUID.timeval time;
			Native.LibUUID.time(this.uuid, out time);

			return(time);
		}

		/*
		 * This is a slight bit ugly, to format the time the way I wanted to
		 * do it.
		 */
		public string get_time() throws UUIDError {
			Native.LibUUID.timeval timestamp = this.get_raw_time();
			Native.LibUUID.time_t unixTime =
				(Native.LibUUID.time_t)timestamp.tv_sec;

			GLib.StringBuilder sb = new GLib.StringBuilder();
			char[] timeStr = new char[27];
			Native.LibUUID.unixtime_to_string(ref unixTime, timeStr);

			int spaces = 0;

			foreach(char c in timeStr) {
				if(c == '\n') break;
				if(c == ' ') spaces++;

				if(spaces == 4) {
					sb.append_printf("+%dus", timestamp.tv_usec);
					spaces++;
				}

				sb.append_printf("%c", c);
			}

			return(sb.str);
		}

		public string get_type_as_string() throws UUIDError {
			Type uuidType = this.UUIDType;
			string retval = "";

			switch(uuidType) {
			case Type.Time:
				retval = "Time";
				break;
			case Type.Random:
				retval = "Random";
				break;
			}

			return(retval);
		}

		public string get_variant_as_string() throws UUIDError {
			Variant uuidVariant = this.UUIDVariant;
			string retval = "";

			switch(uuidVariant) {
			case Variant.NCS:
				retval = "NCS";
				break;
			case Variant.DCE:
				retval = "DCE";
				break;
			case Variant.Microsoft:
				retval = "Microsoft";
				break;
			case Variant.Other:
				retval = "Other";
				break;
			}

			return(retval);
		}
	}
}

Alright, so that's 234 lines of object-oriented goodness. What does that get us, though? The ability to focus more closely on an actual application! Here is a simple UUID generator program which uses this wrapping. This application is considerably more featureful than the last one with the last bindings without the wrapper class, as can be seen by looking at it:

/*
 * uuid.vala - UUID generator program using libwrapuuid.vala and libuuid from
 * e2fsprogs.
 * Copyright (c) 2009 Michael B. Trausch <mike@trausch.us>
 * License: GNU GPL v3 as published by the Free Software Foundation
 */
using Trausch.Vala.UUID;

public class Program {
	public static bool cmdline_force_time;
	public static bool cmdline_force_random;
	public static bool cmdline_version;

	const OptionEntry[] options = {
		{ "force-time", 't', 0, OptionArg.NONE, ref cmdline_force_time,
		  "Force time-based UUIDs", null },
		{ "force-random", 'r', 0, OptionArg.NONE, ref cmdline_force_random,
		  "Force random UUIDs", null },
		{ "version", 'v', 0, OptionArg.NONE, ref cmdline_version,
		  "Show the program version", null },
		{ null }
	};

	public static int main(string[] args) {
		int howMany = 2;

		try {
			GLib.OptionContext ctx = new GLib.OptionContext("- display uuids");
			ctx.set_help_enabled(true);
			ctx.add_main_entries(options, null);
			ctx.parse(ref args);
		} catch(OptionError e) {
			err(e.message, args[0]);
			return(1);
		}

		if(cmdline_version == true) {
			show_version();
		}

		if(cmdline_force_time && cmdline_force_random) {
			err("--force-time and --force-random are mutually exclusive",
				args[0]);
			return(1);
		}

		if(args.length > 1)
			howMany = (args[1].to_int() == 0 ? howMany : args[1].to_int());

		UUID[] uuids = generate_uuids(howMany);

		foreach(UUID u in uuids) {
			print_uuid_info(u);
		}

		return(0);
	}

	public static void err(string errMessage, string prog) {
		stdout.printf("%s: %s\n", prog, errMessage);
		stdout.printf("Run '%s --help' to see a full list of command line"+
					  " options.\n", prog);
	}

	public static void show_version() {
		stdout.printf("""Sample UUID Generator v0.1.2
Copyright © 2009 Michael B. Trausch <mike @trausch.us>
Licensed under the terms of the GNU GPL v3 as published by the FSF.

""");
	}

	public static UUID[] generate_uuids(int howMany) {
		UUID[] retval = new UUID[howMany];
		for(int cur = 0; cur < howMany; cur++) {
			retval[cur] = new UUID();
			Trausch.Vala.UUID.Type newType;

			if(cmdline_force_time == true) {
				newType = Trausch.Vala.UUID.Type.Time;
			} else if(cmdline_force_random == true) {
				newType = Trausch.Vala.UUID.Type.Random;
			} else {
				if(cur % 2 != 0) {
					newType = Trausch.Vala.UUID.Type.Random;
				} else {
					newType = Trausch.Vala.UUID.Type.Time;
				}
			}

			retval[cur].UUIDType = newType;
		}

		return(retval);
	}

	public static void print_uuid_info(UUID uuid) {
		GLib.StringBuilder sb = new GLib.StringBuilder();
		sb.append_printf("UUID:\t\t%s\n", uuid.to_string());
		sb.append_printf("  Type:\t\t%s\n", uuid.get_type_as_string());
		sb.append_printf("  Variant:\t%s\n", uuid.get_variant_as_string());
		if(uuid.UUIDType == Trausch.Vala.UUID.Type.Time)
			sb.append_printf("  Time:\t\t%s\n", uuid.get_time());
		stdout.printf("%s\n", sb.str);
	}
}

One minor note: You will need Vala's trunk revision from the Subversion or bzr repositories for Vala to build this as is (or wait for a version of Vala >= 0.5.4 to be released). To use Vala 0.5.3, remove the parenthesis on line 94—or the program will try to allocate a ton of memory (trust me, the allocation will fail). It is a relatively new language/compiler, so all the kinks haven't yet been worked out. That's not a big deal, though; as I mentioned earlier, Vala development appears to be very active. When I reported the bug earlier today, the Vala people were very fast to fix the bug (it was fixed in under 30 minutes!) and commit the fix to the SVN repository. I was very pleasantly surprised with that.

The latter version of the binding, the wrapper class, and the Vala source for the application itself should be pretty easy to mentally follow if you're used to reading things like C#, C++, or Java, though if any of it needs additional explanation I can certainly respond to requests. Writing it isn't terribly much harder, though the one bug I found was a bit of a kink since the parenthesis removal wasn't any of the things I thought to check. In any event, you should now know enough to get started with Vala. Oh, and if you don't like systems without documentation, and you're good with documentation, you may contribute documentation resources to the Vala project, if you can at all; Vala is currently short on documentation!

Oh, and of course, a few runs of the new version:

Wednesday, 2009-Feb-18 at 18:05:39 - mbt@zest - Linux v2.6.29-rc5
Ubuntu Intrepid:[1-189/10177-0]:uuid-vala> ./uuid --help
Usage:
  uuid [OPTION...] - display uuids

Help Options:
  -?, --help             Show help options

Application Options:
  -t, --force-time       Force time-based UUIDs
  -r, --force-random     Force random UUIDs
  -v, --version          Show the program version

Wednesday, 2009-Feb-18 at 18:05:44 - mbt@zest - Linux v2.6.29-rc5
Ubuntu Intrepid:[1-190/10178-0]:uuid-vala> ./uuid -v -t 1
Sample UUID Generator v0.1.2
Copyright © 2009 Michael B. Trausch <mike@trausch.us>
Licensed under the terms of the GNU GPL v3 as published by the FSF.

UUID:		ad990d72-fe10-11dd-859f-5ae5840a05cf
  Type:		Time
  Variant:	DCE
  Time:		Wed Feb 18 18:05:47+640357us 2009

Wednesday, 2009-Feb-18 at 18:05:47 - mbt@zest - Linux v2.6.29-rc5
Ubuntu Intrepid:[1-191/10179-0]:uuid-vala> ./uuid -r 1
UUID:		a57c6345-df45-4417-9298-e1c07d2151a8
  Type:		Random
  Variant:	DCE

Wednesday, 2009-Feb-18 at 18:05:53 - mbt@zest - Linux v2.6.29-rc5
Ubuntu Intrepid:[1-192/10180-0]:uuid-vala> ./uuid 3
UUID:		b26b2c40-fe10-11dd-859f-5ae5840a05cf
  Type:		Time
  Variant:	DCE
  Time:		Wed Feb 18 18:05:55+728288us 2009

UUID:		5a431431-cc7c-44bf-99ee-8f967e724b17
  Type:		Random
  Variant:	DCE

UUID:		b26b2c42-fe10-11dd-859f-5ae5840a05cf
  Type:		Time
  Variant:	DCE
  Time:		Wed Feb 18 18:05:55+728288us 2009

Wednesday, 2009-Feb-18 at 18:05:55 - mbt@zest - Linux v2.6.29-rc5
Ubuntu Intrepid:[1-193/10181-0]:uuid-vala> ./uuid -t 3
UUID:		b402ee9e-fe10-11dd-859f-5ae5840a05cf
  Type:		Time
  Variant:	DCE
  Time:		Wed Feb 18 18:05:58+400579us 2009

UUID:		b402ee9f-fe10-11dd-859f-5ae5840a05cf
  Type:		Time
  Variant:	DCE
  Time:		Wed Feb 18 18:05:58+400579us 2009

UUID:		b402eea0-fe10-11dd-859f-5ae5840a05cf
  Type:		Time
  Variant:	DCE
  Time:		Wed Feb 18 18:05:58+400579us 2009

Wednesday, 2009-Feb-18 at 18:05:58 - mbt@zest - Linux v2.6.29-rc5
Ubuntu Intrepid:[1-194/10182-0]:uuid-vala> ./uuid -r 3
UUID:		275ca880-5ae7-44b3-bcaf-4d0084d00df3
  Type:		Random
  Variant:	DCE

UUID:		d270e2af-5ee0-456d-8ac6-e1d5685287d0
  Type:		Random
  Variant:	DCE

UUID:		9a0de010-aba3-4276-99e9-dcb6e28455f2
  Type:		Random
  Variant:	DCE

ETA: If you would like to download the final sources, they are online in a simple tarball.

Obviously, we’re not there yet. Using managed code today is good for certain types of larger systems and eventually (IMHO) it’ll be good for smaller ones, after the majority of software is using it and there isn’t some extremely large overhead for small tools on the system.

In any case, that gets me to where I am going: I need to learn fairly low-level X11 programming. Why? As I am looking at AllTray, it currently seems that it operates at multiple levels of abstraction, depending on GTK+ and GDK to handle some things, and talking directly to X11 for others. Since AllTray is a program which should be suitable for running on all window managers/GUI toolkits/desktop environments, the core functionality of it should not carry a dependency on GTK.

Part of the work I am doing in AllTray in my current local branch is to refactor functionality, and part of that means fixing this in a decent way. The code base should operate at a consistent level of abstraction for the core (so as to make the code easier to understand and lower the dependencies it carries), while the user interface for AllTray should be native to the environment the user is working in. That is, if the user is running GNOME, AllTray should use GTK+ for presenting its user interface; if the user is running KDE3, AllTray should use KDE 3 for presenting its user interface. This would follow for every environment for which someone wants to submit code for—it’ll be modular, so at build-time you’d specify the interfaces you want to build support for (or, someone could in theory create a standalone library which follows the planned UI API which AllTray will work with) and those interfaces will be probed and queried at runtime; it’s possible that a similar system can be used to work around the quirks of various window managers/GUI toolkits/desktop environments. (I am thinking about a general-purpose plugin and hooking system for AllTray, but I can only think of two uses for it, so it may be better to just implement those special cases and generalize the idea later if it proves to be worth it.) The idea of a plugin system, of course, being so that people can more easily write code to extend AllTray in the ways in which it should be easily extended. Ahh, I digress.

In any case, does anybody know of a good book that will get a person started with X11 programming? Something new would of course be desirable, but (so far) I have only found really old books. Maybe enough hasn’t changed in X11 in the last several years to warrant books on programming directly for it, but I don’t know enough about it at that level to know…

That having been said, I did get the movement of AllTray to Launchpad done while I was in Toledo—you can now go to its project page on Launchpad and see what’s there. I am going to be working on attempting to study and refactor the code base prior to fixing any reported bugs on the software and continuing its development. Mainly, I think that the important parts of it are going to wind up being the refactoring of the code, making it consistently use a single layer of abstraction. From there, I am planning on adding session management support and fixing bugs.

I will (hopefully soon) have a new Web site up for it and have the old one at Sourceforge redirect to it. Ideally, the site will provide an interface for the information that Launchpad exposes about the project, perhaps with a blog in addition to it. It won’t be my primary thing, since there is another project that has been in the cooker for a while which will hopefully be finished early this year that I’ve been working on, but that doesn’t mean AllTray will be nothing in my schedule. My intention is to finish up working with the refactoring process in the next two months or so, and then spend the next six months making changes and enhancements that will lead up to my first stable release of AllTray, which will be version 0.8.0.

For now, though, a little bit of TV watching and relaxing. I am still somewhat wiped out from the drive back…

Why?

I’ve used AllTray for a long time. It is one of those pieces of software that performs a useful function, but you have to know its quirks to make it do what you want quite often, and you have to work around some of the things that it does for it to retain that usefulness. Users shouldn’t have to do that—they should just be able to start using a program or utility and have it work properly. Also, utilities should be (reasonably) independent of the environment that they run within, which AllTray isn’t. And, good utilities should integrate with the system pretty well, which AllTray does not, since it carries a hard dependency on GDK/GTK. (Kdocker is an alternative piece of software, and while I don’t know how well it works compared to AllTray, I know that it is bound to Qt, which is just as bad.)

The license will most likely be GPLv3, since nothing will actually link to it and it’s not a library. If there is a library component, it will be LGPLv3.

The actual “braindump”

The actual *docking* part isn’t terribly hard, it would seem. With GTK+ it can be done in only a few lines of code (especially in C#). However, C# wouldn’t be the language to use for the this tool, because of the dependency on Mono. My thinking is that the proper language to use would be C itself. It makes me think of how easy everything would be if only the desktop environment and all its software were already written in managed code… the effective cost of using C# would be significantly less than it is already.

Using C as the language isn’t that big a deal to me. What is a big deal to me is that the application be well-integrated and well-behaved. For example, AllTray depends on GTK for some of its tasks, and uses a custom (ugly, IMHO) widget for displaying its title when you over over it in the system tray. There’s another program out there, KDocker, that would appear to be slightly more featureful than AllTray, but it carries with it a dependency on Qt, which I don’t like. Ideally, the core for such a docking utility should be able to have various back-ends that serve the purpose of integration with the system, with a single back-end that would apply without dependencies on anything else for the rare scenario where neither GTK+ nor Qt are available.

What would this thing be required to do?

• Dock any arbitrary application to the system tray, not just a single window from the application. (Using the System Tray Protocol Specification)
• Run in a single instance, managing multiple applications within that instance if necessary.
• Provide session management for those who use it, and act as a session manager for applications that are run underneath it that support session management.
• Be able to use something like inotify to be able to show an indication of when the application’s title for its windows change.
• Use a single icon for the application, with a window list if the application carries more than a single window.
• Use individual backends which provide a native look and feel depending on the desktop environment currently in use.
• Use the application’s icon to represent the application in the system tray.
• Support blinking the icon when the application decides to do something like present a modal dialog box or something (if this is even possible, that much I am not quite sure of).
• Detect the desktop environment currently in use and (if possible) load a backend to integrate with it.
• Carry a dependency on GLib so as to make implementing it quicker and less error-prone.
• NOT carry any hard dependencies on any widget libraries like GTK+ or Qt.
• Return an error to the user if there is no running system tray, and there isn’t one after some sort of timeout (say, when starting the desktop environment, there may not be a system tray right away). (I don’t think that sort of situation would be commonplace, but it’s not really the job of this program to create a system tray if one doesn’t exist, since this is the job of the desktop environment.)

The backends would not have any specific configuration items (the entire configuration would be housed somewhere like ${HOME}/.config/docker, for example). There probably ought to be a backend entrypoint that would show a dialog for configuration, but I haven’t really figured out if that’d be the case yet. The docker should probably do all of its configuration by way of the command line for each invocation, which is suitable for use as a desktop shortcut or panel launcher.

This approach requires that each individual desktop environment have its own backend, which might become a bit of a pain for supporting a great number of desktop environments, but the upshot of that is that the application then feels native no matter what environment the user is using. Obviously support for GNOME and KDE would happen, and most everything else would depend on someone with programming experience for various environments to implement a backend for it.

The backend doesn’t handle, say, docking, since there’s a standard for that already. It would handle things like displaying the tip when hovered over or displaying a menu when right-clicked. So, the job that each backend would do would be pretty lightweight. And as much as possible should be handled in the core, with the backend consisting of just enough “hooks” into the target system to be able to provide a useful interface.

The way that this would have to work, using C as the programming language, would be that the core would have to use dlopen() and friends to load the backend and then use the entrypoints that the backend provides. Since dlopen, dlclose, dlsym, and dlerror are defined by POSIX.1-2001, support for those should be in every reasonably modern Unix-like system that supports at least a subset of POSIX. This of course eliminates Windows, but I am not terribly worried about that since there are probably other Windows-only utilities that serve this purpose.

The primary goal, then, is to replace other DE-dependent and toolkit-dependent dockers with a utility that carries the goal to be universally useful and highly robust. The aim of this goal is to replace AllTray entirely in (at the very least) Ubuntu, and hopefully be seen as useful in other distributions as well.

I’ll need some help…

This is the part where it’ll get interesting. I’ll need some help in actually doing this, in many areas.

For the moment, where I’ll need help (be that from Google, or from people in-the-know that happen upon this post) is to find all of the documentation that I’ll need to understand how to implement the software. I can use software that already exists to try to learn how the underlying system works, to a degree anyway. But some of the software doesn’t handle various situations correctly (or I don’t have a way to see if it does) and so studying them would be less than useful for everything I am trying to do. Namely, the things I need to find out yet are:

• How to run a program in such a way as to be able to reliably know what windows belong to the program and be able to dock them. I suspect that it’d involve using LD_PRELOAD and some monitoring trickery, but I would hope that there is some sort of API to do this already. Also, need to know how to manage the application itself; say it continues running but without any windows, the tray icon should obviously disappear while it’s running like that, until it has windows to manage again.
• How to perform session management and be a session manager for subordinate programs. (I might be able to study the source for KDocker to learn the answer to that, though I am not 100% certain yet of that.) I also have a paper to read yet on the subject that may have all the information that I am looking for, though I am not certain there yet, either.
• How to reliably detect the running desktop environment without resorting to ugly kludges and ad-hoc trickery.
• How to work independent of the currently-running window manager and operate reliably for any standards-compliant WM that is running, without having to special-case for various types of software or various types of window managers.

I suspect that there is useful documentation out there for most (if not all) of those things for X11 and other relevant points. I just haven’t found what I am looking for yet for all of it. Standards documents and system documentation are useful things to find, and maybe an example or two of source code on how to do it as well, are what I am looking for, so that I can try to understand these things better.

At some point down the road, I’ll need help with getting the backends written when they have an API to conform to. I can, in all probability, write a GTK+ backend that will work for XFCE and GNOME since I use GNOME. However, I don’t use KDE or anything that runs on top of Qt, so I will definitely need help from someone that is familiar with writing software for Qt to get a Qt backend written. If XFCE has any sort of non-standard special UI guidelines that are incompatible with GNOME, then I’ll need some help there, too. Any other non-GNOME or non-KDE environments (such as GNUstep, if it supports it anyway) that support the System Tray Protocol Specification will need backends written for them by people that are able to do so.

$ sudo apt-get build-dep libgtksourceview2.0-cil \
 libgecko2.0-cil libgtk2.0-cil libgnome2.0-cil mono \
 libgluezilla

The usage screen for the script:

Mono 2.0 Home Directory Installer

Copyright (c) 2008 M. Trausch <mike@trausch.us>
License: GPLv3

Total download: ~206 MiB, total installed size: ~161 MiB.
Download & build takes ~35 minutes on an AMD Phenom 9500 Quad-Core CPU w/ 8Mbps
downstream Internet connection (and happy servers, of course).

Logs are output to '/tmp/mono2-build.3044/log'.

usage: install-mono2.sh [options]

Available options:
  -s PKG
  --skip PKG	Skip building package PKG (one of libgdiplus, mono, mono-basic,
		gtk-sharp, gnome-sharp, gnome-desktop-sharp, gluezilla,
		monodoc, mono-tools, mono-addins, gecko-sharp,
		gtksourceview-sharp, heap-buddy, nant, monodevelop).  May
		be specified multiple times.

  -d
  --download-only
		Only download the packages, do not install them.

  -r
  --monodevelop-revision
		Specify which SVN revision of MonoDevelop trunk to install.
		Defaults to SVN HEAD.

  -S
  --skip-download
		Skip the download (if you're rebuilding, use this to
		save bandwidth.

  -m
  --monodevelop	Download, build, and install MonoDevelop only.  ONLY USEFUL
		FOR UPDATING MONODEVELOP AFTER YOU'VE RUN THIS SCRIPT TO
		INSTALL MONO 2.0.

  -1
  --single-cpu	Use ONLY single CPU/core for building.  Use this if you have
		trouble building and think it might be related to 'make -j'.

Now, when this script runs, it will download the Mono files (and MonoDevelop from SVN) into /tmp/monosrc. It will do the build in /tmp/mono2-build.XXXXX where XXXXX is the process ID number of the install-mono2.sh script. When you’re finished and happy with the installation, you can safely remove those directories:

$ rm -Rf /tmp/mono2-build.* /tmp/monosrc

If you want to track MonoDevelop within SVN, then it is better to not delete /tmp/monosrc, because the script uses svn to pull from the MonoDevelop Subversion repository. This means that it can, if the directory already exists, save you bandwidth by only pulling a delta from the revision that you have and the current HEAD revision. The astute probably noticed that you can select a revision explicitly as well. I am currently using r115695, and have used it for a grand total of 20 minutes (and debugging, as best as I can tell, works quite wonderfully in it, but it is a work-in-progress, coming directly from the development mainline). In short, the MonoDevelop is great for toying with but not for production use. I may release a version of this script that lets you choose between the latest stable released version of MonoDevelop and an SVN revision, but I haven’t gotten there yet because I wanted to play with the SVN version (which uses features in Mono 2.0… that was the whole point of me wanting to play with Mono 2.0, actually).

In any case, the script is available for download here from my Web site.

A few words about the script: This script is released under the terms of the GNU General Public License, Version 3. It is version 3 ONLY. There is NO SUPPORT for this software; I wrote it for myself, and it may change or it may stagnate. I provide it in the hope that it is useful to someone out there other than myself, but it is AS-IS, WITHOUT WARRANTY, UNSUPPORTED software. If it changes your dog’s gender, bursts your house into flames, destroys your whatchamacallermicroflobbermajag, do not come and yell at me. That having been said, I welcome any modifications (at least to view them). You can modify it for your own use, you can share it (in fact I expect you to!) and you can tinker with and tweak it. Suggestions? Welcome! Comments? Welcome! Flames? Hey, free speech is your right…

Lastly, I RECOMMEND THAT YOU RUN THE SCRIPT ON A CLEAN USER ACCOUNT, WITHOUT YOUR DATA, WITHOUT YOUR ANYTHING IN THE WAY. Just create yourself another user, and login to it using Xnest or something, or log out of your user account completely—I don’t care how you do it. Why do I make this recommendation? Because while nothing bad should happen, it’s always best to sandbox things that you don’t trust. After all, what reason do you have to trust me if you don’t know me? You can also of course read through the script and audit it if you wish. It doesn’t handle EVERY exceptional condition, but it handles just about anything that needs to be and tells you where things went awry if they go wrong. And it logs everything (see the program output when you run it). It also takes advantage of multiple processors or cores if you have them for those (few) builds that will permit it without going nutso, but YMMV. If you have an issue with this aspect of the script, see the usage text. If there are any issues that you can’t solve with that, or installing build-dependencies, then paste the relevant segments of your failing log file to something like pastebin and ping me a comment here, and I can take a look at it and (try to) offer advice, depending on where the fail is.

Oh, yeah, and I used a hell of a lot of bandwidth putting it together and making sure that it did things like downloaded things right and all of that. Sorry for hitting your servers repeatedly, Mono people—and thank you for the software!

So, I signed up for it and decided that I would start with the first problem and work my way through them, using the problems to also try to learn or confirm various things that I know/think are true. I’ve been using the C# programming language for some time now, so I thought, “What the hell, I’ll go ahead and do these and see what I learn.” Having worked the first problem, I found some rather interesting things out about the Mono and .NET runtimes.

Firstly, I learned that the .NET runtime’s implementation of DateTime and/or TimeSpan sucks (my guess is DateTime is what sucks). It is not capable of measuring very small amounts of time, say smaller than one hundredth or one thousandth of a second, and treats those small amounts of time as if they were zero time at all. Which means that under .NET 2.0, I cannot measure things like I can under Mono. Oh, well, I suppose.

I wrote a small program to compute the answer to the first problem, and wrote a small library routine that computes the time spent computing the solution. The program runs each method twice (because the “JITter” compiles methods to native code upon the first invocation of a method), and measures the execution time each time.

So, I implemented 5 different solutions to the problem.

   public static void BruteForceMethod() {
    int total = 0;

    for(int i = 1; i < 1000; i++) {
      if((i % 3 == 0) || (i % 5 == 0)) {
        total += i;
      }
    }

    Console.WriteLine("The answer is {0} (brute force method).", total);
  }

This is what I call the “brute force” method. It is simple: It checks every single number to determine if it is evenly divisible by either 3 or 5. If so, it adds it to the total. Pretty clear.

  public static void SetMethod() {
    int total = 0;

    for(int i = 3; i < 1000; i += 3) {
      total += i;
    }

    for (int i = 5; i < 1000; i += 5) {
      if(i % 3 == 0) continue;
      total += i;
    }

    Console.WriteLine("The answer is {0} (set method).", total);
  }

This method, I call the "set" method, because it breaks down the space a bit; instead of stepping through 1 to 999 using an increment of 1, it steps through 3 to 999 with a step of 3 and then 5 to 999 with a step of 5. In the second pass, it checks to see if the number is _also_ evenly divided by 3, and if so, doesn't add it a second time.

  public static int HelperMod(int target, int modBy) {
    int tmp = target / modBy;
    return(modBy * tmp * (tmp + 1) / 2);
  }

  public static void HelperMethod() {
    int target = 999;
    int total = HelperMod(target, 3) + HelperMod(target, 5) -
      HelperMod(target, 15);

    Console.WriteLine("The answer is {0} (helper method).", total);
  }

These two methods together are the “helper method”. I didn’t come up with this one, one of the other problem solvers did, in Ruby (though their implementation was a slight bit different, the method of operation is the same). This one uses more in-depth math to perform the computation, instead of a larger number of simpler math operations. Specifically, the function:

… where x is the interval and y is the inclusive upper limit on the natural number, provides the result. The middle value is the calculation I extracted from the Ruby code, and the simplified form on the right is how it’d be done if it were being done by hand (and is far less confusing for humans).

  public static void TableMethod() {
    int[] numbers = { 10, 100, 102, 105, 108, 110, 111, 114, 115, 117,
                      12, 120, 123, 125, 126, 129, 130, 132, 135, 138,
                      140, 141, 144, 145, 147, 15, 150, 153, 155, 156,
                      159, 160, 162, 165, 168, 170, 171, 174, 175, 177,
                      18, 180, 183, 185, 186, 189, 190, 192, 195, 198,
                      20, 200, 201, 204, 205, 207, 21, 210, 213, 215,
                      216, 219, 220, 222, 225, 228, 230, 231, 234, 235,
                      237, 24, 240, 243, 245, 246, 249, 25, 250, 252,
                      255, 258, 260, 261, 264, 265, 267, 27, 270, 273,
                      275, 276, 279, 280, 282, 285, 288, 290, 291, 294,
                      295, 297, 3, 30, 300, 303, 305, 306, 309, 310,
                      312, 315, 318, 320, 321, 324, 325, 327, 33, 330,
                      333, 335, 336, 339, 340, 342, 345, 348, 35, 350,
                      351, 354, 355, 357, 36, 360, 363, 365, 366, 369,
                      370, 372, 375, 378, 380, 381, 384, 385, 387, 39,
                      390, 393, 395, 396, 399, 40, 400, 402, 405, 408,
                      410, 411, 414, 415, 417, 42, 420, 423, 425, 426,
                      429, 430, 432, 435, 438, 440, 441, 444, 445, 447,
                      45, 450, 453, 455, 456, 459, 460, 462, 465, 468,
                      470, 471, 474, 475, 477, 48, 480, 483, 485, 486,
                      489, 490, 492, 495, 498, 5, 50, 500, 501, 504,
                      505, 507, 51, 510, 513, 515, 516, 519, 520, 522,
                      525, 528, 530, 531, 534, 535, 537, 54, 540, 543,
                      545, 546, 549, 55, 550, 552, 555, 558, 560, 561,
                      564, 565, 567, 57, 570, 573, 575, 576, 579, 580,
                      582, 585, 588, 590, 591, 594, 595, 597, 6, 60,
                      600, 603, 605, 606, 609, 610, 612, 615, 618, 620,
                      621, 624, 625, 627, 63, 630, 633, 635, 636, 639,
                      640, 642, 645, 648, 65, 650, 651, 654, 655, 657,
                      66, 660, 663, 665, 666, 669, 670, 672, 675, 678,
                      680, 681, 684, 685, 687, 69, 690, 693, 695, 696,
                      699, 70, 700, 702, 705, 708, 710, 711, 714, 715,
                      717, 72, 720, 723, 725, 726, 729, 730, 732, 735,
                      738, 740, 741, 744, 745, 747, 75, 750, 753, 755,
                      756, 759, 760, 762, 765, 768, 770, 771, 774, 775,
                      777, 78, 780, 783, 785, 786, 789, 790, 792, 795,
                      798, 80, 800, 801, 804, 805, 807, 81, 810, 813,
                      815, 816, 819, 820, 822, 825, 828, 830, 831, 834,
                      835, 837, 84, 840, 843, 845, 846, 849, 85, 850,
                      852, 855, 858, 860, 861, 864, 865, 867, 87, 870,
                      873, 875, 876, 879, 880, 882, 885, 888, 890, 891,
                      894, 895, 897, 9, 90, 900, 903, 905, 906, 909,
                      910, 912, 915, 918, 920, 921, 924, 925, 927, 93,
                      930, 933, 935, 936, 939, 940, 942, 945, 948, 95,
                      950, 951, 954, 955, 957, 96, 960, 963, 965, 966,
                      969, 970, 972, 975, 978, 980, 981, 984, 985, 987,
                      99, 990, 993, 995, 996, 999 };
    int total = 0;

    foreach(int num in numbers) total += num;
    Console.WriteLine("The answer is {0} (table method).", total);
  }

In this method, I precomputed the numbers that needed to be added together (using (for i in 3 5; do for j in $(seq 0 $i 999); do echo $j; done; done) | sort | uniq | tr '\n' ',' | sed 's/,/, /g;s/, $/\n/g;', goodness Unix is great) and put them into an array, adding each of the values of the array together to come up with the total.

Finally, there’s this method:

  public static void PrecomputedMethod() {
    int total = 233168;

    Console.WriteLine("The answer is {0} (precomputed).", total);
  }

This method simply contains the answer and prints it.

So, what can be learned from this? Let’s take a look at a single run:

Project Euler: Problem 1
Find the sum of all the multiples 3 OR 5 that are below 1,000.

The answer is 233168 (brute force method).
Execution time: 00:00:00.0184820

The answer is 233168 (brute force method).
Execution time: 00:00:00.0000600

The answer is 233168 (set method).
Execution time: 00:00:00.0001280

The answer is 233168 (set method).
Execution time: 00:00:00.0000100

The answer is 233168 (helper method).
Execution time: 00:00:00.0001300

The answer is 233168 (helper method).
Execution time: 00:00:00.0000070

The answer is 233168 (table method).
Execution time: 00:00:00.0001870

The answer is 233168 (table method).
Execution time: 00:00:00.0000110

The answer is 233168 (precomputed).
Execution time: 00:00:00.0000650

The answer is 233168 (precomputed).
Execution time: 00:00:00.0000070

So, we run through each of these twice, and display the time computed to be the runtime for each method. (I’ll show just how I did that here in a little bit, it was pretty simple.) The obvious thing is that for each method, the first execution was a good deal slower than the second method, usually by more than an order of magnitude. That shows the delay created by JIT compilation of the method. It also shows that the very first brute-force method JIT takes longer than any of the others. My hypothesis there was that the methods called after the first one benefited from some sort of caching (probably at the hardware level) in working with the opcodes for the for loop in the Mono runtime. It wouldn’t make sense for it to be a delay in JIT itself, since by the time we start measuring things, the JIT has already compiled at least 5 different methods, and probably more depending on the internal implementation of things such as System.Console.Write and System.Console.WriteLine. By changing the order of execution and moving the brute force method to be the second one tested, that would seem to be the case:

Project Euler: Problem 1
Find the sum of all the multiples 3 OR 5 that are below 1,000.

The answer is 233168 (set method).
Execution time: 00:00:00.0179120

The answer is 233168 (set method).
Execution time: 00:00:00.0000600

The answer is 233168 (brute force method).
Execution time: 00:00:00.0001400

The answer is 233168 (brute force method).
Execution time: 00:00:00.0000410

The answer is 233168 (helper method).
Execution time: 00:00:00.0001310

The answer is 233168 (helper method).
Execution time: 00:00:00.0000080

The answer is 233168 (table method).
Execution time: 00:00:00.0001760

The answer is 233168 (table method).
Execution time: 00:00:00.0000100

The answer is 233168 (precomputed).
Execution time: 00:00:00.0000670

The answer is 233168 (precomputed).
Execution time: 00:00:00.0000080

Now, the set method contains the longest delay. That is an interesting find; it would seem to confirm that the JIT benefits somewhat from some form of caching. Just what caching, I don’t exactly know, and I don’t know how to find that out just yet.

In any event, we know that the first time we call each of these methods, the JIT converts the method from IL bytecode into native object code, and then executes the native object code. Pretty nice, that. On subsequent runs of the method, no more JIT needs to be done, and so the method’s object code executes directly, since that is retained in memory by the Mono runtime.

Now, let’s take a look at the runtimes for each method type, specifically, the second run of each, from the first run above:

These are interesting in that they show what’s more efficient, relatively speaking. The “brute force” method is clearly the loser. Though if you look at the second output above, you’ll notice that the “set” method lost when it was first in line. Still, though, we can see from the two sets of numbers that the brute force method takes longer than the set method even when run in the same place. That’s not surprising, because it is a stupid for loop compared to two more intelligently designed for loops. The fact that the table method performs fairly similarly to the set method was expected, and shows that they are roughly the same in terms of efficiency.

Perhaps what I found the most interesting was the execution time of the helper method. It’s nearly the same (well, it is the same, in this run) as the static answer, even though there are 3 additional function calls. The JIT compiler can inline methods when it sees fit, and I am going to wager a guess that is exactly what it did, so the helper method winds up being nothing more than a calculation which then gets printed. The interesting part is that both methods compile to what one would expect in IL bytecode (that is, the method calls and such are preserved), so any inlining that is happening would be happening at runtime, when the JIT compiler evaluates HelperMethod() and HelperMod().

Alright, so this made me more curious, and I wanted to see what would happen if there were an AOT compiled version of the code on my system. So I compiled both the executable and the small library that I made to time execution using Mono’s --aot flag. I saw no difference; I am going to guess for the time being that the version of Mono in Intrepid doesn’t do AOT quite right. It’s lacking some of the functionality of Mono 2.0 that enables you to force the AOT binaries to be the only thing executed, and so I am not quite sure what is happening there. It’s not majorly important for my purposes, though, so I’m just not going to pursue that any further now.

The only lingering question left is “how efficient is the JITted code compared to native binary code?” To answer that, of course, the program has to be done in C. Not a big issue, here’s the output from a compiled C version, compiled using gcc -o p1 p1.c, along side with the first run from earlier, with the program info header removed for space reasons.:

The answer is 233168 (brute force method)
Execution time: 00:00:00.0000460 sec

The answer is 233168 (brute force method)
Execution time: 00:00:00.0000300 sec

The answer is 233168 (set method)
Execution time: 00:00:00.0000110 sec

The answer is 233168 (set method)
Execution time: 00:00:00.0000110 sec

The answer is 233168 (helper method)
Execution time: 00:00:00.0000050 sec

The answer is 233168 (helper method)
Execution time: 00:00:00.0000070 sec

The answer is 233168 (table method).
Execution time: 00:00:00.0000160 sec

The answer is 233168 (table method).
Execution time: 00:00:00.0000090 sec

The answer is 233168 (precomputed).
Execution time: 00:00:00.0000030 sec

The answer is 233168 (precomputed).
Execution time: 00:00:00.0000030 sec

The answer is 233168 (brute force method).
Execution time: 00:00:00.0184820

The answer is 233168 (brute force method).
Execution time: 00:00:00.0000600

The answer is 233168 (set method).
Execution time: 00:00:00.0001280

The answer is 233168 (set method).
Execution time: 00:00:00.0000100

The answer is 233168 (helper method).
Execution time: 00:00:00.0001300

The answer is 233168 (helper method).
Execution time: 00:00:00.0000070

The answer is 233168 (table method).
Execution time: 00:00:00.0001870

The answer is 233168 (table method).
Execution time: 00:00:00.0000110

The answer is 233168 (precomputed).
Execution time: 00:00:00.0000650

The answer is 233168 (precomputed).
Execution time: 00:00:00.0000070

Clearly, that is faster. Of course, we already knew that would happen because 100% of the compilation for a C program is done by the C compiler, the assembler, and then the binary is put together by the linker for a specific operating system and CPU architecture combination. But… look at those numbers again, and there is one thing that is unexpected—at least it was to me. The helper method has the same execution time when JITted under Mono as when compiled ahead of time by GNU GCC for one of the invocations. The set method was pretty close, too. And comparing the JITted code execution time with the code output by GCC shows that Mono’s JIT compiler isn’t doing a horrible job with compiling things. It could do better, but it’s not doing bad.

If you want to check it all out yourself, the both the C code and the C# code are available here: Source code for Euler Problem 1 (C and C#). But, having learned all of this, and compared the two programs together, I have to say that I am pretty pleased to see how well Mono performs something like this. It’s surely a very, very small sample, and it’s not an authoritative, tell-all benchmark—just some informal little tests, running things a few more times, and seeing what happens. But it’s telling enough to help figure things out when trying to determine, say, how much the delay on the first invocation to a method will be, or something like that. It’s fascinating to me, anyway. And I think that it shows the potential that the Mono runtime has, too. There’s no (technical) reason to avoid using Mono, clearly.

But, managers like metrics. So, the question remains, how do you metricize jobs like programming jobs, where quantitative measures are relatively meaningless? Quality is what matters in software production, not quantity of code, and quality of software is something that (while widely agreed upon in terms of its definition) is a non-objective measure. Some might wonder how I can call it subjective. Easy: Nearly everyone would agree that writing readable code is a good thing when it comes to programming. However, two programmers can differ widely in their opinion of what’s easily readable and what’s not. Or, programmers A and B might agree that code duplication is a bad thing, but programmer A might take that to mean that code that performs similar tasks should be consolidated and simplified, and programmer B might take it to mean that only duplicate code is bad, and not necessarily duplicate functionality (I have seen that happen before, and honestly the position programmer B takes is easily defensible if programmer B claims that even apparently identical code has different technical purposes, but then things boil down to semantics).

Even worse, programming is largely arbitrary: Ask ten programmers to write code to perform action x, and you’ll likely get ten very different implementations. These variations can even occur among programmers who have similar experience and knowledge, because everyone has a different writing style. So, programming can be likened to writing something in human languages in a way. How are authors of human written-language works “graded” on their writing performance? I’d imagine that it’s probably also something arbitrary, maybe popularity. However we know that popularity is bad compared to various other things (see ¶ 8 here for an example), depending on the goal you are trying to accomplish. So, if the goal is high-quality software, popularity isn’t necessarily a marker of that, and isn’t appropriate either. After all, Windows is the most popular system around—and it’s not what I’d call of remarkably high quality.

I’d like to think that I have an answer to it, but I don’t, really. If I were going to hire a programmer, my test for them would be to take something real-world, fairly trivial in nature in the language I was hiring them for, and ask them to make a modification that suited a particular purpose. If I liked the result, I’d (probably) hire them. If I thought it was nasty and ugly and it didn’t need to be, I probably won’t hire them. Either way, I’d probably award more weight to their code if they can successfully (again, in my view) defend the reason they wrote it the way that they did, and held their explanation well, and maybe even convinced me of it if I wasn’t to begin with. After all, there are some times in a programmer’s life where they will have to write code that they must defend in an argument. That’s the nature of the best—and it’s highly subjective.

What sort of metrics have you seen in the field of programming? Is it lines of code, or average successful implementation time of features/projects, or number of hours that other people have to spend to understand your code? What about in single-programmer environments (though I am not sure that I know anyone out there in that situation)?