So, this is a pretty simple-sounding network, yes? It should be—it is just a handful of computer systems. However, there is a problem—a pretty large one, I think. There is very little in the way of either policy or convention on the network. Some users use their documents directory for storing their documents, others store their documents on their desktop, others somewhere differently altogether on the C drive, others on a network share in a public storage space. There are different versions of software on the various machines, more-or-less updated when someone thinks about it, I think.

This is a perfect scenario which shows why a business network should be centrally managed in some form. Note that I am not saying that each machine should be a bit-for-bit mirror image of the one next to it, though that is certainly a possibility. I think that people should be able to use their own choice of things like email client or Web browser software, because everyone is different. But when you have different client applications fulfilling a role on the individual workstations, you have to take a centralized approach to ensuring that things like the email is all backed up.

Furthermore, if you don’t take a centralized approach to backing up such data, it is very difficult to centralize the network storage. Think about adding a domain controller (that is Windows speak for a central server which handles authentication and authorization, as well as file and printer sharing and things like roaming profiles) to such a network. I expect that with multi-gigabyte mail files, things will be very slow at first—and that likely the only fix for them that is going to be viable in the long term is to centralize more infrastructure.

I am too tired to expand more on my thoughts on what I have learned and where it is heading, but the Reader’s Digest version of the point reads something like this: If you are a small to medium sized business, make sure that you have someone who is competent in both system and network administration, and make sure that they are a part of your business from day one. Like writing software, building up a technical infrastructure without careful thought and design is hazardous and comes with many hidden and unpredictable costs. It is best to head those things off right from the start; to delay only amplifies the cost of fixing the underlying problems and puts oneself in the position where fixing one issue can have a domino-like effect and create more new problems.

For my current situation, I think I am going to have to seriously re-think how this whole setup is done. What I do not yet know is how to quickly and efficiently bring things into shape. A bit of training and education may be required, and certainly the removal of a lot of unnecessarily-granted privileges on the workstations. That, too, should be something caught early-on: do not let every person in a business run with administrator privilege, unless they are an administrator (and even they should only run with administrator privilege when they are actually doing something that requires that privilege). If everyone is an administrator, there is little to no control on how things are done in a network, and it can get messy.

I have a lot more reading to do, as well.

Well, anyway, it is way past my bedtime.  Time for sleep.

The Free Software Foundation objected to UDI for various reasons. Mostly, I think, it was because they were afraid that people who are not them would choose to use drivers that were non-free. As I’ve written about before here, there are people who think that forcing people to use free software is somehow freedom—and I will not go into it in any great depth here, because I have done that in the past. Suffice it to say that forcing anything is not freedom; it cannot be freedom. So, the Free Software Foundation, I think, was really afraid that they would have to do more work to be able to stick to their own requirement of using 100% free software on their own computer systems. (And hey, Roy, if you’re reading—I’m not saying that the FSF is wrong, and I’m not putting myself in a position opposite of that of the FSF. I suspect you think so anyway, but hey, I just figured I would point that out.)

Even if the free software operating systems did not adopt the UDI specification, why didn’t proprietary operating systems? This is perhaps the most puzzling thing to me. It seems that in this event, none of the operating systems—free or proprietary—did what would have made sense. After all, even if only Apple and Microsoft adopted a common device driver specification, that would save a lot of time, effort, and improve user experience all the way around. Apple users would be able to use all the hardware that Microsoft users could use—and the inverse would also be true. The amount of time that device driver authors would have to spend writing and debugging driver code would go way down—free software driver authors would be able to write a driver once, for example, and all systems (including free software systems that chose to support the specification) would benefit.

I could see an objection of a driver specification that was binary-only. However, UDI was not—it mandated an ABI so that drivers that are built for a particular platform were binary-compatible with operating systems on the same platform, but it also mandated an API, so that drivers would be source compatible to any operating system that implemented the specification, on any platform. That by itself would seem to me to be positive motivation to hardware manufacturers to release the source code to drivers so that they can support operating systems that are on platforms that do not exist yet, or have not been considered (or have been considered to be nonviable or unsupported platforms).

So, I have to wonder why a common device driver specification was never implemented in various operating systems. It would seem to be a common sense thing, especially given that there are so many operating systems. It would make the coexistence of operating systems a lot easier, and it would promote choice. It might encourage bits of proprietary code on free software operating systems, but it would also enable people to drop the excuse that “free operating system x does not support device y”, and would as a result potentially increase the number of free software programs and operating systems in use, even if there is a minor cost in terms of certain drivers. And those drivers could always be replaced—a common driver specification would make it easier to understand the structure of drivers generally, and make it easier for lawful, clean-room reverse engineering to be done on those drivers.

Imagine, for example, if drivers for graphics cards, TV tuner cards, video and audio encoding/decoding cards, modems, storage chipsets, motherboard chipsets, USB chipsets, IEEE-1394 chipsets, graphics tablet devices, touch screens, debugging interfaces, network devices, and so forth were all written to a common specification, it would reduce the amount of code which needed testing. It would increase user choice in both hardware and operating systems—something which I still hold is quite likely the most valuable freedom we have. It would increase reliability, since the users of Windows, OS X, Linux, the various BSD systems, and other, not-so-mainstream operating systems would be able to run the same driver code and collectively supply debugging information and perform testing in a multitude of environments. It would increase security, because then common code that is well-known could be used on all platforms and not just the one it was written for. It would do for device drivers what POSIX has done for user-mode application software. I do not believe that I could be convinced that this would be anything other than a good thing.

Also, it could bring back old operating systems.  Imagine what life could be like, for example, if OS/2 had a “UDI driver” written for it, and it could then take advantage of newer drivers never intended for it. Or any other very old operating system which is no longer supported and could still be useful, for any of a number of reasons…

“You can have freedom without choice.”

That someone could even come up with this one is just amazing to me. Note that this is not an exact quote, but it is the summary of Friday’s topic. For example, this summary comes from the idea that Canonical is bad for considering making mainstream non-free software available for Ubuntu based on user preferences. It does not matter who came up with it, of course, but the important thing is that it be called what it is: patently absurd. The ability to choose is a major part of what freedom—or liberty—is. If you cannot make a choice on a matter, then by definition you do not have freedom in the context of that matter. It is quite simple and self-explanatory. Canonical is seeking to increase freedom here, not take it away. Some people actually want to use non-free software; others may not want to use it, but aren’t aware of alternatives. The latter group of people should have our focus with regard to education (but then we should let them make the choice for themselves!).

Note that I am not one of these people: I would rather use free software because of the liberty it gives me that I have come to expect over the years. But I am not going to tell someone else that they are harming me because they would rather use non-free software that is familiar to them. All I can do is show them that there are free alternatives that exist. I cannot—and I will not—make them use it or make them feel bad for not using it. I may not like proprietary software for a variety of reasons, but I will defend people’s right to use it just as I will defend even a stupid person’s right to spew nonsense by way of speech or written word. In other words, “I disapprove of what you say, but I will defend to the death your right to say it,” or perhaps more appropriately, “I [may] disapprove of what [software you run], but I will defend to the death your right to [run] it.” Even I use a package or two that is proprietary in nature (though it is looking like I will not have to do so for much longer, given the efforts to replace these packages with equivalent free software).

It is worth it to note that by adding non-free software to Ubuntu, the free software that is already there does not change. The mere existence of non-free software within its repositories does not make Ubuntu somehow bad or evil. It would add choices that do not currently exist, and that one such as myself or yourself can certainly opt out of—I most likely would, for the most part, as I do not need to depend on non-free application software, and I only use non-free drivers if I have hardware where anything else is nonviable (and only until there are functional free software drivers). Did you know that Ubuntu has an option in the installer to only install free software? Can you say that for your favorite desktop operating system distribution, whatever that might be?

The response to this idea, then, is that without choice, there is very little—if any, really—freedom. The thing that gives us freedom with free software is that we are able to to download the source code, to review/audit it, to change it to fit our needs or fix a problem, and to share those changes. If we cannot do those things, then it is not free software; see the essential freedoms. But non-free software inside a distribution is not something that should not cause you great consternation even if you are among the most dedicated of freedom advocates, for if you are a true advocate of freedom then by definition you must respect a computer user’s freedom of choice. Remember that we choose to run free software because of the benefits it brings to us; we choose to improve upon free software for much the same reason. Eventually, I think that free software will once again become the norm for computer software, on merit alone, for no other reason than the development, release, and usage of free software is a highly practical solution for many things ranging from library code to application software to complete operating systems. It is worth noting that free content—which is similar in concept to free software, which itself is merely a specific application of freedom itself—also appears to making major headway towards becoming mainstream; it is doing so more quickly than free software is, but there is every reason to believe that free software will follow, for it is already.

An Example

Imagine that you are in a store, because you need some milk for dinner some night. You always get 1 gallon of 2%. But, the store has stopped carrying it, because more people buy whole milk and they were throwing away the 2% milk—demand was low, supply got to be too high, so they just stopped carrying it altogether. You leave the store and head to the next in the same town and you find the same thing there. You have a choice of stores to go to, and you have made the choice to go buy yourself some milk. But there is only one type of milk. You no longer have the choice to buy 2% where you are, and so effectively, your freedom to buy it has been taken away. (Of course, you can make 2% milk from whole milk (and make whole from 2% even, or even butter), but I suspect just as many people want to do that as want to write their own free software that they demand simply must exist, but doesn’t yet).

Now, the point here is that there is more than one freedom in play: the freedom of the store to stock (or not stock) various products, which affects your freedom as a consumer to buy the product you want. In the case of software, and choice, if the software you are running gives you all the choices you want, then it fits your needs. If it does not, then you are not going to be able to use it the way you want. Now you have two choices: you can do the work that it would take to make your desired choice possible, or you can use another system (free or proprietary) that will give you the choice that you want. Many people will choose the latter, especially if they are non-programmers. Though I’ve seen programmers also choose to use proprietary systems for something that they could themselves implement. That is their choice, of course. After all, if you really wanted 2% milk, you would have the same choice: make it yourself, or drive to the next town over which might have it available for you (assuming that there is some in stock and that the stores neighboring towns have not also decided to stop stocking 2% milk).

Ubuntu One: The Reason Behind This

This discussion came up because someone on identi.ca made the claim that Canonical is forcing proprietary software into Ubuntu by way of the Ubuntu One client software. I cannot even begin to state just how woefully incorrect this point of view is. First off: the only thing added to Ubuntu is the ability to connect to Ubuntu One, and the software that was added to Ubuntu do to that is licensed under Version 3 of the GNU General Public License. The claim made in response to that was that Ubuntu One is only partly free software, because the server is somewhere else and has not been released. As we shall soon see, that claim is nonsensical—it depends on an extremely naïve view of how software actually works in order to make sense, really.

So, first things first: Ubuntu One, which was added to Ubuntu 9.04, is not proprietary software. The proof rests in the fact that it GNU GPL v3.0, and we know a priori that software licensed under the GPL is free software, so we do not need to go further on that point.

Now, because the software in question added to Ubuntu is free software, we can read it. The essential freedoms granted to us by truly free software ensure this, and the GPLv3 is indeed a truly free software license because it grants those freedoms. Because we are able to study the software and see how it communicates with the server. Once we know how to communicate with the server, we can write that up and design a server that communicates exactly the same way. From there, it is just a matter of patching the sync dæmon that is in Ubuntu to talk to an arbitrary, Ubuntu One compatible server. To determine how to do that, one need only read the Python source code contained in the python-ubuntuone-storageprotocol and python-ubuntuone-client packages. If you do not know Python well, you might expect to spend several days doing that, but if it bothers you so tremendously that you are going to practically start a flame war over it, you may find it worth it to do so.

Of course, the other side to that is this: if you really want Ubuntu One to talk to an arbitrary server that runs free software, and you want that free software to be written, you can fund the effort to write the free software. Approach a proficient developer somewhere out there on the Internet and ask them how much they’d charge to write a server for Ubuntu One. You might not be able to afford the fund the project entirely, but if you get a number from someone, you can start a coordinated effort to raise the funds. If you are lucky enough to be able to fund the whole project, then do so: it is but one way that you can help provide something back to the community. This does not apply to just an implementation of the Ubuntu One protocol, it could apply to anything that you see that is missing and needs to be created. Or you could spend time learning what you need to learn to pick up the project yourself, if you care for the project that deeply. The most important attribute that a person can have in order to get started with development is motivation—James Westby reminded me of this a couple of years ago, something which I had forgotten.

Perceptions: Another (Possible) Reason

It was suggested to me that another possible reason that people would object to having non-free software inside an operating system distribution such as Ubuntu is that they are afraid that the proprietary options have higher quality, or offer superior features, or provide functionality that is not offered by any existing free software. Thus, they have this perception that by adding such non-free software into a distribution like Ubuntu, people will automatically use and prefer it over free software. This simply is not the case. Sure, some people will use iTunes if it is available on Ubuntu. Maybe many people would. I might even do so, if it were legally available for me to use that way and if it supports the purchase of DRM-free music. However, if there were a free software client for the iTunes store, I’d much prefer to use that. To my knowledge, however, there is no such thing that exists.

If there is not a free software alternative for a non-free component inside a distribution of software, if you are offended by that, then by all means, create a free software alternative for it! As mentioned above, you can start on such a project’s development, or you can look for people that would be interested in volunteering for it and coordinating them, or you can put up funds to pay developers to implement it. If you have money, this can be the easy part: find someone who is willing to accept payment for the service of implementing the free software alternative for whatever it is that someone else has funded, wrote, and released as proprietary software. It is not like free software is developed without cost (and if you think that it is, then you seriously do not understand what free software is or anything about the world of free software and have no standing to be getting mad when a company spends money writing software and does not release it as free software. You can try to write companies that write such software and ask them if they will give you any form of written specifications for the software, or an interface definition, or something along those lines. The worst thing that could happen is that you will be told “no”. And do so nicely, or they’ll be more inclined to tell you “bugger off” instead of simply “no”.

“Allowing users to choose proprietary software is anti-freedom.”

Nothing could be farther from the truth; it is the same, in fact, as the above statement that one can have freedom without choice. For example, if Ubuntu adopts iTunes and makes it so that you can “sudo aptitude install itunes” in the future, that is not a bad thing! How can it be—It contributes to the ability to choose, and thereby contributes to the freedom of the end-user. If you are a die-hard free software supporter and do not want to run non-free software on your system, then there is a very simple solution for you: simply don’t install it.  That is a valid solution to the problem. There are tools already available that can be run as a cron job and report on any non-free software that you might have accidentally (or even intentionally) installed. If you are worried about additional non-free software getting into Ubuntu, then help enhance those tools. Or write a GUI front-end for something like the virtual RMS program and work to get that included into Ubuntu as well, perhaps something that can run every time you login to the computer, or that runs as a persistent process that watches the package database on your distribution of choice for updates and then checks to see if newly installed software is non-free and alerts the user. Of course, it’d be most effective as an opt-in system, and not an opt-out one where it would just be annoying.

There is no way, then, that freedom is actually reduced in this way when another choice becomes available. If iTunes were to be included in the repositories (and I suspect it would be, like the restricted, universe and multiverse repositories, a separate opt-in repository; perhaps simply “proprietary” would be fitting), this does not reduce your ability to choose to run a free software media player and manager like Banshee, or Rhythmbox, or even AmaroK if you are so inclined to run that KDE stuff.

Once upon a time, FUD (fear, uncertainty, and doubt) was the tool of Microsoft. We (the free software world) completely hated it when Microsoft would put out FUD, because we would then have to fight that FUD by way of explanation and demonstration. Well, some time ago, a subgroup of the free software world decided to start using FUD themselves—it was done with Mono, and it is being done now with just a survey asking people what sort of software they would like to see in Ubuntu. Now, those of us who are left who are advocates of liberty—both personal and societal—are stuck potentially fighting two battles. One with Microsoft’s FUD—such as the constant notion that you have to pay for software—and one with the “free software evangelists” FUD, who have even gone so far as to say that people should not use certain types of free software (the one who calls himself “The Open Sourcer” even still today tells people to remove certain truly free software from their systems). The truth is somewhere in the middle, between these two ends of the spectrum.

Conclusion

Back to the point at hand: to say that giving a person a choice is a constraint on that person’s freedom, that is doublespeak.; it is saying that “slavery is freedom,” albeit to a lesser degree than that very melodramatic extreme—it simply does not make sense. The concept just does not make sense unless the words that are used to express the concept are dramatically redefined to mean things vastly different from what standard English dictionaries define them to be. The only reason that one has to try to convince someone that additional choice is a constraint on freedom is to try to convince people of things that are not true; to install fear, uncertainty, and doubt into people. This is the sort of behavior that—no matter what community it originates from—is completely immoral, unethical, and absolutely unacceptable. It’s dishonest, and for those of you who know me personally, you know what I think of dishonesty.

Let’s get something straight, people:  Freedom is a lot more than just free software or choices.  Freedom is a concept that most people who have gotten on the “free software” bandwagon of late just don’t understand!  It is awfully frightening; even RMS feels that freedom cannot be had without choice—his writings of the past 20+ years make it clear that he feels that choice is key, and these people are calling him “leader” without even knowing what it is he is fighting for!

It seems that the "free software" crowd these days assumes that if you do not hang off of their every word, accept their every inconsistent argument, and bow to their ways of making every last line of code free and GPL’d, you are the inconsistent one (because you’re inconsistent with them).  Even more to the point, I am utterly sick of running into people who learned about “free software” bloody yesterday and are now going “Oh, you bad NVIDIA user, you’re evil, you’re wrong, you’re hurting me just by using NVIDIA.”  Sure, asshole.  I’ll remember that the next time I send data off to a project working on free drivers—I’m hurting you, yep.  So, then, why am I bothering to contribute my time, tell me?  And when people rant but then get offended when you tell them that they’re ranting and to hurry up and get to the point, and they get offended by that? What is that, anyway? "Oh, I’m so thin skinned I can’t even listen to you tell it like it is?" is that what that is? I don’t know.  In case you couldn’t tell by the category, this post is a rant. (though much larger than the rants that are possible on services like identi.ca).

I am going to guess that it is time for me to pick up my book on freedom and start working on it again.  The world needs to be reminded what freedom really is; far too many have too limited a view of what freedom is and only apply it to certain situations, certain circumstances, or certain things.  Or think it is some boundless fountain of infinite choice.  Where are the people that don’t sit at those two extremes‽

And in the meantime, I have to figure out, seriously, why I bother.  My clients want to move to free software stacks, and I am doing so for them over time (gasp! by using their proprietary software to study it, learn about it, and reproduce it—but I suppose that is “harming” the new generation of free software users, too, right?).

What am I missing here?  I don’t know.  I know this: It is late and I am cranky. Night.

The server was down for a while because of the move, and then we had to wait for Comcast to do its thing and get us a new cable modem (why they don’t just let you move it from one place to the other is beyond me…).  And now, we’re making various tweaks to try to fix things up here while we’re here—a form of rent, I suppose.

Christmas was good.  Ben got lots of stuff, I got some PostgreSQL manuals in dead-tree form.  I’d like them in PDF on a Kindle DX, but with those being so awfully expensive… well, it’s better just to have something I can hold in my hand.

Anyway, there really isn’t all that much to write right now.  Many of my projects have stalled over the holiday, and they’ll be picking up again soon.  Libvfcgi is moving along slowly (no thanks to myself; I have barely had any time to work on it!) and there is one other person who is working on it at the moment and doing a quite excellent job.  I’d like to have 0.0.2 out soon, but we’ll see how that works.  First I have to get my new workstation setup so that I can work.  Off to try to work on the wireless network here!

Let’s see if we make it past another 24 hours without something strange going on…

Through something of a comedy of errors, the server suffered some strange software issues that prevented it from working this weekend.  There was a bug in a recent update to the server software (running testing software) and that caused longer downtime than it should have due to various interactions between things on the server.  The good news is that this is mostly fixed.

Additionally, this downtime has taught me that there is yet still more to do in terms of getting the server able to stand back up on its own.  I’ve simplified the server’s setup a bit, and I have to write some scripts and other little glue here and there to tie down some of the things I’m doing so that the server can do things like go down and come back up without issues, all by itself.  Getting that done would be generally a good thing.  First things first, I have to figure out a decently reliable way to shutdown the system without having to do something like kill the containers and not give them the chance to cleanly shut down.  Ideally, there would be some sort of command that could be run on the system that would enable the containers to be shutdown.  This is slightly challenging, because you cannot just chroot into the directory tree that the VMs are running in and kill processes, because things like /proc inside the container aren’t visible to tools running on the host.  Oops.

So, I have some work left yet in terms of getting the server going the way it needs to be again.  After that, I’ll be working on it over the next couple of weekends to try to increase its robustness, and so planned weekend downtime for *.trausch.us and www.mischiefinoverdrive.us can be expected. 

Base Position

Mailing lists should Just Work.  Mailing list management servers are software just like any other software, and they can be made to do what is convenient for us humans, and they should be made to do what is convenient for us humans.  Mailing list software should cater to the needs and desires of all its users, regardless of technical aptitude or choice of MUA (mail user agent, email client) software.  Mailing list software should be updated to operate with the expectations and flexibility expected of software used in the 21st century, and provide more options for interaction both technical users and non-technical users alike can love and use.

It is my personal belief that both mailing list software and MUA software should follow the established philosophy of "do one thing and do it well".  That means that MUA software should be expected to only be capable of being a basic mail user agent, and that mailing list server software should be expected to be capable of everything required to manage a mailing list for end-users, technical users, and list administrators.  A finer-grained permission model in mailing list server software could be adopted to make some of the proposed features even more convenient and get rid of the "administrator" vs. "user" line (i.e., "Jack can unsubscribe users", or "Carla can list messages in the mailing list’s pending approval queue", or whatever).

Furthermore, it is my belief that enabling convenience for end-users does not necessarily mean taking advanced functionality away from technical users: with careful thought and design, both needs can be met with a single standard.  It is also my belief that the most important part of any proposed design is the consideration of users ranging from complete newbies to people that have no time in the world but to learn about every feature that is available and find applications for them.

It is not my belief that mailing list management software should be extended to do things beyond the realm of mailing list management, though I believe that mailing lists are also currently very narrowly defined to be a mirror of public forums such as Usenet, and mailing lists certainly do not need to be restricted to such a model when a little bit of extension can bring a great deal of benefit for all users involved.

In short, mailing list management software should maximize flexibility while minimizing the amount of knowledge required to initially begin using the mailing list system (at the end-user level, at the technical user level, at the administrative level) and regularly interact with the system.  Convenience, ease-of-learning, continued ease-of-use, flexibility in usage patterns/workflows and use cases are all equally important.

Functionality Target Proposal

Let us assume the following statements are true:

1. Mail User Agent software can only be relied upon to have barebones message handling functionality that does not encompass mailing-list aware logic.
2. Users can only be relied upon to have barebones message handling functionality with only message-level context, such that they know that they are responding to a message.

Now, let’s define a few other assumptions that we will take to be true with regard to MUA software:

1. MUA software properly sends either a References header or an In-Reply-To header, which facilitates threading (should not be an issue, since RFC{822,2822,5322} specifies the header and it should be supported universally).
2. MUA software gives the end user no ability to send custom headers as a base feature.
3. MUA software gives the end user no ability to modify headers during message composition other than To, Cc, Bcc, and Subject as a base feature.
4. Even if MUA software can do items 2 and 3 above, it is safe to assume that:
   1. It is terribly inconvenient to do so, or
   2. The user does not know how to do so.

Now, let’s define a few other assumptions that we will take to be true with regard to this ideal mailing list management software:

1. The mailing list management software remembers mailing list threads by remembering message IDs.
2. The mailing list management software may or may not fall back to identification of threads by subject line, timestamp, or some other means, depending on what is most feasible and whether or not there are any accurate huristics that can be applied to make a determination with reasonable confidence.
3. The mailing list management software is remembers messages and posters, and stores information about each message in a manner that makes filtering (e.g., filters for the display of archives, or filters for sending back out to the mailing list) on the server side possible.
4. The mailing list management software can apply a limited number of per-user preferences on the server, including multiple addresses associated with a single subscriber, mode of subscription (digest, normal), interval of digests with at least an option for daily or per-x-number of messages with x being a server-specific configuration item, and perhaps server-side support for bandwidth saving features such as server-side "killfiles".
5. The mailing list management software never requires custom message headers or interfaces other than email for management of the server or per-user preferences, though it can implement such features in addition to a base interface which depends only on an RFC{822,2822,5322} capable mailer.
6. The mailing list management server uses all allowable standard headers for list management such that MUA software implementing those standards can use the information, but does not assume that MUA software is actually compliant to those specifications, instead treating them as optional in nature.

Let’s further define a few modes of operation.  We will discuss potential implementations for these modes later; for now, we are just defining the modes of operation.  If you have a new mode to suggest, please post it in the comments section below, in the same format.  A solution for how to make the mode work is not required, but if desired can be supplied as well if it is in the framework of this post.  We’re talking about the ideal mailing list management software here, not any currently existing one. 

Here is the first (and probably most common) mode of operation, which we will call Mode 1 ("Typical operation"):

1. Bob wants to send an original post to a mailing list (e.g., start a new thread).  He enters To and Subject headers, and composes his message just as if he’s sending to an individual.  Note that current mailing lists already do that, and thus mailing lists are in this respect transparent.
2. Bob’s message hits the mailing list server and is distributed to the subscribers of the mailing list as expected.
3. Alice sees Bob’s message, and wishes to add to the thread of discussion, so she hits "Reply", and enters her message as a reply to Bob’s.  She then hits send.
4. Alice’s message hits the mailing list server and is distributed to the subscribers of the mailing list as expected.

Here is Mode 2 ("Private replies"):

1. Bob wants to send an original post to a mailing list (e.g., start a new thread).  However, the thread is intended to be a private poll, and so replies should go to Bob directly and not be distributed to the mailing list.
2. Bob’s message hits the mailing list server and is distributed to the subscribers of the mailing list as expected.
3. Alice replies to Bob’s mailing list post, which goes to Bob without being distributed to the mailing list.

Here is Mode 3 ("Private subthreads"):

1. Bob wants to send an original post to a mailing list (e.g., start a new thread) normally.
2. Bob’s message hits the mailing list server and is distributed to the subscribers of the mailing list as expected.
3. Alice responds to Bob’s message with a few queries and sends the message to the mailing list.
4. Frank responds to Alice’s queries with a few more to seek clarification.
5. Bob responds to Alice normally.
6. Bob responds to Frank normally.
7. Jo sees Bob’s response to Frank and has a question that should be addressed to Bob directly, without going to the mailing list.  Jo sends an email that should get directly to Bob and not be distributed to the mailing list.
   1. Note: mail filtering (if Bob uses it) should not filter the mail as if it came from the mailing list, but the message should thread properly if mail filtering is not considered or if the message is moved to a folder with the rest of the mailing list messages.
   2. Bob receives the response and replies to the message, which goes to Jo and does not get distributed to the whole of the mailing list.

Here is Mode 4 ("Dead thread"):

1. Bob wants to send an original post to a mailing list that is normally a discussion list, but this is an announcement and there should be no replies.
2. Bob’s message hits the mailing list server and is distributed to the subscribers of the mailing list as expected.
3. Alice responds to Bob’s message with some questions pertaining to the announcement.
4. The mailing list software informs Alice that responses were disabled on the post.

Here is Mode 4a ("Dead subthread"):

1. Bob wants to send an original post to a mailing list normally.
2. Bob’s message hits the mailing list server and is distributed to the subscribers of the mailing list as expected.
3. Alice responds to Bob’s message with some questions.
4. Bob responds to Alice’s message, answering the questions, and noting that the discussion should go elsewhere for any reason, and that the subthread is closed.
5. Richard responds to Bob’s message, trolling.
6. The mailing list software informs Richard that responses were disabled on the subthread.

Here is Mode 4b ("Killed thread"):

1. Bob wants to send an original post to a mailing list normally.
2. Bob’s message hits the mailing list server and is distributed to the subscribers of the mailing list as expected.
3. Alice responds to Bob’s message.
4. Frank responds to Bob’s message.
5. Jo responds to Bob’s message.
6. Frank responds to Alice’s message.
7. Frank responds to Jo’s message.
8. …
9. Eventually, the thread gets out of hand, and everyone but the trolls want the thread dead.  Bob sends a message stating that the thread is dead, and dead it remains.
10. Future messages in reply to the thread are responded to by the mailing list software with a message stating that the thread was killed and replies are disabled on the post.

Here is Mode 5 ("Muted thread"):

1. Bob wants to send an original post to a mailing list normally.
2. Bob’s message hits the mailing list server and is distributed to the subscribers of the mailing list as expected.
3. Conversation ensues and certain users are utterly disinterested and do not want to receive messages on the thread any longer.
4. Jo sends a message to the mailing list indicating disinterest and the mailing list server obliges by not sending Jo any more messages from the thread.

Here is Mode 5a ("Muted subthread"):

1. Bob wants to send an original post to a mailing list normally.
2. Bob’s message hits the mailing list server and is distributed to the subscribers of the mailing list as expected.
3. Conversation occurs.
4. Jo sends a message to the mailing list indicating disinterest in a subthread, and the mailing list server obliges by not sending Jo any more messages from that subthread.

Here is Mode 5b ("Temporary Mute"):

1. A mailing list has "blown up" and some of its users would rather not see any mail from the list for a period of time (say, a day or a week).
2. Users respond to any message in the mailing list and state that they want the list totally muted from their perspective for a specific (but arbitrary) duration.

Here is Mode 6 ("Named Subgroups"):

1. Bob is on a mailing list for a project and has a security issue to discuss.  He sends a message with [security] in the subject.
2. The message is delivered to only people that are in the [security] subgroup; whether the subgroup is open for joining by members of the mailing list or is closed off is up to mailing list server configuration.
3. Replies are contained within the named subgroup.

Here is Mode 7 ("Hidden Addresses"):

1. A group of users would like to have a mailing list, but they do not want their addresses shared with other users when they post, for whatever reason.
   1. Note that this can be used to prevent spammers from subscribing to the list, lurking, and harvesting addresses–very useful today.
2. The mailing list software takes posts and:
   1. Modifies the "From" header of the message to retain the name component, and changes the address component to that of the mailing list.
   2. Replaces the Message-ID header of the message with one generated by the mailing list software for the mailing list itself.
   3. Removes any Received headers.
   4. Removes (but remembers!) any Cc headers that were applied to the incoming message, and honors them until such time as it is told that the Cc should be dropped explicitly, for example by a recipient of the Cc (who sees the Cc line with only their address in it) or a trust list administrator or trusted list helper that can act on the behalf of the Cc recipient ("plz stop sending mailz 2 me k thx").

Possible Implementation (High Level) Notes

Now, between these seven distinct modes and their submodes, all the infrastructure that is necessary to expand the mailing list’s utility are possible to fit nearly any type of usage pattern/use case–transparently, and without any special requirements on MUA software beyond very simple, basic message receiving/display and sending.  At this point, I’m interested in collecting other use cases that I may not have thought about, but that people already use or would imagine would be useful.  The goal, ultimately, is to attempt to come up with a design and specification for a truly transparent mailing list management system that is more flexible than today’s software, that can have per-user preferences that are centralized and MUA-independent, and have features that can be used in the context of messages sent to the mailing list.

One idea is to use in-band signaling on the mailing list to indicate various things.  Unlike list software like majordomo where commands are sent to special mailing addresses, it’s possible that some things can be done by adopting existing conventions and others can be done by adding special information to the message body itself.  For example, Mode 6 is imagined as being part of the already-widely adopted convention that people can filter out based on [tags] in the subject, so an [OT] is used for off-topic, [RFC] for request-for-comments, and so forth.  The advantage of adopting conventions and in-band signaling is that it is inherently portable to nearly all MUA software, regardless of age or capability, as long as either References are In-Reply-To are supported.

Additional Modes

If you know of a workflow or a mode of operation/use case that would not fit in with this proposal, state it (I’m not looking for technical arguments of feasibility right now, just desired modes of operation; a future post and/or wiki page will serve the purpose of overall technical feasibility evalutation).  Consider this to be the "research stage" of a system to-be-specified, with potential for becoming an RFC a published Internet standard with the goal of improving life for all of us: non-technical users, technical users, users married to our workflow (or not).  The only correct system is one that works for everyone in as flexible manner as is possible.

Centralization

Many of these use cases (modes) require that mailing lists are a "gateway" for all of the interaction between users unless things are explicitly taken off-list, which of course users are free to do, if desired.  Some would seem to believe that if everything is centralized that certain use cases become inherently impossible.  While that might have been true given the computing resources that were available in the 1990s, it is not true today, and we shouldn’t pretend that it is; even a uniprocessor server running with a 266 MHz CPU and 64 MB of RAM should be able to run a mail list server implementing these features, serving a moderate number of users and still be fast and responsive.

Sure, lists that are the size of the LKML would probably want to run the mail list server on one machine and the database server on another, or have a distributed environment where the workload can be handed between mail list servers running for the same list (or family of lists), but if the software is correctly designed and implemented with such a configuration in mind, it would be easy for even very large, very active mailing lists to use a featureful system such as this.  For that matter, correctly designed, "centralized" does not have to mean "not highly available", because here I am talking about the service being centralized for a single list, although it would be possible in theory to have the mailing list run on only donated CPU time from volunteers (that said, DNS already makes this possible when combined with a well-planned and well-executed deployment process, though not terribly easily on the IPv4 Internet).

Others state that a centralized setup would take away their freedom to interact with the mailing list in some way: I am saying that this doesn’t have to be the case.  It may be with the way current mailing lists are implemented, but it doesn’t have to be that way at all!  And it can be done in a way that is intuitive–or at the very least, makes sense to users and is easy to learn, if not 100% intuitive.

Again, however, the motivation at this point is a high-level conversation that seeks to be gradually refined into lower and lower levels of abstraction until the final product is source code that can be seen and examined.

Motivation

Why am I motivated now?  Because it is plain as day that a universally applicable solution can be made to exist, and it’s even clearer that there’s no reason for it not to.  I don’t expect that this would be an overnight process, but I think that given ample discussion and interest, we can figure out a way to make everyone happy, even those that are way out with the extreme corner cases.  After all, there is no reason that systems cannot serve both complete end users and extremely adept technical users seeking to use every available feature.

In short, I think it’s time for a change in the way we think about what mailing lists are and how they work, and I think that it’s time that we consider how to make them useful and transparent to us, all of us, regardless of technical knowledge or background, regardless of the end user’s stringent desire to follow RFCs, and without having to be concerned about everything the system can do (while knowing that if you want to learn about all of the features and use them all, you certainly could).  I think it’s time to drop the assumptions that we’ve made for the last several decades with regards to mailing lists and redefine them.

So, who is interested?

Question 1: What is an operating system?

Here is a (very partial, heavy on the Unix-like systems) list of operating systems:  GNU, FreeBSD, OpenBSD, NetBSD, Microsoft Windows, Mac OS, XENIX, AIX, OS/2, MS-DOS, PC-DOS, FreeDOS.  All of these are full operating systems, though some are more compartmentalized than others.  I hear some saying already, "But GNU is not an operating system!"  Incorrect.  When using the Hurd and a GNU userland operating system stack, that is the GNU system.  I hear others saying, "But you left Linux out!"  Correct, I did: Linux is not an operating system by itself.  Many people are under the impression that the kernel is the operating system, and this is simply incorrect.  A kernel is the core of an operating system, but it is not the operating system itself.

If the kernel is not the operating system itself, what is?  An operating system consists of a kernel and software that is used to make that kernel useful—software that provides the user (or the programmer) with an interface to the kernel.  An operating system in the modern sense comprises a kernel and software that surrounds the kernel.  In the case of POSIX, UNIX and UNIX-like operating systems, this includes the C library, a set of utilities (kill, ps, ls, cp, mv, rm, sh, and so forth), and more.  To compare to Windows, the Windows operating system is not just NTOSKRNL.EXE.  That’s the kernel, but there are a suite of executables and libraries that surround it and provide interfaces to it which make the kernel useful.  While it is technically possible to do, law prevents the creation of a "GNU/Windows" or "GNU/NTOS".  It would not be as straightforward as it is to run GNU on a Unix-like system, but it would be possible (Cygwin is as close as it can lawfully get).

So, when we talk about the FreeBSD operating system, we are referring both to the FreeBSD kernel and the utilities and software that make it useful.  "FreeBSD" includes utilities and libraries and all of that, just like the NetBSD or Windows operating systems do.  But, Linux does not.  When we say "Linux", we are necessarily talking about only a kernel, the project that is the brainchild of a man named Linus Torvalds and the army of programmers around him.

Question 2: What is an OS kernel?

An operating system kernel is the core of the operating system.  However, without the rest of the components that build an operating system, a kernel is useless.  To make matters more interesting, a single kernel can be used as a component in several different operating systems.  This is actually the case for many systems, though it is most prominent in the case of systems that are built around the Linux kernel because there are so many of them.

The lifecycle of a kernel is generally something like this:  Set up and initialize itself, load and initialize basic hardware drivers, kick off one (or more) processes that are required to make the system useful, and then loop, handling things like hardware interrupts and system calls from software.  In the case of a Unix-like system, the kernel attempts to run a program called "init" (usually on modern systems, this is found in /sbin/init), which initializes userland and starts running services that make the system useful.  So a Unix-like operating system provides at least a kernel and an init process, as well as programs for logging in from TTYs (or virtual terminals), manipulating files, processes, and the kernel’s state, and so forth.  If a Unix-like system cannot find an init program, it will halt (panic) or return to firmware where a user could tell the kernel to use an alternate program for init (for example, you can pass "init=/bin/bash" to the Linux kernel, and it will start a single program: bash).

Question 3: Where is the line between an OS and the rest?

To answer this question, we must look at several operating systems, and we must be able to know what the difference is between core software and application software.  The line can be blurred, of course, especially when the core software depends on certain application software.  So in order to more accurately tell the difference, we need to know exactly what the core system uses.  A classic example in the UNIX world: some software is used as system software, and is also available for use as application software.  A system cannot be called a Unix without some of it, and a system certainly isn’t Unix-like without it (and remember: Unix is an operating system definition).

For non-technical people, an operating system is a distribution.  That is, Microsoft Windows is an operating system as it is installed; Ubuntu, Debian, and Red Hat Enterprise Linux are also operating systems.  For those who are more technical, though, the line isn’t there:  the line is somewhere between what is shipped for installation and what is the core system.

Let’s compare FreeBSD with Linux.  A typical FreeBSD system contains the FreeBSD kernel, FreeBSD’s C library, FreeBSD’s kernel management utilities, an implementation of init, and implementations of the Unix utilities (ls, ps, cp, kill, etc.).

However, a typical Linux system contains the Linux kernel, the Linux kernel utilities, the GNU C Library, GNU coreutils, GNU findutils, GNU bash, filesystem-specific utiltiies, an implementation of init (could be sysvinit, Upstart, or any number of other ones), and others.

Question 4: When is it GNU/Linux?  When is it not?  When is any system GNU/anything?

The simple answer: most systems aren’t GNU/anything, but they could be.  Take as an example the project to use the FreeBSD kernel (but not the FreeBSD userland).  The system cannot simply be called FreeBSD any longer: a significant portion of the FreeBSD operating system is removed when its userland is replaced.  The project’s operating system is called "GNU/kFreeBSD", not "GNU/FreeBSD" because FreeBSD was a complete operating system before its utilities were removed and GNU implementations dropped in.  This would include the GNU C library and the typical things you see in a Linux installation.  This also means that it’s a different operating system entirely!  Any piece of software that expects to be built on FreeBSD and assumes a complete FreeBSD system might not even build when using the FreeBSD kernel and the GNU userland.

In the case of Linux:  When you’re using the GNU utilities and core system, it’s GNU/Linux.  When you’re not, then it’s not.  For example, Android is a distinct operating system from Ubuntu.  They both use the Linux kernel, but that’s where the similarities stop.  Android is one example of a distinctly different operating system that shares a kernel with another operating system.  Aside from the fact that Ubuntu is most commonly run on x86 or x86-64 systems and Android is most commonly run on ARM systems, if you sat the two side-by-side on the same platform you’d still have different operating systems that do things differently.  This is because Android doesn’t even use the same C library as Ubuntu; many pieces of software that are stated to work on a Unix-like system such as GNU/Linux or FreeBSD assume a richer environment (closer to that specified by operating system standards).  They would have to be ported to Android to run on Android successfully; that’s a major red flag that says "Oh, that’s a different operating system."

Question 5: Does that mean I can create my own operating system?

Yes.  You can.  And you can even re-use the kernel from any other operating system that is licensed in such as way as to permit you to do so.

Let’s think about this in terms of OS X and what Microsoft could potentially do for Windows, as an example.  Apple used to have its own operating system called "System", which was then called "MacOS", and today we call it "Mac OS Classic".  This system was a cooperatively multitasking operating system that ran software specially built for it.  It was very popular, and it was also not as robust as modern operating systems.  It was pretty interesting, though: it did not have a command-line interface.  Nearly everything that was in the System Folder was the operating system itself, aside from installed fonts, extensions, and system enablers (though often extensions and enablers would patch the operating system in memory, or hook into it in some way so as to attach itself to it).  Apple abandoned that system years ago for OS X.  They took a BSD kernel and wrote a stack around it, reusing some components but using a very different architecture to put things together.  The kernel for OS X is called Darwin, and it’s a very important—in fact, the central—piece of the OS X operating system stack.  But it is not OS X; it is but a component.

How could Microsoft do something like this?  Let’s say that Microsoft went and did something similar, taking the BSD kernel and porting Windows’ software stack to run around it.  They also call it Windows.  Let’s say that they modified the kernel to load Microsoft’s old drivers and that they had incorporated a PE loader and ported their software to this new operating system (but retained API and binary compatibility with their legacy operating system that used a very different composition of kernel and userland, by necessity).  Would this mean that they were running FreeBSD?  No.  It would mean that their operating system were based on (at the very least) the FreeBSD kernel.  Though they could keep the FreeBSD stack and package it as a set of extensions to their new operating system, and claim the ability to run FreeBSD (and in a limited fashion, some Linux, if they retain the Linux compatibility mode of the FreeBSD kernel and system) software.  Now that would be an interesting market position for Microsoft to be in, wouldn’t it?  Microsoft could even make their new system compliant in some way with POSIX and the SUS and get branded as a UNIX® system.  But the new system would be more like Windows than it would be FreeBSD if they didn’t keep FreeBSD as-it-were, since they’d likely replace the core stack to fit their desires and needs, like Apple did.

Conclusion

I hope that this has clarified things a bit.  However, if it has not, a good place to start reading is the Wikipedia articles for "operating system" and "Kernel (computer science)".  They are a long articles, but they make for a great starting point for research and clarification of just what an operating system is.  Both articles look at different operating systems (and their kernels).  As to whether or not you can determine what an operating system is, you will want to become intimately familiar with many different operating systems (and types of operating systems; there are families of them as people create more and more systems both original and reimplementations of others) before you start trying to figure it out.  Look at native code and managed code operating systems—also realize that the distinction of what is the operating system can sometimes be relatively hard to determine accurately.

If you want to see what a bare operating system looks like, though, check out one of these operating systems’ source code:  FreeBSD, NetBSD, OpenBSD.  Each of them in their version control systems has the complete operating system, which is much more than just a kernel.  You can then, should you want, figure out exactly what packages on a typical GNU/Linux system take the place of the remainder of the operating system.  It’d be an interesting exercise in learning—if learning is truly what you’re after, and not just argument.

That’s right.  Now, there are many things that the man has done that have been effective, and the man has worked hard to ensure that we have freedom in the vast world of software.  But recently, he has stated that Miguel de Icaza “is basically a traitor to the free software community,” and no matter how much good a person has done, I cannot stand by such a fallacious statement.

What is freedom, people?  The two cornerstones of freedom are education and choice.  A person who is uneducated cannot choose, and so one depends on the other.  Some people will choose to use either free software or proprietary software exclusively, and others will mix the two together, with varying priorities that they place on their choices.  As long as people are aware of what’s out there and can determine what is best for the way they use their own systems, freedom is available.  Yes, that means some people may choose to run Windows.  Many will not, and if one makes a conscious choice, that is for them to do for themselves.  My concern is that people need to know about the choices in order to make a choice to begin with.

Now, de Icaza has done a great deal of work for the open source community.  He’s one of the key people who has been behind Midnight Commander and GNOME.  And of course, Mono.  He has given us a great deal of good quality, free software that many people can choose to use, and he has given the ability to use a VM-based runtime environment that previously was not cross-platform, as well, on nearly any platform that can be chosen.  The GNU people are even working on a similar one.  But this is not the reason that de Icaza is supposedly a traitor to the community.  The reason?  de Icaza is helping Microsoft out with their new “Open Source Labs”.

Guess what, Mr. Stallman?  The fact that Microsoft is starting to enter the world of free software cannot be considered to be a bad thing.  You did a great deal of work to ensure that free software that is adequately licensed can stay that way, and the turn the copyright system around on those who have sought out to corrupt it.  You have put free software on the radar around the world.  You have done a great many things that others would not have the heart, nor the motivation, to carry out.  It would appear, however, your time is up:  the world can and will move on without you, and I declare that it is time for you to go away.  When you start attacking people in the very world that you helped to create, you become obsolete and no longer serve your purpose.  I dub St. Ignucius of the Church of Emacs to be a heathen who has forgotten his message, forgotten his values, and is now a harmful creature; hardly a saint, more like a devil.  Overtaken with fits of arrogance.  Lately, you have caused a great deal of infighting within the community.  As I am sure you are well aware, infighting serves very little useful purpose but to tear apart communities.  And now you make a bold, arrogant, and false statement that serves only that purpose, but to an extent the likes of which we have never seen from you.

I say that we shun him—and his kind. Things are starting—albeit slowly—to come around to the way we want them, and he would prefer to attack it.  Why?  What logical reason is there to do so?  Because he’s not the controller of it?  Because it grew to be something that even companies that he helped cultivate an irrational, religious-like hatred of, are beginning to see is useful (not only to them, but to all of us)?  What is his new goal, that all software should be GPL’d and if it isn’t it is evil?  We should not tolerate these sorts of counterproductive behaviors.  Stallman has reduced himself to a Schestowitz, a creature who deserves no respect.  A troll.  I suppose in his old age, he has decided that attention is more important than our freedoms.

Richard Matthew Stallman, you are no longer relevant to our community, our world, the world of people who truly believe in and advocate for freedom, that of choice and that of the rights to study, improve, and distribute software.  Thank you for your services, for they were needed to get us where we are today.  However, you have chosen to do us no further good, only ill.  And for that, I say damn you.  Leave the Free Software Foundation and permit it to continue to work for freedom, or take it down and let the GNU project carry on to produce software without all of the utter crap that you have begun to spew of late.  I have no problem with the GNU project.  I love the GNU project.  And I am grateful for the work that you, independently and through the Free Software Foundation, have done so that we can enjoy the freedom to choose a free software system to do our work and our play.  But you are no longer helping; you have become a bully, an old troll.  You do not help free software any more.  Your recent action is a great offense to free software, and very much condemning a road which will lead to more free software.  Go away, troll.

Long live freedom—true freedom—and free software.  And thank you, Miguel de Icaza, for all of the great software that you have given us, as well, for your continued work to make software more useful and portable.  May you never wind up irrelevant and petty as Stallman has wound up being.