So, this is a pretty simple-sounding network, yes? It should be—it is just a handful of computer systems. However, there is a problem—a pretty large one, I think. There is very little in the way of either policy or convention on the network. Some users use their documents directory for storing their documents, others store their documents on their desktop, others somewhere differently altogether on the C drive, others on a network share in a public storage space. There are different versions of software on the various machines, more-or-less updated when someone thinks about it, I think.

This is a perfect scenario which shows why a business network should be centrally managed in some form. Note that I am not saying that each machine should be a bit-for-bit mirror image of the one next to it, though that is certainly a possibility. I think that people should be able to use their own choice of things like email client or Web browser software, because everyone is different. But when you have different client applications fulfilling a role on the individual workstations, you have to take a centralized approach to ensuring that things like the email is all backed up.

Furthermore, if you don’t take a centralized approach to backing up such data, it is very difficult to centralize the network storage. Think about adding a domain controller (that is Windows speak for a central server which handles authentication and authorization, as well as file and printer sharing and things like roaming profiles) to such a network. I expect that with multi-gigabyte mail files, things will be very slow at first—and that likely the only fix for them that is going to be viable in the long term is to centralize more infrastructure.

I am too tired to expand more on my thoughts on what I have learned and where it is heading, but the Reader’s Digest version of the point reads something like this: If you are a small to medium sized business, make sure that you have someone who is competent in both system and network administration, and make sure that they are a part of your business from day one. Like writing software, building up a technical infrastructure without careful thought and design is hazardous and comes with many hidden and unpredictable costs. It is best to head those things off right from the start; to delay only amplifies the cost of fixing the underlying problems and puts oneself in the position where fixing one issue can have a domino-like effect and create more new problems.

For my current situation, I think I am going to have to seriously re-think how this whole setup is done. What I do not yet know is how to quickly and efficiently bring things into shape. A bit of training and education may be required, and certainly the removal of a lot of unnecessarily-granted privileges on the workstations. That, too, should be something caught early-on: do not let every person in a business run with administrator privilege, unless they are an administrator (and even they should only run with administrator privilege when they are actually doing something that requires that privilege). If everyone is an administrator, there is little to no control on how things are done in a network, and it can get messy.

I have a lot more reading to do, as well.

Well, anyway, it is way past my bedtime.  Time for sleep.

Many projects that are all about being open and transparent provide mechanisms for their users to get in touch with the developers.  Launchpad has become an excellent place to do these things; Launchpad’s Answers and Bugs components are truly excellent ways to get in touch with people that manage software there.  The system is so good that I’d advocate that every project try to move to it.  Launchpad does have its issues, though they are often quickly fixed (and can be fixed even quicker now that it is free software).  But between Bazaar (a truly amazing and flexible distributed version control system) for code hosting and the other components of Launchpad such as the Answers, Bugs, and Translations components, Launchpad provides virtually everything that a project needs to communicate effectively with its users aside from the project’s Web site.  And users seem to naturally be able to reach out using it.

In the nearly one year that I’ve been working with AllTray, I’ve talked—well, written to—several people who have asked questions and acted in their own ways to work to improve the software.  Development on AllTray has been made much easier just because people are willing to speak up.  Everything from the random “thank you” to “hey, are you going to do this?” or “I’d like to see feature x in the new version” or whatever.  It’s truly great.  I can’t say that every project has users quite like AllTray’s, because AllTray is something of a niche application, but AllTray’s users are a great model for other projects.  There has been virtually no negativity from users and everything has been constructive.

To anyone who manages a project that doesn’t use Launchpad, I’d strongly encourage them to do so.  Its users are great, and as far as management of the overall project goes, it’s very easy for project people to communicate back and manage communications in a single spot.  It’s also great that users can ask a question and that the project can do things like go, “oh, hey, this is really a bug,” or vice versa with bugs→questions.  And if you don’t use Launchpad because you don’t want to use Bazaar (I hear that users of git are pretty dedicated to it; I can understand that, but it’s just not for me—it’s too complex, and I like that Bazaar just stays out of my way), consider using LP for everything but code hosting.  Or consider using Launchpad’s code hosting to mirror your project from git, it does this quite nicely.  Or, find some way to do what Launchpad does wherever you do project management.  It works really well, and it’s a big help.

And to AllTray’s users. A big Thank You.  You guys and gals are great.  Keep communicating!

One piece of software that is almost ubiquitous (at least, outside of the not-so-wonderful world of Microsoft Windows) is a small program known as m4.  You may or may not have heard of it, and you have probably never used it outside of the GNU build system if you’ve used that at all.  m4 is fantastic, if hard to learn, but it is available and it is a pretty much standard component of most UNIX-like systems.  It is a macro processor that dates back to V7 UNIX and it’s been standardized as part of the POSIX body of standards, and GNU has its own version with extensions.

Essentially, what you do with m4 is create macros that process text, and then you can use those macros to simplify things.  In other words, m4 makes it possible to follow the DRY principle when it comes to writing text documents, configuration files, and other text-based files.  For example, I’ve written some macros to make it easier to work with djbdns configuration files, which are available on my software page.  Instead of writing a line that looks like this:

+alltray.trausch.us:173.15.213.185:600

I can write a line that looks like this:

DD_A_FULL(alltray, 173.15.213.185)

Or, for multiple hosts that share the same address, I can (actually, I do; this is from my configuration file) do this:

DD_A_IP(173.15.213.185)
DD_A(allspice)
DD_A(alltray)
DD_A(mike)
DD_A(morganne)
DD_A(phone)
DD_A(projects)
DD_A(sip)
DD_A(vcs)
DD_A(www)
DD_A(gallery)
DD_A(wiki)
DD_AAAA_IP(2001:470:1f11:3f::1)
DD_AAAA(allspice)
DD_AAAA(alltray)
DD_AAAA(mike)
DD_AAAA(morganne)
DD_AAAA(phone)
DD_AAAA(projects)
DD_AAAA(sip)
DD_AAAA(vcs)
DD_AAAA(www)
DD_AAAA(gallery)
DD_AAAA(wiki)
DD_AAAA(spicerack)

The nifty part is that the AAAA lines actually expand to:

:allspice.trausch.us:28:\040\001\004\160\037\021\000\077\000\000\000\000\000\000\000\001:600

Which, as you can imagine, is quite a pain to manage manually.  However, using m4 and some helper programs that are written in C for formatting IPv6 addresses for AAAA records and formatting SRV records for things like XMPP services, managing my configuration has become so easy that I don’t really have to think very hard to do it.  I don’t have to seek out any online generators for SPF records, AAAA records, or SRV records.  I don’t have to worry about repeating a TTL in every line, either, because I set up a domain like so:

DD_DOMAIN(trausch.us)
DD_TTL(600)
DD_SOA(spicerack,mbt.zest.trausch.us,1800,600,21600,600)
DD_NS(173.15.213.185,spicerack)
DD_NS(,primary.staffasap.com)
DD_PTR(173.15.213.185,spicerack)

And for a section that should have a longer TTL, I do so by saying DD_TTL(longer_ttl) and then writing the new lines.  I didn’t try to turn the configuration file format into something that was BIND-like (after all, part of the reason I left BIND was I hated managing its configuration files) but I did make the configuration easier for me to manage.  Now, when I need a new address in the file, or change anything about my domain, it’s as simple as adding or deleting macro calls, and the rest is handled for me.  (I also modified my Makefile that generates the djbdns "data.cdb" file so that m4 is called automatically when I update the data.m4 file.)

M4 can be a major pain to use until you learn it.  And while I do sometimes run into the occasional pitfall with it, I’ve used it for adding preprocessing capability to C# and Vala code to make my life easier, and I’ve used it for various other things in the past outside of the GNU build system.  I absolutely love it.  It is not for everyone—indeed, not everyone has a use for a macro processor—but if ever the need arises, it is well worth your time to learn m4 and use it any time you need a macro processor.  It sure beats having to learn macro processors that are specific to a particular environment, and thanks to Cygwin, you can use GNU m4 on Windows, too, if you happen to be so unlucky as to have to live life with 20% of your CPU cycles (and who knows how much RAM) paid as a tax to fend off viruses and malware. 

And no, m4 isn’t just for programming, nor is it just for programmers.  Anyone who works with a great deal of text is likely to have a good use case for it.  People have been known to avoid using things like PHP and alleviate server-side stress by using a set of macros and generating all of their pages statically (without having to statically manage them all).  This can be good if you have to work with a server that is extremely light on resources and shouldn’t be running things like server-side scripting languages and fetching data from databases.  Instead, you can do all of the database lookups and page formulations ahead of time, meaning that the so-called "slashdot effect" doesn’t ever cause you any trouble on nearly any type of server since it is only serving static pages.  Or writing documents that require things like legal boilerplate and the like; though the days of text processing and typesetting for the masses have generally gone away, sadly.  There are a potentially unlimited number of applications that m4 could be used for—and while m4 is somewhat difficult to learn and can be frustrating at times, it’s standard and it has decades of history, like UNIX in general.

Well, I have recently been doing it differently. Presumably, just about anything I want will still be available somewhere on the Internet at a given point in time, so, why store a copy at all? When I download something anymore, I just extract it from the tarball over the Internet. For example, I downloaded Tinymail today, and the way I did it was:

wget -q -O- http://tinymail.org/files/releases/pre-releases/v0.0.7/libtinymail-0.0.7.tar.bz2|tar xvjf -

There is only one problem with doing it this way: While GNU tar knows how to autodetect the format of an archive when it is stored on the local filesystem, it does not appear to be able to do so when getting the archive in from a pipe. I would like to look into whether or not detecting the archive format as tar, tar.gz, or tar.bz2 can be done over a pipe, because it would be very nice to use the same invocation of tar that I would use anyway.

Oh, well. Anyway, it still saves me disk space and cobwebs, so it’s all good.