eaten by a grue

My laptop has had an interesting couple of days. The main filesystem went read-only a couple of nights ago after a couple of random journal errors. After being fsck'd and cleaned up it did it again, so I reinstalled it and restored it from backup yesterday. Then last night it overheated, leading me to open the case and clean the wall of dust out of the fan. Its back together now, but a couple of lost of tiny parts means I have no indicator lights and no trackpoint. Fortunately the trackpad still works, but its taking a little getting used to. On the other hand, its not burning my lap or my hands anymore, so its probably an overall victory though its not quite feeling that way yet.

One of the things I did lose in the rebuild, due to it not living in one of my backup locations (which is /etc/ and /home/rob) is my cute little mobile roaming script. I rewrote it on the bus on the way home yesterday and thought that perhaps its interesting enough to post here.

The basic idea is that every day I switch between at least two networks. My home network has a PC in the hall cupboard which among other things runs a web proxy and a mail server. Its also the firewall, so web and mail traffic can't go out directly. Work on the other hand, implements transparent proxying (with some network authentication) and has a SMTP server, but naturally it has a different address. I also occassionally use other networks (friend's places, coffee shops, etc) which usually have no facilities at all, requiring me to fend for myself.

My laptop runs a SMTP server (Postfix) of course, because that's just what you do on Unix. I also run a Squid proxy which I point all my local HTTP clients at. This way, when I move networks, I only have to reconfigure the local proxy rather than tweak every web client I have.

I spent a long time looking for a decent roaming reconfiguration package, but I never managed to find one. Some would try to do network detection and too often get it wrong. Some would have overly complicated and/or feature deficient configuration languages. I vaguely recall that I really liked one of them but it was tightly integrated with NetworkManager, which I don't use because it could never seem to keep the network alive for more than a few minutes (and it appears to be pretty much tied to the GUI, which is painful when I need network on the console).

So, in the finest open source tradition, I rolled my own. The script itself is trivial; its just a tiny template expander. I'll list the script in a moment, but first I'll talk about its operation.

The script, which I call location, takes a location name on the command line (like home or work), runs over a (hardcoded) list of config files, reads them in, modifies them, and spits them out to the same file. It makes modifications according to templates that may exist in the file. If the file has no template, then location ends up emitting the unchanged file.

In any file you want it to modify, you add an appropriate template. This is the template I have in my /etc/postfix/main.cf:

### START-LOCATION-TEMPLATE
##@ home relayhost = lookout.home
##@ work relayhost = smtp.monash.edu.au
##! /etc/init.d/postfix restart
##! sleep 1
##! /usr/bin/mailq -q
### END-LOCATION-TEMPLATE

When it finds itself inside a template, location stops its normal operation of outputting the lines of the file as-is and instead starts parsing. Interesting lines begin with ##, anything else is ignored. Its the third character that determines how the line is interpreted. So far I have the following functions:

• #: do nothing, just output the line
• @: emit if at location. If the location specified on the command line matches the first argument to @, then the rest of the line is added to the file as-is.
• !: run command. Calls a shell to run the specified command after the file has been generated.
• >: interpolate line. Include the rest of the line in the file, but expand any %variable%-type markers. So far only %location% is defined, and is replaced with the location specified on the command line.

(I'll provide an example of that last one in a moment).

So in the case of main.cf, lets say we ran location with home as the location. This would result in the template section being written to the output file as:

### START-LOCATION-TEMPLATE
##@ home relayhost = lookout.home
relayhost = lookout.home
##@ work relayhost = smtp.monash.edu.au
##! /etc/init.d/postfix restart
##! sleep 1
##! /usr/bin/mailq -q
### END-LOCATION-TEMPLATE

The listed commands are then run, which cause Postfix to be restarted and the mail queue to be flushed:

/etc/init.d/postfix restart
sleep 1
/usr/bin/mailq -q

Naturally Postfix interprets the template parts of the file as comments, so nothing to worry about. The next time location is run, the "bare" relayhost line is ignored, so it doesn't get in the way.

The config for Squid is similar. Because Squid's config file is huge, I don't quite trust my script to handle the whole thing sanely, so at the bottom of squid.conf I've added:

include /etc/squid/location.conf

And in location.conf I have:

### START-LOCATION-TEMPLATE
##@ home cache_peer lookout.home parent 8080 0 default
##@ home never_direct allow all
##! /etc/init.d/squid restart
### END-LOCATION-TEMPLATE

By default Squid will try and hit the internet directly, which is fine for work and unknown locations. For home, i need to force it to always go to an upstream proxy, which is what those the cache_peer and never_direct directives will achieve.

The proxy at work used to be an authenticating proxy, so I had to specify both a peer and a username/password combination. This made the required amount of variable config a little unwieldy to be include in a template, which is where the > function came from. location.conf used to have this:

##> include /etc/squid/upstream.%location%.conf

Which would arrange for upstream.home.conf, upstream.work.conf, etc to be included depending on the location. There's every chance this will come in useful again one day, so I've left the code in there for now.

Here's the script in its entirety:

#!/usr/bin/env perl

use 5.010;

use warnings;
use strict;

my @files = qw(
    /etc/squid/location.conf
    /etc/postfix/main.cf
);

use autodie qw(:default exec);

use FindBin;

if ($< != 0) {
    exec "/usr/bin/sudo", "$FindBin::Bin/$FindBin::Script", @ARGV;
}

say "usage: location <where>" and exit 1 if @ARGV != 1;

my ($location) = @ARGV;

for my $file (@files) {
    say "building: $file";

    my @out;
    my @cmd;

    open my $in, "<", $file;

    my $in_template = 0;
    while (my $line = <$in>) {
        chomp $line;

        if ($line =~ m/^### START-LOCATION-TEMPLATE/) {
            $in_template = 1;
            push @out, $line;
            next;
        }

        if ($line =~ m/^### END-LOCATION-TEMPLATE/) {
            $in_template = 0;
            push @out, $line;
            next;
        }

        if (!$in_template) {
            push @out, $line;
            next;
        }

        my ($tag) = $line =~ m/^##([#@!>])/;
        if (!$tag) {
            next;
        }

        given ($tag) {
            when ('#') {
                push @out, $line;
                next;
            }

            when ('@') {
                push @out, $line;

                my ($want, $rest) = $line =~ m/^##@ (\w+) (.*)/;
                if ($want eq $location) {
                    push @out, $rest;
                }

                next;
            }

            when ('!') {
                push @out, $line;

                my ($cmd) = $line =~ m/^##! (.*)/;
                push @cmd, $cmd;

                next;
            }

            when ('>') {
                push @out, $line;

                my ($rest) = $line =~ m/^##> (.*)/;

                $rest =~ s/%location%/$location/g;

                push @out, $rest;
            }
        }
    }

    die "$file: unclosed location template" if $in_template;

    close $in;

    open my $out, ">", $file;
    say $out $_ for @out;
    close $out;

    for my $cmd (@cmd) {
        say "running: $cmd";
        system $cmd;
    }
}

Because its so trivial and I only run it a couple of times a day, I just run it when I get to work (location work) or when I get home (location home). If I felt inclined I could probably hook it up to my network stuff but I think that would be more trouble than its worth.

On occassion I have to use Windows on the same machine. I have no idea how to achieve something similar there, so I just reconfigure my browser. Fortunately I don't go there often, and almost never from work. This is why I like open source. I can make my system work in exactly the way I want and usually with a minimum of fuss.

Day two of training today. I had to run off early, but not before getting a crash course in Moose. I've been watching Moose for a couple of years, and have a project in mind for it, but haven't got around to doing anything with it yet. Doing a few basic exercises with it was awesome just to see what it can do, but I did manage to get frustrated by three things within the first ten minutes.

1. The first thing I noticed is that every attribute accessor created is publically available. There isn't really a way to make an attribute that is read/write from inside the class but read-only from outside. The fine manual suggests this:

   has 'attr' => (
       is     => 'ro',
       writer => '_set_attr',
   );
   

   This works ok, but its still possible for code outside the class to call _set_attr directly. Until we get lexical subs its impossible to make the writer method invisible to the outside world, but until then I'd still like it to be possible for Moose to produce an accessor that can check its caller.

   In a similar vein, its not possible to create proper protected or private attributes. Private attributes can sort of be done by assigning directly to the object hash:

   $self->{attr} = 1;
   

   I don't like this because it because it makes assumptions about the internal implementation of the objects (and with Moose I'd like to remain as ignorant as possible on this point), but also because it provides no type or constraint checking.

   Protected attributes (that is, attributes private to a class and its subclass) seem to be completely impossible.

2. By default, a Moose class quietly accept any and all parameters passed to its constructor, regardless of whether or not they correspond to an attribute in the class or its parents. This confused me for a moment as I've come from Params::Validate which allows you to declare parameter types and constraints much like Moose attribute declarations, but dies if you provide a parameter that is not declared. The fine inhabitants of #moose on irc.perl.org pointed me at MooseX::StrictConstructor, which does what I want - dies if undefined parameters are provided.

   It gets better though. I was declaring an attribute that I wanted to be impossible to initialise via the constructor as I planned to set its initial value in BUILD, and to allow the user to provide a value only to ignore it is confusing. The manual explains that specifying init_arg => undef in the attribute definition will arrange for that parameter passed to the constructor to be ignored, but again, it does it quietly.

   It turns out (again via #moose) that combining MooseX::StrictConstructor with init_arg => undef yields the desired results. I can live with that, but I would never have anticipated that result from the documentation. Hmph.

3. Moose doesn't provide any syntactic sugar for class attributes/methods. A quick search just now turns up MooseX::ClassAttribute which will probably be as much as I'll need, at least initially, but I was surprised that core Moose didn't have anything for this. Are class attributes so uncommon?

At the end of the day though, these are all pretty minor nits. Moose is awesome. Its very actively maintained and developed by a number of incredibly smart people, so its not going away any time soon. I'm looking forward to having the time to do something serious with it.

Work is sending me on a Perl Training Australia course this week, so I'm getting to hang out with Paul and Jacinta and get a good refresher on Perl OO. I wouldn't say I needed it, but I've been enjoying the discussion and it never hurts to make sure that your accumulated understanding matches the current reality.

One of the exercises involved a class representing a coin with methods to flip the coin. One of the things we were asked to do at one point was to create an array of coins and do various things to them. My first instinct to create the array was to do this:

my @coins = (Coin->new) x 10;

I was saddened but not surprised to find that this doesn't work. As the following test demonstrates the left hand side is only evaluated once and then just copied, so I ended up with an array containing ten references to the same object:

$ perl -E '$c = 0; @x = ($c++) x 10; say @x'
0000000000

The best I could come up with is this, which I don't think reads anywhere near as well:

my @coins = map { Coin->new } (1..10);

We briefly discussed whether it would be worth developing a core patch to do something like it, but realistically the only option that preserves a reasonable amount of backward compatibility is to only reevaluate the left side for a very specific set of types, namely code references, giving something like this:

my @coins = (sub { Coin->new }) x 10;

Given that that really doesn't read particularly better than the version using map, and not knowing if anything smarter is possble (and how to do it if it is), and knowing that the core developers aren't particularly keen on new features to existing constructs at the best of times, I've opted to leave it for now but keep my eyes open for things like this.

One part of another exercise had me dealing with decks of cards. Internally I represented suits as integers, with the following list to assist with the as_string method:

my @suits = qw(hearts spades clubs diamonds);

When I got to adding the initialiser for the class, naturally I wanted to be able to specify a string. The usual thing I'd do here is create a hash from @suits with the values as the integer array indexes. This time I came up with this one-liner to determine the index of a value in an array:

my $index = do { my $found; grep { $found = 1 if $_ eq $needle; $found ? 0 : 1 } @haystack };

It plays on the fact the grep in scalar context returns the number of matches; that is, the number of times the code block evaluates true. All this does is arranges it such that the block is true for every array index before the wanted value but false for every index after (and including) the wanted index. If $index == @haystack, then it wasn't found.

Its certainly not optimal - a binary search is always going to be quicker, and you'd nearly always want to use the hash method if you were doing it many times, but it was certainly fun to write a cute oneliner to do it.

A couple of months ago I started work in a new job, doing various programmery things for the Monash student intranet thing The job description says "web programmer", but there hasn't been much web yet. At this point I'm mostly concentrating on the glue code needed to hook up various Google services to our environment. 99% of the code I'm writing and working on is Perl.

I've been using Perl for ages though. For the last nine-and-a-bit years I've been a mail sysadmin at Monash, and while the "core" of our systems has always been full-on professional mail software packages (both proprietary and open-source), all the bits in between have always been Perl. We've written all sorts of stuff, from full web environments and workflow packages to all the traditional sysadmin tools like log parsers, report generators, config builders and everything in between. There's never been any question for us - Perl is just so obvious for this kind of work.

Previously though, Perl was merely a tool that I used to get the job done. In many ways though its now become the job itself. For any given day I can be reasonably confident that most of it will be reading or writing Perl code, whereas before I'd only bust it out when I needed it. Additionally, I haven't really done this type of work before so I'm getting lots of ideas for stuff I want to play with on my own time and also finding gaps in my knowledge that I want to fill out. So now I'm finding just about every moment I'm at the computer I'm doing something with Perl, far more than ever before.

So, I've decided that it would be really good to finally join the mob rather than just hang around the edges looking in. I'm signing up for the Iron Man challenge to keep me honest, and I'm moving my code to Github for a bit more visibility. This is going be an interesting change for me, as in both blogging and coding I'm used to producing something large and fully-formed before showing the world, but obviously that doesn't work if you need to post once a week. I've started making a list of little Perl things I can write about in a couple of paragraphs, so hopefully I'll be able to keep it fresh.

Additionally, I've committed myself to writing everything I possibly can in Perl. I have a long history with C as well, and for the longest time always reached for it for anything closer to the hardware/OS (a fuzzy line, but typically that means server-type things). No more. From now on unless there's a very specific reason why Perl is unsuitable, I'll be choosing Perl for my code.

So that's it. Hi :)

As is often the case with me, what started as a small hack to blosxom to make it do tags and per-tag feeds turned into me rewriting it from the bottom up. I quite like what its become, though I doubt its of much use to anybody but myself. Give me a yell if you want the code.

Anyway, now the whole site has proper tags, and you can use them to subscribe to just bits of my ramblings rather than the whole lot, which should make a huge difference considering how much I don't actually write. Oh, and there Atom feeds too if you like that sort of thing.

To celebrate this momentous occasion, I've moved the whole mess to a new and somewhat relevant domain, eatenbyagrue.org. I think I got all the redirects right, so existing subscriptions should work ok.

That's all. Back to work shortly.

I love blosxom, so I persist with it, but its a nightmare to refactor. It could have done what it does just as well without being quite as clever. I wonder if that's why development on it is mostly dead. In any case, the refactoring goes well, and I hope soon to have something that does exactly what I want (which I'll talk about more soon; better to spend my time on code right now).

Tonight I wrote a simple tag plugin for blosxom (the blog engine I use), imported all the categories from the old Wordpress blog as tags, and wrote some CSS to make it work properly. There's still a bit more to do, notably making the tags into links that take you to ther posts tagged the same thing as well as getting per-tag feeds going (though the RSS plugin needs a lot of work anyway).

The plugin, for the curious:

package tags;

use vars qw($tags);

sub start { 1 }

sub story {
    my ($pkg, $currentdir, $head_ref) = @_;

    $tags = '';
    if ($meta::tags) {
        my @tags = split /\s+/, $meta::tags;
        $tags = "<div class='tags'>tags: <ul>";
        $tags .= "<li>$_</li>" for @tags;
        $tags .= "</ul></div>";
    }
}

1;

Obviously, its really the meta plugin that does most of the heavy lifting.

So now at the top of a post I write something like:

meta-tags: site perl

and tags pop out. Lovely!

After my lack of motivation I've had a couple more interesting ideas and so I've started very slowly poking at them, being very careful to not overdo it in an attempt to avoid the burnout. So far it seems to be working!

The short is that my mate Sam and I have been pondering for a year or more the idea of building an arcade cabinet that runs emulators for various old systems (MAME and such). He's a high school woodworking teacher and cabinetmaker by trade so he's perfectly qualified to build the box. I know a thing or two about computers, so I can do that bit. The problem so far is that we've never really had any good place to build it - I have a garage but no tools, Sam had no space at all.

He's just recently moved house and is finishing getting his own garage all kitted out with workbenches and drop saws and drill presses and other things that scare me and he's now ready to build something, so we're taking another look at it. I've managed to scrounge enough parts to build a reasonably good rig, though the ridiculous weather is making me reluctant to go out to the shed to work on it. Arcade Gaming Australia have all the buttons, joysticks and other bits that we'll need. There's only one thing left - an awesome UI for choosing games and things. Yay software!

So tonight I've been sitting under the air conditioner fiddling with Clutter. Its a library for making fancy interfaces by using lots of 3D stuff under the hood. As far as I can tell the most well known example of the type of thing its for is Apple's Cover Flow. Just from playing with some of the samples I already have some idea of how I'd like a game selector to look, so I've started experimenting using the Perl bindings.

The basic idea is that you set up a bunch of actors, which are basic visual elements - some text or an image for example. You can specify various transformations for an actor, eg scaling, rotating, etc. After that, you place your actors somewhere on the stage, which is roughly analogous to a window.

Next is where I get a little confused, but not so much that I can't get something done. You setup a timeline, which has two paramaters - number of frames, and frames per second. You hook up an "alpha" to the timeline, which is a function that gets called every frame, and returns a number that I don't fully understand the purpose of yet. The number is used to drive "behaviours" attached to each actor, which makes them do something depending on the current distance through the timeline. A behaviour might be to move the actor around the stage, rotate it, or something more clever.

There's also an input layer, but I haven't really started looking at that yet.

So here's the fruits of my evening. It takes a random image and rolls it around a window.

#!/usr/bin/env perl

use 5.10.0;

use strict;
use warnings;

use Glib qw( :constants );
use Clutter qw( :init );

say "usage: roll image" and exit -1 if !@ARGV;

my $stage = Clutter::Stage->get_default;
$stage->set_color(Clutter::Color->parse("DarkSlateGray"));
$stage->signal_connect('key-press-event' => sub { Clutter->main_quit });
$stage->set_size(800, 600);

my $actor = Clutter::Texture->new($ARGV[0]);
$actor->set_anchor_point($actor->get_width / 2, $actor->get_height / 2);
$actor->set_position($stage->get_width / 2, $stage->get_height / 2);
$stage->add($actor);

my $timeline = Clutter::Timeline->new(100, 26);
$timeline->set(loop => TRUE);

my $alpha = Clutter::Alpha->new($timeline, sub {
    my ($alpha) = @_;
    return int($alpha->get_timeline->get_progress * Clutter::Alpha->MAX_ALPHA);
});

my $rotate = Clutter::Behaviour::Rotate->new($alpha, "z-axis", "cw", 0.0, 359.0);
$rotate->apply($actor);

my $path = Clutter::Behaviour::Path->new($alpha, [ $actor->get_width,                     $actor->get_height                      ],
                                                 [ $actor->get_width,                     $stage->get_height - $actor->get_height ],
                                                 [ $stage->get_width - $actor->get_width, $stage->get_height - $actor->get_height ],
                                                 [ $stage->get_width - $actor->get_width, $actor->get_height                      ],
                                                 [ $actor->get_width,                     $actor->get_height                      ]);
$path->apply($actor);

$timeline->start;

$stage->show;

Clutter->main;

Hard to show it here, but here you go:

Of course I have no idea if this is the "right" way to do it, but it seems to perform well enough so it will do for now. Next is to make a little photo thumbnail viewer, using the arrow keys to scroll through the photos and a little zooming magic.

A few random things I've been working on lately:

• Started migrating this blog into ikiwiki. Stuck on a problem with file create/modify times not being preserved, which makes complete sense but is annoying. I think a plugin is required, much like one I wrote for blosxom once upon a time.

• Made progress with Test::MockTerm. The terminal itself works, with open() and POSIX::isatty() being overidden correctly. I'm currently planning the interface for sending and handling control characters (and from there emulating Term::ReadKey). Its hairy.

• Made a new release of XML::Quick after a couple of years. The test suite now runs (the were bugs in the suite itself) and an ancient bug was taken care of to boot.

• Dug out XML::Spice, something I started a couple of years ago as an answer to Python's stan XML generation library. It already has some funky magic in place that lets you call the generator function to create some piece of XML which totally complete and usable, but then call the generator again with that returned chunk as one of the arguments to have it embed that chunk into another chunk of XML. At this point both are valid, but it can do some funky stuff like move and reprefix namespace declarations to make the result more concise without changing the semantics. It does this without having to reparse the original XML. You have to read the tests to find out more. In any case, its not finished yet and perhaps never will be since I'm long over XML.

• Also dug out HTML::Calendar::Render (no link yet), which you give a bunch of events and it creates a calendar using HTML tables. This was done for work, at a time when we temporarily needed an alternative view to our corporate calendar system. Producing calendars in HTML is nothing new, but this is prettier than any of others I've seen on CPAN. It works really hard to produce stuff that looks like your calendar in something like Outlook, where overlapping events appear next to each other at half-width. This one I'm kinda interested in getting into some kind of release quality. The plan is to split out the tree generator from the renderer itself, so you can add modules that might render it with or without HTML tables, in PDF, or via cairo. Coming soon I guess.

I've been getting into Perl again in a big way. I'd been getting in the mood for code again, and started looking through my old stuff to see if anything tickled my fancy. I found some Perl stuff, which I've been chucking in git, and was poking about it, when a question about IO::Prompt came up. Since I'm theoretically responsible for that module, it seemed like a good place to kick off.

I started work again on Test::MockTerm, this time making a pure-Perl version rather than relying on IO::Pty, which had a few issues that made it unsuitable for what I needed. As mentioned in that old post of mine though, without real terminal handles I have no way to test the code paths that expect them.

During my break, Perl 5.10.0 was released, but the hoped for patch that would allow me to override the -t file test operator never made the cut, so I'm still stuck without a solution. The short-term workaround will therefore be to modify IO::Prompt to use POSIX::isatty, which does essentially the same thing. Hopefully it'll work on Windows when that time comes.

Longer term, there needs to be a way to make -t overridable in some way, so I started poking at the code a bit, and thought about extending tied handles to do the work.

For the uninitiated, Perl has a facility called "tying" where you can essentially tuck an object reference into a variable. When operations are performed on that variable (like storing a value in it), Perl arranges to call methods on the object to do the work. perltie explains the details.

You can tie a filehandle to an object in the same way, so its the ideal way to do what I want. Except that in the BUGS section, we see:

Tied filehandles are still incomplete. sysopen(), truncate(), flock(), fcntl(), stat() and -X can't currently be trapped.

-X is the general way to refer to the file test operators. Damn.

Tonight I sent off a patch that implements stat() and -X for tied filehandles.

It was quite a challenge to write, but very fulfilling. One of the really interesting things about hacking on Perl is that the culture of writing comprehensive tests and documentation is deeply entrenched, so no matter what you do, you end up with high-quality work at the end of it because its provably correct.

Unfortunately even if it is accepted it seems unlikely to me that the patch will appear anytime soon. I don't know what the policy is on new features on the current maintenance branch, but I wouldn't be surprised if it was slated for inclusion in 5.12. It took over three years for 5.10 to appear after 5.8, so I won't have this code available in a stable Perl for quite a while yet, but at least I have something to point to.

Next is to get Test::MockTerm finished, which I'll get back into tomorrow.

So a couple of weeks ago I got the chance to chat to Damian about IO::Prompt and my completion patch. While he rejected the patch because the interface was sucky (and I agree), he accepted my offer to take on maintenance duties for the module. Thats not it for completion though; we're currently designing a much better completion and history interface, which I'll write more about that some other time. My first trick will be to get a test suite up and running.

IO::Prompt doesn't currently have a test suite, and I'm not confident that I'll be able to make any significant changes without breaking whats there, so the current functionality has be recorded. The difficult thing about it is that we're testing something terminal based, so we have to pretend to type something, and then watch not only what the API returns, but also watch what appears on the screen.

This turns out to be quite complicated. The module opens /dev/tty directly, both for reading and writing, so we need to intercept the calls to open (via CORE::GLOBAL::open) and returns some filehandles we can manipulate directly. My first cut used basic scalar handles, but then I ran into further trouble when I found that the module uses -t to see if its talking to a terminal. Obviously my scalar handles are not terminals, so I needed a way to convince -t otherwise.

After a deep tour into the guts of Perl itself (a fascinating and scary place) I determined that there's really no pleasant way of overriding -t, though there is a patch under consideration for 5.10, and I did figure out a really evil way that might do it by twisting the optree in ways that I wouldn't dare give to the world. So the only other option is to somehow produce filehandles that are in fact terminals.

IO::Pty provides the answer, by allowing me to get pseudo-terminals from the operating system. I kinda didn't want to go there, because it ties the implementation to systems that have terminals, which doesn't include Windows, but I've since decided that it'll be fine for now since the current code hits /dev/tty directly, and that doesn't exist on Windows either.

Time passes. I play with this module, figure out the difference between master and slave and make a note of it because its stupid and I can never remember, and finally produce Test::MockTerm. Its not CPAN-ready yet, its currently a build helper for IO::Prompt, but I think it may have a life of its own someday. Using it, I write some basic tests for IO::Prompt, and run it .. and it hangs, waiting to read from standard input.

After further perusal of the code, it seems that IO::Prompt only reads directly from /dev/tty when the -tty or -argv options are specified. Otherwise, it arranges to read from standard input. However, it does this not be simply using the STDIN handle, but by using the first file in ARGV, and if that doesn't work, trying to open - (using the single-arg scalar-and-filehandle form of open). I think (more testing required) Damian did it this way because STDIN may have been redirected away from wherever ARGV and - pointed initially.

This presents an interesting problem. I now need to arrange for opening - to actually cause my pseudo-terminal input handle to be used instead. But, I've already overridden open, and you can't have multple overrides, so I need some kind of multiplexing/dispatch thing to figure out which open replacement to use.

Except I don't. I've just now had a good idea. What if you specified /dev/tty explicitly on the command line as the input source? Wouldn't we want that intercepted also? And isn't that in the scope of what Test::MockTerm should do? The answer is yes. I'm going to modify my code to look for /dev/tty in the one-arg form of open, as well as to look for - and use the same handles. That should take care of it. Epiphany!

So thats where I'm at for now. This has been an incredibly challenging project so far, and I haven't actually written any real tests yet! I intend for this code to be released in IO::Prompt 0.99.5 or 0.99.6, depending on how long it takes.

Damian dropped in on Monday to word us up on the finer points of VIM insanity. So now I have an editor that can do anything, one way or another. And if it can't, Perl can, and the two of them work well enough to get things done. As a test, I implemented an insert-mode LDAP lookup which uses Perl to do the LDAP stuff, leaving the editor bits to VIMs internal scripting language.

The Perl interface for talking back into VIM itself (eg reading and setting variables) is pretty horrendous, but I've already written a proof-of-concept for a module that will make life far more Perly, eg instead of this:

VIM::Msg(VIM::Eval("a:arg"));

we can do this:

print $a{arg};

I know which I prefer :)

When and if I get this module into some kind of usable state, it'll go up to CPAN as VIM::Sane.

I also deleted my entire .vimrc and .vim/ and started again. Its now clean and nicely commented, and makes me happy. I finally know what I'm doing :)

I also got a chance on Monday to talk Perl with Damian at lunch, and got an answer to my Class::Std problems as well as lots of IO::Prompt related stuff. But I'm out of time now, will talk about that more later.

The Midnight clone I keep talking about (and I am getting around to writing about, really) is actually a port to Perl of a Java application. My initial goal was to get the thing working, then make it more Perl like.

One interesting pattern that is used throughout the original is special classes that act only as enumerated types - they simply create a bunch of instances of themselves with specific names that act as constants. These classes are known in Java as "typesafe enumerations".

So I needed something to replace them with. My first attempt was to simply create empty packages with a pile of use constant declarations in them. This worked well enough, but the original enumeration classes had additional features - they needed to stringify properly (via a toString method), they needed to compare equal only to themselves (something use constant can't do, since its constants resolve to just plain scalars) and the constants can be objects with methods declared on them.

Now I don't know if all these extras stop the classes are "correct" by some theoretical definition of what a typesafe enumeration should be (if such a definition exists), but there's no denying that this stuff is useful, and Perl has always been about practicality over usefulness. Plus, I've started to think of places where I could use something similar in other code of written - pretty much anywhere that a variable could have a limited number of possible values, like state variables.

So I implemented something generic and useful, that I'm now quite proud of: Class::Constant. It makes it dead simple to declare a basic C-style enumeration, but has enough wackiness to do some really crazy things. I'll talk a bit more about some real examples once I start writing about my project, real soon now. Until then, this is just highly ineffective advertising :)

I gave Daniel a demo of the Midnight clone the other day, and one of the first things he did was to try and do a tab complete in the little command-line interface. It was immediately obvious that any modern command-based interface needs completion and history, so I set out to find a way to provide it.

A quick CPAN search didn't really uncover anything. A couple of the Term::Readline variants claim to have support, but the interface seemed rather clunky (reasonable, since it comes from C). I use and love IO::Prompt, largely because of its trivial interface. The answer became clear - IO::Prompt requires tab completion.

The whole thing took three evenings to implement. It was pretty straightforward. I started by adding support for a -complete option, which took a list of possible values. After that it was just a case of hooking up the tab key, comparing the current input to the values in the list, and replacing part or all of it with the matched value. This worked wonderfully well, and did great things for my confidence - I'm always a little unsure if I'm doing something the correct way when I go to work on someone elses code - particularly when its written by a known hero like Damian Conway :P

Adding the characteristic "beep" when you only get a partial match was next. Trivial, of course - just emit ASCII code 0x7 at the proper time; the terminal takes care of the rest. A -bell option went in alongside - I'm a firm believer in being as flexible as possible.

The next hallmark of tab completion is displaying possible values when there is more than match for the current input. Since I already knew about the available matches, its no effort to print them out, but it wants to look nice too. A little column calculator went in to make things pretty. I also added a "show all <N> matches?" prompt when its likely that showing them all will scroll your terminal. Obviously, getting a prompt is no problem (this is a prompting module, after all :P ), but I also found that prompt() is totally reentrant - it doesn't restore the previous terminal settings when it exits, opting instead to return it to "cooked" mode. I haven't looked in any depth, so I don't know if IO::Prompt or Term::ReadKey is at fault here. Either way, it caused tab and other keys to not be detected correctly after the "show all" prompt. The workaround was to simply chuck the terminal back into raw mode, and it coped nicely.

So that just about finished it, but then while writing examples I started to realise that the whole thing was actually pretty useless. The reason: the vast majority of command line inputs are actually a set of individual and sometimes unrelated fragments. Think about completion for files. If I had to provide an array of every possible file that could be chosen in a given situation, I'd have to provide a list of every file on the system, each with full path. Obviously, this is ridiculous. What's wanted is a way to complete only portions of the input line, with the possible values selected by looking at the surrounding context. More than a humble array can achieve. What was needed was a callback.

I made it so that -complete could handle a coderef as well as an array - the simple array code might still be usable in places, and its certainly easier to understand. I figured it would be enough to simply pass the current input to the callback, and have it look at the contents and return a list of possible values based on it.

This worked, but had problems. The callback code was pretty complex, and when there were multiple possible values, displaying them was awful, because my code only knew how to complete entire input lines, not fragments of lines. So the callback would have to return the full input line with each possible outcome. Perhaps an example: we're writing a program that has to load files, using a "load /path/to/file" command. We want to do shell-style completion for the file/path portion of the input. To work correctly, the callback has to look for the "load " at the start, then split up the file path, look inside the directory, and return any files there. But, it has to return the full input line, so it returns something like:

• load /path/to/file/foo
• load /path/to/file/bar
• load /path/to/file/baz

As well has having piles of redundant information, if my code were to display them, it would show the entire lines, when it should have just shown foo, bar and baz (just like your shell). Obviously, IO::Prompt needed to be smarter - it had to understand fragments and do the splitting itself.

This actually took me about two days of thought to figure out - the bus to work is a great place for pondering. The solution was to have prompt() split the input and pass all of those to the callback, and only do completion on the final item in the split. So in the above example, the callback would return qw(foo bar baz), and thats that. A -split option was added that takes a regex to be passed to Perl's split builtin.

Implementing this took quite a bit of internal gymnastics to achieve because I was having to essentially write the same code for two cases (full line and split fragments). Generalising aspects of the code (mostly the list matching and the inpup replacement code) was proving quite finnicky, until I made the leap of logic that told me I'd made the correct choice with the interface. If we're trying to complete for a full line, that line becomes a split with no delimiting character, and one fragment.

Five minutes later, it was done. And the callback is trivial. The most complex callback you'll ever likely write for this sort of thing is one to do file paths, because of the special cases - there's files and subdirs. I included that in the examples. It weighs in at just eight lines.

All in all, a roaring success. Damian now has the patch. Haven't heard back yet, but I'll see him next week so I'll hassle him then if necessary.