On Promises

I saw a chunk of code today that made my eyes pop out because of its lack of promise. Well, not its lack of promise, but its absence of MT::Promise. The MT::Promise class, new in Movable Type 3.0, solves a common coding problem so it's worth getting the word out about what it does.

What made my eyes pop out was this chunk of code, in the definition of a container tag for looping over categories:

  my $needs_entries = ($ctx->stash('uncompiled') =~ /<\$?MTEntries/) ? 1 : 0;

  if ($needs_entries) {
    my @entries = MT::Entry->load(@args);
    $ctx->{__stash}->{'entries'} = delay (sub{\@entries});
    $ctx->{__stash}->{'category_count'} = scalar @entries;
  } else {
    $ctx->{__stash}->{'category_count'} = MT::Entry->count(@args);
  }

What's happening here is that the tag is trying to determine whether it contains a use of the MTEntries tag, and if so, it's getting some data (a list of entries, and a count of them) for that plugin. Otherwise, it avoids getting that data and gets the count of those entries in a more efficient way. The assumption is that if MTEntries isn't used within this tag, then no part of the computation will expect the 'entries' stash variable to be filled in. That assumption in itself may not be valid. At the same time, this code gets the 'category_count' data regardless of whether it thinks that value will be needed.

To determine whether the MTEntries tag is used within the present tag, the above code uses a regular expression match on the textual contents of the tag, but this is somewhat fragile. After all, what if something else matches that pattern? What if someone adds a new tag that also needs the entries? Finally, and most damning of all, the above code is structured in a substantially different way than the more naive scenario where it unconditionally loads the entries—and since the code is more complicated, it's more prone to programmer errors.

Luckily, this is the exact problem that the MT::Promise class was designed to solve. A promise is an object which delays computing some value until such time as that value is actually needed. If it's never needed, the computation is never performed.

Now, passing around a promise for a value is a lot like passing around the value itself—except that you don't need to compute the value until it's actually needed. In designing an interface for a subroutine or an MT tag, you may want to have a value available, but not know ahead of time whether it will be needed. In such a case, you need to make a promise.

In the Movable Type core, we made use of this feature to speed up rebuilds. We recognized that some users have configured their templates to list Monthly archives and, for each month, to list the entries that fall under that archive. (This applies to Daily archives as well, but not to Weekly, for subtle reasons) The interface between the MTArchiveList tag and the MTEntries tag required the former to load a stash variable called 'entries' with the list of entries for that month. But loading the entries required a database hit (typically a disk access), and marshaling a lot of data, and yet not all users would use this list of entries. The conundrum arose because it was difficult to know ahead of time whether the MTEntries tag was used within the MTArchiveList tag.

By using a promise, we don't have to know ahead of time. And what's more, when using a promise, you write your code in much the same way as you would in the naive scenario; the only change you have to make is to indicate that you'd like to use a promise, saying in effect, "Here's the expression to get the entries, but don't go and evaluate that baby until the value is needed!" To do so, you simply change this:

    $variable = function_call($arg1, $arg2, $arg3);

to this:

    $variable = delay(sub { function_call($arg1, $arg2, $arg3) });

Now, mind you, whoever is using $variable has to be aware that you're doing this. The consumer of this value will need to "force" the promise:

    $underlying_data = force($variable);

Forcing the promise is what ultimately causes the delayed code to be evaluated.

In Movable Type 3.0, we changed the usage of $ctx->stash('entries') across the board so it would accept a promise, but we retained backward compatibility by making use of Perl's wacky typing rules. Since Perl has type information available at runtime, we could to overload the semantics of this stash variable: if 'entries' contains an MT::Promise object, then we treat it as a promise, and force it; if you stick an array reference in there (as you were wont to do in pre-3.0 versions) then we treat it as the entry list itself. This trick will work in most places where you'd want to use a promise. Since the MT::Promise object generally has a different type than the underlying data, you can usually tell whether you've got a promise, which needs to be forced before it can be used, or a raw value, which is already ready for use. The only case where that wouldn't work is where the object you want to promise is itself a promise—an unlikely scenario, but not outside the bounds of propriety. If you are in that situation, making promises on promises, I recommend you retain a lawyer.

In Movable Type's template system we've so far only used promises in the convention of one stash variable—but the new MT::Promise class can be used in a variety of situations. I encourage you to experiment with the class and to explore ways of using it. I'd be very pleased if the usual ingenuity of the Movable Type developer community came up with some unexpected uses for promises.

Caveats and Mazeltovs

Note that if the code in your promise depends on a global variable, you'll be at the mercy of whatever changes take place on that global before the first time the promise is forced. Generally speaking, you won't want to have to worry about the values of some globals. If you really need to access the value of a global from the time when your promise was created, you can copy it into a local variable:

    my $local_copy = $global_var;
    $the_promise = delay(sub { $local_copy });

Now you'll be at the mercy of any changes to the local variable, but it's pretty easy to make sure that the local one doesn't change, since it has a small scope. Keep in mind that such things as the stash variables of MT::Template::Context, as well as members of singleton classes, are effectively global, because their values could change outside of your local scope.

One more note about promises. You might think that delaying the evaluation of an expression like this would lead to some inefficiencies. In fact, it's not bad. Even if you force a promise more than once, it only runs the delayed code one time, so the amount of computation performed is pretty close to what it would have been if you had run the code up front—except that you don't have to run the code at all, if you never need it.