LibX 2.0 Tuple Spaces

Modules in LibX 2.0 communicate with each other using Tuple Spaces.

Rationale

Tuple Spaces accomplish the following goals:

  • Low coupling between modules: changes of one module’s implementation should not require changes to other modules, unless the structure of the tuples consumed or produced by a module changes.
  • Independent composition: as much as possible, a module’s functioning should not depend on the presence or absence of other modules. Of course, a Libapp must contain a necessary combination of modules (typically, a scraper, a display, and zero or more processors.)
  • Execution order independence:  a Libapp should function independent of the order in which modules are executed. If needed, the execution order may be subject to  minimal priority constraints while preserving low coupling and independent composition.
  • Simplicity: see http://www.python.org/dev/peps/pep-0020/ (“only one obvious way to do something”, etc.) – do not provide multiple ways of doing something.
  • Suitability for Metaprogramming: a module’s function can be described by the structure of the tuples it produces (writes), and the templates that guard its operation. When expressed as metadata, these description allows modules to be combined into application in a principled and checkable way.

Tuple Syntax and API

Tuples and templates are expressed in JSON. The wildcard is expressed using the variable libx.space.WILDCARD. LibX supports exclusion in templates, so libx.space.NOT can be given in a template to match only tuples in which a given property is absent.

The API provides a ‘write’ and a ‘take’ operation.  For instance,

libx.space.write({ a : 5, b : "a string", c : true });

would place a tuple (a=5, b=”a string”, c=true) into the space. A call to take removes a tuple if its template matches. The following templates would match the tuple above:

(a=5, b="a string", c=*)
(a=*, b="a string")
(a=*)
( )

The following templates would not match:

(a=2, b=*, c=*)
(a=5, b="a string", c=true, d=false)
(a=5, NOT b)
(c=false)

Since JavaScript does not provide blocking operations, the API for take uses a callback object, like so:

libx.space.take({
   template: { a : 5, b : libx.space.WILDCARD },

   ontake: function (tuple) {
        // tuple.a == 5
        // tuple.b == "a string"
        // tuple.c == true
   }
});

Care must be taken with the asynchronous or synchronous nature of these calls. There are two complementary cases. 

When a take operation occurs, a tuple matching the provided template may or may not already exist in the tuple space.  If a matching tuple exists, it is removed and the callback executed. In this case, the callback ‘ontake’ will run *before* libx.space.take() returns. If no matching tuple exists, the template is added to a set of pending requests, and libx.space.take() will return without executing the callback.

A complementary situation exists for libx.space.write(). If no take is pending that matches the tuple being inserted, the tuple is simply added and write returns. However, if a take is pending, its ‘ontake’ callback will be executed before write returns.

LibX 2.0 module may be guarded by one or more templates. The presence of these templates will wrap the entire module in a libx.space.take call. In other words, the module’s body becomes the function body of an ontake callback, like so:

libx.space.take({
    template: { <guarded-by template goes here> },
    ontake: function (tuple) {
        <the module's body goes here. 'tuple' is available>
    }
});

The actual implementation of guard templates is a bit more involved, however. Guard templates actually execute a module repeatedly whenever a matching tuple appears. In addition, the guarantee that a tuple is only consumed once by each module that has a guard template. This semantics is easily accomplished by augmenting the template with a ‘processed_by_this_module : libx.space.NOT’ property, which is added to a tuple post removal.

Use of templates and tuples

Tuples carry the information that is passed between modules. Writing a tuple is synonymous to outputting data, while taking one is synonymous with input. A template is an input specification.

We will define a metadata specification for modules that describes the structure of tuples produced and consumed. This metadata specification will allow us to check if modules are being combined in a libapp in a manner that makes sense, such that there is at least one module whose input template specification matches the structure of any output tuple.