NaTcl : Native Client Tcl Port

In April 2011, Google opened NaCl to outside developers. The motivation for porting Tcl to it stemmed from a general frustration about not (easily) having Tcl in browsers. To the non-JS world, NaCl comes across as an opening to alternate languages.

(for other -- and promising ! -- methods to bring Tcl into the browser landscape, see Steve Landers' paper.)

But the real trigger was Colin Mc Cormack's unwinking enthusiasm and support, backed by his deep knowledge of the whole field (WubTk in perspective).

The salient issue that comes to mind when thinking about an NaCl port of Tcl is clearly the isolation from the OS. First, one may ask, How are we supposed to do interesting things in such a neutered environment ?

The answer is, of course: use the browser (and its JS context) as a proxy to the real world. Despite the limitations mentioned above, it can still do many things: GUI (of course); fetch intra-domain URLs; access app-restricted local config or user-selected normal files.

Bottom line: we don't need those missing syscalls anyway !

Given the unavailibility of syscalls at link level, two approaches were considered:

• cut "high" : separate Tcl's language and data manipulation core from more peripheral OS-related primitives;

• cut "low" : take it as a whole, faking syscalls.

While the first approach is cleaner, it implies a fair amount of code surgery, which in turn makes it hard to keep in sync with the mainstream codebase. Cutting "low", on the other hand, means a very small set of changes, at the expense of error message clarity ('no such file or directory' instead of 'invalid command name "open"').

After a couple of nanoseconds weighing the options, cutting low sounded like the way to go. This means that the starting point of the porting effort is a list of trivial syscall/libc definitions, typically setting errno to something not-too-alien, and returning the adequate value for failure (NULL or -1). See naclMissing.c.

Once syscall plugging was done, a few ancillary adaptations followed:

• Compatibility headers defining the (unused) structs passed to the emulated syscalls, not provided by the NaCl toolchain's includes. See naclCompat.h

• Toplevel bootstrapping glue calling Tcl_CreateInterp(), wrapping init.tcl, and passing data back and forth to JS. See naclMain.c

• JS support code. See loader.js.

• Incremental build system adaptations: parameterize and call ../unix/configure; patch the generated Makefile.

The bootstrapping code circumvents the absence of local filesystem access by stringifying the contents of init.tcl. This was preferred over a full-fledged VFS by the same reasoning as above: to keep it incremental, refrain from pulling in a significant mass of code.

Note that init.tcl is the only file needing this special handling, because once the interp is initialized, Tcl scripts can take over. For example, [source] is emulated (in init.tcl) by a Tcl coroutine that yields back to JS while the requested URL is fetched by the browser (with a vanilla XHR).

A further motivation for this approach is size and modularity: .nexes tend to be hefty, so as soon as at least two NaTcl-based applications exist (wishful thinking), it is best to share the generic Tcl .nexe in the browser's cache and let the individual apps [source] their specific code (which may be cached too) at init time.

Once we have a working Tcl interpreter, properly adapted to the peculiar syscall-less link environment, the next step is to integrate it into the JS context's lifecycle. This task is outstandingly easy when Everything Is A String ;-). To be fair, JS also takes part in this, with its own eval() function. Indeed, we can set up a very simple "JS trampoline":

•  (JS) String result = natcl.eval("some Tcl code");
   (JS) eval(result);
  

It is important to note that these two lines are not in a tight "while(true)" loop; instead, they are typically invoked from within a JS event handler, which in turn may be set up by (a side effect of) the "eval(result)" line. As a consequence, as long as "some Tcl code" takes a small time to complete (or to [yield]), the JS interpreter and associated browser-borne GUI stay responsive. The coupling between NaTcl and the browser is thus identical to the Tcl/Tk one.

(to be completed with current Nacl+Chrome)

Bottom line:

• the NaTcl balls demo uses roughly thrice the CPU used by the original pure-Javascript code at the same frame rate.

• pure Tcl code, not hampered by the I/O with the JS context, runs marginally slower than native Tcl on the same platform.

One thing about the string I/O bottleneck: the NaCl team promised the advent of TypedArrays in the Pepper API, which will allow to populate JS data with native values (like lists and integers) from within NaCl. This points to a promising optimization of the transmission of a bunch of coordinates, directly from Tcl's Lists and Integers to JS's. TBC, when Google delivers.

The "balls" demo shows that, with NaTcl in hand, a JS newbie (like me) can whip up a non-ridiculous coupling with the HTML5 canvas. The fundamental reason is that while the String is a handy common ground, each side knows to back it with more efficient representations.

Now, within this general string-coupling strategy, many forms of Tcl-side syntax and JS-side tricks are obviously possible. In particular, if you replace the JS newbie with a JS+Tcl expert like Colin, you get NaTk (based on ideas from WubTk). Learn more about it in Steve's paper.