the 1010 query language is state-based:  we start with a base table, and then transform it sequentially via a series of "ops", possibly "linking in" additional tables.

the ops are:

    base    define a base table
    sel     select rows
    sort    sort
    willbe  define a new column
    link    link in a table optionally under a selection expression
    asof    asof link
    tabu    create one or more tabulation columns
    ret     return the table as defined by the current state of the cache (Z K)

aggregation functions are those like avg, sum, first, last, &c.

g_sum is an example of a group function.  it takes three arguments G, S, and X.  you can interpret g_sum[G;S;X] as "sum X grouped by G where S".

example query:

we define two tables:  stations (keys) and readings (data).

then:

    base`readings
    sel"value<10"
    willbe[`f]"value+100"
    link[`stations;`id;`id]""
    willbe[`g]"gsum[state;value<5;value]"
    sel"g<6"
    sort[`f]`down
    t:tabu[`name`state;.+(`vsum`vcount;((sum;`value);(count;`value)))]

    show t
  name        state   vsum  vcount
  ---------   -----   ----  ------
  ASHEVILLE   NC        10       2
  FLAGSTAFF   AZ         8       2
  HUNTINGTON  WV         7       1
  HUNTSVILLE  AL         6       1
  PADUCAH     KY         3       1

link syntax:  link[to-table;from-cols;to-cols;boolean-valued-expression]

asof syntax:  link[to-table;from-cols;to-cols;boolean-valued-expression]

tabu syntax:  tabu[break-cols;.+(newcols;..(aggregation;sourcecol)..)]

conceptually, the "result" (i.e. side-effect) of each op (other than 'tabu' and 'get') is a new base-table for the next op.

wherever we may use a column we may use an expression involving at least one column or a dictionary of such expressions in the case of break columns for 'tabu'.

we have five scripts:

1010_1.k contains the code for the optimized version.
show.k contains a pretty-printer function for k tables.
e1010.k loads 1010_1.k and show.k and runs an example.
1010_0.k contains an unoptimized version of the query functions and runs an example.
data.k contains the data for the examples.

thus:

  \l e1010

or

  \l 1010_0

the optimized version uses six globals:  I, C, W, L, Z, and K.

I is the current permutation vector.

W is the willbe column cache.

L is the link-index cache.

C is the column dictionary.  C[`column] is one of:

    base     = <`table> or <"table">
    link     = (<foreign_table>;<local_cols>;<foreign_cols>;<boolean-valued-expression>)
    willbe   = ,<expression> or (<"expression">;<I>)

Z is the operations cache.

K is the index into the current state of the operations cache.

the 'base' function creates an entry in C consisting of either:

- a symbol of the base-table (if that table is a variable in memory),
- a string of the base-table (if that table is a directory of .l files on disk).

the 'sel' function selects rows based on a boolean-valued expression.

the 'sort' function implements multi-column sort.

Both 'sel' and 'sort' modify the global permutation vector I.

the 'willbe' function creates an entry in C consisting of the associated expression and, if a sticky function is used, the current state of I.

the 'link' function creates an entry in C consisting of the linktable, the components of the link-index vector (x?/:y), and the selection expression.

the 'tabu' function calculates and returns an aggregation.  tabu does not change state (i.e. the cache).

the 'col' function implements column reference.  the value of a column c depends on the representation of c in the C dictionary (see above).

if c is a base column, then it is either a column in a dictionary in memory or a .l file.

if c is a linked-in column, then we must either compute the link-index to align it to the base-table and then cache the result in L, or extract its cached value from L.

if c is a willbe column then it is either an expression which we must parse and execute, or in the case of a sticky expression it is an expression stored together with the state of the permutation vector I as it was at the time when the willbe was defined.  like the case of the linked-in column, the result is cached the first time it is computed.