Skip to content
Query.Farm
Talk with Us

2. Your first table function

Step two: add a table function to the worker from step 1. A scalar function transforms a column; a table function produces rows, so it is called in a FROM clause. About 10 minutes.

table shape
args β†’ N rows

A table-valued source: scalar arguments in, a whole set of rows out.

Replace calcscalar.ts with calc.ts, keeping double and adding series:

calc.ts
calc.ts
// Copyright 2025, 2026 Query Farm LLC - https://query.farm

// calc is the worker built across the vgi-typescript tutorial: one scalar
// function and one table function in a single catalog.
//
// The scalar `double` transforms a column in place. The table function `series`
// *generates* rows from an argument, so it is called in a FROM clause rather
// than an expression. One worker can serve any mix of shapes.
//
//   bun run calc.ts
//   # then, in a Haybarn shell:
//   ATTACH 'calc' (TYPE vgi, LOCATION 'bun run /abs/path/calc.ts');
//   SELECT calc.double(21);
//   SELECT * FROM calc.series(3);

import {
  Worker,
  defineScalarFunction,
  defineTableFunction,
  batchFromColumns,
  toSchema,
  int,
} from "@query-farm/vgi";

// ── scalar: double(n) ───────────────────────────────────────────────────────

export const double = defineScalarFunction({
  name: "double",
  description: "Doubles a BIGINT",
  params: { n: int },
  returns: int,
  compute: (batch) => {
    // Columns are erased `Iterable<unknown>`; cast at the use site, where the
    // declared params make the value type known.
    const ns = batch.getChildAt(0)! as Iterable<bigint | null>;
    return Array.from(ns, (v) => (v == null ? null : v * 2n));
  },
});

// ── table: series(count) ────────────────────────────────────────────────────

// The output schema is fixed, so it can be built once at module scope. A table
// function whose columns depend on its arguments would build it in onBind.
const seriesSchema = toSchema({ n: int });

const BATCH_SIZE = 1024;

export const series = defineTableFunction({
  name: "series",
  description: "Generates the integers 0..count-1",

  args: { count: int },
  argDocs: { count: "How many numbers to generate" },
  // Enforced at bind, not just advertised: series(-1) fails before any row is
  // produced, rather than looping or silently returning nothing.
  argConstraints: { count: { ge: 0 } },

  onBind: () => ({ outputSchema: seriesSchema }),

  // Runs once per scan, after bind. Arguments are fixed for the whole scan, so
  // this is where they are read β€” decoding them per batch would be waste.
  //
  // BigInt() is not decoration. A table function's `args` arrive as JS numbers
  // even for an int64 argument, while the same type reaches a *scalar*
  // function's columns as bigint. Normalize once here and the rest of the
  // function can do bigint arithmetic without a runtime "Invalid mix of BigInt
  // and other type" surprise on the first batch.
  initialState: ({ args }) => ({ i: 0n, count: BigInt(args.count) }),

  // process is the pull loop: DuckDB calls it repeatedly and consumes lazily.
  // Emit what you can, then finish() to signal end-of-stream. Nothing has to be
  // materialized up front, which is what makes a table function a *generator*.
  process: (_params, state, out) => {
    if (state.i >= state.count) return out.finish();
    const end = state.count - state.i > BigInt(BATCH_SIZE)
      ? state.i + BigInt(BATCH_SIZE)
      : state.count;
    const ns: bigint[] = [];
    for (let k = state.i; k < end; k++) ns.push(k);
    out.emit(batchFromColumns({ n: ns }, seriesSchema));
    state.i = end;
  },
});

export const worker = new Worker({
  catalog: {
    name: "calc",
    comment: "Tutorial worker: a scalar and a table function",
    schemas: [{ name: "main", functions: [double, series] }],
  },
});

if (import.meta.main) worker.run();

Five things to notice:

  • State is separate from arguments. initialState builds the per-scan cursor; arguments are fixed for the whole scan, so they are read once, here, rather than per batch.
  • process is a pull loop. DuckDB calls it repeatedly and consumes lazily. Emit what you can, then out.finish() to signal end-of-stream. Nothing has to be materialized up front β€” that is what makes a table function a generator.
  • argConstraints are enforced, not just advertised. { count: { ge: 0 } } travels into the registered spec, so series(-1) fails at bind rather than looping or silently returning nothing.
  • The output schema is built once. toSchema({ n: int }) at module scope, because it never varies. A function whose columns depend on its arguments would build it inside onBind.
  • BigInt(args.count) is load-bearing. See the warning below.
A table function's arguments arrive as numbers, not bigints

An int argument reaches a scalar function’s columns as a bigint. The same int argument reaches a table function’s args as a JS number.

That asymmetry is invisible until the first batch, where mixing the two throws β€œInvalid mix of BigInt and other type in subtraction” β€” at runtime, from inside process, with a stack that points at your arithmetic rather than at the cause. Normalize once in initialState:

initialState: ({ args }) => ({ i: 0n, count: BigInt(args.count) }),

Then the rest of the function is bigint throughout and the question never comes up again.

Why a generator?

A table function is pulled, not called: the engine asks the worker for output until it signals completion. process is the pull handler. series knows up front how many rows it owes, so its state is just a counter β€” but a generator reading a paginated API would keep its cursor in the same state object and decide for itself when it is done.

Re-attach the new file and query it in a FROM clause:

LOAD vgi;
ATTACH 'calc' (TYPE vgi, LOCATION 'bun run ./calc.ts');

SELECT * FROM calc.series(3);

Output

n
0
1
2

Both functions live in one worker, so they compose in a single query:

SELECT calc.double(n) AS doubled FROM calc.series(3);

Output

doubled
0
2
4

The batching is real, not decorative β€” series(5000) streams through in chunks of 1024:

SELECT count(*), sum(n) FROM calc.series(5000);

Output

count_star() sum(n)
5000 12497500

And the constraint does its job:

SELECT * FROM calc.series(-1);
Invalid Input Error: VGI Worker Exception: ArgumentValidationError: argument 'count' must be >= 0

What just happened: one calc worker now serves two functions. series ran in the FROM clause β€” DuckDB pulled batches from process until it called finish() β€” and double transformed them, all in the same process.

Troubleshooting
  • Binder Error: Failed to attach database: database with name "calc" already exists β€” calc is still attached from step 1. Run DETACH calc; or open a fresh shell.
  • Function "series" is a table function but it was used as a scalar function β€” table functions go in FROM, not SELECT.
  • Invalid mix of BigInt and other type β€” the BigInt(args.count) normalization above.
  • Empty result β€” series(0) is legitimately zero rows. Try series(5).
  • The scan never ends β€” a generator that never reaches its finish() is an infinite scan. process must make progress toward the condition it checks.