2. Your first table function
Step two: add a table function to the worker from
step 1. A scalar function transforms a column; a table
function produces rows, so it is called in a FROM clause. About 10 minutes.
Step 1 β Add the function
Section titled βStep 1 β Add the functionβReplace calcscalar.ts with calc.ts, keeping double and adding series:
calc.ts
// Copyright 2025, 2026 Query Farm LLC - https://query.farm
// calc is the worker built across the vgi-typescript tutorial: one scalar
// function and one table function in a single catalog.
//
// The scalar `double` transforms a column in place. The table function `series`
// *generates* rows from an argument, so it is called in a FROM clause rather
// than an expression. One worker can serve any mix of shapes.
//
// bun run calc.ts
// # then, in a Haybarn shell:
// ATTACH 'calc' (TYPE vgi, LOCATION 'bun run /abs/path/calc.ts');
// SELECT calc.double(21);
// SELECT * FROM calc.series(3);
import {
Worker,
defineScalarFunction,
defineTableFunction,
batchFromColumns,
toSchema,
int,
} from "@query-farm/vgi";
// ββ scalar: double(n) βββββββββββββββββββββββββββββββββββββββββββββββββββββββ
export const double = defineScalarFunction({
name: "double",
description: "Doubles a BIGINT",
params: { n: int },
returns: int,
compute: (batch) => {
// Columns are erased `Iterable<unknown>`; cast at the use site, where the
// declared params make the value type known.
const ns = batch.getChildAt(0)! as Iterable<bigint | null>;
return Array.from(ns, (v) => (v == null ? null : v * 2n));
},
});
// ββ table: series(count) ββββββββββββββββββββββββββββββββββββββββββββββββββββ
// The output schema is fixed, so it can be built once at module scope. A table
// function whose columns depend on its arguments would build it in onBind.
const seriesSchema = toSchema({ n: int });
const BATCH_SIZE = 1024;
export const series = defineTableFunction({
name: "series",
description: "Generates the integers 0..count-1",
args: { count: int },
argDocs: { count: "How many numbers to generate" },
// Enforced at bind, not just advertised: series(-1) fails before any row is
// produced, rather than looping or silently returning nothing.
argConstraints: { count: { ge: 0 } },
onBind: () => ({ outputSchema: seriesSchema }),
// Runs once per scan, after bind. Arguments are fixed for the whole scan, so
// this is where they are read β decoding them per batch would be waste.
//
// BigInt() is not decoration. A table function's `args` arrive as JS numbers
// even for an int64 argument, while the same type reaches a *scalar*
// function's columns as bigint. Normalize once here and the rest of the
// function can do bigint arithmetic without a runtime "Invalid mix of BigInt
// and other type" surprise on the first batch.
initialState: ({ args }) => ({ i: 0n, count: BigInt(args.count) }),
// process is the pull loop: DuckDB calls it repeatedly and consumes lazily.
// Emit what you can, then finish() to signal end-of-stream. Nothing has to be
// materialized up front, which is what makes a table function a *generator*.
process: (_params, state, out) => {
if (state.i >= state.count) return out.finish();
const end = state.count - state.i > BigInt(BATCH_SIZE)
? state.i + BigInt(BATCH_SIZE)
: state.count;
const ns: bigint[] = [];
for (let k = state.i; k < end; k++) ns.push(k);
out.emit(batchFromColumns({ n: ns }, seriesSchema));
state.i = end;
},
});
export const worker = new Worker({
catalog: {
name: "calc",
comment: "Tutorial worker: a scalar and a table function",
schemas: [{ name: "main", functions: [double, series] }],
},
});
if (import.meta.main) worker.run();
Five things to notice:
- State is separate from arguments.
initialStatebuilds the per-scan cursor; arguments are fixed for the whole scan, so they are read once, here, rather than per batch. processis a pull loop. DuckDB calls it repeatedly and consumes lazily. Emit what you can, thenout.finish()to signal end-of-stream. Nothing has to be materialized up front β that is what makes a table function a generator.argConstraintsare enforced, not just advertised.{ count: { ge: 0 } }travels into the registered spec, soseries(-1)fails at bind rather than looping or silently returning nothing.- The output schema is built once.
toSchema({ n: int })at module scope, because it never varies. A function whose columns depend on its arguments would build it insideonBind. BigInt(args.count)is load-bearing. See the warning below.
An int argument reaches a scalar functionβs columns as a bigint. The same int argument
reaches a table functionβs args as a JS number.
That asymmetry is invisible until the first batch, where mixing the two throws
βInvalid mix of BigInt and other type in subtractionβ β at runtime, from inside process, with a
stack that points at your arithmetic rather than at the cause. Normalize once in initialState:
initialState: ({ args }) => ({ i: 0n, count: BigInt(args.count) }),
Then the rest of the function is bigint throughout and the question never comes up again.
A table function is pulled, not called: the engine asks the worker for output until it signals
completion. process is the pull handler. series knows up front how many rows it owes, so its
state is just a counter β but a generator reading a paginated API would keep its cursor in the same
state object and decide for itself when it is done.
Step 2 β Call it
Section titled βStep 2 β Call itβRe-attach the new file and query it in a FROM clause:
LOAD vgi;
ATTACH 'calc' (TYPE vgi, LOCATION 'bun run ./calc.ts');
SELECT * FROM calc.series(3);
Output
| n |
|---|
| 0 |
| 1 |
| 2 |
Both functions live in one worker, so they compose in a single query:
SELECT calc.double(n) AS doubled FROM calc.series(3);
Output
| doubled |
|---|
| 0 |
| 2 |
| 4 |
The batching is real, not decorative β series(5000) streams through in chunks of 1024:
SELECT count(*), sum(n) FROM calc.series(5000);
Output
| count_star() | sum(n) |
|---|---|
| 5000 | 12497500 |
And the constraint does its job:
SELECT * FROM calc.series(-1);
Invalid Input Error: VGI Worker Exception: ArgumentValidationError: argument 'count' must be >= 0
What just happened: one calc worker now serves two functions. series ran in the FROM clause
β DuckDB pulled batches from process until it called finish() β and double transformed them,
all in the same process.
Binder Error: Failed to attach database: database with name "calc" already existsβcalcis still attached from step 1. RunDETACH calc;or open a fresh shell.Function "series" is a table function but it was used as a scalar functionβ table functions go inFROM, notSELECT.Invalid mix of BigInt and other typeβ theBigInt(args.count)normalization above.- Empty result β
series(0)is legitimately zero rows. Tryseries(5). - The scan never ends β a generator that never reaches its
finish()is an infinite scan.processmust make progress toward the condition it checks.
Next steps
Section titled βNext stepsβ- The other three shapes β Function patterns β table-in-out, aggregate and buffering, each with a runnable worker.
- When each callback fires β Function lifecycle.
- Exact contracts β Table functions.