Skip to content
Query.Farm
Talk with Us

1. Your first scalar function

The first tutorial step: build a worker with one scalar function and call it from SQL — about 10 minutes, for first-time VGI users with Bun or Node.js 22.15+.

What's a “worker”?

A worker is a small program DuckDB launches as a subprocess and talks to over Apache Arrow. It exposes one or more typed functions, and DuckDB calls them like built-ins. It is an ordinary TypeScript program — nothing is compiled into DuckDB.

scalar shape
1 row → 1 value

Runs on each row independently and returns a single value — a pure per-row transform.

Why this is powerful

double is intentionally trivial — DuckDB can already do n * 2. The point is that compute is ordinary TypeScript: drop in an npm package, a model, a fetch, a parser DuckDB has never heard of, and DuckDB calls it like a native SQL function.

Create a project and add the SDK:

mkdir calcscalar && cd calcscalar
bun init -y
bun add @query-farm/vgi @query-farm/apache-arrow @query-farm/vgi-rpc

All three go in explicitly. @query-farm/apache-arrow and @query-farm/vgi-rpc are peerDependencies of the SDK, not transitive installs — the SDK marks them external so a single shared copy is used, which is what prevents the duplicate-Protocol type clash you would otherwise hit the moment you import anything from vgi-rpc directly.

Bun reports a blocked postinstall. Ignore it.

bun add prints “Blocked 1 postinstall”. The script it blocked is a development-only fixup for the SDK’s own monorepo — it checks for a sibling vgi-rpc-typescript checkout and exits immediately when there isn’t one, so it is a no-op in your project either way.

Then calcscalar.ts:

calcscalar.ts
// Copyright 2025, 2026 Query Farm LLC - https://query.farm

// calcscalar is the worker built in step 1 of the vgi-typescript tutorial: one
// scalar function, served over stdio, callable from DuckDB as calc.double().
//
// A scalar function is the simplest shape — one row in, one value out, with no
// state and no finalize phase. DuckDB hands the worker a whole Arrow column and
// expects a column of the same length back.
//
//   bun run calcscalar.ts
//   # then, in a Haybarn shell:
//   ATTACH 'calc' (TYPE vgi, LOCATION 'bun run /abs/path/calcscalar.ts');
//   SELECT calc.double(21);

import { Worker, defineScalarFunction, int } from "@query-farm/vgi";

// `int` is the Int64 type alias. compute() therefore reads and returns bigint —
// DuckDB's BIGINT does not fit in a JS number, so the SDK never narrows it.
export const double = defineScalarFunction({
  name: "double",
  description: "Doubles a BIGINT",
  params: { n: int },
  returns: int,

  // compute runs once per input BATCH, not per row. Arguments are positional:
  // column 0 is the first argument, whatever the params key is called.
  compute: (batch) => {
    // A column is an erased `Iterable<unknown>` — the facade cannot know the
    // value type, because arrow-js and flechette parameterize differently. The
    // cast at the use site is the intended pattern, and under `strict` it is
    // required: without it `Array.from`'s mapper gets `unknown`.
    const ns = batch.getChildAt(0)! as Iterable<bigint | null>;
    // One output value per input row. null in → null out.
    return Array.from(ns, (v) => (v == null ? null : v * 2n));
  },
});

// Functions are served through a catalog, and the catalog's name is the name
// DuckDB ATTACHes. They must match: attaching under any other name fails.
export const worker = new Worker({
  catalog: { name: "calc", schemas: [{ name: "main", functions: [double] }] },
});

if (import.meta.main) worker.run();

Four things to notice:

  • params is the signature. It declares the argument types and their order. DuckDB calls scalar functions positionally, so the keys are names for documentation and introspection — inside compute you read columns by position.
  • A whole column at a time. compute receives an Arrow batch, not one row. It runs once per batch and returns one output value per input row.
  • int means bigint. DuckDB’s BIGINT does not fit in a JS number, so the SDK never narrows it. Values arrive and leave as bigint.
  • The catalog name is the ATTACH name. They are not independent: attaching under any other name fails with “No worker handles catalog …”.
Columns are erased — cast at the use site

batch.getChildAt(0) is a VgiColumn: an Iterable<unknown> whose get() returns unknown. That is deliberate — the two Arrow backends the SDK can run on parameterize their column types differently, so the facade erases the value type rather than picking one.

The consequence is that under strict you must say what you are reading:

const ns = batch.getChildAt(0)! as Iterable<bigint | null>;
return Array.from(ns, (v) => (v == null ? null : v * 2n));

Without the cast, Array.from’s mapper receives unknown and the call does not typecheck. Your declared params are what make the real type known at that point.

New to Apache Arrow?

Apache Arrow is a language-independent columnar memory format. Rather than rows of objects, data lives in arrays: a contiguous, typed sequence of values for a single column. VGI hands your function a whole column, and operating on it at once is what keeps it fast across the process boundary.

There is nothing to build — bun run executes TypeScript directly. Run the worker once to confirm it starts:

bun run calcscalar.ts

It prints a couple of startup lines to stderr and then exits with “Input stream closed before first IPC message”. That is the correct outcome: the worker speaks Arrow over stdin/stdout and there is nobody on the other end yet. What you are checking for is the absence of an import or syntax error.

VGI functions run inside a DuckDB-compatible engine. We’ll use Haybarn — start it from the folder holding your worker:

npx haybarn@rc

This opens Haybarn’s in-memory SQL shell — the memory H prompt. (First run downloads the CLI; see other install options.)

Haybarn serves the vgi extension

Haybarn distributes the vgi extension through its own channel, so INSTALL vgi FROM community; works out of the box. vgi isn’t in DuckDB’s public community repository, so stock DuckDB can’t INSTALL it today — Haybarn is the supported path. The extension is the same one Python, Go and TypeScript workers all talk to.

At the memory H prompt:

INSTALL vgi FROM community;
LOAD vgi;

ATTACH 'calc' (TYPE vgi, LOCATION 'bun run ./calcscalar.ts');

LOCATION is the command DuckDB runs, not a path to a binary — which is why a TypeScript worker needs no build step at all.

Now call it:

SELECT calc.double(21);

Output

double(21)
42

…or over a whole column:

SELECT calc.double(n) FROM (VALUES (1), (2), (3)) AS t(n);

Output

double(n)
2
4
6

Nulls pass straight through, because compute maps them to null:

SELECT calc.double(n) FROM (VALUES (5), (NULL)) AS t(n);

Output

double(n)
10

What just happened: ATTACH ran bun run ./calcscalar.ts as a subprocess and registered calc.double in your SQL session. DuckDB handed your TypeScript the n column as a single Arrow array, compute ran over the whole thing, and the result streamed back — no row-by-row round trips. Swap the body for any TypeScript you like and the SQL above doesn’t change.

You’ve built and run your first VGI function in TypeScript. 🎉

Troubleshooting
  • IO Error: VGI worker not found or not executableLOCATION is a command run from the directory the engine was started in. Check that bun is on PATH and the relative path resolves.
  • No worker handles catalog 'x' — the name in ATTACH must equal the catalog.name in the worker. It is not a free alias.
  • Cannot find package '@query-farm/apache-arrow' — the peer dependencies were not installed. Add all three packages, not just @query-farm/vgi.
  • The ATTACH hangs — run bun run ./calcscalar.ts directly. It looks like it hangs waiting for input; you’re checking for a startup error on stderr.
  • Catalog Error: Scalar Function with name double does not exist! — the name comes from the function’s name field, qualified by the catalog name.