One ATTACH, wherever it runs.
A VGI worker is a normal program, in any language, that answers SQL queries over Arrow IPC —
ATTACH is how DuckDB reaches one (more on the protocol itself:
Architecture →). Where that worker actually runs
is your decision: a process on your laptop, an executable stored in a database, a container
pulled from a registry, a command run over ssh, or a service someone else operates. The SQL
that connects to it doesn't change.
Before you can attach to a worker, something has to decide where it runs — and that's really two questions, not one. First: does it run locally, tied to a single DuckDB session, or as a distributed service other sessions can share? Second — only if you picked distributed — who operates it: your own infrastructure, or Orchard, Query Farm's hosted VGI platform?
The first decision
Local or distributed?
Local — your machine, your session.
ATTACH spawns the worker as a subprocess —
uv run for Python, npx for TypeScript,
or a Docker/OCI image VGI pulls and runs itself — talking to DuckDB over a pipe on the same
machine. Simplest setup, zero infrastructure, free. The tradeoff: the worker starts and
stops with that one DuckDB session, so it's not something two people or two services share.
It's still parallel, though — DuckDB can spawn more than one local worker process for a
single query and split the work across them, the same way it parallelizes any other scan.
Distributed — a persistent service, reachable
over HTTP. For anything beyond one person on one machine — a data vendor's
subscribers, a company-wide catalog, anything meant to stay running — the worker runs as a
long-lived HTTP service instead, and any DuckDB instance with network access can
ATTACH to it. That's what makes one worker shareable across a
team, or sellable to a subscriber base, at all. What DuckDB reaches doesn't have to be a
single process, either — a load balancer can front several replicas of the same worker,
scaling it out the same way any other HTTP backend does, invisibly from DuckDB's side.
Local execution, portable package
A worker can travel inside a database
A database:// location points at a package table instead of a
command or network address. VGI selects the row for the worker name, package version, and
current DuckDB platform; verifies the stored digest; then copies or extracts the package
into its local cache and starts the entrypoint as an ordinary subprocess. A later attach
to the same immutable package reuses that materialized copy.
This is distribution, not remote execution. The database carries the program bytes — a raw executable or an archive containing several files — while each DuckDB process runs its own local worker. That makes a registry database useful for reproducible snapshots, controlled internal distribution, and environments where fetching a release or container at query time is undesirable.
-- The registry is an ordinary attached DuckDB database.
ATTACH 'worker-registry.duckdb' AS worker_registry;
-- VGI selects the requested worker, this machine's platform, and the version.
ATTACH 'models' AS models (
TYPE vgi,
LOCATION 'database://worker_registry/main/packages/models?package_version=2.1.0'
);
SELECT * FROM models.main.predict('features.parquet');
See Workers in a database for the package
table contract, the vgi_worker_package() helper, archives,
platforms, cache cleanup, and the security model.
Still local, just not nearby
Running a worker elsewhere over SSH
LOCATION is just a command, so “local” doesn't have to mean
“this machine.” Nothing about a subprocess worker requires the command to launch
something on the same host — it only has to be a command whose standard input and output
carry the VGI wire protocol back and forth, and ssh does exactly
that: run a command on a remote host and tunnel its stdio back to you.
ATTACH 'metrics' AS metrics (
TYPE vgi,
LOCATION 'ssh gpu-box -- uv run metrics_worker.py'
);
SELECT * FROM metrics.main.gpu_utilization;
DuckDB doesn't know the difference. It launched a process, that process speaks VGI, and where the bytes actually come from — this machine or one reachable over your SSH config — is a detail the command line carries, not something the protocol needs to understand. Whatever you'd normally rely on for SSH access — host aliases, an agent, a bastion — applies exactly as it would for any other command you'd run that way. It's a distributed query without standing up an HTTP service at all.
Reference
What LOCATION accepts
The scheme in front decides how DuckDB reaches the worker. Everything below is the full set.
| Form | How the worker is reached |
|---|---|
uv run worker.py — any command
|
Subprocess. DuckDB spawns the command and speaks Arrow IPC over its stdin/stdout. This is the fallback for anything without a scheme, which is why ssh gpu-box -- … works: it is a command like any other. |
launch:<command>
|
Launcher. The same command, spawned once and reused — worth it when startup is expensive (a JVM, a loaded model). The worker outlives the session that started it, and later attaches connect to the running one over a Unix socket. It doesn't have to handle those one at a time, either: the launcher protocol also supports workers that serve multiple attaches concurrently via threads in that same process, instead of queuing behind a single connection. |
unix:///path/to/worker.sock
|
Connect-only. A worker already listening on a Unix domain socket, started and supervised by something other than DuckDB. |
tcp://host:port
|
Connect-only. A worker already listening on a raw TCP socket, for a runtime or network where a Unix socket is not an option. No HTTP in front of it, so keep it to loopback or a trusted network. |
https://worker.example.com/
|
HTTP. A worker deployed as a service (http:// too). Any number of DuckDB sessions attach to the same one, and it sits behind ordinary web infrastructure. |
oci://ghcr.io/org/image:tag
|
Container. Pulled and run through your container runtime, wired over stdin/stdout exactly like a subprocess and pooled the same way. docker:// is an alias; any registry works. |
github://owner/repo@tag/asset
|
GitHub release. Downloads that release asset, verifies its SHA-256, caches it, and runs it as a subprocess — a container-free way to ship a worker. Pin the hash with #sha256=…. POSIX only. |
github-auto://owner/repo@tag
|
The same thing with the asset name built from your platform by convention, so there is no filename to write out. |
database://catalog/schema/table/worker?package_version=…
|
Database package. Reads a platform-specific executable or archive from a committed DuckDB table, verifies its SHA-256, materializes it in an immutable local cache, and runs it as a subprocess. The package travels with the database; execution stays on the attaching machine. |
worker:/workers/example.js
|
Web Worker. In-browser DuckDB-WASM only: the worker runs as a Web Worker, reached over a SharedArrayBuffer ring instead of a socket. |
LOCATION is dynamically typed: where a form takes options — a
container runtime or volume mounts, say — pass a struct carrying the address and its
options together instead of a plain string.
The second decision
Who operates it?
Once a worker is a network service rather than a subprocess, the next question is who keeps it running.
Your infrastructure
Your own servers, Kubernetes, or cloud account. Full control, and your data never leaves your network — but you own the operations: deploys, scaling, uptime.
Orchard
Query Farm's VGI platform as a service. Subscribe, get a token, attach a URL — the worker is already running, patched, and someone else's problem to keep up. Browse connectors →. Orchard is in early access — reach out to get set up.
Cloud Run, Lambda, Cloudflare Workers, or a warm pool behind a load balancer all work for the self-hosted half — the protocol was built to survive being load-balanced. How the hosting side works →
In one session
Every one of them attaches the same way
Whichever path a worker came from, adding it is one ATTACH. Only
the LOCATION changes, and the catalogs land side by side in the
same session — so a container, a laptop process, and a hosted URL join to each other in one
query.
-- Local: DuckDB spawns a Python worker as a subprocess
ATTACH 'calc' AS calc (TYPE vgi, LOCATION 'uv run calc_worker.py');
-- Container: pulled from a registry and run for you, nothing to install
ATTACH 'sklearn' AS sk (TYPE vgi, LOCATION 'oci://ghcr.io/query-farm/vgi-sklearn:latest');
-- Database: restore a verified platform build from a package table
ATTACH 'models' AS models (
TYPE vgi,
LOCATION 'database://worker_registry/main/packages/models?package_version=2.1.0'
);
-- Another machine: LOCATION is a command, and ssh is a command
ATTACH 'metrics' AS metrics (TYPE vgi, LOCATION 'ssh gpu-box -- uv run metrics_worker.py');
-- Hosted: a URL, plus a token from Orchard
CREATE SECRET orchard (TYPE vgi, TOKEN 'orch_live_...');
ATTACH 'kafka' AS kafka (TYPE vgi, LOCATION 'https://orchard.query.farm/kafka');
-- Five workers, five delivery paths, one SQL surface
SELECT * FROM kafka.main.messages LIMIT 10;
Keep going