One ATTACH, wherever it runs.
A VGI worker is a normal program, in any language, that answers SQL queries over Arrow IPC —
ATTACH is how DuckDB reaches one (more on the protocol itself:
Architecture →). Where that worker actually runs
is your decision: a process on your laptop, a container pulled from a registry, a command run
over ssh, or a service someone else operates. The SQL that connects to it doesn't change.
Before you can attach to a worker, something has to decide where it runs — and that's really two questions, not one. First: does it run locally, tied to a single DuckDB session, or as a distributed service other sessions can share? Second — only if you picked distributed — who operates it: your own infrastructure, or Orchard, Query Farm's hosted VGI platform?
The first decision
Local or distributed?
Local — your machine, your session.
ATTACH spawns the worker as a subprocess —
uv run for Python, npx for TypeScript,
or a Docker/OCI image VGI pulls and runs itself — talking to DuckDB over a pipe on the same
machine. Simplest setup, zero infrastructure, free. The tradeoff: the worker starts and
stops with that one DuckDB session, so it's not something two people or two services share.
It's still parallel, though — DuckDB can spawn more than one local worker process for a
single query and split the work across them, the same way it parallelizes any other scan.
Distributed — a persistent service, reachable
over HTTP. For anything beyond one person on one machine — a data vendor's
subscribers, a company-wide catalog, anything meant to stay running — the worker runs as a
long-lived HTTP service instead, and any DuckDB instance with network access can
ATTACH to it. That's what makes one worker shareable across a
team, or sellable to a subscriber base, at all. What DuckDB reaches doesn't have to be a
single process, either — a load balancer can front several replicas of the same worker,
scaling it out the same way any other HTTP backend does, invisibly from DuckDB's side.
Still local, just not nearby
Running a worker elsewhere over SSH
LOCATION is just a command, so “local” doesn't have to mean
“this machine.” Nothing about a subprocess worker requires the command to launch
something on the same host — it only has to be a command whose standard input and output
carry the VGI wire protocol back and forth, and ssh does exactly
that: run a command on a remote host and tunnel its stdio back to you.
ATTACH 'metrics' AS metrics (
TYPE vgi,
LOCATION 'ssh gpu-box -- uv run metrics_worker.py'
);
SELECT * FROM metrics.main.gpu_utilization;DuckDB doesn't know the difference. It launched a process, that process speaks VGI, and where the bytes actually come from — this machine or one reachable over your SSH config — is a detail the command line carries, not something the protocol needs to understand. Whatever you'd normally rely on for SSH access — host aliases, an agent, a bastion — applies exactly as it would for any other command you'd run that way. It's a distributed query without standing up an HTTP service at all.
Reference
What LOCATION accepts
The scheme in front decides how DuckDB reaches the worker. Everything below is the full set.
| Form | How the worker is reached |
|---|---|
uv run worker.py — any command
|
Subprocess. DuckDB spawns the command and speaks Arrow IPC over its stdin/stdout. This is the fallback for anything without a scheme, which is why ssh gpu-box -- … works: it is a command like any other. |
launch:<command>
|
Launcher. The same command, spawned once and reused — worth it when startup is expensive (a JVM, a loaded model). The worker outlives the session that started it, and later attaches connect to the running one over a Unix socket. It doesn't have to handle those one at a time, either: the launcher protocol also supports workers that serve multiple attaches concurrently via threads in that same process, instead of queuing behind a single connection. |
unix:///path/to/worker.sock
|
Connect-only. A worker already listening on a Unix domain socket, started and supervised by something other than DuckDB. |
tcp://host:port
|
Connect-only. A worker already listening on a raw TCP socket, for a runtime or network where a Unix socket is not an option. No HTTP in front of it, so keep it to loopback or a trusted network. |
https://worker.example.com/
|
HTTP. A worker deployed as a service (http:// too). Any number of DuckDB sessions attach to the same one, and it sits behind ordinary web infrastructure. |
oci://ghcr.io/org/image:tag
|
Container. Pulled and run through your container runtime, wired over stdin/stdout exactly like a subprocess and pooled the same way. docker:// is an alias; any registry works. |
github://owner/repo@tag/asset
|
GitHub release. Downloads that release asset, verifies its SHA-256, caches it, and runs it as a subprocess — a container-free way to ship a worker. Pin the hash with #sha256=…. POSIX only. |
github-auto://owner/repo@tag
|
The same thing with the asset name built from your platform by convention, so there is no filename to write out. |
worker:/workers/example.js
|
Web Worker. In-browser DuckDB-WASM only: the worker runs as a Web Worker, reached over a SharedArrayBuffer ring instead of a socket. |
LOCATION is dynamically typed: where a form takes options — a
container runtime or volume mounts, say — pass a struct carrying the address and its
options together instead of a plain string.
The second decision
Who operates it?
Once a worker is a network service rather than a subprocess, the next question is who keeps it running.
Your infrastructure
Your own servers, Kubernetes, or cloud account. Full control, and your data never leaves your network — but you own the operations: deploys, scaling, uptime.
Orchard
Query Farm's VGI platform as a service. Subscribe, get a token, attach a URL — the worker is already running, patched, and someone else's problem to keep up. Browse connectors →. Orchard is in early access — reach out to get set up.
Cloud Run, Lambda, Cloudflare Workers, or a warm pool behind a load balancer all work for the self-hosted half — the protocol was built to survive being load-balanced. How the hosting side works →
In one session
Every one of them attaches the same way
Whichever path a worker came from, adding it is one ATTACH. Only
the LOCATION changes, and the catalogs land side by side in the
same session — so a container, a laptop process, and a hosted URL join to each other in one
query.
-- Local: DuckDB spawns a Python worker as a subprocess
ATTACH 'calc' AS calc (TYPE vgi, LOCATION 'uv run calc_worker.py');
-- Container: pulled from a registry and run for you, nothing to install
ATTACH 'sklearn' AS sk (TYPE vgi, LOCATION 'oci://ghcr.io/query-farm/vgi-sklearn:latest');
-- Another machine: LOCATION is a command, and ssh is a command
ATTACH 'metrics' AS metrics (TYPE vgi, LOCATION 'ssh gpu-box -- uv run metrics_worker.py');
-- Hosted: a URL, plus a token from Orchard
CREATE SECRET orchard (TYPE vgi, TOKEN 'orch_live_...');
ATTACH 'kafka' AS kafka (TYPE vgi, LOCATION 'https://orchard.query.farm/kafka');
-- Four workers, four transports, one SQL surface
SELECT * FROM kafka.main.messages LIMIT 10;Keep going