Vector Gateway Interface
Turn APIs, models, and code into SQL.
VGI lets Haybarn and DuckDB attach a remote worker as if it were a normal SQL catalog. Your data, service, model, or Python/TypeScript/Go/Rust code stays in its own runtime; SQL users get tables and functions they can discover, query, and join.
What VGI does
A gateway from real systems to SQL
DuckDB sees
ATTACH 'my_api' AS my_api (
TYPE vgi,
LOCATION 'https://worker.example.com'
);
SELECT *
FROM my_api.main.customers;The gist
A SQL interface for the systems DuckDB cannot reach by itself
VGI is for the moment when your data or logic is not already a file, table, or native DuckDB extension. You write or use a worker that exposes an API, model, service, dataset, or library. Haybarn attaches to that worker, discovers what it exposes, and makes it queryable from SQL.
Expose a hosted service as a SQL catalog.
Discover schemas and tables from DuckDB.
Join remote workers with local files and tables.
Ship workers in the language and runtime your team already uses.
VGI
The DuckDB interface
VGI is the extension and Arrow-based protocol DuckDB uses to attach something outside DuckDB as a catalog. It handles discovery, planning, types, and moving batches back and forth.
VGI worker
The service that exposes data
A worker is the process or hosted service you build or reuse. It owns the API calls, model code, auth, schemas, tables, and functions that DuckDB users query through VGI.
Under the hood
A protocol, not a framework
A small DuckDB extension speaks the VGI protocol from inside DuckDB. Lightweight worker libraries speak it from Python, TypeScript, Go, Rust, Java, or C++. The two sides exchange Apache Arrow batches over pipes, sockets, shared memory, or HTTP.
The important part: your code runs outside DuckDB. That means no DuckDB headers, no FFI tangle, and no native rebuild every time your runtime or DuckDB version changes.
DuckDB
SQL calls tables and functions
VGI
Arrow request and response
Worker
Your service or code runs here
Technical view
How the pieces fit together
DuckDB on one side, your worker on the other, Apache Arrow IPC between them. Pick the language SDK and transport that match your stack.
Hover over each step to learn more.
SQL caller
DuckDB session
A query calls tables or functions exposed by an attached VGI catalog.
DuckDB side
VGI extension
The extension plans the call, validates types, and speaks the VGI protocol from inside DuckDB.
Wire layer
Arrow transport
Requests and responses move as Arrow batches over pipes, sockets, shared memory, or HTTP.
Your code
Worker runtime
Python, TypeScript, Go, Rust, Java, or C++ code runs in its own process or service.
The DuckDB-side is a small C++ extension. Your code lives in a separate process, written in any of the supported languages. The two ends speak Apache Arrow IPC over a transport you choose — pipes for the spawned-subprocess case, Unix sockets for an existing local worker, shared memory for high-throughput co-located workloads, HTTP for remote ones.
What you can build
Use cases
Expose external APIs
Wrap REST, gRPC, or any HTTP service as SQL-accessible tables and functions. Query remote data alongside local tables without building connector plumbing.
Call models from SQL
Score rows through PyTorch, ONNX, or any model that lives in your own runtime. Keep the model where it belongs and call it from DuckDB.
Reuse your code
Make pandas transforms, Go services, TypeScript validators, and internal business logic available to SQL users without compiling native DuckDB extensions.
The 10-second tour
From hosted worker to SQL catalog
-- 1. Install + load VGI
INSTALL vgi FROM community;
LOAD vgi;
-- 2. Attach a hosted worker as a SQL catalog
ATTACH 'volcanos' AS volcanos (
TYPE vgi,
LOCATION 'https://vgi-volcanos.fly.dev/'
);
-- 3. Query the remote service like normal tables
SELECT name, country, primary_type
FROM volcanos.smithsonian.pleistocene_volcanoes
LIMIT 10;Dive deeper
Where to next
Build with VGI
Building something with VGI?
We help teams design, build, and ship VGI-based extensions. From a one-day spike to a long-term engagement.