The database you already run.
The data product you haven’t built yet.
One standard way to connect to both.
Proxy existing database drivers or build an ADBC service for anything your code can reach. Write it in Rust, Go, Python, or TypeScript. You define the query; Apache Arrow carries the results.
Apache 2.0 · Built by Query.Farm
New to ADBC? Start here.
ADBC stands for Arrow Database Connectivity. It is an open standard from the Apache Arrow project for connecting applications to databases. Like ODBC or JDBC, it gives developers a familiar set of operations: connect, run a query, and read the results.
ADBC is built around Apache Arrow: a format that organizes data into typed columns. Results arrive in batches that analytical tools and data frames can consume efficiently, reducing the need for format conversions.
Grainlift brings that interface to your service or idea. ADBC drivers typically connect to databases. Grainlift can proxy those connections—or let you build a service that behaves like an ADBC driver to its clients, backed by anything your code can reach.
Get to know Apache Arrow ADBC ↗
ADBC for the connection. Arrow for the data.
Both paths use the same Grainlift client driver. Choose where the data comes from; your applications keep speaking ADBC.
Runs the downstream ADBC driver
Built with a Grainlift framework
Database drivers usually interpret a query as SQL. In a custom Grainlift service, your backend decides how to interpret the statement text: SQL, a domain-specific expression, a command, or a natural-language request.
You implement the parser, routing, or model call. The application still executes a statement through ADBC and reads typed Arrow results.
See it in practice: the Rust example acceptsQUERY ↗QUERYGenerate Arrow batchesThe runnable Rust example
prices: AAPL, MSFTFetch a market-data APIReturn symbols, prices, and timestamps
Forecast next weekRun your forecasting modelReturn dates and predicted values
Less integration work.
More places your data can go.
Deliver your data product through a standard ADBC connection. Customers can consume Arrow results in their applications and pipelines.
Centralize database drivers, credentials, and target permissions. Keep native driver installations out of every notebook, job, and application image.
Put your pricing logic, research dataset, or model outputs behind an ADBC interface. Implement the queries and capabilities your backend can actually support.
Frameworks in four languages share the Grainlift contract. Define what a request means and where its data comes from; reuse authentication, handle ownership, bounded results, and cleanup.
Implement Grainlift’s Backend, BackendConnection, and BackendStatement traits.
Implement Backend, Connection, and Statement interfaces; return an Arrow record reader.
Implement Worker, Connection, and Statement hooks; yield lazy PyArrow batches.
Implement backend classes with typed results and the Grainlift service lifecycle.
The Rust path reuses the Grainlift server library; Go, Python, and TypeScript have dedicated backend frameworks. Start from the source repositories and runnable examples. The TypeScript package is prerelease and not yet on npm; Python package publication is a separate release step.
Go supports HTTP, HTTPS, loopback TCP, mTLS TCP, and Iroh with the published VGI-RPC Go v0.30.0 dependency. Iroh requires a separately installed VGI Iroh bridge. See each framework’s repository for transport setup and requirements. Your backend implements SQL behavior, transactions, and other capabilities; unsupported operations return ADBC NOT_IMPLEMENTED. The frameworks do not supply a general SQL engine.
Database proxy throughput and custom-service latency, measured through the native ADBC driver. Select a workload to see the numbers in context.
queries / second
DuckDB through Grainlift · HTTPThe workload. A 200,000-row dataset; 2,048 rows and a 128-byte payload per row returned and verified on each query. Each of 32 independent sessions ran two warmups and 50 measured iterations. One short run per backend over HTTP.
The environment. Client, server, and databases on the same EC2 host: 48 ARM Neoverse-N1 CPUs, 92.6 GiB RAM, Amazon Linux 2023. DuckDB 1.5.5; SQLite and PostgreSQL ADBC drivers 1.12.0; PostgreSQL 14. No query errors in these runs.
Read the workload, memory figures & raw evidence ↗mean query latency
Custom Rust service · TCP / mTLSThe workload. A synthetic Rust service returns 4,096 rows with 64-byte payloads in eight 512-row batches. One reused ADBC connection; ten warmups, 1,000 measured queries, and 100 expected errors per run. Means average three repetitions; successful-query latency excludes expected errors and initial connection setup.
The environment. Same-host EC2 loopback, 48 ARM Neoverse-N1 CPUs, 92.6 GiB RAM, Amazon Linux 2023. Release Rust build, verified mTLS, and bounded result connection reuse. No downstream database. All reported cases passed with zero unexpected errors.
Read the methodology & raw evidence ↗These are different workloads, not a proxy-versus-framework comparison. Same-host measurements do not establish WAN performance or production capacity. There is no direct-ADBC baseline here. Go and TypeScript performance is not represented by these results.
Grainlift and VGI are complementary products, built for different ways of using data. You can use both in the same platform.
Grainlift gives applications a standard ADBC connection to databases and custom services. VGI extends DuckDB and Haybarn with remote tables and functions that participate in SQL queries.
They share VGI-RPC and Apache Arrow underneath. That common foundation supports both database-style access from applications and custom capabilities inside a query engine.
Explore the VGI product →
| Your goal | Grainlift | VGI |
|---|---|---|
| Reach your users | Connect notebooks, applications, and pipelines through an ADBC driver. | Let DuckDB and Haybarn users query remote tables and call functions from SQL. |
| Expose your backend | Proxy an existing database driver, or interpret requests in your own data service. | Build a worker that exposes data and computation through VGI’s table and function interfaces. |
A pricing team could expose its engine through Grainlift for applications that submit pricing requests, and through a VGI worker for analysts who call a pricing function inside a DuckDB query. Both implementations can reuse the team’s underlying pricing logic.
Each interface needs its own implementation. Sharing VGI-RPC does not make a Grainlift endpoint a VGI catalog, or make a VGI worker an ADBC service. Grainlift does not require DuckDB.
What will you connect next?Put an existing database behind Grainlift, or build the service only your team can build.
Apache Arrow, Arrow, Apache Arrow ADBC, ADBC, and the Apache Arrow logo are trademarks or registered trademarks of The Apache Software Foundation in the United States and other countries.