DuckDB Extensions

Connect DuckDB to any database with an Arrow Database Connectivity (ADBC) driver — SQLite, PostgreSQL, Snowflake, Flight SQL, and more — using Arrow's columnar wire format end to end. Read, write, and inspect remote schemas without round-tripping through CSV/Parquet.

ATTACH an Arrow Flight server as a DuckDB catalog and query it in SQL. Best fit: custom Flight services you build in Python / Go / Java, plus Flight-native systems like Dremio and InfluxDB 3. Capabilities (pushdown, mutation, schema discovery) follow what your Flight server implements.

Read and write Redis from DuckDB SQL. Strings, hashes, and lists; cursor-paginated SCAN/HSCAN; and a single-query bulk-dump pattern that drains an entire Redis keyspace into a DuckDB table. Best fit: operational queries, hot-cold data joins, and one-off migrations — not application-grade Redis access.

Read Apache Kafka topics from DuckDB SQL. Scan a topic into a query, snapshot it into a table, JOIN against local data, or inspect cluster metadata — all without a separate Kafka client. Authentication, partitions, and offsets are configured through the underlying librdkafka named-parameter surface.

Two-way real-time messaging from DuckDB SQL. Subscribe to WebSocket and Redis Pub/Sub channels, query received events as ordinary tables, and queue outgoing messages with status tracking. Best fit: in-process pipelines, dashboards, and ad-hoc inspection — buffers are in-memory, scoped to the DuckDB process.

Make HTTP GET and POST requests directly from DuckDB SQL. Returns a JSON object with status, reason, and body you can extract inline. Best fit: ad-hoc API calls, low-volume per-row enrichment, and joining a SQL query with a small REST endpoint. Synchronous and buffered — not a streaming or high-throughput HTTP client.

Expose a running DuckDB process over HTTP so clients can POST a SQL query and read JSON results. Single shared key or HTTP Basic auth, an embedded query UI, and zero-config startup. Best fit: small internal API surfaces and ad-hoc dashboards in front of a read-only DuckDB — not a hardened multi-tenant gateway.

Use Unix pipes as DuckDB filenames. Append | to a path and DuckDB executes it as a shell pipeline and reads from stdout; prefix with | to write to stdin. Lets read_csv / read_json / COPY chain into curl, awk, custom scripts — anything that speaks stdio.

Evaluate JSONata expressions against JSON values from inside DuckDB SQL. Best fit: reshaping nested JSON into a new shape, predicate filters on arrays, and concise traversals that get verbose in chains of json_extract. Implemented as a single scalar function — no ATTACH interface, no secrets.

JSON Schema validation and default-value enrichment in SQL. Validate that JSON columns conform to a schema, lint the schema itself, generate JSON-Patch operations that fill in missing defaults, and apply those defaults in place. Targets JSON Schema draft-07.

Cryptographic hashing, HMAC, and secure random bytes inside DuckDB. Adds BLAKE3, BLAKE2b, SHA-3, the SHA-2 family, MD5, and SHA-1 alongside an order-deterministic aggregate hash for dataset checksums and tamper-evident audit trails.

Fast, deterministic, non-cryptographic hash functions — xxHash, MurmurHash3, RapidHash. For partitioning, sharding, cache keys, and probabilistic-filter inputs; not for security.

Fuzzy string matching and edit-distance for DuckDB, powered by the RapidFuzz C++ library. Five ratio variants for record matching plus eight distance/similarity algorithms (Jaro, Jaro-Winkler, Hamming, Indel, Levenshtein/OSA, LCS, prefix/postfix) — every one in distance, similarity, and normalized forms.

String case transformation and English inflection inside SQL. snake_case ↔ camelCase ↔ PascalCase ↔ kebab-case, plus pluralize / singularize / table-name / class-name / module-name helpers. Apply to single strings, structs, or whole result sets.

Time-Sorted Unique Identifier generator for DuckDB. 32-character hex strings that sort chronologically by plain string comparison while remaining unique under bulk generation. Stable surrogate keys, audit-log primaries, and time-range pagination keys.

Bring Apache DataSketches into DuckDB. Approximate distinct-counts (HLL, CPC, Theta), quantiles (KLL, TDigest, REQ, classic Quantiles), and frequent items — every sketch serializes to a portable BLOB you can store in a column, ship between processes, and merge later without rescanning.

Probabilistic set-membership filters as DuckDB columns. Quotient, XOR (8/16-bit), Binary Fuse (8/16-bit), and DuckDB-style Bloom filters — answer 'is X in this set?' in O(1) using a fraction of the memory of the source set, with tunable false-positive rates and zero false negatives.

Monte Carlo, synthetic data, and what-if analysis directly in SQL. 22 named probability distributions, each with the same uniform surface — sample, PDF, CDF, quantile, hazard, plus full distribution properties — for ~444 functions total. Best for ad-hoc statistical work next to warehouse data; for production-grade simulation many users will still want a Python notebook with NumPy and SciPy.

Static, space-efficient MARISA trie data structures for DuckDB. Build a compact searchable string set as a single BLOB, then run lookups, prefix searches, and predictive completions against it. The lookup primitive for autocomplete, spell-check, and routing tables.

Encode lat/lng to a pentagonal cell ID and aggregate, join, or spatially filter by that ID. A5 is a pentagon-based discrete global grid system — newer than Uber's H3 (hexagonal) and Google's S2 (square), with more uniform cell area across the globe. 31 resolution levels from continental scale down to sub-square-meter precision.

Compact geometry storage for DuckDB. A first-class GEOSILO column type that delta-encodes coordinates as scaled integers and pairs with ZSTD for ~21% smaller geometry storage than DuckDB's GEOMETRY (and ~70% smaller raw blobs against WKB). Native ST_* overloads for bounding-box, area, length, and introspection skip the WKB decode entirely.

Linearize multi-dimensional numeric arrays via Hilbert and Morton (Z-order) space-filling curves. Use as ORDER BY when writing Parquet to enable row-group skipping on multi-column predicates — the same Z-Order / Hilbert-Order optimization Databricks pioneered for Delta Lake.

Render Jinja2-compatible templates from SQL using the embedded MiniJinja engine. Generate HTML, config files, notifications, and dynamic SQL — full Jinja syntax (loops, filters, conditionals, autoescaping) with JSON context.

Render Tera templates from SQL — Rust's Jinja2-style templating engine. Variable substitution, filters, loops, conditionals, autoescape, and template file inclusion driven by JSON context.

Run JavaScript inside DuckDB SQL via the embedded QuickJS-NG engine. Modern ES2020 syntax, scalar and table-valued forms, parameter passthrough — for the cases where SQL doesn't have the right primitive and Python is too heavy.

Run Rhai scripts inside DuckDB SQL. Rhai is a small, sandboxed, Rust-native scripting language with dynamic typing — best when SQL alone can't express the logic but a full UDF would be overkill. Constant scripts compile once and cache; per-row context goes in via a JSON object.

Load DuckDB SQL macros (scalar and table) from a remote URL. Define a macro in a Git repo, gist, or any HTTP-served file; pull it into a session with one call. The point is centralized, versioned macro distribution — one source of truth for shared SQL helpers across machines and teammates.

Schedule SQL queries to run on a cron expression — but only while the DuckDB process is alive. Six-field expressions with second-level resolution, list/inspect/cancel from SQL. Best fit: periodic refreshes, recurring exports, scheduled webhook alerts, and maintenance tasks inside a long-running DuckDB process.

Hook into DuckDB's internal events — query begin/end, transactions, connections — and ship them as JSON to any external program via stdin. Audit logging, error alerting, observability integration, and SIEM feeds, all configured with PRAGMAs.

ASCII and Unicode plots inline in DuckDB query results — bars, sparklines, and density plots that fit in a single VARCHAR column. Useful for eyeballing distributions and trends in the DuckDB CLI without leaving the terminal.

Better SQL autocomplete for the DuckDB CLI. Replaces the built-in edit-distance autocompleter with VS Code's fuzzy-matching algorithm — substring matches, camel-case awareness, and cross-database / cross-schema table-name completion.