Redis
Read and write Redis from DuckDB SQL. Strings, hashes, and lists; cursor-paginated SCAN/HSCAN; and a single-query bulk-dump pattern that drains an entire Redis keyspace into a DuckDB table. Best fit: operational queries, hot-cold data joins, and one-off migrations β not application-grade Redis access.
Install
-- Install the extension
INSTALL redis FROM community;
-- Load it into your session
LOAD redis;
-- Connect via a secret
CREATE SECRET IF NOT EXISTS redis (
TYPE redis,
PROVIDER config,
host 'localhost',
port '6379'
);
-- Read and write strings
SELECT redis_set('user:1', 'John Doe', 'redis') AS ok;
SELECT redis_get('user:1', 'redis') AS name;
-- Discover keys matching a pattern
SELECT * FROM redis_keys('user:*', 'redis'); Technical Overview
Why Use Redis from DuckDB?
Read and write a Redis instance directly from SQL β no app-layer Python or Node.js glue between DuckDB and the cache. Best for operational queries, hot-cold data joins, and one-off migrations; for application-grade Redis access, keep a real client.
π What this extension is for
Redis is the most common in-memory cache and quick-lookup store. This extension makes it queryable from inside DuckDB so you don't have to build a Python or Node.js bridge for the common SQL-side workflows.
- β’ Hot-cold data joins: Keep recent or frequently-accessed records in Redis, the bulk in DuckDB; join across the boundary with a regular SQL
JOINand per-rowredis_get/redis_hgetalllookups. - β’ Bulk migration and audits: Dump an entire Redis keyspace into a DuckDB table with
redis_hscan_over_scanβ one query, server-friendly cursor pagination, ready for offline analysis or export. - β’ Cache priming: Push computed values from DuckDB into Redis with
redis_set/redis_hsetso downstream applications hit a warm cache. - β’ Operational queries: Count keys by pattern, sample hashes, find the largest values, or audit a keyspace without writing a Redis-aware tool. Use
redis_scanfor production-safe pagination instead ofKEYS.
π How it works
Every redis_* function takes a trailing secret argument naming a DuckDB secret of TYPE redis. The extension opens a connection per call (with internal pooling) and speaks the Redis Serialization Protocol over plain TCP. There is no ATTACH interface β this is purely a function-based extension.
- β’ Strings:
redis_get(GET),redis_set(SET),redis_mget(MGET) β single-value and batched key/value access. - β’ Hashes:
redis_hget(HGET),redis_hset(HSET),redis_hgetall(HGETALL),redis_hscan(HSCAN) β for the schema-light record pattern most apps use. - β’ Lists:
redis_lpush(LPUSH),redis_lrange,redis_lrange_tableβ push and read; the_tablevariant emits one row per element instead of a comma-joined string. - β’ Discovery:
redis_keys(KEYS) for development,redis_scan(SCAN) for production.redis_hscan_over_scanchains both into a streaming(key, field, value)row source β the most useful single function in the extension. - β’ Key management:
redis_type(TYPE),redis_exists(EXISTS),redis_del(DEL) for inspect / probe / remove.
π‘οΈ Production caveats
Be aware before pointing this at a busy Redis instance. The extension is honest-by-design about what it doesn't yet do.
- β’ Use SCAN, not KEYS:
KEYSblocks the Redis server for the duration of the scan β fine in dev, dangerous on production keyspaces. Preferredis_scanandredis_hscan_over_scan, which use cursor-paginatedSCANunder the hood. - β’ Strings, hashes, and lists only: No SET / ZSET / STREAM / PUB-SUB / TRANSACTIONS / SCRIPTING today. See Redis data types for what's supported elsewhere.
- β’ Single-node connections: No Redis Cluster topology awareness. Point the secret at a single endpoint.
- β’ Plaintext auth: The
passwordfield in the secret is sent over plain TCP β TLS is not yet wired up. Use the extension on trusted networks until that lands. See Redis security for the upstream guidance. - β’ Experimental status: The function surface may change as more Redis primitives land. Pin a known-good extension version in production CTAS jobs.
π― Common Use Cases
Drain a Redis keyspace into DuckDB
One CREATE TABLE AS wrapping redis_hscan_over_scan β usually the fastest path from "every user:* hash in production" to a queryable Parquet/CSV-ready dataset.
Join cached lookups against bulk data
Per-row redis_get / redis_hgetall lookups inside a SELECT against a wide DuckDB table. Works because DuckDB pipelines the lookups; for very wide joins, consider materializing the Redis side via redis_hscan_over_scan first.
Audit a production keyspace
Count keys by prefix, sample hashes, find outliers β using redis_scan so the audit doesn't stall the Redis server. Easier than writing a Python script for each one-off question.
Prime a cache from analytical output
Compute aggregates in DuckDB, then push them with redis_set / redis_hset so the application layer hits a warm cache. Single SQL query end-to-end.
Deep Dive
Technical Details
What you can do with one query
The single most useful pattern: drain an entire Redis keyspace into a DuckDB table without writing a script:
CREATE TABLE redis_users_dump AS
SELECT key, field, value
FROM redis_hscan_over_scan(
/* scan_pattern */ 'user:*',
/* hscan_pattern */ '*',
/* count */ 100,
/* secret */ 'redis'
);
redis_hscan_over_scan chains the upstream SCAN and HSCAN cursor commands into one streaming (key, field, value) row source. Server-friendly cursor pagination throughout β no KEYS blocking, no Python in the middle.
This extension is for SQL-side workflows where data integration matters more than every Redis feature. It supports strings, hashes, and lists; no SET / ZSET / STREAM / PUB-SUB / TRANSACTIONS / SCRIPTING. Connections are single-node (no Redis Cluster). Auth is currently plaintext β TLS isnβt wired up yet. The status is experimental; the function surface may change.
For application-grade Redis access, keep a real client. For one-off migrations, hot-cold joins, and operational queries from SQL, this is the right tool.
Authentication
Every redis_* function takes a trailing secret argument naming a DuckDB secret of TYPE redis. This keeps credentials out of query text and lets multiple environments (dev/staging/prod) live as separate secrets β pick the right one per query:
CREATE SECRET prod_redis (
TYPE redis,
PROVIDER config,
host 'cache.prod.internal',
port '6379',
password :prod_password
);
SELECT redis_get('counter', 'prod_redis');
See CREATE SECRET and the DuckDB Secrets Manager for the secrets infrastructure, and the Secrets section below for the parameter table the redis secret type accepts.
SCAN vs KEYS β why it matters
redis_keys is the convenience function β it returns all matching keys as a table. Itβs fine for development and small keyspaces, but it issues the KEYS command, which blocks the Redis server for the duration of the scan. On a large production instance this can stall traffic.
redis_scan is the production primitive β cursor-paginated, non-blocking, server-side filtered. The SCAN protocol returns a cursor with each batch of keys; you re-pass the cursor to fetch the next page. Cursor 0 means iteration is complete:
SELECT redis_scan('0', 'user:*', 100, 'redis');
-- "1024:user:1,user:2,...,user:50"
redis_hscan_over_scan (used in the lead snippet above) chains both into one streaming pipeline: for every key matching scan_pattern, run HSCAN hscan_pattern and emit (key, field, value) rows. The fastest way to drain a Redis hash dataset into DuckDB.
Whatβs in the box
| Group | Functions | Redis commands |
|---|---|---|
| Strings | redis_get, redis_set, redis_mget | GET, SET, MGET |
| Hashes | redis_hget, redis_hset, redis_hgetall, redis_hscan | HGET, HSET, HGETALL, HSCAN |
| Lists | redis_lpush, redis_lrange, redis_lrange_table | LPUSH, LRANGE |
| Discovery | redis_keys, redis_scan, redis_hscan_over_scan | KEYS, SCAN, HSCAN |
| Key management | redis_type, redis_exists, redis_del | TYPE, EXISTS, DEL |
The extension speaks the Redis Serialization Protocol over plain TCP. Recipes for each group live in the Cookbook.
Reference
Extension Contents
Quick reference to all available functions and settings organized by category.
| Name | Description | |
|---|---|---|
| Cache / In-memory store | ||
| redis | Redis connection details | |
| Hashes HGET / HSET / HGETALL / HSCAN β for field-keyed records stored under one Redis key. The | ||
| redis_hget() | Get the value of a hash field | |
| redis_hgetall() | List all (field, value) pairs of a hash as a table | |
| redis_hscan() | Cursor-paginated scan of a hash's fields | |
| redis_hscan_over_scan() | For every key matching a SCAN pattern, run HSCAN and emit (key, field, value) rows | |
| redis_hset() | Set the value of a hash field | |
| Keys DEL / EXISTS / TYPE / SCAN / KEYS β keyspace introspection and management. Use SCAN for production traffic; KEYS blocks the server on large keyspaces. | ||
| redis_del() | Delete a key | |
| redis_exists() | Check whether a key exists | |
| redis_keys() | List all keys matching a pattern as a table | |
| redis_scan() | Cursor-paginated SCAN of the keyspace | |
| redis_type() | Get the Redis type of a key (string / list / hash / set / zset / stream / none) | |
| Lists LPUSH / LRANGE β interact with Redis lists. Use | ||
| redis_lpush() | Push a value onto the head of a list | |
| redis_lrange() | Get a range from a list as a comma-separated string | |
| redis_lrange_table() | Get a range from a list as a table β one row per element | |
| Strings GET / SET / MGET against Redis string keys. The simplest key-value layer β read and write opaque values. | ||
| redis_get() | Get the value of a string key | |
| redis_mget() | Get multiple values in one round-trip | |
| redis_set() | Set the value of a string key | |
API Reference
Function Documentation
Detailed documentation for each function including signatures, parameters, and examples.
redis_del
Signature
Parameters (Positional)
| Parameter | Type | Mode | Description |
|---|---|---|---|
key | VARCHAR | Positional | |
secret_name | VARCHAR | Positional |
Returns
Description
Delete a key. Returns TRUE if the key existed and was removed.
Examples
SELECT redis_del('mykey', 'my_redis_secret'); Related Functions
redis_exists
Signature
Parameters (Positional)
| Parameter | Type | Mode | Description |
|---|---|---|---|
key | VARCHAR | Positional | |
secret_name | VARCHAR | Positional |
Returns
Description
Check whether a key exists.
Examples
SELECT redis_exists('mykey', 'my_redis_secret'); Related Functions
redis_get
Signature
Parameters (Positional)
| Parameter | Type | Mode | Description |
|---|---|---|---|
key | VARCHAR | Positional | |
secret_name | VARCHAR | Positional |
Returns
Description
Get the value of a string key.
Examples
SELECT redis_get('mykey', 'my_redis_secret'); SELECT redis_get(key_column, 'my_redis_secret') FROM my_table; Related Functions
redis_hget
Signature
Parameters (Positional)
| Parameter | Type | Mode | Description |
|---|---|---|---|
key | VARCHAR | Positional | |
field | VARCHAR | Positional | |
secret_name | VARCHAR | Positional |
Returns
Description
Get the value of a hash field.
Examples
SELECT redis_hget('myhash', 'field1', 'my_redis_secret'); Related Functions
redis_hgetall
Signature
Parameters (Named)
| Parameter | Type | Mode | Description |
|---|---|---|---|
key | VARCHAR | Named | |
secret_name | VARCHAR | Named |
Returns
Description
List all (field, value) pairs of a hash as a table.
Examples
SELECT * FROM redis_hgetall('myhash', 'my_redis_secret'); Related Functions
redis_hscan
Signature
Parameters (Positional)
| Parameter | Type | Mode | Description |
|---|---|---|---|
key | VARCHAR | Positional | |
cursor | VARCHAR | Positional | |
pattern | VARCHAR | Positional | |
count | BIGINT | Positional | |
secret_name | VARCHAR | Positional |
Returns
Description
Cursor-paginated scan of a hash's fields.
Examples
SELECT redis_hscan('myhash', '0', '*', 100, 'my_redis_secret'); Related Functions
redis_hscan_over_scan
Signature
Parameters (Named)
| Parameter | Type | Mode | Description |
|---|---|---|---|
scan_pattern | VARCHAR | Named | |
hscan_pattern | VARCHAR | Named | |
count | BIGINT | Named | |
secret_name | VARCHAR | Named |
Returns
Description
For every key matching a SCAN pattern, run HSCAN and emit (key, field, value) rows. The high-throughput discovery primitive for migration / analysis.
Examples
SELECT * FROM redis_hscan_over_scan('user:*', '*', 100, 'my_redis_secret'); Related Functions
redis_hset
Signature
Parameters (Positional)
| Parameter | Type | Mode | Description |
|---|---|---|---|
key | VARCHAR | Positional | |
field | VARCHAR | Positional | |
value | VARCHAR | Positional | |
secret_name | VARCHAR | Positional |
Returns
Description
Set the value of a hash field.
Examples
SELECT redis_hset('myhash', 'field1', 'value1', 'my_redis_secret'); Related Functions
redis_keys
Signature
Parameters (Named)
| Parameter | Type | Mode | Description |
|---|---|---|---|
pattern | VARCHAR | Named | |
secret_name | VARCHAR | Named |
Returns
Description
List all keys matching a pattern as a table. Convenient but blocks the Redis server on large keyspaces β prefer redis_scan in production.
Examples
SELECT * FROM redis_keys('*', 'my_redis_secret'); SELECT * FROM redis_keys('user:*', 'my_redis_secret'); Related Functions
redis_lpush
Signature
Parameters (Positional)
| Parameter | Type | Mode | Description |
|---|---|---|---|
key | VARCHAR | Positional | |
value | VARCHAR | Positional | |
secret_name | VARCHAR | Positional |
Returns
Description
Push a value onto the head of a list.
Examples
SELECT redis_lpush('mylist', 'value1', 'my_redis_secret'); Related Functions
redis_lrange
Signature
Parameters (Positional)
| Parameter | Type | Mode | Description |
|---|---|---|---|
key | VARCHAR | Positional | |
start | BIGINT | Positional | |
stop | BIGINT | Positional | |
secret_name | VARCHAR | Positional |
Returns
Description
Get a range from a list as a comma-separated string.
Examples
SELECT redis_lrange('mylist', 0, -1, 'my_redis_secret'); SELECT redis_lrange('mylist', 0, 10, 'my_redis_secret'); Related Functions
redis_lrange_table
Signature
Parameters (Named)
| Parameter | Type | Mode | Description |
|---|---|---|---|
key | VARCHAR | Named | |
start | BIGINT | Named | |
stop | BIGINT | Named | |
secret_name | VARCHAR | Named |
Returns
Description
Get a range from a list as a table β one row per element. Easier to work with in SQL than the comma-separated form.
Examples
SELECT * FROM redis_lrange_table('mylist', 0, -1, 'my_redis_secret'); Related Functions
redis_mget
Signature
Parameters (Positional)
| Parameter | Type | Mode | Description |
|---|---|---|---|
keys | VARCHAR | Positional | |
secret_name | VARCHAR | Positional |
Returns
Description
Get multiple values in one round-trip. Keys are passed as a comma-separated string.
Examples
SELECT redis_mget('key1,key2,key3', 'my_redis_secret'); Related Functions
redis_scan
Signature
Parameters (Positional)
| Parameter | Type | Mode | Description |
|---|---|---|---|
cursor | VARCHAR | Positional | |
pattern | VARCHAR | Positional | |
count | BIGINT | Positional | |
secret_name | VARCHAR | Positional |
Returns
Description
Cursor-paginated SCAN of the keyspace. Returns 'cursor:keys_csv'. Iterate by re-passing the returned cursor.
Examples
SELECT redis_scan('0', '*', 100, 'my_redis_secret'); SELECT redis_scan('0', 'user:*', 10, 'my_redis_secret'); Related Functions
redis_set
Signature
Parameters (Positional)
| Parameter | Type | Mode | Description |
|---|---|---|---|
key | VARCHAR | Positional | |
value | VARCHAR | Positional | |
secret_name | VARCHAR | Positional |
Returns
Description
Set the value of a string key.
Examples
SELECT redis_set('mykey', 'myvalue', 'my_redis_secret'); Related Functions
redis_type
Signature
Parameters (Positional)
| Parameter | Type | Mode | Description |
|---|---|---|---|
key | VARCHAR | Positional | |
secret_name | VARCHAR | Positional |
Returns
Description
Get the Redis type of a key (string / list / hash / set / zset / stream / none).
Examples
SELECT redis_type('mykey', 'my_redis_secret'); Related Functions
Security
Secrets
DuckDB secrets for securely storing credentials and encryption keys used by the redis extension.
redis
Description
Redis connection details. All redis_* functions take a final `secret` argument naming a secret of this type.
Parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
| host | VARCHAR | Required | Redis server hostname or IP. |
| port | VARCHAR | Required | Redis server port (typically '6379'). |
| password | VARCHAR | Optional | Authentication password if the server requires AUTH. |
Examples
Local Redis without auth
CREATE SECRET IF NOT EXISTS redis (
TYPE redis,
PROVIDER config,
host 'localhost',
port '6379'
); Redis Cloud with TLS endpoint
CREATE SECRET IF NOT EXISTS redis (
TYPE redis,
PROVIDER config,
host 'redis-1234.ec2.redns.redis-cloud.com',
port '16959',
password 'xxxxxx'
); Practical Examples
Cookbook
Real-world recipes and patterns for common use cases.
Connect
Redis credentials live in a DuckDB secret. Every redis_* function takes a final secret argument naming this secret:
CREATE SECRET IF NOT EXISTS redis (
TYPE redis,
PROVIDER config,
host 'localhost',
port '6379',
password 'optional_password'
);
For production, parameterize the password from the environment so it doesnβt appear in cached query plans:
SET VARIABLE redis_password = getenv('REDIS_PASSWORD');
CREATE SECRET prod_redis (
TYPE redis,
PROVIDER config,
host 'cache.prod.internal',
port '6379',
password :redis_password
);
See CREATE SECRET for the full secrets DSL and the Secrets section on this page for the redis parameter table.
String operations
-- Set a value (Redis SET)
SELECT redis_set('user:1', 'John Doe', 'redis') AS ok;
-- Get a value (Redis GET)
SELECT redis_get('user:1', 'redis') AS user_name;
-- Bulk write from a column
INSERT INTO users (id, name)
SELECT id,
redis_set('user:' || id::VARCHAR, name, 'redis')
FROM new_users;
-- Multi-get in one round-trip (Redis MGET)
SELECT redis_mget('user:1,user:2,user:3', 'redis');
Maps to GET, SET, MGET. See redis_get / redis_set / redis_mget for the function signatures.
Hash operations
-- Set fields (Redis HSET)
SELECT redis_hset('user:1', 'email', '[email protected]', 'redis');
SELECT redis_hset('user:1', 'age', '30', 'redis');
-- Get one field (Redis HGET)
SELECT redis_hget('user:1', 'email', 'redis') AS email;
-- All fields and values as a table (Redis HGETALL)
SELECT * FROM redis_hgetall('user:1', 'redis');
Maps to HGET, HSET, HGETALL. See redis_hget / redis_hset / redis_hgetall.
List operations
-- Push values (Redis LPUSH)
SELECT redis_lpush('queue:jobs', 'job-1', 'redis');
SELECT redis_lpush('queue:jobs', 'job-2', 'redis');
-- Read as a comma-joined string
SELECT redis_lrange('queue:jobs', 0, -1, 'redis');
-- Or as a table β one row per element
SELECT * FROM redis_lrange_table('queue:jobs', 0, -1, 'redis');
Maps to LPUSH and LRANGE. redis_lrange_table is the table-shaped variant β pick it when you want one DuckDB row per list element instead of a single comma-joined string from redis_lrange.
Discovery (SCAN, KEYS)
KEYS is convenient but blocks the Redis server on large keyspaces. In production, prefer cursor-paginated SCAN:
-- KEYS β fine for development / small keyspaces
SELECT * FROM redis_keys('user:*', 'redis');
-- SCAN β paginated, non-blocking
SELECT redis_scan('0', 'user:*', 100, 'redis');
-- β "1024:user:1,user:2,...,user:50"
-- Pass the returned cursor (1024) back in to fetch the next page; cursor "0" means done.
See redis_keys and redis_scan.
Bulk hash discovery (redis_hscan_over_scan)
The killer feature for migration and bulk analysis β combines SCAN with HSCAN to emit (key, field, value) rows for every hash matching a key pattern, in one query:
-- Every field of every user:* hash, as rows
SELECT key, field, value
FROM redis_hscan_over_scan(
/* scan_pattern */ 'user:*',
/* hscan_pattern */ '*',
/* count */ 100,
/* secret */ 'redis'
)
ORDER BY key, field;
Pipe directly into a CREATE TABLE AS for offline analysis or migration:
CREATE TABLE redis_users_dump AS
SELECT key, field, value
FROM redis_hscan_over_scan('user:*', '*', 100, 'redis');
See redis_hscan_over_scan for the full signature.
Key management
-- Type, existence, deletion
SELECT redis_type('user:1', 'redis'); -- 'hash'
SELECT redis_exists('user:1', 'redis'); -- TRUE
SELECT redis_del('user:1', 'redis'); -- TRUE if existed and removed
Maps to TYPE, EXISTS, DEL. See redis_type / redis_exists / redis_del.
Platform Support
Compatibility
Extension availability may vary by platform and DuckDB version. Check below to ensure this extension supports your environment before installation.
Quick Facts
Platforms
Supported platform architectures
Compiled binary sizes
| Platform | Architecture | Size |
|---|---|---|
| Linux | aarch64 | 2.83 MB |
| Linux | x86_64 | 3.21 MB |
| Linux (musl) | x86_64 | 2.51 MB |
| macOS | Apple Silicon | 1.96 MB |
| macOS | Intel | 2.28 MB |
| Windows | x86_64 | 7.58 MB |
Gzipped download size from the DuckDB community-extensions registry.
SQL access to your Redis instance
Install Redis to read/write strings, hashes, and lists from inside DuckDB queries β and to drain an entire keyspace into a table with one CREATE TABLE AS.