πŸ“•

Redis

Read and write Redis from DuckDB SQL. Strings, hashes, and lists; cursor-paginated SCAN/HSCAN; and a single-query bulk-dump pattern that drains an entire Redis keyspace into a DuckDB table. Best fit: operational queries, hot-cold data joins, and one-off migrations β€” not application-grade Redis access.

Install

-- Install the extension
INSTALL redis FROM community;

-- Load it into your session
LOAD redis;

-- Connect via a secret
CREATE SECRET IF NOT EXISTS redis (
  TYPE redis,
  PROVIDER config,
  host 'localhost',
  port '6379'
);

-- Read and write strings
SELECT redis_set('user:1', 'John Doe', 'redis') AS ok;
SELECT redis_get('user:1', 'redis') AS name;

-- Discover keys matching a pattern
SELECT * FROM redis_keys('user:*', 'redis');

Technical Overview

Why Use Redis from DuckDB?

Read and write a Redis instance directly from SQL β€” no app-layer Python or Node.js glue between DuckDB and the cache. Best for operational queries, hot-cold data joins, and one-off migrations; for application-grade Redis access, keep a real client.

πŸ“• What this extension is for

Redis is the most common in-memory cache and quick-lookup store. This extension makes it queryable from inside DuckDB so you don't have to build a Python or Node.js bridge for the common SQL-side workflows.

  • β€’ Hot-cold data joins: Keep recent or frequently-accessed records in Redis, the bulk in DuckDB; join across the boundary with a regular SQL JOIN and per-row redis_get / redis_hgetall lookups.
  • β€’ Bulk migration and audits: Dump an entire Redis keyspace into a DuckDB table with redis_hscan_over_scan β€” one query, server-friendly cursor pagination, ready for offline analysis or export.
  • β€’ Cache priming: Push computed values from DuckDB into Redis with redis_set / redis_hset so downstream applications hit a warm cache.
  • β€’ Operational queries: Count keys by pattern, sample hashes, find the largest values, or audit a keyspace without writing a Redis-aware tool. Use redis_scan for production-safe pagination instead of KEYS.

πŸ”Œ How it works

Every redis_* function takes a trailing secret argument naming a DuckDB secret of TYPE redis. The extension opens a connection per call (with internal pooling) and speaks the Redis Serialization Protocol over plain TCP. There is no ATTACH interface β€” this is purely a function-based extension.

  • β€’ Strings: redis_get (GET), redis_set (SET), redis_mget (MGET) β€” single-value and batched key/value access.
  • β€’ Hashes: redis_hget (HGET), redis_hset (HSET), redis_hgetall (HGETALL), redis_hscan (HSCAN) β€” for the schema-light record pattern most apps use.
  • β€’ Lists: redis_lpush (LPUSH), redis_lrange, redis_lrange_table β€” push and read; the _table variant emits one row per element instead of a comma-joined string.
  • β€’ Discovery: redis_keys (KEYS) for development, redis_scan (SCAN) for production. redis_hscan_over_scan chains both into a streaming (key, field, value) row source β€” the most useful single function in the extension.
  • β€’ Key management: redis_type (TYPE), redis_exists (EXISTS), redis_del (DEL) for inspect / probe / remove.

πŸ›‘οΈ Production caveats

Be aware before pointing this at a busy Redis instance. The extension is honest-by-design about what it doesn't yet do.

  • β€’ Use SCAN, not KEYS: KEYS blocks the Redis server for the duration of the scan β€” fine in dev, dangerous on production keyspaces. Prefer redis_scan and redis_hscan_over_scan, which use cursor-paginated SCAN under the hood.
  • β€’ Strings, hashes, and lists only: No SET / ZSET / STREAM / PUB-SUB / TRANSACTIONS / SCRIPTING today. See Redis data types for what's supported elsewhere.
  • β€’ Single-node connections: No Redis Cluster topology awareness. Point the secret at a single endpoint.
  • β€’ Plaintext auth: The password field in the secret is sent over plain TCP β€” TLS is not yet wired up. Use the extension on trusted networks until that lands. See Redis security for the upstream guidance.
  • β€’ Experimental status: The function surface may change as more Redis primitives land. Pin a known-good extension version in production CTAS jobs.

🎯 Common Use Cases

Drain a Redis keyspace into DuckDB

One CREATE TABLE AS wrapping redis_hscan_over_scan β€” usually the fastest path from "every user:* hash in production" to a queryable Parquet/CSV-ready dataset.

Join cached lookups against bulk data

Per-row redis_get / redis_hgetall lookups inside a SELECT against a wide DuckDB table. Works because DuckDB pipelines the lookups; for very wide joins, consider materializing the Redis side via redis_hscan_over_scan first.

Audit a production keyspace

Count keys by prefix, sample hashes, find outliers β€” using redis_scan so the audit doesn't stall the Redis server. Easier than writing a Python script for each one-off question.

Prime a cache from analytical output

Compute aggregates in DuckDB, then push them with redis_set / redis_hset so the application layer hits a warm cache. Single SQL query end-to-end.

Deep Dive

Technical Details

What you can do with one query

The single most useful pattern: drain an entire Redis keyspace into a DuckDB table without writing a script:

CREATE TABLE redis_users_dump AS
SELECT key, field, value
FROM redis_hscan_over_scan(
  /* scan_pattern  */ 'user:*',
  /* hscan_pattern */ '*',
  /* count         */ 100,
  /* secret        */ 'redis'
);

redis_hscan_over_scan chains the upstream SCAN and HSCAN cursor commands into one streaming (key, field, value) row source. Server-friendly cursor pagination throughout β€” no KEYS blocking, no Python in the middle.

Best fit, scoped honestly

This extension is for SQL-side workflows where data integration matters more than every Redis feature. It supports strings, hashes, and lists; no SET / ZSET / STREAM / PUB-SUB / TRANSACTIONS / SCRIPTING. Connections are single-node (no Redis Cluster). Auth is currently plaintext β€” TLS isn’t wired up yet. The status is experimental; the function surface may change.

For application-grade Redis access, keep a real client. For one-off migrations, hot-cold joins, and operational queries from SQL, this is the right tool.

Authentication

Every redis_* function takes a trailing secret argument naming a DuckDB secret of TYPE redis. This keeps credentials out of query text and lets multiple environments (dev/staging/prod) live as separate secrets β€” pick the right one per query:

CREATE SECRET prod_redis (
  TYPE redis,
  PROVIDER config,
  host 'cache.prod.internal',
  port '6379',
  password :prod_password
);

SELECT redis_get('counter', 'prod_redis');

See CREATE SECRET and the DuckDB Secrets Manager for the secrets infrastructure, and the Secrets section below for the parameter table the redis secret type accepts.

SCAN vs KEYS β€” why it matters

redis_keys is the convenience function β€” it returns all matching keys as a table. It’s fine for development and small keyspaces, but it issues the KEYS command, which blocks the Redis server for the duration of the scan. On a large production instance this can stall traffic.

redis_scan is the production primitive β€” cursor-paginated, non-blocking, server-side filtered. The SCAN protocol returns a cursor with each batch of keys; you re-pass the cursor to fetch the next page. Cursor 0 means iteration is complete:

SELECT redis_scan('0', 'user:*', 100, 'redis');
-- "1024:user:1,user:2,...,user:50"

redis_hscan_over_scan (used in the lead snippet above) chains both into one streaming pipeline: for every key matching scan_pattern, run HSCAN hscan_pattern and emit (key, field, value) rows. The fastest way to drain a Redis hash dataset into DuckDB.

What’s in the box

GroupFunctionsRedis commands
Stringsredis_get, redis_set, redis_mgetGET, SET, MGET
Hashesredis_hget, redis_hset, redis_hgetall, redis_hscanHGET, HSET, HGETALL, HSCAN
Listsredis_lpush, redis_lrange, redis_lrange_tableLPUSH, LRANGE
Discoveryredis_keys, redis_scan, redis_hscan_over_scanKEYS, SCAN, HSCAN
Key managementredis_type, redis_exists, redis_delTYPE, EXISTS, DEL

The extension speaks the Redis Serialization Protocol over plain TCP. Recipes for each group live in the Cookbook.

Reference

Extension Contents

Quick reference to all available functions and settings organized by category.

Name Description
Cache / In-memory store
redis Redis connection details
Hashes

HGET / HSET / HGETALL / HSCAN β€” for field-keyed records stored under one Redis key. The _over_scan variant pairs SCAN with HSCAN for high-throughput bulk reads.

redis_hget() Get the value of a hash field
redis_hgetall() List all (field, value) pairs of a hash as a table
redis_hscan() Cursor-paginated scan of a hash's fields
redis_hscan_over_scan() For every key matching a SCAN pattern, run HSCAN and emit (key, field, value) rows
redis_hset() Set the value of a hash field
Keys

DEL / EXISTS / TYPE / SCAN / KEYS β€” keyspace introspection and management. Use SCAN for production traffic; KEYS blocks the server on large keyspaces.

redis_del() Delete a key
redis_exists() Check whether a key exists
redis_keys() List all keys matching a pattern as a table
redis_scan() Cursor-paginated SCAN of the keyspace
redis_type() Get the Redis type of a key (string / list / hash / set / zset / stream / none)
Lists

LPUSH / LRANGE β€” interact with Redis lists. Use redis_lrange_table when you want one row per list element (easier than parsing the comma-separated form).

redis_lpush() Push a value onto the head of a list
redis_lrange() Get a range from a list as a comma-separated string
redis_lrange_table() Get a range from a list as a table β€” one row per element
Strings

GET / SET / MGET against Redis string keys. The simplest key-value layer β€” read and write opaque values.

redis_get() Get the value of a string key
redis_mget() Get multiple values in one round-trip
redis_set() Set the value of a string key

API Reference

Function Documentation

Detailed documentation for each function including signatures, parameters, and examples.

redis_del

Scalar Function Keys
Signature
redis_del(key: VARCHAR, secret_name: VARCHAR) β†’ BOOLEAN
Parameters (Positional)
Parameter Type Mode Description
key VARCHAR Positional
secret_name VARCHAR Positional
Returns
Description

Delete a key. Returns TRUE if the key existed and was removed.

Examples
1
SELECT redis_del('mykey', 'my_redis_secret');

redis_exists

Scalar Function Keys
Signature
redis_exists(key: VARCHAR, secret_name: VARCHAR) β†’ BOOLEAN
Parameters (Positional)
Parameter Type Mode Description
key VARCHAR Positional
secret_name VARCHAR Positional
Returns
Description

Check whether a key exists.

Examples
1
SELECT redis_exists('mykey', 'my_redis_secret');

redis_get

Scalar Function Strings
Signature
redis_get(key: VARCHAR, secret_name: VARCHAR) β†’ VARCHAR
Parameters (Positional)
Parameter Type Mode Description
key VARCHAR Positional
secret_name VARCHAR Positional
Returns
Description

Get the value of a string key.

Examples
1
SELECT redis_get('mykey', 'my_redis_secret');
2
SELECT redis_get(key_column, 'my_redis_secret') FROM my_table;

redis_hget

Scalar Function Hashes
Signature
redis_hget(key: VARCHAR, field: VARCHAR, secret_name: VARCHAR) β†’ VARCHAR
Parameters (Positional)
Parameter Type Mode Description
key VARCHAR Positional
field VARCHAR Positional
secret_name VARCHAR Positional
Returns
Description

Get the value of a hash field.

Examples
1
SELECT redis_hget('myhash', 'field1', 'my_redis_secret');

redis_hgetall

Table Function Hashes
Signature
redis_hgetall(key := VARCHAR, secret_name := VARCHAR) β†’ None
Parameters (Named)
Parameter Type Mode Description
key VARCHAR Named
secret_name VARCHAR Named
Returns
Description

List all (field, value) pairs of a hash as a table.

Examples
1
SELECT * FROM redis_hgetall('myhash', 'my_redis_secret');

redis_hscan

Scalar Function Hashes
Signature
redis_hscan(key: VARCHAR, cursor: VARCHAR, pattern: VARCHAR, count: BIGINT, secret_name: VARCHAR) β†’ VARCHAR
Parameters (Positional)
Parameter Type Mode Description
key VARCHAR Positional
cursor VARCHAR Positional
pattern VARCHAR Positional
count BIGINT Positional
secret_name VARCHAR Positional
Returns
Description

Cursor-paginated scan of a hash's fields.

Examples
1
SELECT redis_hscan('myhash', '0', '*', 100, 'my_redis_secret');

redis_hscan_over_scan

Table Function Hashes
Signature
redis_hscan_over_scan(scan_pattern := VARCHAR, hscan_pattern := VARCHAR, count := BIGINT, secret_name := VARCHAR) β†’ None
Parameters (Named)
Parameter Type Mode Description
scan_pattern VARCHAR Named
hscan_pattern VARCHAR Named
count BIGINT Named
secret_name VARCHAR Named
Returns
Description

For every key matching a SCAN pattern, run HSCAN and emit (key, field, value) rows. The high-throughput discovery primitive for migration / analysis.

Examples
1
SELECT * FROM redis_hscan_over_scan('user:*', '*', 100, 'my_redis_secret');

redis_hset

Scalar Function Hashes
Signature
redis_hset(key: VARCHAR, field: VARCHAR, value: VARCHAR, secret_name: VARCHAR) β†’ VARCHAR
Parameters (Positional)
Parameter Type Mode Description
key VARCHAR Positional
field VARCHAR Positional
value VARCHAR Positional
secret_name VARCHAR Positional
Returns
Description

Set the value of a hash field.

Examples
1
SELECT redis_hset('myhash', 'field1', 'value1', 'my_redis_secret');

redis_keys

Table Function Keys
Signature
redis_keys(pattern := VARCHAR, secret_name := VARCHAR) β†’ None
Parameters (Named)
Parameter Type Mode Description
pattern VARCHAR Named
secret_name VARCHAR Named
Returns
Description

List all keys matching a pattern as a table. Convenient but blocks the Redis server on large keyspaces β€” prefer redis_scan in production.

Examples
1
SELECT * FROM redis_keys('*', 'my_redis_secret');
2
SELECT * FROM redis_keys('user:*', 'my_redis_secret');

redis_lpush

Scalar Function Lists
Signature
redis_lpush(key: VARCHAR, value: VARCHAR, secret_name: VARCHAR) β†’ VARCHAR
Parameters (Positional)
Parameter Type Mode Description
key VARCHAR Positional
value VARCHAR Positional
secret_name VARCHAR Positional
Returns
Description

Push a value onto the head of a list.

Examples
1
SELECT redis_lpush('mylist', 'value1', 'my_redis_secret');

redis_lrange

Scalar Function Lists
Signature
redis_lrange(key: VARCHAR, start: BIGINT, stop: BIGINT, secret_name: VARCHAR) β†’ VARCHAR
Parameters (Positional)
Parameter Type Mode Description
key VARCHAR Positional
start BIGINT Positional
stop BIGINT Positional
secret_name VARCHAR Positional
Returns
Description

Get a range from a list as a comma-separated string.

Examples
1
SELECT redis_lrange('mylist', 0, -1, 'my_redis_secret');
2
SELECT redis_lrange('mylist', 0, 10, 'my_redis_secret');

redis_lrange_table

Table Function Lists
Signature
redis_lrange_table(key := VARCHAR, start := BIGINT, stop := BIGINT, secret_name := VARCHAR) β†’ None
Parameters (Named)
Parameter Type Mode Description
key VARCHAR Named
start BIGINT Named
stop BIGINT Named
secret_name VARCHAR Named
Returns
Description

Get a range from a list as a table β€” one row per element. Easier to work with in SQL than the comma-separated form.

Examples
1
SELECT * FROM redis_lrange_table('mylist', 0, -1, 'my_redis_secret');

redis_mget

Scalar Function Strings
Signature
redis_mget(keys: VARCHAR, secret_name: VARCHAR) β†’ VARCHAR
Parameters (Positional)
Parameter Type Mode Description
keys VARCHAR Positional
secret_name VARCHAR Positional
Returns
Description

Get multiple values in one round-trip. Keys are passed as a comma-separated string.

Examples
1
SELECT redis_mget('key1,key2,key3', 'my_redis_secret');

redis_scan

Scalar Function Keys
Signature
redis_scan(cursor: VARCHAR, pattern: VARCHAR, count: BIGINT, secret_name: VARCHAR) β†’ VARCHAR
Parameters (Positional)
Parameter Type Mode Description
cursor VARCHAR Positional
pattern VARCHAR Positional
count BIGINT Positional
secret_name VARCHAR Positional
Returns
Description

Cursor-paginated SCAN of the keyspace. Returns 'cursor:keys_csv'. Iterate by re-passing the returned cursor.

Examples
1
SELECT redis_scan('0', '*', 100, 'my_redis_secret');
2
SELECT redis_scan('0', 'user:*', 10, 'my_redis_secret');

redis_set

Scalar Function Strings
Signature
redis_set(key: VARCHAR, value: VARCHAR, secret_name: VARCHAR) β†’ VARCHAR
Parameters (Positional)
Parameter Type Mode Description
key VARCHAR Positional
value VARCHAR Positional
secret_name VARCHAR Positional
Returns
Description

Set the value of a string key.

Examples
1
SELECT redis_set('mykey', 'myvalue', 'my_redis_secret');

redis_type

Scalar Function Keys
Signature
redis_type(key: VARCHAR, secret_name: VARCHAR) β†’ VARCHAR
Parameters (Positional)
Parameter Type Mode Description
key VARCHAR Positional
secret_name VARCHAR Positional
Returns
Description

Get the Redis type of a key (string / list / hash / set / zset / stream / none).

Examples
1
SELECT redis_type('mykey', 'my_redis_secret');

Security

Secrets

DuckDB secrets for securely storing credentials and encryption keys used by the redis extension.

redis

Description

Redis connection details. All redis_* functions take a final `secret` argument naming a secret of this type.

Parameters
Parameter Type Required Description
host VARCHAR Required Redis server hostname or IP.
port VARCHAR Required Redis server port (typically '6379').
password VARCHAR Optional Authentication password if the server requires AUTH.
Examples
1

Local Redis without auth

CREATE SECRET IF NOT EXISTS redis (
  TYPE redis,
  PROVIDER config,
  host 'localhost',
  port '6379'
);
2

Redis Cloud with TLS endpoint

CREATE SECRET IF NOT EXISTS redis (
  TYPE redis,
  PROVIDER config,
  host 'redis-1234.ec2.redns.redis-cloud.com',
  port '16959',
  password 'xxxxxx'
);

Practical Examples

Cookbook

Real-world recipes and patterns for common use cases.

Connect

Redis credentials live in a DuckDB secret. Every redis_* function takes a final secret argument naming this secret:

CREATE SECRET IF NOT EXISTS redis (
  TYPE redis,
  PROVIDER config,
  host 'localhost',
  port '6379',
  password 'optional_password'
);

For production, parameterize the password from the environment so it doesn’t appear in cached query plans:

SET VARIABLE redis_password = getenv('REDIS_PASSWORD');

CREATE SECRET prod_redis (
  TYPE redis,
  PROVIDER config,
  host 'cache.prod.internal',
  port '6379',
  password :redis_password
);

See CREATE SECRET for the full secrets DSL and the Secrets section on this page for the redis parameter table.

String operations

-- Set a value (Redis SET)
SELECT redis_set('user:1', 'John Doe', 'redis') AS ok;

-- Get a value (Redis GET)
SELECT redis_get('user:1', 'redis') AS user_name;

-- Bulk write from a column
INSERT INTO users (id, name)
SELECT id,
       redis_set('user:' || id::VARCHAR, name, 'redis')
FROM new_users;

-- Multi-get in one round-trip (Redis MGET)
SELECT redis_mget('user:1,user:2,user:3', 'redis');

Maps to GET, SET, MGET. See redis_get / redis_set / redis_mget for the function signatures.

Hash operations

-- Set fields (Redis HSET)
SELECT redis_hset('user:1', 'email', '[email protected]', 'redis');
SELECT redis_hset('user:1', 'age',   '30',               'redis');

-- Get one field (Redis HGET)
SELECT redis_hget('user:1', 'email', 'redis') AS email;

-- All fields and values as a table (Redis HGETALL)
SELECT * FROM redis_hgetall('user:1', 'redis');

Maps to HGET, HSET, HGETALL. See redis_hget / redis_hset / redis_hgetall.

List operations

-- Push values (Redis LPUSH)
SELECT redis_lpush('queue:jobs', 'job-1', 'redis');
SELECT redis_lpush('queue:jobs', 'job-2', 'redis');

-- Read as a comma-joined string
SELECT redis_lrange('queue:jobs', 0, -1, 'redis');

-- Or as a table β€” one row per element
SELECT * FROM redis_lrange_table('queue:jobs', 0, -1, 'redis');

Maps to LPUSH and LRANGE. redis_lrange_table is the table-shaped variant β€” pick it when you want one DuckDB row per list element instead of a single comma-joined string from redis_lrange.

Discovery (SCAN, KEYS)

KEYS is convenient but blocks the Redis server on large keyspaces. In production, prefer cursor-paginated SCAN:

-- KEYS β€” fine for development / small keyspaces
SELECT * FROM redis_keys('user:*', 'redis');

-- SCAN β€” paginated, non-blocking
SELECT redis_scan('0', 'user:*', 100, 'redis');
-- β†’ "1024:user:1,user:2,...,user:50"
-- Pass the returned cursor (1024) back in to fetch the next page; cursor "0" means done.

See redis_keys and redis_scan.

Bulk hash discovery (redis_hscan_over_scan)

The killer feature for migration and bulk analysis β€” combines SCAN with HSCAN to emit (key, field, value) rows for every hash matching a key pattern, in one query:

-- Every field of every user:* hash, as rows
SELECT key, field, value
FROM redis_hscan_over_scan(
  /* scan_pattern  */ 'user:*',
  /* hscan_pattern */ '*',
  /* count         */ 100,
  /* secret        */ 'redis'
)
ORDER BY key, field;

Pipe directly into a CREATE TABLE AS for offline analysis or migration:

CREATE TABLE redis_users_dump AS
SELECT key, field, value
FROM redis_hscan_over_scan('user:*', '*', 100, 'redis');

See redis_hscan_over_scan for the full signature.

Key management

-- Type, existence, deletion
SELECT redis_type('user:1', 'redis');     -- 'hash'
SELECT redis_exists('user:1', 'redis');   -- TRUE
SELECT redis_del('user:1', 'redis');      -- TRUE if existed and removed

Maps to TYPE, EXISTS, DEL. See redis_type / redis_exists / redis_del.

Platform Support

Compatibility

Extension availability may vary by platform and DuckDB version. Check below to ensure this extension supports your environment before installation.

Quick Facts

Software License MIT
Pricing Free
Written In C++
Source Available Yes
View on GitHub
Usage
9,047+
loads in last 30 days

Platforms

Linux
Linux (musl)
macOS
Windows
WASM
Supported platform architectures
Linux: x86_64, aarch64
Linux (musl): x86_64
macOS: Intel, Apple Silicon
Windows: x86_64
WASM: Not available
Compiled binary sizes
Platform Architecture Size
Linux aarch64 2.83 MB
Linux x86_64 3.21 MB
Linux (musl) x86_64 2.51 MB
macOS Apple Silicon 1.96 MB
macOS Intel 2.28 MB
Windows x86_64 7.58 MB

Gzipped download size from the DuckDB community-extensions registry.

DuckDB Versions

Release calendar
Supported
v1.4.4 v1.5.2

SQL access to your Redis instance

Install Redis to read/write strings, hashes, and lists from inside DuckDB queries β€” and to drain an entire keyspace into a table with one CREATE TABLE AS.