Consistent hashing functions in the Hashfuncs DuckDB extension
Sample data for the examples
Each browser example starts with these sample tables; sample files are shared across examples on this page. Run this setup first when trying the examples in your own DuckDB session.
-- Fixtures for the hashfuncs cookbook / technical-details examples.
--
-- The hashfuncs extension itself is pure CPU (no catalog, no network),
-- so these fixtures only exist to satisfy the *table references* the
-- documentation makes ("FROM events", "FROM users", ...). The numbers
-- chosen are tiny and deterministic so re-running the suite is stable.
--
-- Idempotent: every CREATE uses OR REPLACE.
-- The bitfilters cookbook section pairs hashfuncs with the bitfilters
-- extension. Load it best-effort so xor8_filter / xor8_filter_contains
-- resolve at parse time. If the community binary isn't available on the
-- current platform we still want the rest of the fixtures to apply.
INSTALL bitfilters FROM query_farm;
LOAD bitfilters FROM query_farm;
-- cookbook.mdx — "Hash partitioning" and "Independent hashes via seeds"
CREATE OR REPLACE TABLE users AS
SELECT i AS user_id
FROM range(0, 8) t(i);
-- cookbook.mdx — "Cross-system routing (Cassandra / Spark)"
CREATE OR REPLACE TABLE rows_to_route AS
SELECT 'k' || i::VARCHAR AS key
FROM range(0, 4) t(i);
-- cookbook.mdx — "String cache keys"
CREATE OR REPLACE TABLE query_log AS
SELECT i AS user_id,
'select ' || i::VARCHAR AS query_text
FROM range(0, 4) t(i);
-- cookbook.mdx — "Bloom / XOR / Cuckoo filter input"
CREATE OR REPLACE TABLE allowed_users AS
SELECT 'user' || i::VARCHAR || '@example.com' AS email
FROM range(0, 4) t(i);
-- cookbook.mdx — second bitfilters block reads from `filter_table f`
-- (a one-row table holding a pre-built xor8 filter).
CREATE OR REPLACE TABLE filter_table AS
SELECT xor8_filter(xxh3_64(email)) AS filter
FROM allowed_users;
-- cookbook.mdx — "Independent hashes via seeds" uses `FROM keys`
CREATE OR REPLACE TABLE keys AS
SELECT 'k' || i::VARCHAR AS key
FROM range(0, 4) t(i);
-- cookbook.mdx + technical-details.mdx — "events" with event_id, payload
-- Used by sharding / dedup / "magic moment" examples.
CREATE OR REPLACE TABLE events AS
SELECT i AS event_id,
'evt-' || i::VARCHAR AS payload,
'user' || i::VARCHAR || '@example.com' AS email
FROM range(0, 8) t(i);
-- technical-details.mdx — "double-hash with two seeds" reads `FROM data`
CREATE OR REPLACE TABLE data AS
SELECT 'k' || i::VARCHAR AS key
FROM range(0, 4) t(i);
Function category
Consistent hashing
1 functionAssign keys to a number of buckets so that changing the bucket count moves as few keys as possible. jump_hash moves only about 1/(n + 1) of the keys when growing from n to n + 1 buckets, where hash % n would move almost all of them.
Signature
Arguments (Positional)
| Argument | Type | Mode | Description |
|---|---|---|---|
Argument
key
|
Type
<numeric>
2 concrete typesBIGINTUBIGINT
|
Mode Positional | Description |
Argument
buckets
|
Type
BIGINT
|
Mode Positional | Description |
Description
Jump consistent hash (Lamping and Veach): assigns a 64-bit key to one of buckets buckets so that growing from n to n + 1 buckets moves only about 1/(n + 1) of the keys — all of them into the new bucket — where hash % n would move almost every key. Use it to reshard with minimal data movement.
The key is a UBIGINT or BIGINT, read as its 64 bits; hash the value first, e.g. jump_hash(xxh3_64(x), n). Returns the bucket as INTEGER from 0 to buckets - 1, the same bucket Guava's Hashing.consistentHash returns. buckets must be between 1 and 2147483647.
SELECT jump_hash(xxh3_64('user-42'), 16);
Output
| jump_hash(xxh3_64('user-42'), 16) |
|---|
| 10 |
SELECT jump_hash(xxh3_64('user-42'), 16), jump_hash(xxh3_64('user-42'), 17);
Output
| jump_hash(xxh3_64('user-42'), 16) | jump_hash(xxh3_64('user-42'), 17) |
|---|---|
| 10 | 10 |
Related functions
- xxh3_64() — 64-bit xxHash3 (XXH3_64) — the modern xxHash variant tuned for vectorized CPUs (SSE2/AVX2)
- iceberg_bucket() — The partition Apache Iceberg's `bucket[N]` transform assigns a value: `(murmur3_x86_32(value) & Integer
- kafka_partition() — The partition Kafka's default partitioner assigns a keyed record: `toPositive(murmur2(key)) % partitions`