DuckDB Extension
Lindel
Linearize multi-dimensional numeric arrays with Hilbert and Morton (Z-order) space-filling curves.
On this page
Technical Overview
Multi-dimensional data, sorted on one integer
Lindel — linearization and delinearization — maps fixed-size numeric arrays to unsigned integer keys using Hilbert or Morton (Z-order) curves. Sort by the key when writing Parquet to organize rows for queries that filter multiple dimensions.
Organize rows across multiple dimensions
Parquet readers use per-row-group min/max statistics to skip data that cannot match a predicate. Ordering by a space-filling-curve key can group nearby coordinates and make these statistics more selective across several columns. Results depend on the coordinate transform, data distribution, row-group size, and query filters. Neither tight bounds on every column nor smaller files are guaranteed.
Hilbert and Morton
Both curves map several integer coordinates to one sortable position. Compare their encoding cost and the resulting file layout on your data.
- • Hilbert: Follows a curve with no long jumps between consecutive positions on its integer grid. A useful starting point for locality-oriented sorting; nearby input points are not guaranteed to become adjacent rows.
- • Morton / Z-order: Interleaves the coordinate bits. The mapping is simpler than Hilbert but has jumps at quadrant boundaries.
- • Supported inputs and output width: Accepts signed and unsigned 8-bit integers in 1–16 dimensions, 16-bit integers in 1–8, 32-bit integers or FLOAT in 1–4, and 64-bit integers or DOUBLE in 1–2. The output is the smallest unsigned type that holds all component bits, up to UHUGEINT. HUGEINT and UHUGEINT are not input element types.
Prepare and decode coordinates deliberately
Lindel carries the component bit patterns; it does not choose a numeric distance metric or coordinate transform.
- • Shift and quantize for numeric locality: Signed and floating-point inputs are supported directly, but their bit patterns are not normalized for numeric distance. For valid latitude and longitude, round((lat + 90) * 1e6)::UINTEGER and round((lon + 180) * 1e6)::UINTEGER make negative coordinates safe to cast and retain six decimal places in degrees. This is angular precision, not uniform distance in meters.
- • Choose consistent units and ranges: Set the scale of each dimension to reflect the precision and relative importance your queries need. Keep those transforms consistent across files and batches. Use explicit fixed-size ARRAY casts and handle NULL coordinates before encoding.
- • Decode with the original shape and key type: The decoder needs the dimension count, return_float, and return_unsigned flags. For unsigned integers use false, true; for signed integers use false, false; for floats use true, false. Preserve the encoded key’s SQL type because its width determines the component width. Community build 6435106 rejects one-dimensional signed-integer decoding; v1.5 source commit a92ec61 (extension version 2026100701) fixes it. Older community binaries may still be affected.
- • A write-time ordering: Apply ORDER BY inside COPY to persist the layout. Queries still filter the original columns, and updated files or new batches need their own sorting. Measure scanned rows or bytes and query time against your existing layout; Lindel does not create a runtime index or promise a fixed speedup.
Deep Dive
Technical Details
What you can do with one query
Replace a multi-column ORDER BY with one space-filling-curve sort key. For nonnegative integer coordinates, compute the key and order by it:
SELECT x, y, z, hilbert_encode([x, y, z]::UINTEGER[3]) AS hilbertFROM pointsORDER BY hilbert;Use that ordering when writing Parquet to group nearby coordinates. This can improve row-group pruning for queries that filter several dimensions. It does not guarantee that every nearby pair becomes adjacent or that every row group has tight bounds on every column.
Lindel accepts signed integers, unsigned integers, and floating-point values, but encodes their bit patterns directly. It does not normalize signed or floating-point numeric distance. For spatial sorting, choose a coordinate range and resolution, then shift and quantize values to nonnegative integers before encoding.
For latitude in [-90, 90] and longitude in [-180, 180], for example, use round((lat + 90) * 1e6)::UINTEGER and round((lon + 180) * 1e6)::UINTEGER. The offsets make negative coordinates safe to cast; the scale retains six decimal places in degrees. This is an angular grid, not a uniform distance grid on the Earth.
Hilbert vs Morton
Both hilbert_encode and morton_encode take a fixed-size numeric array and return one unsigned integer.
- Hilbert follows a space-filling curve with no long jumps between consecutive positions on its integer grid. It is a useful starting point for locality-oriented sorting.
- Morton / Z-order interleaves the coordinate bits. Its simpler mapping has jumps at quadrant boundaries.
Compare the resulting file layout and encoding cost on your own data. Neither curve guarantees better pruning for every distribution or query.
Type coverage and output width
Both encoders support these element types and array sizes:
| Element type | Dimensions |
|---|---|
TINYINT, UTINYINT |
1–16 |
SMALLINT, USMALLINT |
1–8 |
INTEGER, UINTEGER, FLOAT |
1–4 |
BIGINT, UBIGINT, DOUBLE |
1–2 |
HUGEINT and UHUGEINT are not supported as input array element types. The result uses the smallest unsigned integer type that fits element bit width × number of dimensions, up to 128 bits:
| Input array | Total bits | Output type |
|---|---|---|
UTINYINT[2] |
16 | USMALLINT |
UINTEGER[2] |
64 | UBIGINT |
UINTEGER[3] |
96 | UHUGEINT |
DOUBLE[2] |
128 | UHUGEINT |
Cast arrays explicitly: [10, 20] is a LIST; [10, 20]::UINTEGER[2] is a fixed-size ARRAY. Array elements cannot be NULL.
Choose coordinate units and precision deliberately. A degree of latitude, a meter of altitude, and a second of time are different measures; Lindel does not decide their relative importance for you. Keep the same transform when writing subsequent batches.
Decoding
hilbert_decode and morton_decode recover the component values using the matching curve. Three trailing arguments specify the number of dimensions, whether to return floats, and whether to return unsigned integers:
SELECT hilbert_decode( hilbert_encode([5, 8]::UINTEGER[2]), 2, -- num_elements false, -- return_float true -- return_unsigned);-- [5, 8]For signed integers, use false, false; for floating-point values, use true, false. Preserve the encoded key’s original SQL type: the decoder uses its width together with the dimension count to determine the component width. If you shifted or quantized coordinates before encoding, decoding returns those transformed coordinates.
Community build 6435106 rejects one-dimensional signed-integer decoding. The v1.5 source fix a92ec61 (extension version 2026100701) supports signed round-trips in one dimension and restores four-dimensional 8-bit decoding. Older installed community binaries may still have these limitations. Signed two-dimensional, unsigned one-dimensional, and floating-point one-dimensional round-trips work in the older build too.
Why ordering can help Parquet
Parquet stores per-row-group min/max statistics. For a query such as WHERE lat BETWEEN 40 AND 41 AND lon BETWEEN -74 AND -73, a reader can skip a row group if its statistics rule out either condition. Grouping points by a Hilbert or Morton key can make those ranges more selective across multiple columns.
The improvement depends on coordinate preparation, data distribution, row-group size, and query predicates. Measure scanned rows or bytes, query time, and file size against the existing layout. Sorting does not guarantee smaller files or a particular speedup.
This is a write-time ordering, not a runtime index. Use ORDER BY inside the query passed to COPY; for changing data, sort each rewritten file or batch. Queries still filter on the original columns.
Compared to alternatives
ORDER BY a, bprioritizesa, thenbwithin equal values ofa. It can be a better fit when queries mostly filter the leading column. A curve order balances multiple coordinates in a different way.- Partitioning by year or region separates data into files or directories. Curve sorting can complement partitioning by organizing rows inside each partition.
- A spatial index supports a different access pattern. Lindel computes keys; it does not create an index or automatically translate spatial predicates into encoded-key ranges.
Install
INSTALL lindel FROM community;
LOAD lindel;
Quick Start
Encode a multi-dimensional point to one sortable integer
-- A fixed-size numeric ARRAY becomes one unsigned integer
SELECT hilbert_encode([10, 20]::UINTEGER[2]) AS hilbert;
-- 884
Order rows by the curve — this is the Parquet write-time sort key
-- Sort nonnegative coordinates by their Hilbert key.
-- Use this ORDER BY inside COPY to persist the layout.
SELECT x, y, hilbert_encode([x, y]::UINTEGER[2]) AS hilbert
FROM (VALUES (3, 5), (1, 1), (7, 0), (2, 6), (5, 3), (0, 4)) AS t(x, y)
ORDER BY hilbert;
Compare Morton (Z-order) on the same coordinates
SELECT x, y, morton_encode([x, y]::UINTEGER[2]) AS morton
FROM (VALUES (3, 5), (1, 1), (7, 0), (2, 6), (5, 3), (0, 4)) AS t(x, y)
ORDER BY morton;
Reference
Extension Contents
Quick reference to all available functions and settings organized by category.
| Name | Type | Description |
|---|---|---|
|
Hilbert
Hilbert-curve encoding — best locality preservation. Slightly slower to compute than Morton but produces tighter row groups when used as ORDER BY in Parquet, leading to better predicate skipping at query time. |
||
| hilbert_decode() | Object type: Scalar function | Recover component values from a Hilbert-encoded key. |
| hilbert_encode() | Object type: Scalar function | Encode a fixed-size numeric array into an unsigned integer using the Hilbert curve. |
|
Morton (Z-order)
Morton-curve encoding (also known as Z-order). Faster to compute, slightly weaker locality. The classic Z-Order optimization that Delta Lake popularized. |
||
| morton_decode() | Object type: Scalar function | Recover component values from a Morton-encoded key. |
| morton_encode() | Object type: Scalar function | Encode a fixed-size numeric array into an unsigned integer by interleaving its coordinate bits (Morton / Z-order). |
No extension contents match that search.
API Reference
Function Reference
Practical Examples
Cookbook
Real-world recipes and patterns for common use cases.
What this is for
Lindel computes Hilbert and Morton sort keys from multi-dimensional numeric arrays. Ordering rows by one of these keys can group related coordinates when writing Parquet, helping readers use min/max statistics for multi-column predicates. The benefit depends on the data, coordinate transform, and row-group size.
Prepare latitude and longitude
For valid latitude in [-90, 90] and longitude in [-180, 180], shift both ranges above zero and quantize to six decimal places before casting to unsigned integers:
SELECT lat, lon, hilbert_encode([ round((lat + 90) * 1e6), round((lon + 180) * 1e6)]::UINTEGER[2]) AS hilbertFROM sourceORDER BY hilbert;The offsets matter: casting a negative coordinate directly to UINTEGER fails. The scale sets angular precision, not uniform distance in meters. Direct DOUBLE encoding is supported, but it uses floating-point bit patterns without normalizing numeric distance.
Write the sorted layout to Parquet
COPY ( SELECT * FROM source ORDER BY hilbert_encode([ round((lat + 90) * 1e6), round((lon + 180) * 1e6) ]::UINTEGER[2])) TO 'spatial.parquet' (FORMAT PARQUET);Query the original columns normally:
SELECT * FROM 'spatial.parquet'WHERE lat BETWEEN 40 AND 41 AND lon BETWEEN -74 AND -73;A reader can skip a row group when its statistics cannot match the predicates. Curve sorting can improve that pruning; it does not guarantee tight bounds for every group or column. Benchmark with enough rows to produce multiple row groups.
Use up to four 32-bit dimensions
All components must share one array element type. Four UINTEGER coordinates produce a 128-bit UHUGEINT key:
SELECT hilbert_encode([10, 20, 30, 40]::UINTEGER[4]) AS hilbert;For inputs with different units, choose a consistent scaling and offset for each dimension before encoding. Four DOUBLE dimensions are not supported: their combined width exceeds 128 bits.
Try Morton / Z-order
Use the same prepared coordinates with morton_encode:
SELECT lat, lon, morton_encode([ round((lat + 90) * 1e6), round((lon + 180) * 1e6)]::UINTEGER[2]) AS mortonFROM sourceORDER BY morton;Morton interleaves bits and has a simpler mapping than Hilbert. Compare encoding time and the resulting query performance before choosing an ordering.
Decode unsigned, signed, or floating-point values
Use the same curve to decode, and match the dimension count and type flags:
SELECT hilbert_decode( hilbert_encode([10, 20]::UINTEGER[2]), /* num_elements */ 2, /* return_float */ false, /* return_unsigned */ true) AS unsigned_roundtrip;-- [10, 20]SELECT hilbert_decode( hilbert_encode([-10, 20]::INTEGER[2]), 2, false, false) AS signed_roundtrip;-- [-10, 20]SELECT morton_decode( morton_encode([-1.5, 2.25]::DOUBLE[2]), 2, true, false) AS float_roundtrip;-- [-1.5, 2.25]Preserve the key’s original SQL type when storing or decoding it. The integer width and dimension count determine the decoded component width. Decoding recovers the values passed to the encoder; it does not undo any earlier quantization or offsets.
Community build 6435106 rejects one-dimensional signed-integer decoding. The v1.5 source fix a92ec61 (extension version 2026100701) supports signed round-trips in one dimension and restores four-dimensional 8-bit decoding. Older installed community binaries may still have these limitations.
Type coverage
| Element type | Dimensions |
|---|---|
TINYINT, UTINYINT |
1–16 |
SMALLINT, USMALLINT |
1–8 |
INTEGER, UINTEGER, FLOAT |
1–4 |
BIGINT, UBIGINT, DOUBLE |
1–2 |
The result is the smallest unsigned integer type that fits the combined bits, up to UHUGEINT. HUGEINT and UHUGEINT are not supported input element types. Use fixed-size arrays with explicit casts, and handle NULL coordinates before encoding.
Platform Support
Compatibility
Extension availability may vary by platform and DuckDB version. Check below to ensure this extension supports your environment before installation.
Quick Facts
Platforms
- Linux x86_64 aarch64
- Linux (musl) Not available
- macOS Intel Apple Silicon
- Windows x86_64
- WASM eh mvp threads
Compiled binary sizes
| Platform | Architecture | Size |
|---|---|---|
| Linux | x86_64 | 4.14 MB |
| Linux | aarch64 | 3.76 MB |
| macOS | Intel | 2.22 MB |
| macOS | Apple Silicon | 1.92 MB |
| Windows | x86_64 | 7.46 MB |
| WASM | eh | 78.9 KB |
| WASM | mvp | 82.0 KB |
| WASM | threads | 66.5 KB |
Compressed download size from the Haybarn extension repository.
DuckDB & Haybarn
Release calendar- DuckDB v1.5.5 Haybarn 1.5.5-rc1 Supported