DuckDB
DataSketches for DuckDB 2.0: sketch aggregates now stay within the memory limit and spill to disk
A grouped KLL query used 39 GB under a 1 GB memory limit. Now it uses 0.77 GB and runs 3.7× faster. Here's how we moved 15 sketch families into DuckDB's spillable memory, and the library bugs we fixed along the way.
8 min read