Arrow helpers

On this page

Schema caching, per-type scalar helpers, and the type rules the annotations follow.

class CachedSchema

source

public final class CachedSchema

Description

Wire-portable IPC-encoded Schema with on-demand deserialisation caching. Used by table-producer states that hold an output schema across exchange ticks — the IPC bytes survive HTTP state-token round-trips (Jackson serialises the public #ipc field), and the cached Schema is reused per process to avoid re-decoding on every emit.

The class is intentionally a mutable POJO with a public no-arg constructor so the StateSerializer reflection path can both instantiate it and populate ipc from JSON.

Members

byte[] ipc

IPC-encoded schema bytes; public so Jackson can serialise it through state tokens.

CachedSchema()

No-arg constructor for the reflection-based state deserialisation path.

CachedSchema(byte[] ipc)

Wrap pre-encoded IPC schema bytes.

CachedSchema(Schema schema)

Encode a Schema to IPC bytes for storage.

Schema get()

Deserialise on first call; cached for the lifetime of this instance.

interface Float64Op

source

public interface Float64Op

Description

Per-row computation producing a float64 value.

interface Int64Op

source

public interface Int64Op

Description

Per-row computation producing an int64 value.

class ScalarHelpers

source

public final class ScalarHelpers

Description

Helpers for scalar function processing: numeric dispatch + per-row mapping.

Phase 2 supports int64 and float64 dispatch with input-side widening from smaller integer / float types. Wider numeric coverage (decimal, uint*, etc.) arrives in later phases.

class Schemas

source

public final class Schemas

Description

Convenience builders for Schemas and their IPC byte encodings — primarily used by scalar functions that return a single fixed-type column.

Members

ArrowType INT64 = new ArrowType.Int(64, true)

Signed 64-bit integer (SQL BIGINT).

ArrowType INT32 = new ArrowType.Int(32, true)

Signed 32-bit integer (SQL INTEGER).

ArrowType UINT32 = new ArrowType.Int(32, false)

Unsigned 32-bit integer (SQL UINTEGER).

ArrowType UINT64 = new ArrowType.Int(64, false)

Unsigned 64-bit integer (SQL UBIGINT).

ArrowType FLOAT64 = new ArrowType.FloatingPoint( org.apache.arrow.vector.types.FloatingPointPrecision.DOUBLE)

Double-precision float (SQL DOUBLE).

ArrowType UTF8 = new ArrowType.Utf8()

Variable-length UTF-8 string (SQL VARCHAR).

ArrowType BOOL = new ArrowType.Bool()

Boolean (SQL BOOLEAN).

ArrowType BINARY = new ArrowType.Binary()

Variable-length byte string (SQL BLOB).

Field nullable(String name, ArrowType type)

Nullable Field builder — the default for fixture output columns.

Field nonNull(String name, ArrowType type)

Non-null Field builder.

Schema of(Field… fields)

Build a Schema from a varargs list of fields.

ArrowType timestampMicros(String tz)

Microsecond-precision timestamp type, optionally with a timezone (null or empty for naive timestamps).

Field list(String name, ArrowType item, boolean nullable)

list<item> Field with a child field literally named "item" (Arrow convention). nullable applies to the list itself; the item is always nullable.

Schema singleResult(ArrowType t)

Single-column nullable result schema.

byte[] singleResultIpc(ArrowType t)

IPC byte encoding of #singleResult(ArrowType).

byte[] singleResultAnyIpc()

Single-column “ANY”-typed result schema. The field carries the vgi_type=any metadata that DuckDB’s catalog-enumeration path recognises and reports as ANY in duckdb_functions().

interface StringOp

source

public interface StringOp

Description

Per-row computation producing a utf8 value (may be null).

Members

VectorSchemaRoot mapInt64(Schema outputSchema, VectorSchemaRoot input, BufferAllocator alloc, FieldVector nullSource, Int64Op op)

Build an int64 result column by invoking op once per row of input. NULL inputs yield NULL outputs (default null handling).

VectorSchemaRoot mapNumericRows(Schema outSchema, BufferAllocator alloc, List<FieldVector> inputCols, int rows, IntToLongFunction longOp, IntToDoubleFunction doubleOp)

Build a single result column of int64 or float64 (chosen by outSchema’s result field type), running longOp or doubleOp per row. Rows where any of inputCols is null yield null output (default null handling).

The two ops are passed as separate lambdas because the per-row computation differs by output type (int sums of int inputs, double sums when any input widens).

VectorSchemaRoot mapInt64Raw(Schema outputSchema, VectorSchemaRoot input, BufferAllocator alloc, Int64Op op)

Build an int64 result column unconditionally (caller handles NULLs).

VectorSchemaRoot mapString(Schema outputSchema, VectorSchemaRoot input, BufferAllocator alloc, FieldVector nullSource, StringOp op)

Build a utf8 result column by invoking op once per row.

long toLong(FieldVector v, int row)

Read a row as a long, widening any signed integer vector.

double toDouble(FieldVector v, int row)

Read a row as a double, widening any numeric vector (integer, float, or decimal). NULL decimal cells read as 0.0.

java.math.BigDecimal toBigDecimal(FieldVector v, int row)

Read a row as a java.math.BigDecimal, widening any numeric vector. Decimal vectors are read losslessly; integers and floats are coerced.

String toString(FieldVector v, int row)

Read a row as a UTF-8 String.

boolean toBool(FieldVector v, int row)

Read a row as a boolean.

class TypeRules

source

public final class TypeRules

Description

DuckDB-compatible numeric type promotion rules.

Mirrors vgi-go vgi.PromoteForAddition: scalar-arithmetic functions promote integer inputs one width up (TINYINT → SMALLINT → INT → BIGINT) and leave floats alone. Mixed int/float promotes to float.

Members

boolean isInteger(ArrowType t)

Tests whether t is an integer type (any width, signed or unsigned).

boolean isFloating(ArrowType t)

Tests whether t is a floating-point type (any precision).

boolean isNumeric(ArrowType t)

Tests whether t is numeric, i.e. integer or floating-point (decimals are excluded; see #isAddable).

boolean isAddable(ArrowType t)

Tests whether t can participate in arithmetic addition — numeric or decimal. Backs TypeBoundPredicate.IS_ADDABLE bind-time checks.

ArrowType promoteForAddition(ArrowType t)

Promote a single input type one width up (for double(value)-style arithmetic). Floats stay; ints climb the tier.

String sqlTypeName(ArrowType t)

Render an Arrow type as the SQL name DuckDB users recognise (e.g. BIGINT, VARCHAR, DECIMAL(38,2)). Falls back to Arrow’s toString for types without a clean SQL equivalent.

ArrowType commonTypeForAddition(ArrowType a, ArrowType b)

Promote two operands to a common arithmetic type (used by add_values). Mirrors vgi.CommonTypeForAddition — float wins, otherwise the widest integer width plus one tier (capped at int64).