Skip to content
Query.Farm
Talk with Us

Table-in-out functions

On this page

Stream a relation through, batch by batch.

interface TableInOutFunction

Section titled “interface TableInOutFunction”
source
type TableInOutFunction interface {
// Name returns the function name used in SQL.
Name() string
// Metadata returns descriptive metadata.
Metadata() FunctionMetadata
// ArgumentSpecs returns the function's argument specifications.
ArgumentSpecs() []ArgSpec
// OnBind resolves the output schema given the bind parameters.
OnBind(params *BindParams) (*BindResponse, error)
// OnInit performs one-time initialization and returns execution parameters.
OnInit(params *InitParams) (*GlobalInitResponse, error)
// NewState creates the initial mutable state for this function execution.
NewState(params *ProcessParams) (interface{}, error)
// Process transforms one input batch into output. Must emit exactly one
// output batch via out.Emit.
Process(ctx context.Context, params *ProcessParams, state interface{}, batch arrow.RecordBatch, out *vgirpc.OutputCollector) error
// Finalize is called after all input batches have been processed.
// Returns batches to emit during the FINALIZE phase.
Finalize(ctx context.Context, params *ProcessParams, state interface{}) ([]arrow.RecordBatch, error)
}

Description

TableInOutFunction is the interface for table-in-out VGI functions. Table-in-out functions transform input tables, with an INPUT phase (Exchange) and an optional FINALIZE phase (Producer).

Methods

function AsTableInOutFunction

Section titled “function AsTableInOutFunction”
source
func AsTableInOutFunction[S any](f TypedTableInOutFunc[S]) TableInOutFunction

AsTableInOutFunction wraps a TypedTableInOutFunc into a TableInOutFunction for registration with Worker.RegisterTableInOut. The adapter:

  • Provides type-safe state casting (returns error instead of panic)
  • Defaults OnInit to DefaultInit() (MaxWorkers: 1) unless OnIniter is implemented

Usage:

func NewEchoFunction() vgi.TableInOutFunction {
return vgi.AsTableInOutFunction[struct{}](&EchoFunction{})
}

interface TableInOutFunctionWithCardinality

Section titled “interface TableInOutFunctionWithCardinality”
source
type TableInOutFunctionWithCardinality interface {
TableInOutFunction
// Cardinality returns an estimated row count for query optimization.
Cardinality(params *BindParams) (*TableCardinality, error)
}

Description

TableInOutFunctionWithCardinality extends TableInOutFunction with cardinality estimation.

interface TypedTableInOutFunc

Section titled “interface TypedTableInOutFunc”
source
type TypedTableInOutFunc[S any] interface {
// Name returns the function name used to invoke it in SQL.
Name() string
// Metadata returns descriptive metadata for the function.
Metadata() FunctionMetadata
// ArgumentSpecs returns the function's argument specifications.
ArgumentSpecs() []ArgSpec
// OnBind resolves the output schema and bind state from the bind parameters.
OnBind(params *BindParams) (*BindResponse, error)
// NewState creates a fresh, typed per-scan state value.
NewState(params *ProcessParams) (*S, error)
// Process transforms one input batch into output rows, emitting them via out.
Process(ctx context.Context, params *ProcessParams, state *S,
  batch arrow.RecordBatch, out *vgirpc.OutputCollector) error
// Finalize returns any trailing batches to emit after the last input batch.
Finalize(ctx context.Context, params *ProcessParams, state *S) ([]arrow.RecordBatch, error)
}

Description

TypedTableInOutFunc is the recommended interface for table-in-out functions. It provides compile-time type safety for state management, eliminating the unsafe state.(*myType) assertions required by the lower-level TableInOutFunction interface.

Implementations may also satisfy OnIniter (custom OnInit). This is detected automatically by AsTableInOutFunction.

struct typedTableInOutAdapter

Section titled “struct typedTableInOutAdapter”
source
type typedTableInOutAdapter[S any] struct {
inner  TypedTableInOutFunc[S]
onInit  func(*InitParams) (*GlobalInitResponse, error)
}

Description

typedTableInOutAdapter implements TableInOutFunction by delegating to a TypedTableInOutFunc[S].

Methods

method ArgumentSpecs

Section titled “method ArgumentSpecs”
source
func (a *typedTableInOutAdapter[S]) ArgumentSpecs() []ArgSpec

ArgumentSpecs forwards to the wrapped typed function’s ArgumentSpecs.

method Finalize

Section titled “method Finalize”
source
func (a *typedTableInOutAdapter[S]) Finalize(ctx context.Context, params *ProcessParams, state interface{}) ([]arrow.RecordBatch, error)

Finalize type-asserts the untyped state to *S and forwards to the wrapped typed function’s Finalize, returning an error on a state type mismatch.

method Metadata

Section titled “method Metadata”
source
func (a *typedTableInOutAdapter[S]) Metadata() FunctionMetadata

Metadata forwards to the wrapped typed function’s Metadata.

method Name

Section titled “method Name”
source
func (a *typedTableInOutAdapter[S]) Name() string

Name forwards to the wrapped typed function’s Name.

method NewState

Section titled “method NewState”
source
func (a *typedTableInOutAdapter[S]) NewState(params *ProcessParams) (interface{}, error)

NewState forwards to the wrapped typed function’s NewState, returning the typed state as an untyped interface{}.

method OnBind

Section titled “method OnBind”
source
func (a *typedTableInOutAdapter[S]) OnBind(params *BindParams) (*BindResponse, error)

OnBind forwards to the wrapped typed function’s OnBind.

method OnInit

Section titled “method OnInit”
source
func (a *typedTableInOutAdapter[S]) OnInit(params *InitParams) (*GlobalInitResponse, error)

OnInit invokes the optional OnIniter hook if present, otherwise returns DefaultInit.

method Process

Section titled “method Process”
source
func (a *typedTableInOutAdapter[S]) Process(ctx context.Context, params *ProcessParams, state interface{}, batch arrow.RecordBatch, out *vgirpc.OutputCollector) error

Process type-asserts the untyped state to *S and forwards to the wrapped typed function’s Process, returning an error on a state type mismatch.