2. Your first table function

A table function turns scalar arguments into zero or more rows. DuckDB repeatedly ticks its producer, so it can stream bounded batches without materializing the whole result.

NumbersFunction.cs

using Apache.Arrow;
using Apache.Arrow.Types;
using QueryFarm.Vgi.Internal;
using QueryFarm.Vgi.Table;
using QueryFarm.VgiRpc.Streaming;

namespace QueryFarm.Vgi.DocsExamples;

public sealed class NumbersFunction : ITableFunction
{
    public string Name => "numbers";

    public string Description => "Generate integers from zero through count - 1";

    public Schema ArgumentsSchema { get; } = new(
        [TableArgFields.PositionalWithRange("count", Int64Type.Default, ge: 0)],
        metadata: null);

    public Schema OutputSchema { get; } = new(
        [new Field("n", Int64Type.Default, nullable: false)],
        metadata: null);

    public ITableFunctionProducer CreateProducer(TableInitParams initParams) =>
        new Producer(initParams.Arguments.Int64(0), initParams.ProjectedSchema);

    private sealed class Producer(long count, Schema outputSchema) : ITableFunctionProducer
    {
        private long _next;

        public void Produce(OutputCollector output)
        {
            if (_next >= count)
            {
                output.Finish();
                return;
            }

            var rows = (int)Math.Min(4096, count - _next);
            var values = new Int64Array.Builder();
            values.Reserve(rows);
            for (var row = 0; row < rows; row++)
            {
                values.Append(_next++);
            }

            output.Emit(new RecordBatch(outputSchema, [values.Build()], rows));
            if (_next >= count)
            {
                output.Finish();
            }
        }
    }
}

The important pieces are:

ArgumentsSchema declares positional and named constant arguments with TableArgFields.
OutputSchema declares the columns returned to SQL.
CreateProducer reads bound arguments once and returns per-call state.
Produce emits one RecordBatch per tick and eventually calls Finish().

A public type currently lives under Internal

TableInitParams.Arguments is the public TableArguments type, currently namespaced under QueryFarm.Vgi.Internal; that is why this example imports the namespace. Limit that dependency to argument reads so it is easy to update if the type moves to the public table namespace.

Batch for throughput and backpressure

Emit thousands of rows at a time, not one row and not the whole data set. The example caps each batch at 4,096 rows. The engine controls subsequent ticks and can stop early for cancellation or a limit.

SELECT * FROM demo.numbers(5);
-- 0, 1, 2, 3, 4

SELECT count(*), sum(n) FROM demo.numbers(1_000_000);

For projection/filter pushdown, statistics, ordering, limits, partitions, and split planning, see Optimizer integration.