Skip to content
Query.Farm
Talk with Us

Catalogs

On this page

Presenting a worker as a database: schemas, tables, views, macros.

type AttachOpaqueData

Section titled “type AttachOpaqueData”
source
export type AttachOpaqueData = Uint8Array;

interface AttachOptionSpec

Section titled “interface AttachOptionSpec”
source
export interface AttachOptionSpec

Description

Declarative spec for a single attach-time option.

The catalog’s catalogsInfo() emits these (serialized) in CatalogInfo.attach_option_specs so the DuckDB extension can validate user-supplied ATTACH options and cast them to the declared type before forwarding to the worker’s catalog_attach handler.

Fields

namestring

Option name — matches the key users pass in the ATTACH statement.

descriptionstring

Human-readable description (shown in discovery UIs).

typeVgiDataType

Arrow data type the extension should cast user input to.

defaultunknownoptional

Default value used when the user omits this option. Passed through to the worker’s catalog_attach handler as-is if no override is given. Use null for “no default” (an unset option will then be absent from the options dict delivered to attach()).

requiredbooleanoptional

The caller must supply this option at ATTACH time. A catalog that cannot be attached without it advertises that fact at discovery, so a client can say so before attempting the attach rather than surfacing a failure that reads like an empty catalog.

Mutually exclusive with default — an option that falls back to a value is by definition satisfiable without the caller.

type AttachOptionValue

Section titled “type AttachOptionValue”
source
export type AttachOptionValue = string | number | bigint | boolean | null | Uint8Array;

Description

Values supported in a CatalogAttach options map. The client infers an Arrow type per value:

string → Utf8 bigint → Int64 number → Float64 (JS numbers are doubles — safe for small ints too) boolean → Bool Uint8Array → Binary null → Null (column with no concrete type; value is null)

If you need types this can’t express (Decimal, Timestamp, Int32 vs Int64, nested structs), drop down to optionsBytes on CatalogAttachOptions and build the RecordBatch yourself.

function buildScanBranchesResult

Section titled “function buildScanBranchesResult”
source
export function buildScanBranchesResult(
branches: ScanBranchInput[],
requiredExtensions: string[] = [],
): { branches: Uint8Array[]; required_extensions: string[] }

Description

Build a multi-branch ScanBranchesResult wire dict from explicit branch definitions. Each branch becomes its own 1-row ScanBranchSchema IPC stream carried in the branches list<binary> column. Mirrors vgi-python’s ScanBranchesResult construction in the test fixture’s table_scan_branches_get.

An empty branches list is serialized verbatim — the C++ side loud-fails on it (that asymmetry is exercised by multi_branch_empty_branches.test).

interface CatalogAttachResult

Section titled “interface CatalogAttachResult”
source
export interface CatalogAttachResult

Fields

attach_opaque_dataUint8Array
supports_transactionsboolean
supports_time_travelboolean
catalog_version_frozenboolean
catalog_versionnumber
attach_opaque_data_requiredboolean
default_schemastring
settingsUint8Array[]
secret_typesUint8Array[]
attach_catalogsUint8Array[]
tagsRecord<string, string>
supports_column_statisticsboolean
global_functionsUint8Array[]
global_function_prefixstring
commentstring | nulloptional
resolved_data_versionstring | nulloptional
resolved_implementation_versionstring | nulloptional

interface CatalogDescriptor

Section titled “interface CatalogDescriptor”
source
export interface CatalogDescriptor

Fields

namestring
schemasSchemaDescriptor[]
defaultSchemastringoptional
settingsSettingDescriptor[]optional
secretTypesSecretTypeDescriptor[]optional
attachCatalogsAttachCatalogInfo[]optional

Companion catalogs (lakehouse federation) the client should ATTACH when this VGI catalog attaches. Surfaced via catalog_attach.attach_catalogs.

commentstringoptional
tagsRecord<string, string>optional
sourceUrlstringoptional

Homepage for this catalog — repo, docs, or dataset landing page. Surfaced through the catalog_catalogs discovery record as CatalogInfo.source_url, which DuckDB exposes via duckdb_databases(). Optional; omitted/undefined advertises a null source_url.

globalFunctionsVgiFunction[]optional

Functions this catalog asks the client to additionally publish into its global (DuckDB system.main) function namespace, on top of the normal schema-qualified registration. Surfaced as IPC-serialized FunctionInfo records on catalog_attach.global_functions.

Every entry must also be declared in exactly one of this catalog’s schemas — the FunctionInfo.schema_name it carries is the bind-dispatch key, not a sentinel. Mirrors vgi-python’s Catalog.global_functions.

globalFunctionPrefixstringoptional

Prefix applied client-side to every globalFunctions entry’s name when it is published globally (e.g. prefix vgi_example + global_scalar → vgi_example_global_scalar). The advertised FunctionInfo.name stays unprefixed. Surfaced as catalog_attach.global_function_prefix.

interface CatalogInfo

Section titled “interface CatalogInfo”
source
export interface CatalogInfo

Fields

namestring
attach_option_specsUint8Array[]
releasesCatalogDataVersionRelease[]
implementation_versionstring | nulloptional
data_version_specstring | nulloptional
source_urlstring | nulloptional

class CatalogInterface

Section titled “class CatalogInterface”
source
export abstract class CatalogInterface

Methods

method catalogNameForAttach

Section titled “method catalogNameForAttach”
source
catalogNameForAttach(_attachOpaqueData: Uint8Array): string | null

Which catalog an attach_opaque_data belongs to, or null if unknown.

Two catalogs served by one worker may declare the same schema and function name, in which case the attachment is the only thing that tells them apart — the bind path uses this to scope function resolution. The default answers from catalogs() when this interface owns exactly one, which is right for every single-catalog implementation; the composite overrides it to route by its backend index byte.

method catalogs

Section titled “method catalogs”
source
abstract catalogs(): string[];

method attach

Section titled “method attach”
source
abstract attach(
  name: string,
  options?: Record<string, unknown>,
  dataVersionSpec?: string | null,
  implementationVersion?: string | null,
): Awaitable<CatalogAttachResult>;

Attach this catalog. The framework already validates ATTACH options (name + type) against the specs the worker advertises in catalogsInfo().attach_option_specs, so options arrives as a validated, decoded {name: value} dict with typed column values. Use {} if no options were supplied.

method catalogsInfo

Section titled “method catalogsInfo”
source
catalogsInfo?(): Awaitable<import(“../generated/vgi-client.js”).CatalogInfo[]>;

List the catalogs this worker exposes with their version metadata. The implementation_version / data_version_spec come from the CatalogInfo returned by the read-only catalogs() signature below — a versioned worker overrides catalogs() / catalogsInfo() to advertise real values; the default derives them from the registered descriptor(s).

method detach

Section titled “method detach”
source
abstract detach(attachOpaqueData: AttachOpaqueData): Awaitable<void>;

method version

Section titled “method version”
source
abstract version(
  attachOpaqueData: AttachOpaqueData,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<number>;

method schemas

Section titled “method schemas”
source
abstract schemas(
  attachOpaqueData: AttachOpaqueData,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<SchemaInfo[]>;

method create

Section titled “method create”
source
create(name: string, onConflict: string, options?: any): Awaitable<void>

method drop

Section titled “method drop”
source
drop(name: string): Awaitable<void>

method schemaGet

Section titled “method schemaGet”
source
schemaGet(
  attachOpaqueData: AttachOpaqueData,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<SchemaInfo | null>

method schemaCreate

Section titled “method schemaCreate”
source
schemaCreate(
  attachOpaqueData: AttachOpaqueData,
  name: string,
  comment?: string | null,
  tags?: any,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method schemaDrop

Section titled “method schemaDrop”
source
schemaDrop(
  attachOpaqueData: AttachOpaqueData,
  name: string,
  ignoreNotFound?: boolean,
  cascade?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method schemaContentsTables

Section titled “method schemaContentsTables”
source
schemaContentsTables(
  attachOpaqueData: AttachOpaqueData,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<TableInfo[]>

method schemaContentsViews

Section titled “method schemaContentsViews”
source
schemaContentsViews(
  attachOpaqueData: AttachOpaqueData,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<ViewInfo[]>

method schemaContentsFunctions

Section titled “method schemaContentsFunctions”
source
schemaContentsFunctions(
  attachOpaqueData: AttachOpaqueData,
  name: string,
  type: string,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<FunctionInfo[]>

method tableGet

Section titled “method tableGet”
source
tableGet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  atUnit?: string,
  atValue?: string,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<TableInfo | null>

method tableCreate

Section titled “method tableCreate”
source
tableCreate(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  columns: Uint8Array,
  onConflict: string,
  notNullConstraints?: number[],
  uniqueConstraints?: number[][],
  checkConstraints?: string[],
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method tableDrop

Section titled “method tableDrop”
source
tableDrop(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  ignoreNotFound?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method tableScanFunctionGet

Section titled “method tableScanFunctionGet”
source
tableScanFunctionGet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  atUnit?: string,
  atValue?: string,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<any>

method tableScanBranchesGet

Section titled “method tableScanBranchesGet”
source
tableScanBranchesGet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  atUnit?: string,
  atValue?: string,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<any>

Return the scan branches for a (possibly multi-source) table. Default delegates to tableScanFunctionGet and wraps the single result as a one-branch list, so every single-source catalog is compatible with the branches-aware C++ extension. Override for genuine multi-source tables.

method tableColumnStatisticsGet

Section titled “method tableColumnStatisticsGet”
source
tableColumnStatisticsGet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData,
): Awaitable<

Return serialized column statistics for a table, or null if none are available. The result is the IPC bytes of a ColumnStatistics RecordBatch (see src/util/statistics.ts for the schema) wrapped with an optional cache TTL. Callers typically override this to pull stats from a descriptor. Returning null signals “no stats” so DuckDB falls back to the function-level table_function_statistics path.

method tableCommentSet

Section titled “method tableCommentSet”
source
tableCommentSet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  comment?: string | null,
  ignoreNotFound?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method tableRename

Section titled “method tableRename”
source
tableRename(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  newName: string,
  ignoreNotFound?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method tableColumnAdd

Section titled “method tableColumnAdd”
source
tableColumnAdd(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  columnName: string,
  columnType: string,
  defaultValue?: string,
  ignoreNotFound?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method tableColumnDrop

Section titled “method tableColumnDrop”
source
tableColumnDrop(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  columnName: string,
  ignoreNotFound?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method tableColumnRename

Section titled “method tableColumnRename”
source
tableColumnRename(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  columnName: string,
  newName: string,
  ignoreNotFound?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method tableColumnDefaultSet

Section titled “method tableColumnDefaultSet”
source
tableColumnDefaultSet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  columnName: string,
  defaultValue: string,
  ignoreNotFound?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method tableColumnDefaultDrop

Section titled “method tableColumnDefaultDrop”
source
tableColumnDefaultDrop(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  columnName: string,
  ignoreNotFound?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method tableColumnTypeChange

Section titled “method tableColumnTypeChange”
source
tableColumnTypeChange(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  columnName: string,
  newType: string,
  ignoreNotFound?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method tableNotNullSet

Section titled “method tableNotNullSet”
source
tableNotNullSet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  columnName: string,
  ignoreNotFound?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method tableNotNullDrop

Section titled “method tableNotNullDrop”
source
tableNotNullDrop(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  columnName: string,
  ignoreNotFound?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method viewGet

Section titled “method viewGet”
source
viewGet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<ViewInfo | null>

method viewCreate

Section titled “method viewCreate”
source
viewCreate(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  definition: string,
  onConflict: string,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method viewDrop

Section titled “method viewDrop”
source
viewDrop(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  ignoreNotFound?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method viewRename

Section titled “method viewRename”
source
viewRename(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  newName: string,
  ignoreNotFound?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method viewCommentSet

Section titled “method viewCommentSet”
source
viewCommentSet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  comment?: string | null,
  ignoreNotFound?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method macroGet

Section titled “method macroGet”
source
macroGet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<MacroInfo | null>

method macroCreate

Section titled “method macroCreate”
source
macroCreate(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  macroType: MacroType,
  parameters: string[],
  definition: string,
  onConflict: string,
  parameterDefaultValues?: Uint8Array | null,
  /**
   * Optional Arrow schema (one nullable field per parameter, in `parameters`
   * order) serialized as IPC bytes, carrying per-parameter descriptions via the

method macroDrop

Section titled “method macroDrop”
source
macroDrop(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  ignoreNotFound?: boolean,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<void>

method schemaContentsMacros

Section titled “method schemaContentsMacros”
source
schemaContentsMacros(
  attachOpaqueData: AttachOpaqueData,
  name: string,
  type: string,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<MacroInfo[]>

method schemaContentsIndexes

Section titled “method schemaContentsIndexes”
source
schemaContentsIndexes(
  attachOpaqueData: AttachOpaqueData,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<IndexInfo[]>

method indexGet

Section titled “method indexGet”
source
indexGet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData
): Awaitable<IndexInfo | null>

method copyFromFormats

Section titled “method copyFromFormats”
source
copyFromFormats(
  attachOpaqueData: AttachOpaqueData,
  transactionOpaqueData?: TransactionOpaqueData,
): Awaitable<CopyFromFormatInfo[]>

List custom COPY ... FROM formats advertised by this catalog. Catalog-level (not schema-scoped). The default returns an empty list, so catalogs that don’t define COPY formats are unaffected. The VGI extension treats an empty list (or a not-implemented method) as “no custom formats”. Mirrors vgi-python’s CatalogInterface.copy_from_formats.

method transactionBegin

Section titled “method transactionBegin”
source
transactionBegin(attachOpaqueData: AttachOpaqueData): Awaitable<Uint8Array | null>

method transactionCommit

Section titled “method transactionCommit”
source
transactionCommit(
  attachOpaqueData: AttachOpaqueData,
  transactionOpaqueData: TransactionOpaqueData
): Awaitable<void>

method transactionRollback

Section titled “method transactionRollback”
source
transactionRollback(
  attachOpaqueData: AttachOpaqueData,
  transactionOpaqueData: TransactionOpaqueData
): Awaitable<void>

class CompositeCatalogInterface

Section titled “class CompositeCatalogInterface”
source
export class CompositeCatalogInterface extends CatalogInterface

Methods

method catalogNameForAttach

Section titled “method catalogNameForAttach”
source
override catalogNameForAttach(attachOpaqueData: Uint8Array): string | null

Route to the owning backend and ask it, so the answer is per-attachment.

method catalogs

Section titled “method catalogs”
source
catalogs(): string[]

method catalogsInfo

Section titled “method catalogsInfo”
source
async catalogsInfo(): Promise<CatalogInfo[]>

List the catalogs this worker exposes with their version metadata. The implementation_version / data_version_spec come from the CatalogInfo returned by the read-only catalogs() signature below — a versioned worker overrides catalogs() / catalogsInfo() to advertise real values; the default derives them from the registered descriptor(s).

method attach

Section titled “method attach”
source
async attach(
  name: string,
  options?: Record<string, unknown>,
  dataVersionSpec?: string | null,
  implementationVersion?: string | null,
): Promise<CatalogAttachResult>

Attach this catalog. The framework already validates ATTACH options (name + type) against the specs the worker advertises in catalogsInfo().attach_option_specs, so options arrives as a validated, decoded {name: value} dict with typed column values. Use {} if no options were supplied.

method detach

Section titled “method detach”
source
async detach(attachOpaqueData: AttachOpaqueData): Promise<void>

method version

Section titled “method version”
source
async version(attachOpaqueData: AttachOpaqueData, transactionOpaqueData?: TransactionOpaqueData): Promise<number>

method schemas

Section titled “method schemas”
source
async schemas(attachOpaqueData: AttachOpaqueData, transactionOpaqueData?: TransactionOpaqueData): Promise<SchemaInfo[]>

method schemaGet

Section titled “method schemaGet”
source
override async schemaGet(attachOpaqueData: AttachOpaqueData, name: string, transactionOpaqueData?: TransactionOpaqueData): Promise<SchemaInfo | null>

method schemaContentsTables

Section titled “method schemaContentsTables”
source
override async schemaContentsTables(attachOpaqueData: AttachOpaqueData, name: string, transactionOpaqueData?: TransactionOpaqueData): Promise<TableInfo[]>

method schemaContentsViews

Section titled “method schemaContentsViews”
source
override async schemaContentsViews(attachOpaqueData: AttachOpaqueData, name: string, transactionOpaqueData?: TransactionOpaqueData): Promise<ViewInfo[]>

method schemaContentsFunctions

Section titled “method schemaContentsFunctions”
source
override async schemaContentsFunctions(attachOpaqueData: AttachOpaqueData, name: string, type: string, transactionOpaqueData?: TransactionOpaqueData): Promise<FunctionInfo[]>

method schemaContentsMacros

Section titled “method schemaContentsMacros”
source
override async schemaContentsMacros(attachOpaqueData: AttachOpaqueData, name: string, type: string, transactionOpaqueData?: TransactionOpaqueData): Promise<MacroInfo[]>

method schemaContentsIndexes

Section titled “method schemaContentsIndexes”
source
override async schemaContentsIndexes(attachOpaqueData: AttachOpaqueData, name: string, transactionOpaqueData?: TransactionOpaqueData): Promise<IndexInfo[]>

method indexGet

Section titled “method indexGet”
source
override async indexGet(attachOpaqueData: AttachOpaqueData, schemaName: string, name: string, transactionOpaqueData?: TransactionOpaqueData): Promise<IndexInfo | null>

method tableGet

Section titled “method tableGet”
source
override async tableGet(attachOpaqueData: AttachOpaqueData, schemaName: string, name: string, atUnit?: string, atValue?: string, transactionOpaqueData?: TransactionOpaqueData): Promise<TableInfo | null>

method tableScanFunctionGet

Section titled “method tableScanFunctionGet”
source
override async tableScanFunctionGet(attachOpaqueData: AttachOpaqueData, schemaName: string, name: string, atUnit?: string, atValue?: string, transactionOpaqueData?: TransactionOpaqueData): Promise<any>

method tableScanBranchesGet

Section titled “method tableScanBranchesGet”
source
override async tableScanBranchesGet(attachOpaqueData: AttachOpaqueData, schemaName: string, name: string, atUnit?: string, atValue?: string, transactionOpaqueData?: TransactionOpaqueData): Promise<any>

Return the scan branches for a (possibly multi-source) table. Default delegates to tableScanFunctionGet and wraps the single result as a one-branch list, so every single-source catalog is compatible with the branches-aware C++ extension. Override for genuine multi-source tables.

method tableColumnStatisticsGet

Section titled “method tableColumnStatisticsGet”
source
override async tableColumnStatisticsGet(attachOpaqueData: AttachOpaqueData, schemaName: string, name: string, transactionOpaqueData?: TransactionOpaqueData): Promise<

Return serialized column statistics for a table, or null if none are available. The result is the IPC bytes of a ColumnStatistics RecordBatch (see src/util/statistics.ts for the schema) wrapped with an optional cache TTL. Callers typically override this to pull stats from a descriptor. Returning null signals “no stats” so DuckDB falls back to the function-level table_function_statistics path.

method viewGet

Section titled “method viewGet”
source
override async viewGet(attachOpaqueData: AttachOpaqueData, schemaName: string, name: string, transactionOpaqueData?: TransactionOpaqueData): Promise<ViewInfo | null>

method macroGet

Section titled “method macroGet”
source
override async macroGet(attachOpaqueData: AttachOpaqueData, schemaName: string, name: string, transactionOpaqueData?: TransactionOpaqueData): Promise<MacroInfo | null>

method copyFromFormats

Section titled “method copyFromFormats”
source
override async copyFromFormats(attachOpaqueData: AttachOpaqueData, transactionOpaqueData?: TransactionOpaqueData): Promise<CopyFromFormatInfo[]>

List custom COPY ... FROM formats advertised by this catalog. Catalog-level (not schema-scoped). The default returns an empty list, so catalogs that don’t define COPY formats are unaffected. The VGI extension treats an empty list (or a not-implemented method) as “no custom formats”. Mirrors vgi-python’s CatalogInterface.copy_from_formats.

method transactionBegin

Section titled “method transactionBegin”
source
override async transactionBegin(attachOpaqueData: AttachOpaqueData): Promise<Uint8Array | null>

method transactionCommit

Section titled “method transactionCommit”
source
override async transactionCommit(attachOpaqueData: AttachOpaqueData, transactionOpaqueData: TransactionOpaqueData): Promise<void>

method transactionRollback

Section titled “method transactionRollback”
source
override async transactionRollback(attachOpaqueData: AttachOpaqueData, transactionOpaqueData: TransactionOpaqueData): Promise<void>

interface CopyFromFormatInfo

Section titled “interface CopyFromFormatInfo”
source
export interface CopyFromFormatInfo

Fields

tagsRecord<string, string>

Function/format tags.

format_namestring

The FORMAT identifier users type (single global namespace).

handlerstring

Registered name of the worker function that performs the read.

optionsUint8Array

Serialized Arrow schema of the format’s options (same encoding as FunctionInfo.arguments), each field carrying type / vgi_doc description.

directionstring

"from" / "to" / "both" — the COPY direction this format serves.

descriptionstring

Intrinsic documentation from the handler’s description.

orderedboolean

COPY … TO only — when true the writer requires rows in source order, so the extension uses a single-threaded sink (REGULAR_COPY_TO_FILE) instead of the default parallel sharded write. Set via Meta.sinkOrderDependent on a CopyToFunction; always false for readers. Mirrors vgi-python’s CopyFromFormatInfo.ordered.

commentstring | nulloptional

Free-text comment (CopyFromFunction’s copyFromComment).

function decodeCatalogInfo

Section titled “function decodeCatalogInfo”
source
const decodeCatalogInfo = (b: Uint8Array): CatalogInfo => decodeASD<CatalogInfo>(CatalogInfoSchema, b)

function decodeFunctionInfo

Section titled “function decodeFunctionInfo”
source
const decodeFunctionInfo = (b: Uint8Array): FunctionInfo => decodeASD<FunctionInfo>(FunctionInfoSchema, b)

function decodeMacroInfo

Section titled “function decodeMacroInfo”
source
const decodeMacroInfo = (b: Uint8Array): MacroInfo => decodeASD<MacroInfo>(MacroInfoSchema, b)

function decodeSchemaInfo

Section titled “function decodeSchemaInfo”
source
const decodeSchemaInfo = (b: Uint8Array): SchemaInfo => decodeASD<SchemaInfo>(SchemaInfoSchema, b)

function decodeTableInfo

Section titled “function decodeTableInfo”
source
const decodeTableInfo = (b: Uint8Array): TableInfo => decodeASD<TableInfo>(TableInfoSchema, b)

function decodeViewInfo

Section titled “function decodeViewInfo”
source
const decodeViewInfo = (b: Uint8Array): ViewInfo => decodeASD<ViewInfo>(ViewInfoSchema, b)

type DefaultValue

Section titled “type DefaultValue”
source
export type DefaultValue = string | number | boolean | null;

function deserializeAttachOptionSpec

Section titled “function deserializeAttachOptionSpec”
source
export function deserializeAttachOptionSpec(
bytes: Uint8Array,
): AttachOptionSpec

Description

Deserialize one AttachOptionSpec from the wire format above.

Reads by column name, so a spec written by a peer that predates the required column deserializes with required: false rather than failing — the same tolerance the Python and C++ readers have.

function deserializeAttachOptionSpecs

Section titled “function deserializeAttachOptionSpecs”
source
export function deserializeAttachOptionSpecs(
specs: Iterable<Uint8Array>,
): AttachOptionSpec[]

Description

Deserialize many specs — the shape CatalogInfo.attach_option_specs holds.

function encodeCatalogInfo

Section titled “function encodeCatalogInfo”
source
const encodeCatalogInfo = (v: CatalogInfo): Uint8Array => encodeASD(CatalogInfoSchema, v)

function encodeCopyFromFormatInfo

Section titled “function encodeCopyFromFormatInfo”
source
const encodeCopyFromFormatInfo

Description

Encode a CopyFromFormatInfo to Arrow IPC bytes (single-row batch).

function encodeFunctionInfo

Section titled “function encodeFunctionInfo”
source
const encodeFunctionInfo = (v: FunctionInfo): Uint8Array => encodeASD(FunctionInfoSchema, v)

function encodeMacroInfo

Section titled “function encodeMacroInfo”
source
const encodeMacroInfo = (v: MacroInfo): Uint8Array => encodeASD(MacroInfoSchema, v)

function encodeSchemaInfo

Section titled “function encodeSchemaInfo”
source
const encodeSchemaInfo = (v: SchemaInfo): Uint8Array => encodeASD(SchemaInfoSchema, v)

function encodeTableInfo

Section titled “function encodeTableInfo”
source
const encodeTableInfo = (v: TableInfo): Uint8Array => encodeASD(TableInfoSchema, v)

function encodeViewInfo

Section titled “function encodeViewInfo”
source
const encodeViewInfo = (v: ViewInfo): Uint8Array => encodeASD(ViewInfoSchema, v)

interface ForeignKeyDef

Section titled “interface ForeignKeyDef”
source
export interface ForeignKeyDef

Fields

columnsstring[]
referencedTablestring
referencedColumnsstring[]
referencedSchemastringoptional

interface FunctionInfo

Section titled “interface FunctionInfo”
source
export interface FunctionInfo

Fields

tagsRecord<string, string>
namestring
schema_namestring
function_typeFunctionType
argumentsUint8Array
output_schemaUint8Array
descriptionstring
examplesCatalogExample[]
categoriesstring[]
supported_expression_filtersstring[]
supports_batch_indexboolean
partition_kindPartitionKind
order_dependentOrderDependence
distinct_dependentDistinctDependence
supports_windowboolean
streaming_partitionedboolean
has_finalizeboolean
source_order_dependentboolean
sink_order_dependentboolean
requires_input_batch_indexboolean
input_from_argsboolean
required_settingsstring[]
required_secretsSecretLookupEntry[]
commentstring | nulloptional
stabilityFunctionStability | nulloptional
null_handlingNullHandling | nulloptional
projection_pushdownboolean | nulloptional
filter_pushdownboolean | nulloptional
sampling_pushdownboolean | nulloptional
late_materializationboolean | nulloptional
order_preservationOrderPreservation | nulloptional
max_workersnumber | nulloptional

type FunctionInfoOptions

Section titled “type FunctionInfoOptions”
source
export type FunctionInfoOptions = FunctionInfo;

Deprecated — Use the FunctionInfo interface (snake_case) directly.

interface MacroDescriptor

Section titled “interface MacroDescriptor”
source
export interface MacroDescriptor

Fields

namestring
macroType“scalar” | “table”
parametersstring[]
definitionstring
parameterDefaultValuesUint8Array | nulloptional
parameterDocsRecord<string, string>optional

Optional mapping of parameter name to a human/agent-facing description. Keys must appear in parameters. Descriptions flow over the wire via the macro arguments_schema’s vgi_doc field metadata (the same channel functions use for per-argument docs), so the DuckDB extension’s vgi_function_arguments() can surface them. Empty/omitted = no docs.

commentstringoptional
tagsRecord<string, string>optional

interface MacroInfo

Section titled “interface MacroInfo”
source
export interface MacroInfo

Fields

tagsRecord<string, string>
namestring
schema_namestring
macro_typeMacroType
parametersstring[]
definitionstring
commentstring | nulloptional
parameter_default_values/** @record-batch */ Uint8Array | nulloptional
arguments_schema/** @arrow-schema */ Uint8Array | nulloptional

type MacroType

Section titled “type MacroType”
source
export type MacroType = “SCALAR” | “TABLE”;

class ReadOnlyCatalogInterface

Section titled “class ReadOnlyCatalogInterface”
source
export class ReadOnlyCatalogInterface extends CatalogInterface

Methods

method catalogs

Section titled “method catalogs”
source
catalogs(): string[]

method attachOptionSpecs

Section titled “method attachOptionSpecs”
source
attachOptionSpecs(_name: string): AttachOptionSpec[]

Attach-time options this catalog declares, for the named catalog.

Empty by default — the declarative descriptor path carries no attach options. A subclass that advertises specs in catalogsInfo() should return the same specs here so attach() enforces the required ones it advertised. Takes the catalog name because one interface may serve several catalogs with different option sets.

method catalogsInfo

Section titled “method catalogsInfo”
source
override catalogsInfo(): CatalogInfo[]

List the catalogs this worker exposes with their version metadata. The implementation_version / data_version_spec come from the CatalogInfo returned by the read-only catalogs() signature below — a versioned worker overrides catalogs() / catalogsInfo() to advertise real values; the default derives them from the registered descriptor(s).

method attach

Section titled “method attach”
source
attach(
  name: string,
  options?: Record<string, unknown>,
  dataVersionSpec?: string | null,
  implementationVersion?: string | null,
): CatalogAttachResult | Promise<CatalogAttachResult>

Attach this catalog. The framework already validates ATTACH options (name + type) against the specs the worker advertises in catalogsInfo().attach_option_specs, so options arrives as a validated, decoded {name: value} dict with typed column values. Use {} if no options were supplied.

method detach

Section titled “method detach”
source
detach(attachOpaqueData: AttachOpaqueData): void

method version

Section titled “method version”
source
version(attachOpaqueData: AttachOpaqueData, transactionOpaqueData?: TransactionOpaqueData): number | Promise<number>

method schemas

Section titled “method schemas”
source
schemas(attachOpaqueData: AttachOpaqueData, transactionOpaqueData?: TransactionOpaqueData): SchemaInfo[]

method schemaGet

Section titled “method schemaGet”
source
override schemaGet(
  attachOpaqueData: AttachOpaqueData,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData
): SchemaInfo | null

method schemaContentsTables

Section titled “method schemaContentsTables”
source
override async schemaContentsTables(
  attachOpaqueData: AttachOpaqueData,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData
): Promise<TableInfo[]>

method tableColumnStatisticsGet

Section titled “method tableColumnStatisticsGet”
source
override tableColumnStatisticsGet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData,
):

Return serialized column statistics for a table, or null if none are available. The result is the IPC bytes of a ColumnStatistics RecordBatch (see src/util/statistics.ts for the schema) wrapped with an optional cache TTL. Callers typically override this to pull stats from a descriptor. Returning null signals “no stats” so DuckDB falls back to the function-level table_function_statistics path.

method schemaContentsViews

Section titled “method schemaContentsViews”
source
override schemaContentsViews(
  attachOpaqueData: AttachOpaqueData,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData
): ViewInfo[]

method schemaContentsFunctions

Section titled “method schemaContentsFunctions”
source
override schemaContentsFunctions(
  attachOpaqueData: AttachOpaqueData,
  name: string,
  type: string,
  transactionOpaqueData?: TransactionOpaqueData
): FunctionInfo[]

method schemaContentsMacros

Section titled “method schemaContentsMacros”
source
override schemaContentsMacros(
  attachOpaqueData: AttachOpaqueData,
  name: string,
  type: string,
  transactionOpaqueData?: TransactionOpaqueData
): MacroInfo[]

method schemaContentsIndexes

Section titled “method schemaContentsIndexes”
source
override schemaContentsIndexes(
  attachOpaqueData: AttachOpaqueData,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData
): IndexInfo[]

method indexGet

Section titled “method indexGet”
source
override indexGet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData
): IndexInfo | null

method macroGet

Section titled “method macroGet”
source
override macroGet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData
): MacroInfo | null

method tableScanFunctionGet

Section titled “method tableScanFunctionGet”
source
override tableScanFunctionGet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  atUnit?: string,
  atValue?: string,
  transactionOpaqueData?: TransactionOpaqueData
): any

method tableGet

Section titled “method tableGet”
source
override async tableGet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  atUnit?: string,
  atValue?: string,
  transactionOpaqueData?: TransactionOpaqueData
): Promise<TableInfo | null>

method viewGet

Section titled “method viewGet”
source
override viewGet(
  attachOpaqueData: AttachOpaqueData,
  schemaName: string,
  name: string,
  transactionOpaqueData?: TransactionOpaqueData
): ViewInfo | null

method copyFromFormats

Section titled “method copyFromFormats”
source
override copyFromFormats(
  attachOpaqueData: AttachOpaqueData,
  transactionOpaqueData?: TransactionOpaqueData,
): CopyFromFormatInfo[]

List custom COPY ... FROM formats advertised by this catalog. Catalog-level (not schema-scoped). The default returns an empty list, so catalogs that don’t define COPY formats are unaffected. The VGI extension treats an empty list (or a not-implemented method) as “no custom formats”. Mirrors vgi-python’s CatalogInterface.copy_from_formats.

interface ScanBranchInput

Section titled “interface ScanBranchInput”
source
export interface ScanBranchInput

Description

One physical source backing a multi-branch scan, in the shape the buildScanBranchesResult helper consumes. Mirrors vgi-python’s ScanBranch (catalog_interface.py).

Each scalar argument carries an explicit Arrow type so the nested arguments batch is built with the wire type the C++ binder expects (e.g. utf8 for file paths, int64 for counts).

Fields

functionNamestring

DuckDB function to call for this branch (e.g. “sequence”, “read_parquet”).

positionalArguments{ value: unknown; type: VgiDataType }[]optional

Positional scalar arguments, each with its Arrow type.

namedArgumentsRecord<string, { value: unknown; type: VgiDataType }>optional

Named scalar arguments, each with its Arrow type.

branchFilterstring | nulloptional

Optional SQL filter text AND’d into every scan of this branch.

writablebooleanoptional

Declares this branch as the INSERT target (at most one per table).

sourceCatalogstring | nulloptional

Catalog-table branch (lakehouse federation): leave functionName empty and set these to scan the base table sourceCatalog.sourceSchema.sourceTable in a companion catalog instead of calling a table function.

sourceSchemastring | nulloptional
sourceTablestring | nulloptional

interface SchemaDescriptor

Section titled “interface SchemaDescriptor”
source
export interface SchemaDescriptor

Fields

namestring
tablesTableDescriptor[]optional
viewsViewDescriptor[]optional
macrosMacroDescriptor[]optional
functionsVgiFunction[]optional
indexesIndexDescriptor[]optional
commentstringoptional
tagsRecord<string, string>optional

interface SchemaInfo

Section titled “interface SchemaInfo”
source
export interface SchemaInfo

Fields

tagsRecord<string, string>
attach_opaque_dataUint8Array
namestring
commentstring | nulloptional
estimated_object_countRecord<string, number> | nulloptional

interface SecretTypeDescriptor

Section titled “interface SecretTypeDescriptor”
source
export interface SecretTypeDescriptor

Fields

namestring
descriptionstring
schemaVgiSchema

function serializeAttachOptionSpec

Section titled “function serializeAttachOptionSpec”
source
export function serializeAttachOptionSpec(spec: AttachOptionSpec): Uint8Array

Description

Serialize an AttachOptionSpec to the wire format the extension expects (one IPC-serialized RecordBatch with a single row).

function serializeAttachOptionSpecs

Section titled “function serializeAttachOptionSpecs”
source
export function serializeAttachOptionSpecs(
specs: Iterable<AttachOptionSpec>,
): Uint8Array[]

Description

Convenience: serialize many specs at once for CatalogInfo.attach_option_specs.

interface SettingDescriptor

Section titled “interface SettingDescriptor”
source
export interface SettingDescriptor

Fields

namestring
descriptionstring
typeVgiDataType
defaultValuestring | number | bigint | booleanoptional

Must be a JS primitive compatible with type (string, number, bigint, boolean).

interface TableDescriptor

Section titled “interface TableDescriptor”
source
export interface TableDescriptor

Fields

namestring
columnsVgiSchemaoptional
functionVgiFunctionoptional
argumentsArgumentsoptional
notNullstring[]optional
uniquestring[][]optional
checkstring[]optional
primaryKeystring[][]optional
foreignKeyForeignKeyDef[]optional
defaultsRecord<string, DefaultValue>optional
columnCommentsRecord<string, string>optional

Per-column comment strings, applied as field metadata comment.

generatedColumnsRecord<string, string>optional

Generated (virtual) columns: map of column name → SQL expression computed from other physical columns. Applied as Arrow field metadata generated_expression, which the DuckDB VGI extension reads at table registration. The backing scan function should not return these columns — DuckDB evaluates the expressions client-side.

statisticsRecord<string, ColumnStatistics>optional

Per-column statistics for the optimizer. Keys are column names; values are ColumnStatistics records with typed min/max and Arrow type info. DuckDB uses these for plan-time filter elimination and join reordering via the catalog_table_column_statistics_get RPC.

statisticsCacheMaxAgeSecondsnumber | nulloptional

Cache TTL for this table’s column statistics. DuckDB caches the result of catalog_table_column_statistics_get for up to this many seconds. undefined/null means cache indefinitely; 0 means never cache (always re-fetch).

requiredFiltersstring[][]optional

Required WHERE-filter groups in conjunctive normal form — an AND (outer list) of OR-groups (inner lists) of dotted-path column references that MUST appear in a WHERE expression for any scan of this table. A group is satisfied when any one of its paths has a filter; every group must be satisfied. So [["accession_number"], ["ticker", "cik"]] means “accession_number AND one of (ticker, cik)”; a single-path group [["country"]] is a plain mandatory filter. Paths are top-level names ("country") or struct subfields ("bbox.xmin", "nested.outer.inner"). Empty/undefined (default) means no enforcement — the zero-cost fast path for every existing table.

Satisfaction is prefix-based: a present filter on a shorter path satisfies any required path it is a prefix of. So a whole-struct filter on bbox satisfies all of bbox.xmin / .xmax / .ymin / .ymax. The VGI DuckDB extension’s optimizer pass consults this at bind time and throws a BinderException listing any unsatisfied groups.

supportsTimeTravelbooleanoptional
inlinedCardinality{ estimate: bigint; max: bigint }optional

Inline cardinality estimate/max surfaced through TableInfo. When set, the C++ extension uses these directly and skips the per-bind table_function_cardinality RPC. Only meaningful for function-backed tables; static-column tables don’t have a function to call.

commentstringoptional
tagsRecord<string, string>optional

interface TableInfo

Section titled “interface TableInfo”
source
export interface TableInfo

Fields

tagsRecord<string, string>
namestring
schema_namestring
columnsUint8Array
not_null_constraintsnumber[]
unique_constraintsnumber[][]
check_constraintsstring[]
primary_key_constraintsnumber[][]
foreign_key_constraintsUint8Array[]
supports_insertboolean
supports_updateboolean
supports_deleteboolean
supports_returningboolean
supports_column_statisticsboolean
required_filtersstring[][]
commentstring | nulloptional
scan_functionUint8Array | nulloptional
insert_functionUint8Array | nulloptional
update_functionUint8Array | nulloptional
delete_functionUint8Array | nulloptional
cardinality_estimatenumber | nulloptional
cardinality_maxnumber | nulloptional
column_statisticsUint8Array | nulloptional
bind_resultUint8Array | nulloptional

type TransactionOpaqueData

Section titled “type TransactionOpaqueData”
source
export type TransactionOpaqueData = Uint8Array;

interface ViewDescriptor

Section titled “interface ViewDescriptor”
source
export interface ViewDescriptor

Fields

namestring
definitionstring
commentstringoptional
columnCommentsRecord<string, string>optional

Per-column comments keyed by the view’s output column name. Unlike tables (whose column comments ride along as Arrow field metadata), a view ships only its SQL definition, so column comments need their own channel. The C++ extension aligns these by name against the bound output columns and feeds them into CreateViewInfo.column_comments_map; names that don’t match a bound column are ignored.

tagsRecord<string, string>optional

interface ViewInfo

Section titled “interface ViewInfo”
source
export interface ViewInfo

Fields

tagsRecord<string, string>
namestring
schema_namestring
definitionstring
column_commentsRecord<string, string>
commentstring | nulloptional