Skip to content
Query.Farm
Talk with Us

Add a custom COPY format

A custom reader is an ITableFunction; a custom writer is an ITableBufferingFunction because it must sink the input before its source phase completes.

worker
  .RegisterCopyFromFormat(new LinesReader(), "demo_lines")
  .RegisterCopyToFormat(new LinesWriter(), "demo_lines");

The framework registers each handler as a normal function and publishes the format metadata. TableBindParams.CopyFrom / TableInitParams.CopyFrom carry the source path and required schema; the buffering bind, process, and combine params carry the COPY TO destination context.

COPY target FROM 'input.demo' (FORMAT 'demo.demo_lines');
COPY (SELECT * FROM target) TO 'output.demo' (FORMAT 'demo.demo_lines');

Declare format options in the handler’s ArgumentsSchema. Set SinkOrderDependent only when the output format needs source row order; doing so disables parallel sink ingest.