Zero-ETL

Zero-ETL means tables appear from ingestion directly, with no connectors and no staging pipeline in between.

The pipeline this removes

The conventional path from a stream to a lakehouse table runs through ETL — extract, transform, load — performed by a connector or job that reads the stream, buffers and transforms records, and writes them into table format, usually on its own deployed schedule, independent of how fast the stream itself moves. Every stage in that path is a place data can lag behind the source, get duplicated, or drift from it — see why Lakestream for how that cost adds up.

How tables appear directly

Ingestion writes the open format directly. Records append to the log as row-oriented WAL objects, and the compaction service folds them into compacted objects — usually columnar Parquet — on its normal schedule, the same write path described in stream–table duality, not an additional one. Committing a compacted object (Parquet only) includes registering it with the destination table's catalog, Iceberg or Delta Lake, so the table gains that data the moment compaction commits it. No connector reads the stream a second time, and no staging area holds a copy in between — and this holds regardless of which protocol wrote the data, since a Kafka-compatible write path and a Pulsar-compatible write path both append through the same kind of log.

Because there is one copy with one write path, governance and lineage apply to that copy directly: access control is set once on the log and its table, and every row in the table traces back to the same offsets a stream consumer would read, rather than to whatever a separate pipeline job happened to produce on its own schedule.

What still needs configuration

Zero-ETL removes the pipeline, not the decisions a pipeline would otherwise encode. What is left is configuration, applied once per log rather than re-run on every record:

• Schema — set once and evolved through compatibility modes, since the schema determines how records materialize into the destination table's columns.
• Retention — how long WAL objects and compacted objects are kept before deletion.
• Table mode and catalog — whether the destination is an internal or external table, and which Iceberg or Delta Lake catalog owns it (see lakehouse tables).