What is Lakestream
An open standard for lakehouse-native streaming: one copy of data, readable as a stream and queryable as a table.
Lakestream is an open standard for lakehouse-native streaming, where a single copy of data is readable as a stream and queryable as a table.
Naming convention
Capitalized "Lakestream" names the standard. Lowercase "a lakestream" names any system that conforms to it — the distinction matters throughout this documentation.
The problem
Streaming systems and lakehouses have grown up as two separate storage stacks. A broker such as Kafka or Pulsar holds the hot, ordered log on attached disks or a replicated log service. A lakehouse built on Iceberg or Delta Lake holds the same events again, written there by a connector, as Parquet files on object storage.
The result is two copies of the same data, two consistency models, and a pipeline in between whose only job is keeping them in sync. That pipeline — not the data itself — is what Lakestream removes. See why Lakestream for how that cost and complexity adds up.
The core model
Lakestream defines the log as something stored once, in open formats, on object storage. That single copy is readable as a stream, in offset order, for low-latency consumption, and queryable as a table, through SQL and any Iceberg- or Delta Lake-compatible engine, with no conversion step between the two.
The standard has three parts:
- The pattern — the concepts in this section: what a log is, how it's stored once, and how streaming and table access both read from it. This is the part that stays stable across implementations.
- The API — the specification that turns the pattern into a concrete contract: catalog operations, storage formats, and conformance requirements an implementation has to meet.
- The implementations — systems that follow the pattern and speak the API. Ursa is the reference implementation; Ursa for Kafka is a Kafka-compatible broker built on it.
Three principles hold the pattern together:
- Stream–table duality — the same bytes serve a tailing stream read and a table scan; there's no second copy to keep in sync.
- Diskless & leaderless — serving nodes keep no local partition state, so failover is reassignment rather than re-replication.
- Zero-ETL — ingestion writes the open format directly, so tables appear from the same write path instead of a downstream pipeline.
Where to go next
- Read the specification for the concrete API and storage contract.
- See Ursa for the reference implementation and how it maps the pattern onto object storage.
- See Ursa for Kafka for a Kafka-compatible broker built on the pattern.