AN OPEN STANDARD FOR LAKEHOUSE-NATIVE STREAMING

The stream is the table.

Lakestream is an open architecture pattern and API for streaming on the lakehouse: logs live as open formats on object storage — readable as a stream, queryable as a table. One copy of data, nothing in between.

READ THE SPEC GET STARTED WITH URSA

WRITE — A STREAM

append-ordered records · WAL objects

ONE COPY

OBJECT STORAGE

READ — A TABLE

columnar Parquet · Iceberg / Delta

topic → WAL objects → compacted Parquet → committed tables

PRINCIPLES

Three properties, one storage layer.

STREAM–TABLE DUALITY

Two views, same bytes

Every log is simultaneously a stream you can tail and a table you can query. The record that serves a tail read is the record committed to the table.

DISKLESS & LEADERLESS

Brokers without state

Durability moves to object storage. Any node serves any partition — no leader elections, no rebalancing storms, no cross-AZ replication bill.

ZERO-ETL

Tables appear, connectors don't

Ingestion writes open formats directly: WAL objects compact into Parquet and commit to Iceberg or Delta. There is no pipeline to build or babysit.

ARCHITECTURE

From protocol to Parquet.

PROTOCOL LAYER

KafkaPulsarmore soon

LAKESTREAM API — THE CONTRACT

StreamCatalogLogLogCursorMaterialization

URSA STORAGE ENGINE

WAL objectsCompactionOxia metadata

OBJECT STORAGE & LAKEHOUSE

S3GCSAzureIcebergDelta

The API is the contract — everything below it is replaceable.

THE SPECIFICATION

One API, three levels.

The Lakestream API is small on purpose: three levels, each usable on its own. An implementation that honors the levels — and the storage contract beneath them — is a conformant lakestream.

LEVEL 2Stream Catalog — streams, layouts, readers and writers

LEVEL 1Log Storage — logs as durable, addressable primitives

LEVEL 0Log & Cursor — append, read, acknowledge

BROWSE THE FULL SPEC →

lakestream-api · io.streamnative.lakestream.api

StreamCatalog catalog = StreamCatalog.open(conf);

Stream orders = catalog.stream("orders");

orders.newWriter().append(record);

// arrives as a stream, instantly tailable

orders.table().scan();

// the same records, as a lakehouse table

IMPLEMENTATIONS