ROADMAP
Roadmap
LIPs in progress, and ideas the maintainers and the LIPs have written down. An idea isn't scheduled work, and it may not have a LIP yet.
In progress
Lakestream Improvement Proposals that are proposed, accepted or implemented, and not yet released.
No LIP is in progress right now.
All LIPs →Ideas
To pick one up, start a thread in the project's Discussions: Ideas. A LIP follows if it changes an interface.
Ursa for Apache Kafka (UFK)
Transactions on diskless topics
Diskless topics reject transactional produce requests today.
Cross-region replication
Replicate diskless topics across regions through Ursa.
Lakehouse tables for classic topics
Ingest classic topics into lakehouse tables through a separate, StreamCatalog-based source reader.
Stream materialization framework
Apply the remaining policy fields
Error handling, commit retries, start position and pausing are stored with a materialization policy but not applied yet.
Dead-letter handling and metrics
The framework has no dead-letter handling or metrics of its own yet.
An admin surface for policies
Policies are set through the StreamCatalog API only; a REST or command-line surface is follow-up work.
Stream-to-stream materialization
Materialize a stream into another stream instead of a table.
One stream to many tables
A stream maps to one table today; the policy record can become a list without breaking changes.
Continuous materialization
Materialization runs as pull-based batches today; a push-based driver was out of scope for the first version.
Per-backend options
Options specific to Iceberg, Delta or ClickHouse tables, each proposed in its own LIP.
Build on Lakestream
An engine for another protocol
Serve another messaging protocol from the same storage, the way UFK serves Kafka.
The Lakestream API in another language
Implement the Lakestream API in a language other than Java.
A materializer for another system
Materialize streams into a system Ursa doesn't support yet. Every new materializer needs a LIP.
Further out
Table-to-stream (change capture)
Capture a lakehouse table's changes as a stream: materialization in the other direction.