Ursa for Kafka

Ursa for Apache Kafka (UFK) is a Kafka distribution built on the Lakestream API and specification. Diskless topics keep their records on object storage through Ursa; every other topic stays Kafka. It is built on Apache Kafka 4.3.1.

A diskless topic bypasses Kafka's local partition log and uses shared object storage from the first write. Classic topics keep Kafka's local-log path in the same cluster.

UFK uses the Lakestream StreamCatalog, Log, and LogCursor APIs. Oxia holds catalog and storage metadata, plus snapshots of Kafka's idempotent-producer state. Kafka clients continue to use the Kafka wire protocol; this does not imply support for every Kafka feature on diskless topics.

What changes for a diskless topic?

• Shared message storage. Serving a partition does not require copying its message log to a replacement broker.
• No Kafka follower replication for its records. Diskless topics use replication factor 1. Durability depends on Ursa, the selected storage backend, and Oxia rather than Kafka ISR replication.
• Zone-aware serving. Brokers select a serving owner from live brokers, optionally within the client's zone. The owner appears as the leader in Kafka metadata; it is not the classic partition's elected data leader.
• Stream–table duality. A standalone compactor writes compacted objects that Kafka can still read. With external materialization configured, it also writes an Iceberg table for analytics.

Failover still takes failure detection, metadata refresh, retries, and potentially producer-state recovery. It is not a zero-latency guarantee. Brokers also still need local storage for KRaft metadata, internal topics, and any classic topics.

How does this differ from tiered storage?

Kafka's KIP-405 tiered storage moves older log segments to remote storage while retaining a local-log write path. For a diskless topic, Ursa is the primary store, not a destination for cold local segments.

Turning it on

Enable ursa.storage.enable=true in the cluster configuration, then create a topic with --config ursa.storage.enable=true and --replication-factor 1. Both levels matter: topic creation is rejected when the diskless storage system is disabled. See configuration for the storage endpoints, runtime, and topic defaults.

Version compatibility

UFK's source is a fork of Apache Kafka, maintained at openlakestream/kafka. Release tags add a fourth number to the Apache Kafka release they are built on: v4.3.1.3 is built on Apache Kafka 4.3.1. This section covers the 4.3-ursa branch, which builds as Kafka 4.3.2-SNAPSHOT, and its v4.3.1.3 release tag, published to Docker Hub as lakestream/kafka:4.3.1.3 with Ursa 1.0.0 bundled; lakestream/kafka:latest tracks the newest release. Each release tag also publishes lakestream/kafka-strimzi for clusters managed by the Strimzi operator. UFK resolves org.openlakestream artifacts from Maven Central: lakestream-api at compile time and ursa-storage-kafka-runtime at runtime, pinned together through one ursaStorageVersion property. Keep these dependencies aligned. UFK carries the Apache-2.0 license.

Where next

• Quickstart — pull the image, run the cluster, and try the Iceberg demo.
• Kafka compatibility — which clients, groups and producers work on a diskless topic.
• Diskless architecture — routing, offsets, owner selection, and retention.
• Deployment — Kubernetes with the Strimzi cluster operator.
• Operating the compactor — why it always runs, and how it scales.
• Monitoring — the metrics and log lines a diskless cluster reports.
• Configuration — broker and topic settings checked against code.
• Limitations — transactions, fetch behavior, and operational boundaries.
• Build from source — build, test and package UFK yourself.