Diskless & leaderless

Diskless and leaderless describe the serving data path. They do not mean that the entire system has no state, no ownership, or no consensus.

Diskless: no broker-local durable partition replicas

In a conventional replicated broker, partition leaders and followers store local durable logs. In Ursa's shared-storage deployment, brokers append through object-backed WAL storage and read retained data from shared files. A replacement broker does not need the previous broker's local log directory.

Brokers still hold write buffers, read caches, open handles, and in-memory producer state. These affect throughput and recovery time, but they are not a partition's only durable copy. The configured storage backend and Oxia remain durability and availability dependencies.

Leaderless: ordering does not require broker replica consensus

Ordinary Ursa appends obtain offset ranges through Oxia sequence-key operations after WAL persistence. Brokers do not need to elect a data-replica leader to allocate those offsets or copy the payload to Kafka followers.

This removes broker-level replica coordination from the diskless data path. It does not remove coordination inside Oxia, the object store's own redundancy, or Kafka's controller and metadata machinery.

How does Ursa for Apache Kafka (UFK) route requests?

Kafka clients still expect partition metadata containing a leader. The diskless integration supplies a selected serving broker in that protocol field:

• DisklessBrokerSelector hashes the topic ID and partition index over a sorted list of eligible live brokers.
• Selection can be zone-aware; when a requested zone has no eligible broker, selection falls back to the unscoped broker set.
• The metadata transformer advertises the selected broker as leader and as the single replica and ISR member.
• Request handling checks current ownership and routes or redirects work accordingly. Different zones can select different serving brokers for the same partition.

Thus "any broker can access shared data" does not mean "every request is accepted anywhere without routing." Diskless topics use replication factor 1 at the Kafka level; that is not a statement that the storage backend has only one physical copy.

Recovery and fencing

When membership changes, selection can change without moving partition files. Clients and brokers still need current metadata, initialized readers and writers, and reachable shared storage.

Idempotent-producer recovery is also real work. The Kafka integration keeps producer state in memory, persists snapshots in Oxia, and can replay the log tail after a snapshot. Version-conditional snapshot writes protect snapshot ownership. This is not a promise of zero replay or instantaneous failover.

Keep three safety mechanisms distinct:

• Log fencing rejects writes through a fenced log state.
• Lifecycle fencing and leases protect stream deletion and storage cleanup while handles are active.
• Kafka routing and producer-state ownership govern which serving path should process a request and update its producer snapshot.

A log fence is not a substitute for all distributed ownership checks. In particular, catalog-opened write leases are per holder, not an exclusive single-writer lease for the entire log.

Cost and operational boundaries

Diskless storage removes Kafka follower-payload replication for diskless topics and avoids copying local partition logs during reassignment. It can decouple serving capacity from retained data volume.

It does not eliminate storage requests, metadata traffic, cross-zone access, background materialization, cache warming, or recovery capacity. Those costs depend on placement, workload, and backend choices; "leaderless" alone is not a cost guarantee.