Diskless & leaderless
Why brokers keep no state, and what that does to failover and cost.
Lakestream brokers are diskless and leaderless: they keep no local state, which changes how failover and cost work.
No local state
In a conventional broker, each partition is owned by a leader node that holds it on local or attached disk, and follower nodes replicate it so the system can survive that node's failure. In the pattern, serving nodes hold no local partition state at all: every durable record lives in object storage, as a WAL object or a compacted object, not on the node that first received it. Durability is the storage layer's job, not the serving node's.
Because no node holds a partition's only copy, there is nothing that needs a dedicated leader to protect it.
Failover without elections
Since every node can read the same object storage, any node can serve any partition — which node currently handles a partition is a metadata assignment, not a data placement decision. When a node fails, recovering its partitions means reassigning them to a different node, not re-replicating their data onto one: the replacement node reads what is already durable in object storage and resumes from the last known offset.
The Stream Catalog Service still exists as the source of truth for offset assignment — the pattern is leaderless for serving nodes, not metadata-free. But reassignment does not require the leader elections or rebalancing that broker peers running consensus over partition data would trigger: there is no quorum of replicas that has to agree before serving resumes. It does still fence the log itself, so the storage layer rejects writes from a node that hasn't yet noticed it lost a partition, even after a new node takes over.
Cost implications
Conventional replication multiplies both storage and network cost: each replica is a full copy on its own disk, and keeping replicas in sync across availability zones means paying cross-AZ transfer for every write. When object storage holds the copy of record, that replication traffic disappears as a broker-level cost — the object store's own redundancy is what makes the data durable, and serving nodes never ship the same bytes to each other to keep a replica in sync. No leader elections or rebalancing also means no spare capacity held in reserve for them, and no operational time spent tuning them.
Ursa for Kafka, a diskless Kafka implementation, is a concrete example: diskless topics run at replication factor 1 at the Kafka protocol level, because durability comes from the storage layer instead of from Kafka-level replica acknowledgment. See diskless architecture for how a produce or fetch request is served under that model.