Log storage (Levels 0–1)

Levels 0 and 1 define the durable primitives: logs, cursors, and log storage — what Level 2 is built from.

Level 1: log storage

LogStorage is the lowest addressable surface: append, read, offset queries, trim, and delete, all keyed by LogId — a typed wrapper around the underlying numeric log identity — with no notion of streams or namespaces above it.

public interface LogStorage extends Closeable {
    CompletableFuture<LogEntryHeader> append(LogId logId, int numberOfRecords, ByteBuf data);

    CompletableFuture<List<LogEntry>> readEntries(LogId logId, long startOffset,
                                                   int maxMessageCount, long maxSizeBytes);

    CompletableFuture<LogOffset> getFirstOffset(LogId logId);
    CompletableFuture<LogOffset> getLastOffset(LogId logId);
}

An entry is the storage-level unit append writes: one call, batching one or more records, with the entry's offset equal to its first record's offset — an entry with n records at offset o spans offsets o through o+n (exclusive). LogOffset is the lighter descriptor getFirstOffset/getLastOffset return: offset, record count, timestamp, entry size, and cumulative size, without the payload. Beyond the four calls above, LogStorage also supports index-based reads — readIndexRange and readEntriesByIndex work against pre-fetched EntryIndex objects instead of re-resolving an offset lookup per call, which is what a cursor holding cached indexes uses to avoid repeating that lookup on every read.

Deletion at this level is explicit about how final it is: softTrim marks entries deleted up to a given offset without necessarily removing them from storage, hardTrim physically removes entries before an offset, and deleteLog removes the log entirely. Retention enforcement composes these primitives rather than being one of them: Log separately exposes a helper to compute a retention boundary (computeRetentionTrimOffset), which a caller then feeds into softTrim or hardTrim.

Level 0: log and cursor

Log is the per-log handle a stream hands out (Stream.getLog) — the unit Ursa's Pulsar-compatible ManagedLedger wraps. It layers entry-index caching, retention computation, binary search over entry headers, and cursor management on top of what LogStorage provides:

public interface Log extends AutoCloseable {
    LogId id();

    CompletableFuture<LogEntryHeader> append(int numberOfRecords, ByteBuf data);

    CompletableFuture<LogEntry> readEntry(long offset);

    void fence();
    void activate();
}

LogCursor, obtained from Log.openCursor, tracks a consumer's read position and acknowledgment state against one log:

public interface LogCursor extends AutoCloseable {
    long readOffset();
    long markDeleteOffset();

    CompletableFuture<List<LogEntry>> readEntries(int maxEntries, long maxSizeBytes);

    CompletableFuture<Void> markDelete(long offset, Map<String, Long> properties);

    CompletableFuture<Void> seek(long offset);
}

readEntries reads from the current offset and advances it; markDelete acknowledges everything up to and including a given offset, moving the mark-delete watermark forward. Several other LogCursor operations — individual out-of-order acknowledgment, state persistence, backlog statistics — are default methods with no-op fallbacks, so an implementation only has to back the ones it actually needs.

Fencing

LogStateManager tracks whether a log is NORMAL or FENCED; Log.fence() moves it to FENCED, after which appends fail, and Log.activate() reverses it. This is the storage layer's single-writer safety mechanism — see diskless & leaderless for why a diskless, leaderless design needs it: a previous writer that hasn't yet noticed a handoff can't keep appending once the log is fenced, regardless of when it finds out.

Nothing on this page knows what a stream is. Stream Catalog is what turns a RoutingKey into a LogId and calls down into Log or LogStorage — everything above that routing decision is Level 2's job, not this one's.