Build from source

Build, test and package Ursa for Kafka from the openlakestream/kafka repository.

Released images are on Docker Hub and the Ursa runtime resolves from Maven Central, so running Ursa for Apache Kafka (UFK) needs no source build — the quickstart pulls an image. Build from source to change UFK, run its tests, or produce an image from a commit of your own.

Prerequisites

JDK 17 or later, and a running Docker daemon
Git

Gradle comes with the repository as the ./gradlew wrapper. The first build downloads the Ursa runtime, org.openlakestream:ursa-storage, from Maven Central; no repository credentials are needed.

git clone --branch 4.3-ursa https://github.com/openlakestream/kafka.git
cd kafka
./gradlew jar

./gradlew jar builds every module and skips tests.

Cloning on Windows

CLAUDE.md in the repository is a symbolic link. Clone with git clone -c core.symlinks=true and note that creating symlinks also needs Developer Mode or administrator rights.

Compile the diskless modules

For faster feedback while changing the diskless path, compile only the modules that carry it:

./gradlew :clients:compileJava :core:compileScala :storage:compileJava \
  :storage:storage-diskless-api:compileJava \
  :storage:storage-diskless-ursa:compileJava :server:compileJava

storage-diskless-api holds the storage-mode interface and the broker-side helpers; storage-diskless-ursa holds the Lakestream implementation that loads in an isolated classloader.

Run tests

The diskless integration tests are kept apart from the rest of the suite. They live under org/apache/kafka/server/ursa/integration/ and org/apache/kafka/storage/diskless/, plus DisklessTopicDefaultsTest. Many start Oxia and an S3 emulator in containers through Testcontainers, so Docker has to be running.

# Only the diskless integration tests
./gradlew test -Pkafka.ci.isolated.tests=only

# Everything else
./gradlew test -Pkafka.ci.isolated.tests=exclude

# A single test class, or a single method
./gradlew :storage:storage-diskless-ursa:test \
  --tests "org.apache.kafka.storage.diskless.handlers.UrsaStorageStateTest"

A plain ./gradlew test runs both groups and takes hours. Running a module's tests also runs checkstyle and SpotBugs on that module.

Before opening a pull request:

./gradlew spotlessCheck    # import order; fix with ./gradlew spotlessApply
./gradlew rat              # license headers

`rat` is quiet in a worktree

./gradlew rat checks only files that git tracks, so git add new files first. In a git worktree it checks nothing at all — run it from a regular clone.

Package a distribution and images

./gradlew releaseTarGz

The tarball carries the Ursa runtime jars in ursa-storage/, which is where a broker loads them from when ursa.storage.class.path is unset.

Two scripts turn that tarball into images. Both run ./gradlew releaseTarGz for you unless --tarball points at an existing one.

# Brokers, Kafka CLI tools and the standalone Ursa compactor
docker/examples/docker-compose-files/cluster/ursa/build-image.sh

# The same distribution in Strimzi's image layout
docker/strimzi/build-image.sh
docker/strimzi/verify-image.sh lakestream/kafka-strimzi:latest

Option	Applies to	Effect
`--tarball <path>`	both	Use this release tarball instead of building one
`--platform <platform>`	both	Target platform, for example `linux/arm64`
`--amd64`	`cluster/ursa`	Alias for `--platform linux/amd64`
`--push`	both	Build with buildx and push; the image is not loaded locally
`--tag <name:tag>`	both	Add another tag; repeatable

GRADLE_ARGS passes extra arguments to the release build. For the Strimzi image, STRIMZI_VERSION and STRIMZI_KAFKA_VERSION choose the quay.io/strimzi/kafka base image, and the default image name is lakestream/kafka-strimzi:<STRIMZI_VERSION>-kafka-<tarball version>.

The broker image resolves the compactor's classpath from org.openlakestream:ursa-storage-compact on Maven Central at the Ursa version the tarball ships. There is no separate compactor image and no local Maven install step. Getting a locally modified Ursa into an image is described on Ursa build from source.

What CI runs

Job	When	What it runs
Validate	Every pull request	`./gradlew check releaseTarGz -x test`, then the `LICENSE-binary` check and a Docker Compose smoke test
Isolated tests	Every pull request that is not a draft	The diskless integration tests, on JDK 17
Full test suite	Pushes to `4.3-ursa`, weekends, and pull requests labelled `run-junit-tests`	The whole Apache Kafka suite, on JDK 17 and 25

Pushing a v* tag builds and publishes both images for linux/amd64 and linux/arm64.

Next steps

Quickstart — run the image you just built through the Compose stack.
Deployment — the Strimzi image on Kubernetes.
Contributing — how changes are proposed and reviewed.
Source: openlakestream/kafka