Table catalogs

A table catalog is a registered, named destination a stream can materialize into. Registering one is a separate step from pointing a stream at it: catalogs are declared once for the deployment, and each stream or namespace then selects one by name.

For the settings that shape the table once the destination is chosen, see lakehouse and table settings.

Catalog types

NONE is not a missing value. It selects a materialization that produces internal compacted objects and nothing else — there is no materializer for it, and the dispatch path builds only the compacted-object writer.

Declaring catalogs

Catalogs are declared as property groups. Everything under a prefix with the same <name> segment becomes one catalog of that name, and the keys are passed to the backend with the prefix stripped.

iceberg.catalog.<name>.<key>=<value>
delta.catalog.<name>.<key>=<value>
clickhouse.catalog.<name>.<key>=<value>

Registration runs once at startup and is idempotent. It is also tolerant: a group that fails to register is logged with the count of registered, skipped and errored groups, and startup continues. Check that line in the log after changing catalog configuration — a typo in a prefix produces a catalog that silently does not exist rather than an error.

Iceberg

iceberg.catalog.analytics.type=rest
iceberg.catalog.analytics.uri=https://catalog.example.com/api/catalog
iceberg.catalog.analytics.warehouse=analytics-warehouse

The keys are the Iceberg catalog's own properties. Which backend is used is decided by type or catalog-impl, and catalog-backend selects among the implementations packaged with Ursa:

HADOOP, TABULAR, NESSIE, POLARIS, S3TABLE, BIGQUERY, BIGLAKE, UNITYCATALOG, HORIZON

When neither type nor catalog-impl is set, the backend defaults to hadoop, and the warehouse falls back to storagePath.

Catalog operations are retried on transient authorization failures, bounded by catalogOpsRetryMaxAttempts and catalogOpsRetryDelayMs.

Delta Lake

delta.catalog.archive.warehouse=s3a://my-bucket/delta

A Delta group is registered as DELTA_UC rather than DELTA when it looks like Unity Catalog: the name is literally unity, or any key in the group begins unity-catalog- or unity_catalog_, or the group sets catalog-impl.

Unity Catalog

Unity Catalog can be declared either as a delta.catalog.unity.* group, or through the flat family:

unityCatalogUri=https://example.cloud.databricks.com
unityCatalogName=main
unityCatalogTokenFile=/var/run/secrets/unity/token

The flat keys are coalesced into a single catalog named unity of type DELTA_UC. Names are normalized by stripping the unityCatalog prefix and converting camelCase to kebab-case, so unityCatalogUri becomes unity-catalog-uri.

Prefer unityCatalogTokenFile over unityCatalogToken so the credential is not in the properties file.

ClickHouse

clickhouse.catalog.metrics.dsn=jdbc:clickhouse://clickhouse:8123/default
clickhouse.catalog.metrics.user=ursa
clickhouse.catalog.metrics.password-ref=/var/run/secrets/clickhouse/password

password-ref names a file holding the password, as an alternative to password.

ClickHouse commits inline rather than writing data files, and tracks the schema version it has applied in a _ursa_schema_versions table. Its evolution policy is the strictest of the shipped sinks: added columns only.

Selecting a catalog

A stream reaches a catalog through its materialization policy, not through the catalog declaration. Set catalog.name to the registered name:

catalog.name=analytics

Resolution order:

1. An enabled stream policy.
2. An enabled namespace policy.
3. Neither, in which case the stream is not materialized.

The named catalog must already be registered. A policy naming a catalog that does not exist fails the materialization task rather than falling back to another one — which is the behaviour you want, because the alternative is records quietly landing in the wrong table.

catalog.default supplies the catalog for streams whose policy names none.

Running several catalogs

Registering more than one catalog lets a single deployment route namespaces to different destinations — a general warehouse for most streams and a restricted one for regulated data, say, each with its own endpoint and credentials.

Give each a stable, unique name; the name is what policies refer to, so renaming one breaks every policy that selects it.

Keep credentials out of catalog records and stream properties. Use the file-backed forms — iceberg.credentialFile, unityCatalogTokenFile, ClickHouse's password-ref — so the secret is mounted rather than configured.

What a table is named

The resolved table name comes from the policy's naming template, not from the catalog. See table naming in the Materialization Spec, and lakehouse tables for how a recreated stream maps onto a table.