Open Data Infrastructure
Apache Iceberg REST Catalog Security Patterns
A REST catalog is a control plane boundary, so the security design has to cover identity, authorization, credentials, audit, and engine behavior.
A REST catalog is easy to draw and dangerous to under-design. It is one endpoint between many engines and the tables that run the business.
The practical problem
The Iceberg REST catalog protocol exists because every engine should not need a custom catalog implementation. That is good architecture. It also moves a lot of trust into one service boundary. If that boundary gets identity, authorization, or credentials wrong, the open lakehouse becomes an open incident.
Security for the REST catalog is not just transport security. The catalog decides which table names exist, which operations are allowed, which metadata pointers a client receives, and how storage credentials are issued or signed. That makes it part of the control plane for Open Data Infrastructure.
Core idea: a REST catalog security model should make the allowed table operation explicit before any engine receives enough context to touch data.
The ODI boundary
ODI treats the catalog as more than lookup. The catalog is where table identity, namespaces, metadata, operation authorization, and storage access meet. That means a secure REST catalog has to separate four concerns that teams often blur together.
- Authentication: which user, workload, service account, or agent is making the request.
- Authorization: which catalog, namespace, table, view, or operation that identity can access.
- Credential scope: which storage capability is issued, for how long, and for which path or object boundary.
- Audit: which operation happened, who requested it, what policy allowed it, and which engine executed it.
If the engine gets broad object-store credentials and policy is only enforced in a separate UI, the architecture has a security story. It does not have security infrastructure.
Patterns that work
The first pattern is to make identity workload-aware. Human users, scheduled jobs, notebooks, and agents should not all arrive as the same generic platform principal. If they do, the audit trail collapses into a blob of "service-account did it" events. Nobody enjoys that incident review.
The second pattern is catalog-side authorization. Let the catalog answer whether the operation is allowed before returning metadata or credentials. This is where projects such as Apache Polaris and Lakekeeper are interesting: they make the catalog a place where governance behavior can be implemented around Iceberg tables, not merely documented near them.
The third pattern is narrow credential issuance. The Iceberg REST protocol notes secure table sharing patterns such as credential vending or remote signing. That is the right shape. Clients should receive the least storage capability needed for the requested work, not a reusable key that turns one table read into a warehouse-wide scavenger hunt.
Failure modes
The most common failure is treating the REST catalog like a metadata proxy and leaving storage authorization somewhere else. The engine may be authorized to call the catalog, but the credentials it receives may still be too broad.
The second failure is policy drift. One engine gets a new integration, another still uses the older catalog path, and a third bypasses the catalog completely for a maintenance job. You do not discover the drift until the audit log does not explain the thing you need to explain.
The third failure is missing operation-level policy. Reading a table, committing a snapshot, changing a schema, creating a branch, and dropping a namespace should not all be the same permission. Open data infrastructure needs portable control, not one giant allow button.
Questions to ask
Ask these questions before you call a REST catalog production-ready:
- Can the catalog identify humans, workloads, and agents separately?
- Can it authorize table operations, not only catalog login?
- Can storage credentials be scoped to the requested operation and time window?
- Can audit records connect identity, policy, table, engine, and storage access?
- Can policy be tested in CI before the catalog configuration changes?
For adjacent architecture, read What Is a REST Catalog?, Access Control in Open Data Infrastructure, and Catalogs as the Control Plane for ODI.
Sources to start with
Start with the primary docs. They are the contracts you can test against, not commentary about the contracts.