Open Data Infrastructure
Cost Allocation for Open Lakehouse Workloads
Open lakehouse economics only works when teams can see who drives storage, compute, catalog, and maintenance cost.
The open lakehouse can lower lock-in without lowering discipline. Somebody still pays for the storage, compute, catalog, metadata, and cleanup jobs.
The practical problem
Closed platforms often bundle costs into a single bill that is easy to receive and hard to understand. Open lakehouse architectures unbundle the stack. That gives teams more control, but it also exposes shared costs that can be ignored until the bill gets loud.
Cost allocation is not accounting theater. It tells teams which workloads create value, which ones create waste, and where platform investment should go next.
Core idea: open lakehouse cost allocation should follow workload behavior, not organizational politics.
The ODI boundary
An open lakehouse has at least five cost buckets: storage, compute, catalog and metadata services, maintenance, and observability. Each bucket has a different allocation problem.
Storage can often be tagged by bucket, table, namespace, or domain. Compute depends on engine telemetry and workload identity. Catalog costs are shared control-plane costs. Maintenance costs, such as compaction and snapshot expiration, are often caused by write behavior but paid by the platform team. That mismatch matters.
Patterns that work
Tag data products, namespaces, and workloads consistently. If a table belongs to the marketing domain and a nightly compaction job rewrites that table, the cost model should be able to connect the two.
Separate showback from chargeback. Start by showing costs with enough detail to change behavior. Chargeback can come later if the organization is mature enough. A bad chargeback model just teaches teams to hide usage.
Allocate maintenance back to the patterns that cause it. Streaming writes, small files, long retention, high delete churn, and aggressive branching can all increase maintenance cost. The platform should make those tradeoffs visible.
Failure modes
The first failure is treating storage as the only lakehouse cost. Compute and maintenance usually decide whether the architecture stays economical.
The second failure is shared-platform opacity. Everyone loves shared services until nobody can explain why the catalog tier doubled in cost.
The third failure is punishing the wrong team. If one domain pays for compaction caused by another system's write pattern, incentives get weird fast.
Questions to ask
- Can every major workload be tied to an owner, domain, and business purpose?
- Which costs are direct, and which are shared control-plane costs?
- Can maintenance cost be traced to table write patterns?
- Which retention policies are worth their storage and operational cost?
- Does the cost model help teams change behavior, or only assign blame?
For economics context, read FinOps for the Open Lakehouse, Data Platform Pricing Models, and The True Cost of Vendor Lock-In.
Sources to start with
Start with the primary docs. They are the contracts you can test against, not commentary about the contracts.