DataFusion Logical Plan Reviews for Governed Query APIs

A governed query API should inspect the query before the engine turns it into work.

Policy has to meet the plan early

Governed query APIs usually fail in one of two ways. They either trust the incoming SQL string too much, or they bolt on controls after planning has already turned intent into execution. Both choices make policy review weaker than it needs to be.

Apache DataFusion gives builders a useful middle layer. A logical plan describes the high-level query operations before physical execution. That is where a governed API can inspect table access, selected columns, filters, joins, limits, and policy rewrites before the query runs.

Logical plans describe the query shape

The DataFusion logical plan documentation describes a logical plan as a structured representation of a query that captures operations such as filtering, sorting, and joining. The EXPLAIN documentation separates logical and physical plans for query inspection.

That separation is useful for ODI because access review belongs before execution. A policy engine can decide whether the plan is allowed, whether it needs projection trimming, whether a predicate must be added, or whether the query should be refused.

Review the plan, then record the decision

• Normalize user input into a logical plan before policy review.
• Inspect relations, projections, filters, joins, aggregates, and user-defined functions.
• Apply policy rewrites as explicit plan transformations, not hidden string edits.
• Store the pre-review plan, post-review plan, policy version, and decision ID.
• Expose refusal reasons that data owners can understand.

This connects to DataFusion and composable query engines, policy enforcement in open data systems, and ODI control planes for AI workloads. A governed API needs evidence about what the query meant.

What breaks first

• String-based filters miss equivalent SQL written in a different shape.
• Policy rewrites happen after optimization, so reviewers cannot see the original request.
• The system denies a query but cannot explain which relation or column caused the denial.
• Audit logs capture the submitted text but not the plan that actually executed.

Questions to ask before shipping the API

• Which logical plan nodes are allowed for each user, tool, and purpose?
• Can the API prove which policy rewrites happened before execution?
• Can owners review a denied plan without reading engine internals?
• Does the audit record include both the requested and approved plan shape?

Sources to start with

These primary sources anchor the technical claims in this guide.

• DataFusion logical plan documentation
• DataFusion EXPLAIN documentation
• DataFusion project documentation

Governance gets much easier when the query has a shape you can inspect.