Open Data Infrastructure
Agentic Data Human Review Queues
How to design human review queues for agentic data changes with priority, evidence packets, source snapshots, policy results, and recovery actions.
Human review is not a button at the end of an agent workflow. It is the control system that decides what deserves production trust.
Review queues need evidence
Agentic data workflows can propose model changes, data corrections, table writes, policy updates, and reconciliation actions. Some can run automatically. Others need human review because the cost of being wrong is too high.
The review queue should not dump a prompt and a diff on a human. It should present a structured evidence packet with source snapshots, proposed changes, policy decisions, lineage, tests, risk level, and compensating actions.
Build the evidence packet
Use table snapshots for data state, OpenLineage for jobs and datasets, policy logs for allow or deny decisions, and tool manifests for agent scope. The reviewer should see what the agent saw, what it changed, why it thinks the change is safe, and how to undo it.
Core idea: human review scales only when the evidence arrives with the request.
Make review operational
Queue items need priority, owner, SLA, escalation, approval state, rejection reason, and post-approval verification. If a reviewer accepts a change, the acceptance should become part of the data product history.
For related ODI patterns, read agentic data write approval queues, compensating actions for failed writes, and Iceberg snapshot references for agent sandboxes.
What breaks first
- The queue has approvals but no source snapshot.
- Reviewers see the proposed output but not the policy decision that allowed it.
- Rejected changes disappear, so the same bad proposal returns later.
- Approval happens, but no verification confirms the promoted state.
Review queue questions
Ask whether a reviewer can approve, reject, request more evidence, trigger a replay, or apply a compensating action without leaving the queue. If review requires detective work, automation will outrun governance.
Sources to start with
These primary sources anchor the technical claims in this guide.
- Apache Iceberg table specification
- OpenLineage object model documentation
- Open Policy Agent decision logs documentation
- NIST AI Risk Management Framework
The human should review the evidence, not reconstruct it.