Open Data Infrastructure
AI-Ready Data Policy Test Fixtures
How policy test fixtures prove agent access decisions across identity, purpose, filters, denied requests, and answer boundaries.
A policy that has never failed a test has not earned much trust.
AI-ready data needs policy fixtures
AI-ready data is not only clean data. It is data packaged with access rules, context, lineage, freshness, and evidence that a system can use at runtime. Policy test fixtures make that practical.
A fixture should define an identity, purpose, allowed data, denied data, row or column filters, expected tool output, and expected answer boundary. It should include positive cases and denial cases. Denial cases are where most policy theater gets exposed.
OPA makes policy test behavior concrete
Open Policy Agent provides policy as code and policy testing. That gives teams a way to test access decisions with structured inputs instead of relying on screenshots of an admin console.
The fixture should travel with the data product. If a model, catalog grant, row filter, or agent tool changes, the policy fixtures should run before the change reaches production.
Core idea: AI policy is not real until the denial path is tested.
The ODI pattern turns policy into infrastructure behavior
Open Data Infrastructure asks whether policy is evaluated where work happens. For agents, that means the policy test has to cover the tool path, retrieval path, query path, and answer path.
Related articles include AI-ready data entitlement drift, agentic AI denial logs, and why agents need governed data access. Fixtures connect those ideas to repeatable checks.
What breaks first
Policy failures usually hide in cases nobody bothered to write down.
- The happy path is tested, but a denied request is never exercised.
- Row filters work in SQL but not in retrieval indexes.
- Purpose is recorded in an application log but not supplied to the policy decision.
- An eval checks answer quality without checking whether the answer should have been allowed.
What a fixture should include
Include identity, role, purpose, source assets, allowed fields, denied fields, filter expectations, expected tool output, expected answer behavior, and the reason each denial exists.
The fixture should be small enough to read and strict enough to catch a policy regression before an agent turns it into a customer-facing answer.
Sources to start with
These primary sources anchor the technical claims in this guide.
- Open Policy Agent documentation
- Open Policy Agent policy testing documentation
- OpenAI evals guide
- NIST AI Risk Management Framework
A policy fixture is a small test with a large job: proving the agent knows when to stop.