Open Data Infrastructure
The Context Window Is Not Your Data Layer
If your agent architecture depends on stuffing everything into prompts, you are building a fragile system with no audit trail.
You can make almost anything look intelligent if you paste enough context into the prompt. You just cannot make it safe, governable, or reliable that way.
Why prompt stuffing fails
The context window is a transport mechanism. It is not a storage system, a permissions system, or an audit system. When teams treat it like a data layer, the same failures show up fast.
- No provenance: you cannot prove where a fact came from.
- No policy boundary: you cannot enforce row and column level permissions reliably.
- No reproducibility: you cannot recreate the exact context that produced an action.
- No lifecycle: context gets stale, but it still gets used.
Core idea: if your context layer is not auditable, your agent layer is not production-ready.
What a real data layer provides
A real data layer provides durability, governance, and a contract for meaning. It is not only a blob store.
- Durable storage: table contracts, schema evolution, and time travel where it matters.
- Metadata and semantics: definitions, ownership, and lineage that machines can use.
- Policy enforcement: consistent permissions across tools and access paths.
- Audit trails: you can reconstruct what the system did and why.
The ODI alternative
ODI treats context as governed infrastructure behavior.
- Open tables: store durable context artifacts in open formats.
- Open catalogs: expose machine-readable metadata and policy.
- Governed access: retrieval and query paths that enforce permissions and record decisions.
- Interoperability: agent tools can change without rewriting the data boundary.
This is why open data infrastructure becomes the foundation for agentic systems. The model can change. The contract layer should not.
Anti-patterns to avoid
These are the patterns that look productive early and create outages later.
- Hidden context stores: private caches with no ownership or retention policy.
- Permissionless retrieval: "we will filter it later" as a security model.
- Prompt-only governance: relying on the model to refuse sensitive data.
- Vendor-specific memory: building durable memory inside a single platform feature.
Prompting is not the data layer. It is the last step in an auditable, governed pipeline.
Sources to start with
Start with governance primitives and open contracts for data and metadata.