Open Data Infrastructure
dbt Core and Open Data Infrastructure
Where dbt Core fits in an ODI stack, and where warehouse-first assumptions need stronger metadata and governance boundaries.
dbt Core is everywhere in modern analytics. The ODI question is not whether dbt works. It is whether your dbt project stays portable when your stack stops being one warehouse.
Why it matters
dbt changed the default operating model for analytics engineering. SQL in Git. Tests. Docs. Lineage graphs. That is real progress.
Most dbt projects grew up inside one platform boundary. ODI puts pressure on that assumption because it encourages multiple engines, open tables, and shared catalogs.
The ODI angle
In ODI, dbt Core is a transformation layer, not the platform. It should sit on top of open storage and portable metadata interfaces, not define the only place meaning exists.
You can make dbt more portable by tightening your SQL dialect subset, keeping governance in shared infrastructure, and exporting lineage and metadata in machine-readable ways.
dbt still matters because it keeps transformation logic explicit, reviewable, and testable. That is exactly what AI-era data systems need.
Core idea: ODI is how you stop transformation logic from turning into a one-vendor dependency.
The architecture test
For analytics engineers, the test is whether your dbt project can survive an engine change without rewriting the business.
- Define the contract between dbt models and open table formats.
- Decide which models must be portable and which can be engine-specific.
- Use tests as guardrails, not as a compliance checkbox.
- Integrate lineage capture and keep it machine-readable and exportable.
- Decide how semantic meaning is shared across tools and agents.
What breaks first
This breaks when dbt becomes the only place your organization knows what data means.
- Macros hide a pile of engine-specific behavior that nobody wants to migrate.
- Lineage is a nice graph in a UI, but it cannot be exported or audited across tools.
- Governance concerns get stuffed into dbt artifacts that do not travel.
- Engine changes become "rewrite everything" projects because portability was never tested.
Questions to ask
Use these questions when you evaluate dbt Core open data infrastructure inside a multi-engine ODI strategy.
- What would it take to run this dbt project on a different engine?
- Are macros and packages tying you to a single vendor's SQL behavior?
- How do you export lineage and metadata for other tools and agents?
- How do you handle schema evolution and backfills with minimal risk?
- Do your tests reflect real failure modes, or only happy-path assertions?
If your answers are "we will deal with it later," later is when you are already stuck.
Sources to start with
Start with dbt docs, then check the boundaries where portability actually breaks.