dbt Freshness Checks in CI (and What CI Still Misses)

Teams often run dbt test on every pull request and assume production is covered. Then a mart goes stale for two days while Airflow stays green — because CI never measured freshness against live data, and production either skipped tests or treated them as warn.

This post covers what belongs in CI for dbt freshness and contracts, what still requires runtime monitoring, and how DataXPipe connects both.

What CI is good at

Pull-request CI should fail fast on breaking producer changes:

1. dbt model contracts (enforced: true) — column names and types match the declared shape
2. Schema / uniqueness tests on models touched by the PR
3. Contract / impact analysis — who downstream depends on this model, and which consumer SLAs break

Those gates prevent “we merged a rename and broke finance’s dashboard” incidents.

What CI is bad at alone

CI runs against a PR branch snapshot (or a thin CI warehouse), not against tonight’s production load. It will not reliably catch:

• Upstream SaaS extract that wrote zero rows after merge
• Replication lag that makes MAX(updated_at) miss the SLA
• severity: warn tests that return exit code 0 in Airflow
• Freshness that was never configured on the source or mart

Those are silent failures: orchestration succeeded; the data product did not.

A practical CI checklist for dbt

Gate	Tooling	Blocks merge?
Model contract / schema	dbt contracts + tests	Yes on error
Breaking change vs prior state	dbt compare / state artifacts	Yes when configured
Consumer freshness / row SLAs	DataXPipe consumer contracts + impact API	Yes when incompatible
Blast radius comment	DataXPipe GitHub Action	Inform + optional fail

Run dbt source freshness in production schedules, not only in CI. CI can validate that freshness definitions exist; production must enforce the clock.

Warn vs error (the silent failure trap)

If every soft failure is warn, your scheduler looks healthy while dashboards rot. Reserve error for:

• Executive KPI marts
• Billing / revenue tables
• Any dataset with a hard consumer SLA

Use warn for exploratory or marketing metrics where a one-hour delay is annoying, not false.

How DataXPipe fits

DataXPipe does not replace dbt tests. It adds:

• Import of manifest.json so lineage and default freshness attach to marts
• Runtime checks tied to run IDs when Airflow finishes
• Pipeline Passports that revoke when freshness or KPI drift fails
• PR blast-radius comments via impact analysis so reviewers see consumer risk before merge

See also: Silent failures when the orchestrator is green and the marketing guide Data contracts in CI.

One-week rollout

1. Pick three marts finance actually opens; set freshness error SLAs.
2. Ensure production runs dbt build (or run + test + source freshness) on a schedule — not CI-only.
3. Add a PR job that validates specs and posts blast radius.
4. Issue a Passport for one executive mart; share the link with the stakeholder who used to page you from Slack.

Green CI and green Airflow are necessary. Trustworthy data is a separate event — make them the same.