Multi-Tenant Pipeline Catalogs

SaaS deployments of DataXPipe serve multiple organizations from a shared Catalog API. Each org registers pipelines, views run history, and queries lineage within its own namespace. This guide covers tenant isolation, API key management, and plan limits.

Tenant model

DataXPipe SaaS introduces an organization as the top-level tenant boundary:

Entity	Scope	Example
Organization	Billing, plan, member access	acme-corp
API key	Org-scoped authentication	dxp_a1b2c3...
Pipeline	Registered within one org	orders_sync
Connection	Warehouse credentials per org	bq-prod
Run / Check result	Linked to org via pipeline	run-20250607-001

Organizations are created during signup. Stripe billing links via stripe_customer_id on the org record.

API key authentication

Production clients authenticate with org-scoped keys prefixed dxp_:

curl https://api.dataxpipe.com/api/v1/pipelines/ `
  -H "X-API-KEY: dxp_org_key_here"

Keys map to an organization and inherit its plan limits. Rotate keys from the product UI at app.dataxpipe.com without redeploying Airflow.

Legacy scaffold keys (platform-key, admin-key, dev-key) remain for local development only.

Plan limits

SaaS plans enforce resource ceilings:

Plan	Pipelines	Connections	API rate
Free	3	2	100 req/min
Team	25	10	500 req/min
Business	Unlimited	Unlimited	2000 req/min

Registration attempts beyond limits return 403 with a plan upgrade message. Monitor usage via GET /api/v1/organizations/me.

Isolation patterns

Row-level isolation (default)

All catalog tables include organization_id. API handlers filter queries by the authenticated org. Cross-tenant reads are impossible at the application layer.

Separate schemas (enterprise)

For regulated industries, deploy dedicated Postgres schemas or databases per enterprise customer. Point DATAXPIPE_DB per deployment; route via subdomain (acme.api.dataxpipe.com).

Shared warehouse, isolated metadata

Most teams share a Snowflake/BigQuery account across business units but isolate Catalog metadata by org. Warehouse RBAC (Snowflake roles, BQ dataset ACLs) complements Catalog isolation.

Onboarding a new tenant

1. Organization created via product UI signup or admin API
2. Admin generates org API key
3. Team registers warehouse connections (POST /connections/)
4. CI/CD registers pipelines with org key in X-API-KEY
5. Airflow workers configured with same key via secrets manager

Document the key rotation procedure in your internal runbook. Keys in committed DAG files are a common leak vector.

Multi-team within one org

Large enterprises often have multiple data teams under one billing org. Use pipeline tags and ownership fields:

pipeline:
  name: orders_sync
  owner: ecommerce-data@acme.com
  tags: [ecommerce, tier-1, pii]

The product UI filters pipelines by tag and owner. Lineage graphs remain org-wide—teams see cross-domain dependencies without separate Catalog instances.

Billing integration

Stripe Checkout and Customer Portal endpoints sync subscription state:

• POST /api/v1/billing/checkout — upgrade to Team or Business
• POST /api/v1/billing/portal — manage payment method
• POST /webhooks/stripe — webhook handler updates org plan

Webhook events (checkout.session.completed, customer.subscription.updated) map Stripe customers to organizations via stripe_customer_id.

Security checklist

• Never share API keys across organizations
• Use OIDC JWT auth for human users in the product UI
• Enable DATAXPIPE_RATE_LIMIT_ENABLED=true on the API
• Audit API key creation and deletion events
• Scope warehouse connections to least-privilege roles

Anti-patterns

Single global API key for all customers. Destroys isolation; one leaked key exposes every tenant.

Hardcoded org IDs in specs. Specs should be org-agnostic; authentication context determines scope at registration time.

Ignoring plan limits in CI. Pipeline count checks in CI prevent deploy failures when teams hit Free tier ceilings unexpectedly.

For Stripe setup details, see SaaS billing with Stripe. For API reference, see Catalog API quick reference.