Skip to content
Playbook

Snowflake, BigQuery, Azure SQL: One Pipeline, Five Sets of Quirks

Role-scoped visibility breaks more dashboards than any other single cause. What the warehouses share, what they do not, and why the profiling step decides your deployment topology.

4 min read

Snowflake, Databricks, BigQuery, Azure SQL, and PostgreSQL are one integration surface with five sets of quirks. The pipeline is identical — connect, discover, profile, generate, validate, package — and what changes per warehouse is authentication, type mapping, and how much the catalog will tell you before you have to go and look.

This is a deployment playbook for pointing a generation pipeline at a governed warehouse and getting back a Power BI artifact your team owns. It covers what is common, what is not, and the two things that decide whether the output is usable.

From the standard

“Power BI semantic models represent a source of data that's ready for reporting and visualization.”
Microsoft Learn, Semantic models in the Power BI service — learn.microsoft.com

The shared pipeline

  1. 1 · CONNECT Driver and credential, read-only role
  2. 2 · DISCOVER Tables, columns, types, keys, relationships The warehouse-specific part of the pipeline lives here, and only here.
  3. 3 · PROFILE Cardinality, null density, candidate dimensions and measures
  4. 4 · GENERATE The semantic model Tables, relationships, DAX measures, visuals, page layout.
  5. 5 · VALIDATE Self-correction loop Errors are fixed and re-checked, bounded by a retry limit.
  6. validated
    6 · PACKAGE A .pbip project folder that opens in Power BI Desktop
Only step two changes between warehouses. The other five are identical.

Only the discovery step is genuinely warehouse-specific. Everything downstream operates on a normalised description of the schema, which is why adding a warehouse is a driver and a metadata query rather than a new pipeline.

What differs per warehouse

Warehouse What to watch for
Snowflake Role-scoped visibility: the schema you discover is the schema the role can see. Profile under the same role the refresh will use, or the model will reference tables the report cannot read.
Databricks Unity Catalog governs by identity. The namespace has three levels — catalogue, schema, table — and flattening it collides names across catalogues.
BigQuery Dataset location is part of the identity. Nested and repeated fields have no direct Power BI equivalent and need an explicit flattening rule.
Azure SQL Closest to a conventional relational source, and the one most likely to carry legacy type quirks that map badly to a semantic model.
PostgreSQL Fastest to stand up, and the usual choice for validating the pipeline before pointing it at a production warehouse.
The pipeline is the same for all five. Only the discovery step differs.

The role-scoped visibility point applies to all five and causes the most confusing failures. A model generated under an administrative role and refreshed under a reporting role produces a report that worked once and then broke, with an error that names a permission rather than the design decision that caused it.

Connect read-only, and mean it

The generation pipeline reads schema and samples data. It has no reason to hold write access to the warehouse, and granting it removes the simplest argument you have in a security review. A read-only role scoped to the specific schemas in play makes the blast radius of a compromised pipeline a disclosure question rather than an integrity one.

Profiling reads data, not just metadata

Cardinality and null density cannot be inferred from column types. The profiling step samples rows, which means it touches real data and belongs inside whatever perimeter that data is governed by. If the warehouse holds regulated data, the pipeline runs where the warehouse runs — this is a deployment topology question, not a connector setting.

Cold build versus warm rebuild

A first build against an unfamiliar schema spends most of its time in discovery and inference. A rebuild of the same source with an unchanged schema skips inference entirely — incremental detection compares the schema, finds it identical, and reuses the model at no inference cost.

That gap is the number to measure in your own environment before setting a refresh cadence, because it changes the economics of scheduling. A nightly rebuild costs almost nothing when the schema is stable and costs a full build every time somebody adds a column.

The output has to survive without the tool

The deliverable is a standard .pbip project folder: tables, relationships, DAX measures, visuals, page layouts. It opens in Power BI Desktop and your BI team edits it like anything else they maintain.

This is the property to insist on regardless of who builds the pipeline. A generated artifact that can only be edited through the generating system is not a dashboard you own — it is a dependency with a dashboard-shaped interface, and the cost of that shows up the first time the tool is unavailable and a number on the board is wrong.

Where this fits, and where it does not

It fits where the same dashboard is being hand-built repeatedly against a governed source that already has a semantic definition worth inheriting. It does not fit where the metric definitions are the unsettled part — a generation pipeline will faithfully encode whatever inconsistency exists in the warehouse, and produce a fast, confident, governed version of the wrong number.

Axionalytics

Production agentic AI for enterprise engineering, data, and revenue teams.

Keep reading

Facing this in your own environment?

Forty-five minutes with the engineers who build these systems. Bring the constraint that has been blocking you — you will leave with an architecture opinion whether or not you work with us.