4 posts tagged with "Semantic Model"

Checklist reporting loves wide pivots. Teams often build them late: SQL OUTER APPLY, Import mode, captions invented in the report. Refresh time climbs. Direct Lake never gets a fair chance. The semantic model pays for a shape that should have been decided in gold.

Pivot once in gold. Broadcast joins plus a single groupBy/pivot in PySpark produce a self-contained fact: one row per reading with stable keys. Captions match the language users already see in PDFs. Materialize for Direct Lake. Stop paying Import tax for wide pivots that only exist because the warehouse view stayed nested.

Figure 1. Direct Lake wants report-ready flat facts, not nested checklist trees that only Import mode can fake. Source: Microsoft Learn: understand relationships.

Telemetry tables named after pipelines invite people to treat events like disposable logs. Reports then reinvent grain, matching, and captions in DAX. Volume pushes Import mode until refresh becomes the weekly outage.

Name the business concept (events). Enrich with the same matching logic used by service-device facts. Land them in gold for Direct Lake. Captions, query folders, and descriptions are part of the semantic handoff. Watermark shortcuts and clean views are part of the bronze-to-gold story, not footnotes.

Figure 1. IoT events are facts. Land them as modeled destinations, not as log piles that only Import mode can fake into a report. Source: Microsoft Learn: Eventstreams overview.

Blank visuals are often type mismatches, not proof that "DAX is hard." An int64 fact key next to a string dimension key fails quietly. Role-playing dims multiply the blast radius: one bad FK type breaks several relationships that all looked fine in the diagram.

Align gold SQL and TMDL together. Date keys and degenerate dims deserve explicit type standards. Add a CI check for relationship endpoint types. Matching types is cheaper than another afternoon of staring at empty cards.

Figure 1. Blank visuals are often int64-versus-string relationship endpoints, not proof that DAX is hard. Source: Microsoft Learn: understand relationships.

Figure 2. Align gold SQL types and TMDL together, then gate relationship endpoint types in CI. Source: Microsoft Learn: Direct Lake storage.

A gold device table that looks clean in Spark can still fail as a platform asset. Optional links to contracts drop unmatched devices. Free-text site fields hide business keys behind tribal regex. Captions and folders are treated as polish. Stakeholders open a report and half the estate is missing without an error.

Gold becomes useful when unmatched keys, captions, and relationships ship as one semantic contract with the table. "All devices visible regardless of match" is an acceptance test, not a slogan.

Figure 1. Gold only becomes a platform asset when keys and relationships behave as a semantic contract the model can trust. Source: Microsoft Learn: understand relationships.

Figure 2. Row counts on the SQL endpoint can look healthy while reports hide unmatched keys. Test the contract, not only Spark success. Source: Microsoft Learn: lakehouse SQL analytics endpoint.