2 posts tagged with "Gold Layer"

Telemetry tables named after pipelines invite people to treat events like disposable logs. Reports then reinvent grain, matching, and captions in DAX. Volume pushes Import mode until refresh becomes the weekly outage.

Name the business concept (events). Enrich with the same matching logic used by service-device facts. Land them in gold for Direct Lake. Captions, query folders, and descriptions are part of the semantic handoff. Watermark shortcuts and clean views are part of the bronze-to-gold story, not footnotes.

Figure 1. IoT events are facts. Land them as modeled destinations, not as log piles that only Import mode can fake into a report. Source: Microsoft Learn: Eventstreams overview.

A gold device table that looks clean in Spark can still fail as a platform asset. Optional links to contracts drop unmatched devices. Free-text site fields hide business keys behind tribal regex. Captions and folders are treated as polish. Stakeholders open a report and half the estate is missing without an error.

Gold becomes useful when unmatched keys, captions, and relationships ship as one semantic contract with the table. "All devices visible regardless of match" is an acceptance test, not a slogan.

Figure 1. Gold only becomes a platform asset when keys and relationships behave as a semantic contract the model can trust. Source: Microsoft Learn: understand relationships.

Figure 2. Row counts on the SQL endpoint can look healthy while reports hide unmatched keys. Test the contract, not only Spark success. Source: Microsoft Learn: lakehouse SQL analytics endpoint.