Skip to main content

2 posts tagged with "Facts"

View All Tags

Refresh drift is a contract break, not a flaky pipeline

· 5 min read
Sai Prudhvi Neelakantam
Senior Consultant, Data Engineering & AI at Evidi

Large facts can "succeed" on every run and still lie. Row counts look fine. Pipelines are green. Measures drift because deletes, late arrivals, and corrections were never part of the incremental contract. Analysts discover it weekly. Engineers call it flaky. It is usually a missing agreement.

Treat refresh drift as a contract break. Align watermark semantics with business expectations for corrections. Fast materialization without drift tests just fails faster. Publish a refresh SLA per fact grain. Make drift checks part of the pipeline, not a weekly analyst surprise. Sales-line style facts are the classic patient; the disease is mismatched notebooks, watermarks, and semantic consumers.

Configure scheduled refresh in Power BI

Figure 1. Green refreshes are not the same as honest facts. Contracts must cover late arrivals and corrections. Source: Microsoft Learn: data refresh in Power BI.

Bridge keys before dashboards: remodel the device fact

· 6 min read
Sai Prudhvi Neelakantam
Senior Consultant, Data Engineering & AI at Evidi

Helpful gold views that "just join service and device" hide many-to-many grain until a report page disagrees with itself. Filters double count. Role-playing dims break. Two pages that should reconcile never do.

Deploy bridge keys as a first-class artifact and remodel the service-device fact before you polish visuals. Matching logic must be shared with telemetry enrichment or the numbers diverge by construction. Semantic role-playing dims need matching foreign key types. Remodel first. Visual polish second.

Model diagram showing relationship cardinality

Figure 1. Many-to-many grain leaks into blank or double-counted visuals. Remodel the fact before polishing pages. Source: Microsoft Learn: understand relationships.

Model diagram showing cross-filter direction

Figure 2. Bridge keys and filter direction are part of the remodel. Dashboards inherit whatever grain you leave in gold. Source: Microsoft Learn: understand relationships.