Skip to main content

One post tagged with "Refresh"

View All Tags

Refresh drift is a contract break, not a flaky pipeline

· 5 min read
Sai Prudhvi Neelakantam
Senior Consultant, Data Engineering & AI at Evidi

Large facts can "succeed" on every run and still lie. Row counts look fine. Pipelines are green. Measures drift because deletes, late arrivals, and corrections were never part of the incremental contract. Analysts discover it weekly. Engineers call it flaky. It is usually a missing agreement.

Treat refresh drift as a contract break. Align watermark semantics with business expectations for corrections. Fast materialization without drift tests just fails faster. Publish a refresh SLA per fact grain. Make drift checks part of the pipeline, not a weekly analyst surprise. Sales-line style facts are the classic patient; the disease is mismatched notebooks, watermarks, and semantic consumers.

Configure scheduled refresh in Power BI

Figure 1. Green refreshes are not the same as honest facts. Contracts must cover late arrivals and corrections. Source: Microsoft Learn: data refresh in Power BI.