Skip to main content

One post tagged with "Testing"

View All Tags

Row counts lie: validate MLVs column by column

· 6 min read
Sai Prudhvi Neelakantam
Senior Consultant, Data Engineering & AI at Evidi

A migration checklist that only compares row counts will greenlight wrong money, wrong dates, and wrong statuses. Row counts catch catastrophic drops. They miss the quiet CAST that turns decimals into floats, the null that became empty string, and the column that silently defaulted.

Push MLV cutovers to per-column tables and type-aware aggregates. Borrow the tiered reconciliation classical data migrations already trust: rows, then keys, then column aggregates. Automate it. Treat the artifacts as release evidence, not as a screenshot buried in chat.

Materialized lake view data quality overview

Figure 1. Row counts catch catastrophes. Column aggregates and quality reports catch the quiet CAST that breaks money. Source: Microsoft Learn: MLV data quality reports.

Materialized lake view detail page

Figure 2. Treat per-column reconciliation artifacts as release evidence for MLV cutovers, not optional notebook scratch. Source: Microsoft Learn: MLV data quality reports.