Skip to main content

One post tagged with "Notebooks"

View All Tags

PII redaction belongs in the notebook config, not in tribal knowledge

· 6 min read
Sai Prudhvi Neelakantam
Senior Consultant, Data Engineering & AI at Evidi

Free-text export columns are where names, phones, and emails hide. The same columns often hold codes and addresses that policy still allows. Blanking everything is lazy. Sprinkling regexp_replace through a notebook is how two engineers redact differently on the same Friday.

Put the redaction contract in one config cell the rest of the notebook reuses. Flags, tokens, patterns, and known-name helpers become reviewable. Residual risk decisions leave chat history and enter the artifact you promote.

Sensitivity label scopes in Microsoft Purview

Figure 1. PII handling is a governance contract. Label scopes and redaction rules belong in reviewable config, not tribal notebook lore. Source: Microsoft Learn: sensitivity labels.

Export notebook from Microsoft Fabric

Figure 2. Keep redaction config next to the notebook that owns the export path so reviewers can see what ships with the artifact. Source: Microsoft Learn: how to use notebooks.