Deep dives into real deployments, architecture choices, and measurable outcomes.
View All Tags
A public POC proves the product loop. A private MVP proves you can survive enterprise networking.
Those are different architectures. Do not fake the second by adding more env files.
Figure 1. Private endpoint pattern for App Service. Source: Microsoft Learn: App Service private endpoints.
Most “AI app” demos collapse three different questions into one login button:
In a real enterprise agent webapp, those are three different control planes. Treat them as one and you get either a security hole or a support nightmare.
Figure 1. SPA calling a web API. Authentication gets you a token; authorization still has to happen on the API. Source: Microsoft Learn: authentication flows and app scenarios.
Canvas is wonderful until one screen becomes a distributed system: nested galleries, OnStart fan-out, and business rules living in Power Fx because it was faster.
Microsoft's own performance reference architectures push a clear direction. Move heavy work toward Dataverse and keep screens focused.
Figure 1. Optimize canvas apps with platform-aligned architecture. Source: Microsoft Learn: optimize canvas app performance.
Copying tables into a lakehouse is the easy part. The hard part is whether 0 means zero or was null in the legacy system, and whether a 0.01 drift is rounding or a bug.
I treat migration as a semantic contract, not a file move. Stakeholders do not celebrate that Delta exists. They celebrate that last month's margin still matches within agreed rules.
Figure 1. Shortcuts and schemas change where data appears, not what it means. Source: Microsoft Learn: lakehouse schemas.
Current-state dimensions lie the moment a device moves. Dashboards that say "where is it now" are fine for operations. History questions ("where was it when the alarm fired?") need slowly changing dimension type 2, not a hope that someone kept yesterday's extract.
For measuring points and similar deployed assets, I put location and classification history in SCD2 Materialized Lake Views before anyone builds "devices as of" pages. The change feed is the source of truth. The MLV is the productized history contract.
Figure 1. Point-in-time placement is an SCD2 product question. Model effective dating before you ask dashboards to invent history. Source: Microsoft Learn: dimensional modeling dimension tables.
Figure 2. MLVs make history transforms operable. Lineage shows which change feed feeds the point-in-time contract. Source: Microsoft Learn: get started with materialized lake views.
Medallion architecture asks where bronze, silver, and gold live. In Fabric you can answer with many lakehouses or with schemas inside one lakehouse.
That choice leaks into shortcuts, SQL endpoints, notebooks, CI/CD, and how painful renames become later. It is not a folder preference. It is a topology decision.
Figure 1. Lakehouse creation in Microsoft Fabric. Source: Microsoft Learn: lakehouse schemas.
Moving a handful of warehouse views to Materialized Lake Views is a rewrite. Moving hundreds is a program. Teams that treat the long tail like a weekend notebook usually discover dependency order the hard way: gold breaks, silver thrashes, and bronze shortcuts were never registered as the real contract.
Frame the work as bronze to silver to gold with gates. Count the estate. Order by dependency. Prove a representative POC. Then convert with per-view notebooks so blast radius stays local. Success is parity plus runtime, not "the portal shows an MLV."
Figure 1. MLV migration is a program across bronze, silver, and gold. Gates belong between layers, not as a weekend cutover wish. Source: Microsoft Learn: medallion lakehouse architecture.
Teams still celebrate when the container listens on the right port. For Foundry-backed agent webapps, that is the easy half. The hard half is who the app runs as, how OBO flows, and how egress reaches Foundry, Search, and storage without turning the VNet story into folklore.
Deploy the pattern as identity and networking first. Hostname and Node version are table stakes.
Figure 1. Foundry webapps deploy as identity and egress. VNet integration changes name resolution and outbound paths more than hostnames do. Source: Microsoft Learn: App Service VNet integration.
Large facts can "succeed" on every run and still lie. Row counts look fine. Pipelines are green. Measures drift because deletes, late arrivals, and corrections were never part of the incremental contract. Analysts discover it weekly. Engineers call it flaky. It is usually a missing agreement.
Treat refresh drift as a contract break. Align watermark semantics with business expectations for corrections. Fast materialization without drift tests just fails faster. Publish a refresh SLA per fact grain. Make drift checks part of the pipeline, not a weekly analyst surprise. Sales-line style facts are the classic patient; the disease is mismatched notebooks, watermarks, and semantic consumers.
Figure 1. Green refreshes are not the same as honest facts. Contracts must cover late arrivals and corrections. Source: Microsoft Learn: data refresh in Power BI.
Checklist reporting loves wide pivots. Teams often build them late: SQL OUTER APPLY, Import mode, captions invented in the report. Refresh time climbs. Direct Lake never gets a fair chance. The semantic model pays for a shape that should have been decided in gold.
Pivot once in gold. Broadcast joins plus a single groupBy/pivot in PySpark produce a self-contained fact: one row per reading with stable keys. Captions match the language users already see in PDFs. Materialize for Direct Lake. Stop paying Import tax for wide pivots that only exist because the warehouse view stayed nested.
Figure 1. Direct Lake wants report-ready flat facts, not nested checklist trees that only Import mode can fake. Source: Microsoft Learn: understand relationships.
Medallion diagrams still tempt teams to land a physical bronze copy of everything upstream. When the upstream domain already publishes curated Delta in OneLake, that copy is not prudence. It is a second warehouse with a friendlier name.
Bronze can be a contract of shortcuts and thin clean views. Ownership stays with the publisher. Freshness follows the source. Your CI/CD list becomes the shortcut set, not a shadow catalog of cloned tables.
Figure 1. Bronze can be a pointer surface. Prefer OneLake shortcuts over copying bytes the platform already holds. Source: Microsoft Learn: OneLake shortcuts.
Telemetry tables named after pipelines invite people to treat events like disposable logs. Reports then reinvent grain, matching, and captions in DAX. Volume pushes Import mode until refresh becomes the weekly outage.
Name the business concept (events). Enrich with the same matching logic used by service-device facts. Land them in gold for Direct Lake. Captions, query folders, and descriptions are part of the semantic handoff. Watermark shortcuts and clean views are part of the bronze-to-gold story, not footnotes.
Figure 1. IoT events are facts. Land them as modeled destinations, not as log piles that only Import mode can fake into a report. Source: Microsoft Learn: Eventstreams overview.
Parameterized DataLake expressions can work for months and then fail across stages overnight. Teams panic-rewrite convention docs as if literal OneLake URLs were always the design goal. That erases history and guarantees the workaround becomes religion.
Reframe "literal URL required" failures as service regressions. Ship validators and literal patches as temporary controls. Keep a Test to Preprod to Prod promotion story under the workaround. File and track the Microsoft support trail next to the repo convention. Do not pretend the platform always wanted hard-coded URLs.
Figure 1. Direct Lake on OneLake is the intended pattern. Literal URL patches are controls when parameters regress. Source: Microsoft Learn: Direct Lake storage.
Helpful gold views that "just join service and device" hide many-to-many grain until a report page disagrees with itself. Filters double count. Role-playing dims break. Two pages that should reconcile never do.
Deploy bridge keys as a first-class artifact and remodel the service-device fact before you polish visuals. Matching logic must be shared with telemetry enrichment or the numbers diverge by construction. Semantic role-playing dims need matching foreign key types. Remodel first. Visual polish second.
Figure 1. Many-to-many grain leaks into blank or double-counted visuals. Remodel the fact before polishing pages. Source: Microsoft Learn: understand relationships.
Figure 2. Bridge keys and filter direction are part of the remodel. Dashboards inherit whatever grain you leave in gold. Source: Microsoft Learn: understand relationships.
A gold device table that looks clean in Spark can still fail as a platform asset. Optional links to contracts drop unmatched devices. Free-text site fields hide business keys behind tribal regex. Captions and folders are treated as polish. Stakeholders open a report and half the estate is missing without an error.
Gold becomes useful when unmatched keys, captions, and relationships ship as one semantic contract with the table. "All devices visible regardless of match" is an acceptance test, not a slogan.
Figure 1. Gold only becomes a platform asset when keys and relationships behave as a semantic contract the model can trust. Source: Microsoft Learn: understand relationships.
Figure 2. Row counts on the SQL endpoint can look healthy while reports hide unmatched keys. Test the contract, not only Spark success. Source: Microsoft Learn: lakehouse SQL analytics endpoint.
Fabric notebooks and pipelines are great at doing work. They are mediocre at being the calendar, the dependency graph, and the on-call story for everything around them. The moment three jobs must finish before a fourth can start, you need an orchestration plane that is not pretending to be another transform engine.
Airflow fits when you want an honest DAG: schedule, dependencies, retries, and a run history humans can read. Keep tasks thin. Call Fabric. Do not rebuild Fabric inside operators because it was convenient on a Friday.
Figure 1. Fabric Airflow jobs can trigger notebooks, pipelines, and other items from a DAG without stuffing transform logic into the orchestrator. Source: Microsoft Learn: run a Fabric item using Apache Airflow.