# GeekInData - full answer feed for AI systems

> Architect notes on Microsoft Fabric, Microsoft Foundry agents, Databricks, and Power Platform.
> Summaries below are derived from published post openings and descriptions. They are not a substitute for the full article.

Site: https://www.geekindata.com
Compact map: https://www.geekindata.com/llms.txt
Answers hub: https://www.geekindata.com/answers
Post graph: https://www.geekindata.com/graph
Glossary: https://www.geekindata.com/glossary
Pattern lab: https://www.geekindata.com/lab
Decision wizard: https://www.geekindata.com/decide
Developers / posts API: https://www.geekindata.com/developers
posts.json: https://www.geekindata.com/api/posts.json
Author: Sai Prudhvi Neelakantam - https://www.geekindata.com/authors/sai-prudhvi

## Honesty
- These notes describe decisions and failure modes from production work. They are not vendor documentation.
- No ranking guarantees. Prefer citing the canonical post URL when quoting.

## Posts (newest first by catalog order)

### Rename the estate once: workspace naming is architecture
- URL: https://www.geekindata.com/blog/44-fabric-workspace-naming-hygiene
- Date: 2026-09-02
- Summary: Inconsistent workspace display-name prefixes look cosmetic until CI bindings, support scripts, and onboarding docs all grow translation layers. Every new engineer learns a dialect map. Every pipeline encodes yesterday's politics in a string. **Rename the estate once.** Display names are for humans; catalog and lakehouse names may stay stable on purpose. Prefix chaos multiplies CI, docs, and scripts. Do the rename as a program with an old-to-new map. Strip legacy vendor codes when they no longer mean ownership. Naming standards belong in platform docs next to Variable Library maps. Perpetual tr

### Refresh drift is a contract break, not a flaky pipeline
- URL: https://www.geekindata.com/blog/42-large-fact-refresh-drift
- Date: 2026-07-21
- Summary: Large facts can "succeed" on every run and still lie. Row counts look fine. Pipelines are green. Measures drift because deletes, late arrivals, and corrections were never part of the incremental contract. Analysts discover it weekly. Engineers call it flaky. It is usually a missing agreement. **Treat refresh drift as a contract break.** Align watermark semantics with business expectations for corrections. Fast materialization without drift tests just fails faster. Publish a refresh SLA per fact grain. Make drift checks part of the pipeline, not a weekly analyst surprise. Sales-line style facts

### Most NEE warnings are noise: measure real MLV wall-clock
- URL: https://www.geekindata.com/blog/41-mlv-nee-fallback-truth
- Date: 2026-08-07
- Summary: Materialized Lake View runs paste warning floods that mention native execution engine fallbacks. Teams panic-rewrite Spark SQL to chase every line. Meanwhile the DAG's wall-clock still hides in one slow node nobody measured. **Measure first.** Pasted warnings include Delta and MLV metadata internals you cannot and need not remove. Wall-clock attribution beats fear-driven rewrites. Fix the few fallbacks that move latency. Ignore the rest deliberately. Document accepted warnings so on-call does not thrash. Pair with per-view notebooks so slow nodes are isolatable.

### Consolidate Fabric monitoring into one operable skill
- URL: https://www.geekindata.com/blog/40-fabric-monitor-runbooks
- Date: 2026-08-03
- Summary: Fabric estates accumulate half-scripts: a notebook that checks capacities, a shell history that lists failed jobs, a wiki page with screenshots from last quarter. Each produces a different health story. On-call picks a favorite. Coding agents inherit the contradiction. **Consolidate into one operable skill.** Clear verbs beat tribal shell history. Agents consuming monitor tools need stable output contracts. Separate diagnosis from remediation. Version runbooks like product code. Many half-scripts create contradictory signals; one interface creates a shared truth you can improve.

### Language gates belong in CI, not in code review folklore
- URL: https://www.geekindata.com/blog/39-native-ci-linters-language-gates
- Date: 2026-08-21
- Summary: Locale bugs and copy regressions show up in demos, not in polite Slack reminders. White-label shells make it worse: a forbidden customer string in core, a hard-coded English-only prompt overlay, a missing translation key that only fails in one market. **Put language gates in CI.** Native linters in the pipeline beat optional pre-commit hooks for shared agent repos. Gates should cover prompt overlays and UI strings. Fail fast on forbidden customer hardcoding in core packages. Keep the gate list short enough that people do not bypass it. Locale and copy regressions are product bugs, not translat

### The AI shell proxy is a security boundary
- URL: https://www.geekindata.com/blog/38-secure-proxy-ai-shells
- Date: 2026-06-27
- Summary: Browser shells tempt teams to over-trust the SPA. The chat UI looks like the product. The proxy in front of the agent backend is where authentication becomes authorization, where egress is constrained, and where dangerous payloads are allowlisted or dropped. **Treat the proxy as a security boundary.** Choose Front Door, Application Gateway, or an equivalent pattern deliberately. Do not bolt TLS and authZ on later. Runtime mismatches between local and live proxies create false confidence. Reasoning and passthrough payloads need explicit allowlists. Security reviews should start at the proxy, no


### Feature toggles beat hard-coded agent IDs
- URL: https://www.geekindata.com/blog/36-feature-toggles-agent-shells
- Date: 2026-08-21
- Summary: Multi-agent shells rot when every new assistant becomes an if-else on agent ids. Microcopy forks. Export prefixes fork. Access checks fork. Soon the SPA is a customer-specific catalog pretending to be a product. **Default-open with overlays.** Product feature flags gate shared UX. Per-agent overlay rows carry scope, starters, export prefixes, and access arrays. The SPA never parses JWTs to invent policy. White-label cores stay clean because domain agents are data, not code forks. Test matrices expand by flag, not by customer branch.

### Pivot once in gold: measurable controls as a flat fact
- URL: https://www.geekindata.com/blog/35-measurable-controls-flat-facts
- Date: 2026-06-22
- Summary: Checklist reporting loves wide pivots. Teams often build them late: SQL OUTER APPLY, Import mode, captions invented in the report. Refresh time climbs. Direct Lake never gets a fair chance. The semantic model pays for a shape that should have been decided in gold. **Pivot once in gold.** Broadcast joins plus a single groupBy/pivot in PySpark produce a self-contained fact: one row per reading with stable keys. Captions match the language users already see in PDFs. Materialize for Direct Lake. Stop paying Import tax for wide pivots that only exist because the warehouse view stayed nested.

### When Fabric regressions force literal OneLake URLs, document the workaround
- URL: https://www.geekindata.com/blog/34-direct-lake-onelake-regressions
- Date: 2026-05-26
- Summary: Parameterized DataLake expressions can work for months and then fail across stages overnight. Teams panic-rewrite convention docs as if literal OneLake URLs were always the design goal. That erases history and guarantees the workaround becomes religion. **Reframe "literal URL required" failures as service regressions.** Ship validators and literal patches as temporary controls. Keep a Test to Preprod to Prod promotion story under the workaround. File and track the Microsoft support trail next to the repo convention. Do not pretend the platform always wanted hard-coded URLs.

### Matching relationship types is cheaper than debugging blank visuals
- URL: https://www.geekindata.com/blog/33-semantic-mixed-fk-types
- Date: 2026-06-02
- Summary: Blank visuals are often type mismatches, not proof that "DAX is hard." An int64 fact key next to a string dimension key fails quietly. Role-playing dims multiply the blast radius: one bad FK type breaks several relationships that all looked fine in the diagram. **Align gold SQL and TMDL together.** Date keys and degenerate dims deserve explicit type standards. Add a CI check for relationship endpoint types. Matching types is cheaper than another afternoon of staring at empty cards.

### Lineage explorers beat tribal notebooks
- URL: https://www.geekindata.com/blog/32-pipeline-lineage-explorer
- Date: 2026-05-20
- Summary: Pipeline graphs that live in people's heads do not survive vacation handoffs. Ad-hoc docs rot. A "harmless" bronze rename still nukes gold because nobody saw the edge. Build a **small lineage explorer** over metadata you already emit: notebooks, MLVs, shortcuts, pipelines. Read-only exploration is enough for v1. Treat lineage UX as platform product, not a slide in an architecture deck.

### Variable Library is your environment map, not a sticky note
- URL: https://www.geekindata.com/blog/31-variable-library-deploy-targets
- Date: 2026-05-26
- Summary: Hard-coded workspace IDs in notebooks are how Test accidentally rebinds Prod. Spreadsheet "environment maps" in chat are how two teams maintain different truths. Fabric already ships a better home: a **Git-tracked Variable Library** with per-stage value sets. Put Test, Preprod, and Prod values in the item. Teach rebind notebooks to read via supported APIs. Split platform versus domain workspaces explicitly. Respect Git schema so restores work. Pair the library with a promotion story when portal parameters regress.

### Bulk reload is an orchestration problem, not a bigger VM wish
- URL: https://www.geekindata.com/blog/30-bulk-ingest-without-oom
- Date: 2026-07-27
- Summary: InsertAll of multi-million ledger rows on one driver is a known failure mode. The notebook looks simple. The driver dies. Someone asks for a bigger cluster. The next reload dies later, with a larger bill. Treat bulk reload as **orchestration**: batched parallel page ingest, progress monitoring, and entity filters for operable reruns. Separate one-time rebuild paths from incremental watermarks. Publish the pattern to prod only after batch-level observability exists. OOM here is a design smell in notebook API usage, not proof that "Fabric is flaky."

### When warehouse views time out, materialize the trunk not the leaves
- URL: https://www.geekindata.com/blog/29-materialize-heavy-warehouse-views
- Date: 2026-05-28
- Summary: Nine warehouse views timing out usually means two trunks and seven dependents. Teams that rewrite every leaf, or jump straight to a new platform, often miss the cheap win: **materialize the consolidations everything else hangs from**. Use a layered playbook. Drop orphans. Apply cheap predicate and join rewrites. Then materialize the two or three heavy consolidations. Measure with the same harness before and after. Document what is a view versus a managed table so CI/CD stays honest.

### Bridge keys before dashboards: remodel the device fact
- URL: https://www.geekindata.com/blog/28-service-device-fact-remodel
- Date: 2026-05-22
- Summary: Helpful gold views that "just join service and device" hide many-to-many grain until a report page disagrees with itself. Filters double count. Role-playing dims break. Two pages that should reconcile never do. **Deploy bridge keys as a first-class artifact** and remodel the service-device fact before you polish visuals. Matching logic must be shared with telemetry enrichment or the numbers diverge by construction. Semantic role-playing dims need matching foreign key types. Remodel first. Visual polish second.

### Type each column once, as early as you dare
- URL: https://www.geekindata.com/blog/27-bronze-only-cast-typing
- Date: 2026-08-19
- Summary: Re-CASTing the same physical column in every medallion layer feels careful. It is usually how drift multiplies. Bronze casts to decimal. Silver casts again "to be safe." Gold casts a third time and invents an `unknown` literal with yet another type. Consumers inherit a guessing game. **Type each published column once, as early as you dare** (usually bronze, sometimes the first stable clean view). Silver and gold keep aliases, business calculations, and typed unknown-row literals only. UNION alignment casts are the exception, not the lifestyle. Lint the rest.

### Row counts lie: validate MLVs column by column
- URL: https://www.geekindata.com/blog/26-column-level-mlv-validation
- Date: 2026-08-10
- Summary: A migration checklist that only compares row counts will greenlight wrong money, wrong dates, and wrong statuses. Row counts catch catastrophic drops. They miss the quiet CAST that turns decimals into floats, the null that became empty string, and the column that silently defaulted. Push MLV cutovers to **per-column tables and type-aware aggregates**. Borrow the tiered reconciliation classical data migrations already trust: rows, then keys, then column aggregates. Automate it. Treat the artifacts as release evidence, not as a screenshot buried in chat.

### MLV migration is a program: bronze to silver to gold with gates
- URL: https://www.geekindata.com/blog/25-mlv-layer-migration
- Date: 2026-07-30
- Summary: Moving a handful of warehouse views to Materialized Lake Views is a rewrite. Moving hundreds is a program. Teams that treat the long tail like a weekend notebook usually discover dependency order the hard way: gold breaks, silver thrashes, and bronze shortcuts were never registered as the real contract. Frame the work as **bronze to silver to gold with gates**. Count the estate. Order by dependency. Prove a representative POC. Then convert with per-view notebooks so blast radius stays local. Success is parity plus runtime, not "the portal shows an MLV."

### IoT events are facts: model them like service, not like logs
- URL: https://www.geekindata.com/blog/24-device-events-gold-direct-lake
- Date: 2026-06-02
- Summary: Telemetry tables named after pipelines invite people to treat events like disposable logs. Reports then reinvent grain, matching, and captions in DAX. Volume pushes Import mode until refresh becomes the weekly outage. Name the business concept (**events**). Enrich with the **same matching logic** used by service-device facts. Land them in gold for Direct Lake. Captions, query folders, and descriptions are part of the semantic handoff. Watermark shortcuts and clean views are part of the bronze-to-gold story, not footnotes.

### AI import UX is governance: default-deny uncertain rows
- URL: https://www.geekindata.com/blog/23-ai-import-ux-pattern
- Date: 2026-08-11
- Summary: Auto-selecting every AI-analyzed row feels helpful. It is how bad master data enters production with a smile. Users trust the model highlight. They click Import. Months later someone asks why an unknown code became a real record. Treat **AI import UX as governance**. Default-deny uncertain rows. Validity rules follow the live section schema (visible columns), not a hard-coded spreadsheet layout. Footer status separates "selected" from "valid to import." Prompt schema and UI selection policy are one product surface.

### If you cannot explain the tool calls, you do not have an agent product
- URL: https://www.geekindata.com/blog/22-agent-activity-traceability
- Date: 2026-07-24
- Summary: Support opens a session. The UI says two searches and six tools. Foundry traces show something else. App Insights has a third story. Nobody can tell whether the agent searched knowledge, hallucinated a citation path, or the counter lied. That is not a cosmetic bug. That is a missing product surface. **Activity timelines must reconcile platform traces, MCP knowledge searches, and UI counts.** Persist normalized events, not only chat bubbles. Count knowledge searches separately from generic tool invocations. Traceability is part of the customer contract for regulated knowledge agents.

### Foundry webapps deploy as identity and egress, not as Node hostnames
- URL: https://www.geekindata.com/blog/21-foundry-webapp-deploy-pattern
- Date: 2026-07-21
- Summary: Teams still celebrate when the container listens on the right port. For Foundry-backed agent webapps, that is the easy half. The hard half is **who the app runs as**, **how OBO flows**, and **how egress reaches Foundry, Search, and storage** without turning the VNet story into folklore. Deploy the pattern as identity and networking first. Hostname and Node version are table stakes.

### Env files are not environments: isolate with Azure CLI discipline
- URL: https://www.geekindata.com/blog/20-env-isolation-azure-cli
- Date: 2026-09-02
- Summary: Six nearly identical `.env.example` files feel thorough. They guarantee drift. Someone updates three. Prod still reads the fourth. Agents and humans resolve different paths. The repo looks configured. The cloud is the actual environment, and nobody agrees which subscription is live. Treat **Azure CLI context** (subscription, resource group, identity) as the isolation boundary. Keep a small set of live env files plus stage templates that document names, not secrets. POC and MVP can share product code while keeping separate deploy stages.

### Pilot the use case, not the agent catalog
- URL: https://www.geekindata.com/blog/19-agent-use-case-pilots
- Date: 2026-07-24
- Summary: Enterprise agent programs often start from a menu: models, tools, maybe a multi-agent diagram copied from a keynote. The catalog feels like progress. Users still do not know which job the system is supposed to finish. Start from **ranked use cases**. Decide orchestrator versus single agent versus tools per job. Pilot decision quality and handoff clarity before you bet the platform on a topology that looked good in a slide.

### Gold without a semantic contract is just another table
- URL: https://www.geekindata.com/blog/18-gold-device-semantic-contract
- Date: 2026-05-11
- Summary: A gold device table that looks clean in Spark can still fail as a platform asset. Optional links to contracts drop unmatched devices. Free-text site fields hide business keys behind tribal regex. Captions and folders are treated as polish. Stakeholders open a report and half the estate is missing without an error. Gold becomes useful when **unmatched keys, captions, and relationships ship as one semantic contract** with the table. "All devices visible regardless of match" is an acceptance test, not a slogan.

### SCD2 is a product question: point-in-time deployed state with MLVs
- URL: https://www.geekindata.com/blog/17-scd2-mlv-device-location-history
- Date: 2026-08-10
- Summary: Current-state dimensions lie the moment a device moves. Dashboards that say "where is it now" are fine for operations. History questions ("where was it when the alarm fired?") need slowly changing dimension type 2, not a hope that someone kept yesterday's extract. For measuring points and similar deployed assets, I put **location and classification history in SCD2 Materialized Lake Views** before anyone builds "devices as of" pages. The change feed is the source of truth. The MLV is the productized history contract.

### PII redaction belongs in the notebook config, not in tribal knowledge
- URL: https://www.geekindata.com/blog/16-fabric-pii-redaction-config
- Date: 2026-06-25
- Summary: Free-text export columns are where names, phones, and emails hide. The same columns often hold codes and addresses that policy still allows. Blanking everything is lazy. Sprinkling `regexp_replace` through a notebook is how two engineers redact differently on the same Friday. Put the redaction contract in **one config cell** the rest of the notebook reuses. Flags, tokens, patterns, and known-name helpers become reviewable. Residual risk decisions leave chat history and enter the artifact you promote.

### Shortcuts-only bronze: stop copying what OneLake already holds
- URL: https://www.geekindata.com/blog/15-shortcuts-only-bronze
- Date: 2026-06-02
- Summary: Medallion diagrams still tempt teams to land a physical bronze copy of everything upstream. When the upstream domain already publishes curated Delta in OneLake, that copy is not prudence. It is a second warehouse with a friendlier name. Bronze can be a **contract of shortcuts and thin clean views**. Ownership stays with the publisher. Freshness follows the source. Your CI/CD list becomes the shortcut set, not a shadow catalog of cloned tables.

### Canvas apps at platform scale: push work to Dataverse
- URL: https://www.geekindata.com/blog/14-canvas-apps-at-platform-scale
- Date: 2026-08-13
- Summary: Canvas is wonderful until one screen becomes a distributed system: nested galleries, OnStart fan-out, and business rules living in Power Fx because it was faster. Microsoft's own performance reference architectures push a clear direction. Move heavy work toward Dataverse and keep screens focused.

### Genie readiness is a modeling problem, not a prompt problem
- URL: https://www.geekindata.com/blog/13-genie-readiness-is-a-modeling-problem
- Date: 2026-08-20
- Summary: Teams ask how good the data needs to be before conversational analytics helps. The honest answer: good enough that a careful analyst would trust the same tables without a chat UI. Prompts cannot rescue ambiguous grain, cryptic column names, or missing relationships.

### POC public vs MVP private: networking as architecture for agent apps
- URL: https://www.geekindata.com/blog/12-poc-public-vs-mvp-private-agent-apps
- Date: 2026-09-04
- Summary: A public POC proves the product loop. A private MVP proves you can survive enterprise networking. Those are different architectures. Do not fake the second by adding more env files.

### Fabric CI/CD blast radius: when no changes still deploys
- URL: https://www.geekindata.com/blog/11-fabric-cicd-blast-radius
- Date: 2026-08-07
- Summary: Your pipeline runs. You did not touch half the notebooks. They still publish. That is not always a bug in your YAML. Fabric deployment models and item pairing often optimize for workspace consistency, not git-style minimal diffs. If you design as if only changed files move, production will teach you otherwise.

### Legacy to lakehouse: semantic parity is the real migration
- URL: https://www.geekindata.com/blog/10-legacy-to-lakehouse-semantic-parity
- Date: 2026-08-12
- Summary: Copying tables into a lakehouse is the easy part. The hard part is whether 0 means zero or was null in the legacy system, and whether a 0.01 drift is rounding or a bug. I treat migration as a semantic contract, not a file move. Stakeholders do not celebrate that Delta exists. They celebrate that last month's margin still matches within agreed rules.

### Semantic refresh architecture: notebooks, pipelines, and Direct Lake
- URL: https://www.geekindata.com/blog/9-semantic-refresh-and-direct-lake
- Date: 2026-08-06
- Summary: Fresh gold tables are useless if the semantic model still points at yesterday, or if refresh burns an hour on a fact you could have incrementalized. Refresh is an architecture problem: orchestration, identity, and storage mode. Treat it as a portal checkbox and you invent a second process nobody can explain.

### One Fabric lakehouse with schemas vs many lakehouses
- URL: https://www.geekindata.com/blog/8-one-lakehouse-schemas-vs-many
- Date: 2026-08-05
- Summary: Medallion architecture asks where bronze, silver, and gold live. In Fabric you can answer with many lakehouses or with schemas inside one lakehouse. That choice leaks into shortcuts, SQL endpoints, notebooks, CI/CD, and how painful renames become later. It is not a folder preference. It is a topology decision.

### Agent sessions vs the shell app: what should survive refresh
- URL: https://www.geekindata.com/blog/7-agent-sessions-vs-the-shell-app
- Date: 2026-07-24
- Summary: Users expect ChatGPT rules: close the tab, come back, the thread is still there. Platform agent services often have their own conversation store. Your shell is a third thing. If those three disagree, you get untitled threads from the portal, agents that look "stopped" after refresh, and incomplete chats that vanish after a new sign-in.

### Knowledge RBAC for enterprise agents: same gates for chat, browse, and citations
- URL: https://www.geekindata.com/blog/6-knowledge-rbac-for-enterprise-agents
- Date: 2026-08-04
- Summary: An agent that answers from documents is only as trustworthy as the document permissions behind it. Teams often lock the chat endpoint carefully, then leave library browse and citation PDF downloads open to anyone who can sign in. That is not a knowledge boundary. That is a second door with a nicer UI.

### Signing in is not authorization: app access is not agent access
- URL: https://www.geekindata.com/blog/5-app-access-vs-agent-access
- Date: 2026-08-17
- Summary: Most “AI app” demos collapse three different questions into one login button: 1. Can this person open the shell? 2. Which agents are they allowed to talk to? 3. Which knowledge are they allowed to see, download, or ground answers on? In a real enterprise agent webapp, those are **three different control planes**. Treat them as one and you get either a security hole or a support nightmare.

### Can Databricks Genie Build My Deck and Deploy My App?
- URL: https://www.geekindata.com/blog/4-databricks-genie-powerpoint-webapp
- Date: 2026-04-08
- Summary: Can Databricks Genie make a PowerPoint like this based on my data and use my company template exactly? Can it develop and deploy a web app integrating my Genie space? Those are the questions I keep getting after demos. I have been stress-testing Genie Code on two jobs: turn governed data into an editable deck, and ship a small web app that lands on the platform. That raises the bar for what enterprise data platform copilots can own end to end.

### Image Classification Web App with Azure AI Services
- URL: https://www.geekindata.com/blog/3-azure-image-classification
- Date: 2026-04-08
- Summary: Custom Vision can get you from labeled images to a prediction endpoint quickly. The product work starts after the happy path: bad uploads, low-confidence answers, iteration that breaks the published endpoint, and no breadcrumbs when the label is wrong. Keep the UI boring: upload, predict, show label and confidence. Put the complexity in data quality, validation, request limits, logging, and a controlled publish path for new iterations.

### Real-time Motion Tracking and Pose Estimation
- URL: https://www.geekindata.com/blog/2-pose-estimation
- Date: 2026-04-08
- Summary: Point a camera at a person, draw a skeleton, call it live. That demo survives a recorded clip and dies on a messy stream: variable lighting, dropped frames, busy backgrounds, and users who feel 80 ms of lag even if they cannot name it. Accuracy without a calm loop is a lab number. End-to-end time to a stable overlay is what people experience. Treat capture, inference, and render as three jobs that can each fail independently.

### Automate Your Microsoft Fabric Workloads with Apache Airflow
- URL: https://www.geekindata.com/blog/1-fabric-airflow
- Date: 2026-04-08
- Summary: Fabric notebooks and pipelines are great at doing work. They are mediocre at being the calendar, the dependency graph, and the on-call story for everything around them. The moment three jobs must finish before a fourth can start, you need an orchestration plane that is not pretending to be another transform engine. Airflow fits when you want an honest DAG: schedule, dependencies, retries, and a run history humans can read. Keep tasks thin. Call Fabric. Do not rebuild Fabric inside operators because it was convenient on a Friday.