7 posts tagged with "Experiments"

Materialized Lake View runs paste warning floods that mention native execution engine fallbacks. Teams panic-rewrite Spark SQL to chase every line. Meanwhile the DAG's wall-clock still hides in one slow node nobody measured.

Measure first. Pasted warnings include Delta and MLV metadata internals you cannot and need not remove. Wall-clock attribution beats fear-driven rewrites. Fix the few fallbacks that move latency. Ignore the rest deliberately. Document accepted warnings so on-call does not thrash. Pair with per-view notebooks so slow nodes are isolatable.

Figure 1. Attribute MLV time with job details and DAG reality. Do not treat every NEE warning as a rewrite mandate. Source: Microsoft Learn: Spark detail monitoring.

Enterprise agent programs often start from a menu: models, tools, maybe a multi-agent diagram copied from a keynote. The catalog feels like progress. Users still do not know which job the system is supposed to finish.

Start from ranked use cases. Decide orchestrator versus single agent versus tools per job. Pilot decision quality and handoff clarity before you bet the platform on a topology that looked good in a slide.

Figure 1. Pilot the job the shell must serve. Agent topology follows the use case, not a catalog of models looking for work. Source: Microsoft Learn: authentication flows and app scenarios.

Users expect ChatGPT rules: close the tab, come back, the thread is still there. Platform agent services often have their own conversation store. Your shell is a third thing.

If those three disagree, you get untitled threads from the portal, agents that look "stopped" after refresh, and incomplete chats that vanish after a new sign-in.

Figure 1. The shell acquires tokens and calls APIs. Conversation durability is a separate design. Source: Microsoft Learn: authentication flows.

Pipeline graphs that live in people's heads do not survive vacation handoffs. Ad-hoc docs rot. A "harmless" bronze rename still nukes gold because nobody saw the edge.

Build a small lineage explorer over metadata you already emit: notebooks, MLVs, shortcuts, pipelines. Read-only exploration is enough for v1. Treat lineage UX as platform product, not a slide in an architecture deck.

Figure 1. Lineage that lives in people's heads does not survive a rename. Explore metadata before you change identities. Source: Microsoft Learn: lineage in Fabric.

Figure 2. A thin read-only explorer over notebooks, MLVs, shortcuts, and pipelines is platform product, not a slide. Source: Microsoft Learn: lineage in Fabric.

Point a camera at a person, draw a skeleton, call it live. That demo survives a recorded clip and dies on a messy stream: variable lighting, dropped frames, busy backgrounds, and users who feel 80 ms of lag even if they cannot name it.

Accuracy without a calm loop is a lab number. End-to-end time to a stable overlay is what people experience. Treat capture, inference, and render as three jobs that can each fail independently.

Figure 1. Vision pipelines succeed when the frame path stays explicit: capture, analyze, present. Pose work has the same loop, just with landmarks instead of boxes. Source: Microsoft Learn: image analysis overview.

Custom Vision can get you from labeled images to a prediction endpoint quickly. The product work starts after the happy path: bad uploads, low-confidence answers, iteration that breaks the published endpoint, and no breadcrumbs when the label is wrong.

Keep the UI boring: upload, predict, show label and confidence. Put the complexity in data quality, validation, request limits, logging, and a controlled publish path for new iterations.

Figure 1. Labels and confidence are the user contract. The training UI is where iteration happens; the web app should not invent a second story. Source: Microsoft Learn: Custom Vision overview.

Can Databricks Genie make a PowerPoint like this based on my data and use my company template exactly? Can it develop and deploy a web app integrating my Genie space?

Those are the questions I keep getting after demos. I have been stress-testing Genie Code on two jobs: turn governed data into an editable deck, and ship a small web app that lands on the platform. That raises the bar for what enterprise data platform copilots can own end to end.

Figure 1. Genie Code in Databricks workspace context. Source: Databricks Genie.