Agent Pilots: Should You Rank Use Cases Before Picking an Agent Topology?

Enterprise agent programs often start from a menu: models, tools, maybe a multi-agent diagram copied from a keynote. The catalog feels like progress. Users still do not know which job the system is supposed to finish.

Start from ranked use cases. Decide orchestrator versus single agent versus tools per job. Pilot decision quality and handoff clarity before you bet the platform on a topology that looked good in a slide.

Figure 1. Use case before topology. Rank the jobs, choose the runtime on purpose, pilot the scorecard, and leave the catalog as inventory.

Context and constraint

A common failure: the first draft of a Foundry-backed assistant plan is "stand up many agents and see." The demos are busy. The operating model is not. Job type A wants specialists. Job type B wants one careful agent until volume proves otherwise.

Rank jobs by user value, risk, and evidence availability. For each job, choose single agent, orchestrator plus specialists, or mostly tools with thin generation. Code interpreter and export features are use-case capabilities, not defaults for every agent. Agent-to-agent boundaries should follow human job splits. Mirroring existing source structure is a trap: it recreates yesterday's org chart as tomorrow's agent graph.

Figure 2 is the Learn framing for user-facing app scenarios. The shell is a product surface. Agent topology still has to follow the job.

Figure 2. Pilot the job the shell must serve. Agent topology follows the use case, not a catalog of models looking for work. Source: Microsoft Learn: authentication flows and app scenarios.

Design

1. Rank use cases before endpoints

Write the job, the user, the evidence sources, the decision or artifact produced, and the failure cost. Sort by value and readiness. Only then pick models and tools. A beautiful endpoint list without ranked jobs is procurement theater.

2. Match topology to the job shape

Workflows with distinct specialist skills often want an orchestrator plus specialists. A narrow, single-purpose Q&A job may stay single-agent until concurrency and specialization pressure appear. Do not deploy multi-agent runtime complexity to impress architecture review when a single agent with tools clears the pilot bar. Figure 3 is that match.

Figure 3. Match topology to the job shape. A narrow job can stay single-agent. A multi-skill job earns an orchestrator. Tools-first jobs keep generation thin and privileges least.

3. Treat tools as product features per use case

Code interpreter, export, browse, and MCP knowledge search are not a badge to pin on every agent. Enable them where the job needs them. Disable or hide them where they create risk without value. Feature lists that ignore use cases become attack surface and support surface.

4. Draw A2A boundaries like human handoffs

If two human roles would not share a desk and a queue, do not merge them into one agent "for simplicity," and do not split them into five agents because the repo has five folders. Boundaries should match responsibility, escalation, and audit needs.

5. Start from outcomes and evidence, not existing structure

Old source inventories and prior assistant intents bias you toward material that exists rather than decisions users need. Inventory evidence for the ranked jobs. Gaps become data workstreams, not prompt wishlists.

6. Pilot on decision quality and handoff clarity

Success criteria: can a user finish the job, can a reviewer see why, can the next human step pick up cleanly. Token spend and latency matter, but they are not the headline metric for a pilot gate. Write the scorecard before the build.

When to promote from pilot to platform bet

Promote when the scorecard passes with real users, evidence gaps have owners, and the topology choice still matches the job under light load. Do not promote because the catalog looks empty or because a vendor roadmap slide showed multi-agent. Platform bets fund shared identity, tracing, and evaluation harnesses. Those investments pay off only when use cases are real enough to stress them.

What "ranked" means in practice

Ranking is not a sticky-note workshop that vanishes. It is a short ordered list in the repo with owner, risk, evidence sources, and topology choice per row. Revisit when a pilot fails or when volume forces a split from single agent to orchestrator. The list is allowed to change. Shipping agents that never appear on the list is not.

Figure 4 is the practical gate that keeps the catalog from growing zombies.

Figure 4. No new agent identity until a use-case card exists with owner, scorecard, and topology choice. Catalog growth without cards is how platforms accumulate demo zombies.

Common mistakes

Catalog-first platforms. Many agents, unclear jobs, identical prompts with different names.

Multi-agent by default. Orchestration cost without specialization benefit.

Tools on for everything. Export and interpreter enabled where policy never approved them.

Repo-folder A2A. Engineering structure mistaken for product structure.

Source-inventory archaeology as roadmap. You automate the archive, not the work.

Token dashboards as pilot success. Cheap wrong answers still fail the user.

Use as-is or adapt

Use as-is

• Ranked use-case list gates platform topology decisions.
• Per use case: single agent vs orchestrator vs tools-first, with rationale.
• Tools and exports enabled only where the use case requires them.
• A2A boundaries follow human job splits and audit lines.
• Pilot scorecard emphasizes decision quality and handoff clarity.
• Existing source inventories are inputs to evidence inventory, not the agent graph.

Adapt before you use

Use-case ranking slows the first demo and saves the second rewrite. Single-agent pilots may need a later split when volume grows; that is cheaper than premature orchestration. Strict tool least-privilege annoys builders who want everything enabled in dev. Keep a lab profile separate from the pilot profile you show stakeholders. Evidence-gap work can dwarf model work. That usually means you found the real program.

Primary references: Microsoft Foundry multi-agent and agent concepts on Microsoft Learn, plus your product principles for human handoff and approval. Pair platform docs with a written use-case scorecard in the repo.

A practical gate: no new agent identity in the catalog until a use-case card exists with owner, scorecard, and topology choice. Catalog growth without cards is how platforms accumulate zombies that still show up in demos.

Pilot the use case. Choose topology on purpose. Leave the catalog as inventory, not strategy.