Key takeaways
- A hosted agent moves the runtime: your code, in a container, with its own identity and endpoint. It does not move sign-in, app roles, or which user may open which session.
- Move an agent off a prompt agent only when the agent itself needs code a prompt agent cannot express. A nicer deploy story is not a reason.
- The credential your backend calls with sets the isolation scope. Call with the app's own identity and no user header, and every user lands in one scope.
- Per-user isolation for your own users needs a user header and a custom role, and even then the platform does not fence delegated users from each other. Your session map does.
- Guardrails can fail open while everything returns HTTP 200. Test a blocked prompt per protocol before you rely on it.
- One version per endpoint, no traffic splitting: rollout and version pinning stay with your front door.
With Foundry hosted agents generally available, the obvious question for anyone running an agent web app is: can we drop our backend now? Foundry will run our code, give it an identity and an endpoint, and scale it to zero. Why keep an App Service in front of it?
I have not moved an agent to the hosted runtime yet. On the agent apps I build, agents are prompt agents behind our own backend, and "hosted" exists as a parked runtime value in the agent catalog, behind a flag. This post is why it is parked, what would make me turn it on, and why the backend stays either way.
Named model: runtime moves, front door stays
- Runtime plane (can move to Foundry): the code that runs the agent, its sandbox, its scaling, its agent identity, its guardrails and egress rules.
- Front door plane (stays yours): who the user is, what they may use, which sessions and conversations are theirs, which version they get, and what the product shows them.
- The seam between them: one call per user, with a credential that tells Foundry who that user is.
Figure 1. Hosted agents replace the right-hand panel. Nothing on the left moves with it.
Context and constraint
Foundry Agent Service has three agent types (overview). Prompt agents are configuration: instructions, a model and tools, and Foundry runs them. Hosted agents are your code, in any framework, packaged as a container that Foundry runs with a managed endpoint, scaling and a dedicated Microsoft Entra agent identity (hosted agents).
By front door I mean the backend between the browser and Foundry: it signs the user in, checks app roles, decides which agents and sessions they can reach, streams the response and renders it. Post 21 and post 38 cover why that layer exists at all. This post is about whether hosted agents change that answer. Product status below is as of October 2026.
Design
1. Decide on what the agent needs, not on deploy convenience
Learn lists when to choose hosted agents over prompt agents. Reduced to the questions I ask about one agent:
- Does it need its own orchestration, or a framework graph, that instructions and tools cannot express?
- Does it need a protocol or channel the prompt agent does not speak?
- Does it need files and working state in a sandbox that survive across turns?
- Does it run long, or in the background, past a normal request?
- Does it need its own egress rules on outbound calls?
If every answer is no, a prompt agent is less to own. A hosted agent brings an image to build and patch, a registry to secure, and CPU and memory to size. Sizing is per session and billing counts CPU and memory across all active sessions, so, as Learn puts it, oversizing multiplies cost by your concurrency. Idle sessions scale to zero after a timeout you set, and a cold resume is part of your user experience.
If one answer is yes for one agent, that agent moves. Not the app.
Figure 2. The question is per agent. The answer sets one catalog field, not the architecture.
2. Runtime is a per-agent field, not an app-wide switch
This is the design choice I would keep regardless of the platform roadmap. Each agent in our catalog carries how it runs. Today every entry says prompt agent. The hosted value exists, behind a flag, with nothing using it. When an agent needs it, one entry changes and the front door routes that agent's calls to its hosted endpoint. Sign-in, roles, citations, traces and the conversation list are designed not to care where the agent runs, so they should not change. Tool approvals are the part I would retest, because custom code can raise them differently.
The alternative, deciding "we are a hosted agents app" up front, couples the product to a runtime choice that only some agents need. Post 36 has the flag and overlay pattern I use for this.
3. The credential your backend calls with is a design decision
Hosted agents isolate sessions by caller identity by default: each caller lists and manages only its own sessions (isolate sessions per user). That sounds like the platform did your authorization for you. It did not. It depends on who Foundry thinks the caller is, and your backend decides that:
- Forward the user. The backend calls on behalf of the signed-in user, so Foundry sees that user. Isolation is per user, and tools can act with the user's own access through per-connection user delegation (user delegation).
- Call as the app, with no user header. Foundry sees one caller. Every user's sessions sit in one scope, and there is no user context for tools to delegate.
- Call as the app, with a user header. The backend sends a stable user identifier in
x-ms-user-identity. Sessions are scoped to that value. The calling identity needs a data action that no built-in role includes, so you create a custom role for it; without it, the call returns 403. The delegation page adds that a service calling for many users should send this value per user, so users do not share OAuth sign-in state for tools.
The third shape has a sharp edge. Learn's warning: within delegation, the platform does not fence one delegated user from another. It only separates delegated from non-delegated callers. Route two users to the same session ID and they can see each other's data. The sessions page says it plainly: the isolation key "is a partitioning value, not an authentication or authorization mechanism" (manage sessions). Learn also notes that an identity with the Foundry User role on the project can list and manage every session on the agent. Know who holds it.
I have seen the same trap with prompt agents. When our backend calls with its own managed identity, Foundry sees one caller, so the list of a user's conversations has to come from our own per-user index, not from Foundry. Hosted agents do not remove that index. They make it more important.
4. A session is not a conversation
Hosted agents separate two things that chat UIs tend to merge (manage sessions). A session is the sandbox: compute and a persisted filesystem. A conversation is the history of messages and tool calls. On the Responses protocol, reusing a session ID does not replay history; continuity comes from the previous response or a conversation ID. On the Invocations protocol, the platform stores no history at all.
So the front door still owns the conversation index: which conversations belong to which user, their titles, and how to resume one after the stream drops. Post 7 covers why that clock is separate from the UI, and post 47 covers where the index lives. Hosted agents add a third thing to map: which session belongs to which conversation, and to which user.
One more rule from the same page: files uploaded to a session land in the agent's home directory. Treat them as untrusted, and never load code, plugins or configuration from there.
5. Rollout stays with you
A hosted agent endpoint serves one version at a time, routes all traffic to it, and does not split traffic between versions (hosted agents). Each session binds to one version when it is created, and version_indicator pins a session to a concrete version so later turns keep it after you publish a newer one.
That gives you the parts of a rollout, not the rollout. A canary, a pilot group on the new version, or a rollback that leaves running conversations alone are decisions about users, so they belong to the front door that knows the users. The eval gate in post 48 runs before a version is published; the front door decides who meets it first.
6. Test the block, not the deploy
Guardrails on hosted agents attach to the agent definition, and the guardrails page lists several ways they fail without an error (guardrails):
- A policy name that does not exist: on many subscriptions the agent deploys, reports active, and filters nothing.
- The Invocations protocol without
invocations_moderation: the policy is inert, requests pass unfiltered, and the agent returns HTTP 200. - The Invocations WebSocket protocol: content moderation is not available.
- A policy change: running sandboxes do not pick it up. You deploy a new agent version and start or resume a session.
Streaming adds a front door job. A prompt blocked at input is rejected before any event. But on the Invocations protocol, output blocked partway through a stream arrives as an error event inside an HTTP 200 response, and earlier text may already be on screen. Learn says it directly: a 200 status alone does not mean the response passed the policy. The UI has to treat that event as terminal and say the answer was stopped, not leave half an answer looking complete.
Egress controls for hosted agents are in preview and, in Learn's words, complement your own network controls rather than replace them.
Figure 3. Each of these returns success. The test is a prompt that should be blocked and a user who should be refused.
7. What would make me turn the flag on
The day one agent needs a framework graph, sandbox files across turns, or work that outlives a request, that agent's catalog entry changes to hosted. Before it does, three checks: the backend calls with a credential that carries the user, a blocked prompt is blocked on the protocol that agent uses, and a second user cannot reach the first user's session through our API. None of those checks is about the hosted runtime. All of them are about the front door.
Common mistakes
Dropping the backend because the runtime moved. Sign-in, roles and the session map were never in the runtime.
Calling as the app with no user header. Every user shares one isolation scope.
Reusing one session ID across users. The platform does not fence delegated users from each other.
Treating the session ID as the conversation. It does not replay history.
Trusting a guardrail because the deploy succeeded. Test a blocked prompt per protocol.
Treating HTTP 200 as success in a stream. Handle the error event.
Sizing every session for the worst case. The bill multiplies by concurrency.
Making runtime an app-wide switch. Make it a field per agent.
Use as-is or adapt
Use as-is
- The five questions per agent, and moving only the agent that answers yes.
- Runtime as a field in the agent catalog, behind a flag until an agent needs it.
- A per-user conversation and session index in your own store, checked on every call.
- One session per user, never shared across delegated users unless you designed for it.
- A blocked-prompt test per protocol as part of every release.
Adapt before you use
- Forwarding the user versus a user header from the app: it depends on whether your users sign in with Entra and on what your tools need to act as the user.
- How you pin versions for a pilot group, given there is no traffic splitting.
- Idle timeout and session sizing, measured on a representative workload rather than guessed.
- Whether a stopped stream retracts the partial text or marks it, depending on what your users may already have copied.
Related
- Microsoft Foundry series (all posts on production agent apps)
- Microsoft Foundry: Is Deploy as Web App Enough, or Do You Need a Backend Proxy? (why the front door exists)
- Secure AI Shells: Should the Browser Talk to Foundry, or Only Through a Proxy? (the proxy boundary)
- Microsoft Foundry: How Do You Persist Agent Sessions Outside the Chat Shell? (conversation clocks)
- Foundry Agent Webapps: Cosmos or Azure SQL for Backend State, and When? (where the index lives)
- Agent Shells: How Do You Feature-Toggle Agent Capabilities Across Environments? (flags and overlays)
- Agent Evals: Can a Score Block a Release? (gating a version)
Let Foundry run the agent. Keep owning who gets to talk to it.