OpenAI Agents API vs Azure Foundry Hosted Agents: Where Should Your Agent Run?

AI agents are moving through the same maturity curve we have seen many times before.

First, everything lives in the app. Then teams extract a few reusable helpers. Then a managed service appears and promises to handle the boring parts. Finally, everyone realizes the hard question was never "can I call the model?" It was "where should this thing run, and who owns the operational risk?"

That is exactly where agentic systems are now.

OpenAI has introduced the Agents API in public beta, exposing a managed version of the Codex-style harness behind a developer API. Microsoft Foundry has also been pushing hard in the same direction. Its July and August updates made Hosted Agents, Toolboxes, Voice Live integration, and several SDK improvements much more relevant for production work. At the same time, Python, .NET, and TypeScript developers still have strong reasons to keep parts of the agent loop inside their own application using SDKs and normal service boundaries.

So the practical question is not "which one is best?"

The better question is:

Where should the agent loop live?

That decision affects your tool security, debugging story, cloud boundary, latency, cost controls, data handling, and how painful it will be to change models later.

The four runtime choices

For most teams, the options now fall into four buckets.

The first is a fully managed agent runtime. OpenAI's Agents API fits here. You define the agent, task, tools, model, and execution environment through an API, and OpenAI runs the harness for you. This is attractive when you want to avoid building the orchestration loop, session management, tool dispatch, sandbox integration, and lifecycle plumbing yourself.

The second is an application-embedded SDK approach. This is where OpenAI Agents SDK, Responses API patterns, Semantic Kernel, LangGraph, Pydantic AI, or your own orchestration layer run inside your service. You own the loop. You own persistence. You own retries. You also get very direct control over the business transaction boundary.

The third is a cloud-hosted bring-your-own-code agent. Azure Foundry Hosted Agents are a good example. You package your agent as code, commonly as a container serving a supported contract, and let Foundry handle managed deployment, scaling, identity integration, networking, and observability hooks. This is a middle ground: you still own the agent implementation, but the platform owns more of the runtime surface.

The fourth is a conventional service with agentic calls inside it. This is still underrated. Not every workflow deserves a long-running autonomous runtime. Sometimes the right architecture is an ASP.NET Core API, a Python worker, or an Azure Function that calls a model, invokes a small number of tools, validates the result, and returns.

If that sounds less exciting, good. Boring architecture is often what survives production.

Start with the failure mode, not the feature list

The mistake I see in agent architecture discussions is starting with capabilities:

• Does it support tools?
• Does it support MCP?
• Does it have memory?
• Can it run code?
• Can it call web search?
• Can it work with Python or .NET?

Those questions matter, but they are not the first questions.

Start with failure instead.

What happens when the tool call half succeeds? What happens when the model chooses the wrong tool? What happens when the user closes the browser but the workflow should continue? What happens when the downstream API rate-limits you? What happens when the model consumes four times the expected tokens? What happens when the agent sees data it should not have seen?

The runtime choice becomes much clearer when you map those failures to ownership.

If the workflow is mostly exploratory, developer-facing, or task-oriented, a managed runtime can remove a lot of plumbing. If the workflow sits inside a regulated business transaction, you may want the agent loop closer to your own service and database so you can enforce invariants with normal code. If the workflow needs enterprise networking, Azure identity, regional controls, and operational visibility inside an Azure estate, Foundry Hosted Agents become more interesting.

The question is not whether managed agents are "production ready" in the abstract. The question is whether their operational boundary matches your production boundary.

When the OpenAI Agents API makes sense

The Agents API is most attractive when the harness itself is not your differentiator.

Think about internal research agents, code-maintenance assistants, operational runbooks, support investigation workflows, or developer tools where the main value is connecting a strong model to a controlled set of tools and letting it work through a task.

In those cases, building your own durable agent harness can become accidental platform work. You start with a simple loop. Then you add tool schemas. Then retries. Then sandboxing. Then compaction. Then session history. Then subagents. Then credential handling. Then logs. Suddenly the "small agent feature" is a platform team.

A managed Agents API can be a good fit when you want:

• faster time to a working agent;
• a managed harness rather than a custom orchestration loop;
• built-in session and environment concepts;
• tool execution isolated from your main application;
• less operational code around agent lifecycle.

The trade-off is control.

You need to understand where data is processed, which compliance modes are supported, how credentials are represented, how tool execution is isolated, and how much of the runtime behavior you can observe and test. For some workloads, that trade-off is perfectly fine. For others, especially customer-data-heavy or region-sensitive workflows, it may be a blocker until the platform controls match your requirements.

My default view: use the Agents API when you want a managed agent product surface, not just a model call. Do not use it merely because it is new.

When Azure Foundry Hosted Agents make sense

Azure Foundry Hosted Agents become compelling when your organization already runs a meaningful part of its platform on Azure and wants agents to behave like other managed workloads.

This matters more than it sounds.

Production agent systems are not only model prompts and tool definitions. They need identity, network boundaries, deployment pipelines, environment variables, monitoring, private connectivity, policy, and cost visibility. Those are usually not optional in enterprise systems.

Foundry's current direction is interesting because it treats agents as deployable workloads rather than chat widgets. Hosted Agents let you bring your own code or framework, package it for the platform, and still get a managed runtime around it. Toolboxes also point toward a healthier pattern: register tools once, manage them centrally, and attach them to agents without copying tool glue into every project.

For .NET and Azure teams, this is especially relevant. You can keep the agent close to familiar operational primitives: Azure Container Apps, managed identity, Application Insights, private networking, and normal CI/CD. For Python teams, the Foundry SDK and hosted patterns give a path from local experimentation to managed deployment without pretending that a notebook is an architecture.

Hosted Agents are a good fit when you want:

• ownership of the agent code and framework;
• Azure-native deployment and governance;
• private networking and enterprise identity integration;
• central tool registration through platform-managed concepts;
• observability that can connect to the rest of your Azure estate.

The trade-off is platform complexity.

You are now designing a cloud workload. That means manifests, containers, deployment scripts, role assignments, logs, scaling, and environment management. If your agent is simple, this may be unnecessary weight. If your agent is business-critical, this weight is usually the price of sleeping better.

When the SDK-in-your-app approach still wins

Do not underestimate the value of keeping the agent loop inside your own application.

If the workflow is tightly coupled to your domain model, your authorization rules, or a database transaction, embedding the agent inside your app can be the cleanest architecture. The model becomes a reasoning component inside a service, not a semi-independent actor outside it.

This is often the right move for:

• customer-facing flows with strict authorization rules;
• workflows that need strong schema validation before every state change;
• systems where rollback and idempotency are domain-specific;
• scenarios where you already have background workers and durable queues;
• teams that want to test the agent loop like any other application code.

In .NET, that might mean an ASP.NET Core API or Worker Service using Semantic Kernel, the OpenAI SDK, Azure SDKs, or a small custom orchestration layer. In Python, it might be FastAPI, Celery, Durable Functions, Pydantic AI, the OpenAI Agents SDK, or your own task runner.

The main downside is that you own the boring parts.

That includes persistence, context management, tool retries, timeout behavior, traces, redaction, cost accounting, and migration paths when model behavior changes. But owning those parts is not always bad. If those concerns are part of your product's correctness, you probably want them in your codebase anyway.

A practical decision model

Here is the simple model I would use before choosing a runtime.

First, classify the agent's authority.

If it only reads data and produces recommendations, managed runtime options are easier to adopt. If it can write to systems, spend money, change customer state, or trigger external communication, keep the control plane closer to your own authorization and audit systems.

Second, classify the workflow duration.

Short request-response flows often fit inside your app. Longer-running workflows with multiple turns, tool calls, checkpoints, and resumability may justify a managed agent runtime or a hosted agent platform.

Third, classify the tool boundary.

If tools are generic and reusable, platform-managed tool registries or managed agent APIs make sense. If tools encode sensitive business rules, keep them behind your service layer and expose only narrow, policy-checked operations to the agent.

Fourth, classify the operational boundary.

If your production operations already live in Azure, Foundry Hosted Agents may integrate better with your deployment and monitoring model. If your agent is primarily OpenAI-centric and you want the Codex-style harness as a service, the Agents API may reduce build time. If your system is multi-cloud or deeply custom, an SDK approach may preserve flexibility.

Finally, classify your tolerance for platform churn.

Agent platforms are moving quickly. That is not a criticism; it is the stage of the market. But it means you should isolate your business logic from the runtime. Put your tool contracts, domain validation, and state transitions behind interfaces you control. Treat the agent runtime as a replaceable orchestration layer, not the place where all business rules go to hide.

A reference architecture that works in practice

For many production teams, I would start with a split architecture.

Keep the business tools behind your own API. Make those tools boring: authenticated, scoped, idempotent where possible, and heavily logged. The agent should not get raw database access just because it is convenient.

Then choose the agent runtime based on the workflow:

• use a managed Agents API for task-oriented agents where the harness is mostly generic;
• use Azure Foundry Hosted Agents when the agent is a deployable workload in an Azure-governed environment;
• use an SDK inside your app when the workflow is part of a tightly controlled business process.

Add an evaluation and observability layer from the beginning. Capture the user goal, selected tools, tool inputs, tool outputs, model output, token usage, latency, and final decision. Redact sensitive data before it leaves the boundary where it belongs. Run regression tests against real examples before changing models or prompts.

This is where many teams get agent architecture wrong. They start by making the model more capable. Production systems usually improve faster when you make the environment more constrained and observable.

What I would avoid

I would avoid building a custom general-purpose agent platform unless you are sure that is your product or internal platform mission.

It is tempting. Engineers like frameworks. Agent loops look simple at first. But the serious work is not the first tool call. It is the hundredth edge case: partial failure, duplicate execution, prompt injection through tool output, trace correlation, credential scoping, runaway cost, stale memory, and model migration.

I would also avoid putting high-authority tools directly into a managed agent runtime without a mediation layer. The agent does not need "update customer" as a broad capability. It needs "submit a proposed update that passes policy and validation." That difference matters.

And I would avoid treating memory as magic. Durable memory is useful, but it is also a data-management feature. It needs expiry, provenance, correction, access control, and a way to inspect what the agent thinks it knows.

The takeaway

Managed agents are becoming real infrastructure. That is good news.

OpenAI's Agents API can remove a lot of harness work when you want a managed agent surface. Azure Foundry Hosted Agents can make sense when you want bring-your-own-code agents that fit Azure operations. SDK-based approaches still win when the agent is deeply embedded in your application and business rules.

The best architecture is not the one with the most agent features. It is the one where ownership is clear.

Who owns the tools? Who owns the data boundary? Who owns the failure handling? Who owns the traces? Who owns the cost controls? Who can explain why the agent did what it did?

Answer those questions first. The runtime choice gets much easier after that.