Every capability, every provider, and the production playbook nobody hands you — including how to take the exact same agent code from OpenAI to vLLM, NVIDIA Triton, Together AI, and Azure AI Foundry.

When teams sit down to build their first serious agentic system, a strange thing happens: most of them reach for the OpenAI Agents SDK first. Not because it's the only option — LangGraph, CrewAI, Autogen, and a dozen others exist — but because it gets out of your way. It is small, typed, unopinionated about your infrastructure, and provider-agnostic by design. You can prototype an agent in ten lines and still ship the same code to production behind your own GPUs.

This guide is the long version of that promise. We'll walk through every major capability of the SDK, then spend the second half on the part that actually matters: running it in production. Along the way we'll cover the scenario that motivates so many adoptions — building an MVP on OpenAI and later moving the model on-prem to a vLLM or Triton backend, or to Azure AI Foundry, without rewriting your agents.

The code in this guide targets the current openai-agents Python package. Install it with pip install openai-agents (Python 3.10+).

Part 1 — Why this SDK keeps winning the first decision

Before the API tour, it's worth being precise about why the Agents SDK is the default starting point for so many projects.

It's a thin orchestration layer, not a framework that owns your app. The two primitives — Agent and Runner — manage the turn loop (call the model, run tools, follow handoffs, enforce guardrails, persist sessions). Everything else is plain Python objects you can inspect, clone, and test.

It's provider-agnostic. It speaks the OpenAI Responses API and Chat Completions API natively, and through one line of configuration it talks to any OpenAI-compatible endpoint, plus 100+ models via adapters. This is the feature that de-risks the whole project: your model choice is not a one-way door.

It's typed and small. Pydantic-backed schemas, real generics on context, and a tiny surface area mean your IDE and type checker do real work for you.

Tracing is built in. From the first run you get a structured trace of model calls, tool calls, handoffs, and guardrails — locally and in the OpenAI Traces dashboard — with first-class integrations for Logfire, Langfuse, Braintrust, Datadog, W&B, MLflow, and many more.

Production concerns are first-class, not afterthoughts. Retries, timeouts, human-in-the-loop approvals, durable execution integrations (Temporal, Dapr, Restate, DBOS), and zero-data-retention controls all ship in the box.

Keep these five points in mind — the rest of the guide is really just them, expanded.

Part 2 — The core: Agents and the Runner

An Agent

An Agent is an LLM configured with instructions, tools, guardrails, handoffs, and an optional structured output type.

from agents import Agent, Runner, function_tool


@function_tool
def get_weather(city: str) -> str:
    """Return the weather for a city."""
    return f"The weather in {city} is sunny."


agent = Agent(
    name="Assistant",
    instructions="You are concise and helpful.",
    model="gpt-5.5",
    tools=[get_weather],
)

result = Runner.run_sync(agent, "What's the weather in Istanbul?")
print(result.final_output)

from agents import Agent, Runner, function_tool


@function_tool
def get_weather(city: str) -> str:
    """Return the weather for a city."""
    return f"The weather in {city} is sunny."


agent = Agent(
    name="Assistant",
    instructions="You are concise and helpful.",
    model="gpt-5.5",
    tools=[get_weather],
)

result = Runner.run_sync(agent, "What's the weather in Istanbul?")
print(result.final_output)

from agents import Agent, Runner, function_tool


@function_tool
def get_weather(city: str) -> str:
    """Return the weather for a city."""
    return f"The weather in {city} is sunny."


agent = Agent(
    name="Assistant",
    instructions="You are concise and helpful.",
    model="gpt-5.5",
    tools=[get_weather],
)

result = Runner.run_sync(agent, "What's the weather in Istanbul?")
print(result.final_output)

The agent loop

When you call Runner.run(), the SDK runs a loop:

• Call the LLM for the current agent with the current input.

• Inspect the output:

  • If it's a final output (text of the desired type with no tool calls), stop and return the RunResult.

  • If it's a handoff, switch the current agent and re-loop.

  • If there are tool calls, execute them, append the results, and re-loop.

• If the loop exceeds max_turns, raise MaxTurnsExceeded (pass max_turns=None to disable).

There are three entry points:

• Runner.run(...) — async, returns RunResult.

• Runner.run_sync(...) — synchronous wrapper.

• Runner.run_streamed(...) — async, returns RunResultStreaming; you consume .stream_events() as the model produces tokens and items.

Agent configuration worth knowing

instructions can be a static string or a function (dynamic instructions) that receives context and the agent, so you can inject the user's name, locale, or the current date per run.

output_type turns plain text into a typed, structured output (a Pydantic model, dataclass, TypedDict, etc.). Under the hood this enables structured outputs.

model_settings carries temperature, top_p, tool_choice, parallel_tool_calls, and more.

tool_use_behavior controls whether tool results loop back to the model ("run_llm_again", the default), stop at the first tool ("stop_on_first_tool"), stop at named tools (StopAtTools(...)), or run a custom function that decides.

clone() duplicates an agent with overrides — perfect for A/B variants.

from openai.types.shared import Reasoning
from agents import Agent, ModelSettings

research_agent = Agent(
    name="Research agent",
    model="gpt-5.5",
    model_settings=ModelSettings(reasoning=Reasoning(effort="high"), verbosity="low"),
)

from openai.types.shared import Reasoning
from agents import Agent, ModelSettings

research_agent = Agent(
    name="Research agent",
    model="gpt-5.5",
    model_settings=ModelSettings(reasoning=Reasoning(effort="high"), verbosity="low"),
)

from openai.types.shared import Reasoning
from agents import Agent, ModelSettings

research_agent = Agent(
    name="Research agent",
    model="gpt-5.5",
    model_settings=ModelSettings(reasoning=Reasoning(effort="high"), verbosity="low"),
)

The run result: what Runner actually returns

This is the part most tutorials skip, so let's be precise. Runner.run() / run_sync() return a RunResult; Runner.run_streamed() returns a RunResultStreaming. Both inherit from RunResultBase, which exposes the surfaces you'll use every day:

result = await Runner.run(agent, "Summarize this thread.")

print(result.final_output)          # the answer (str or your output_type)
print(result.last_agent.name)       # who finished after any handoffs
for item in result.new_items:       # rich, typed transcript of the run
    print(type(item).__name__)

result = await Runner.run(agent, "Summarize this thread.")

print(result.final_output)          # the answer (str or your output_type)
print(result.last_agent.name)       # who finished after any handoffs
for item in result.new_items:       # rich, typed transcript of the run
    print(type(item).__name__)

result = await Runner.run(agent, "Summarize this thread.")

print(result.final_output)          # the answer (str or your output_type)
print(result.last_agent.name)       # who finished after any handoffs
for item in result.new_items:       # rich, typed transcript of the run
    print(type(item).__name__)

Note that final_output is typed as Any (handoffs can change which agent — and which output_type — finishes), and in streaming mode it stays None until you finish consuming stream_events(). RunResultStreaming adds stream_events(), current_agent, is_complete, and cancel().

Token tracking and the Usage object

Every run automatically tracks token usage. You read it from result.context_wrapper.usage, which is a Usage object:

result = await Runner.run(agent, "What's the weather in Tokyo?")
usage = result.context_wrapper.usage

print("Requests:    ", usage.requests)        # number of LLM API calls in this run
print("Input tokens:", usage.input_tokens)    # total prompt tokens
print("Output tokens:", usage.output_tokens)  # total completion tokens
print("Total tokens:", usage.total_tokens)    # input + output

# Subset details:
print("Cached input tokens:", usage.input_tokens_details.cached_tokens)
print("Reasoning tokens:   ", usage.output_tokens_details.reasoning_tokens)

result = await Runner.run(agent, "What's the weather in Tokyo?")
usage = result.context_wrapper.usage

print("Requests:    ", usage.requests)        # number of LLM API calls in this run
print("Input tokens:", usage.input_tokens)    # total prompt tokens
print("Output tokens:", usage.output_tokens)  # total completion tokens
print("Total tokens:", usage.total_tokens)    # input + output

# Subset details:
print("Cached input tokens:", usage.input_tokens_details.cached_tokens)
print("Reasoning tokens:   ", usage.output_tokens_details.reasoning_tokens)

result = await Runner.run(agent, "What's the weather in Tokyo?")
usage = result.context_wrapper.usage

print("Requests:    ", usage.requests)        # number of LLM API calls in this run
print("Input tokens:", usage.input_tokens)    # total prompt tokens
print("Output tokens:", usage.output_tokens)  # total completion tokens
print("Total tokens:", usage.total_tokens)    # input + output

# Subset details:
print("Cached input tokens:", usage.input_tokens_details.cached_tokens)
print("Reasoning tokens:   ", usage.output_tokens_details.reasoning_tokens)

Three things matter for getting billing right

Usage is aggregated across the entire run — every model call, including those triggered by tool loops and handoffs. If a single Runner.run() makes three LLM calls, usage reflects all three combined.

reasoning_tokens is a subset of output_tokens, not an addition to it. On the OpenAI Responses API, reasoning tokens are billed as output tokens, so they're already counted inside output_tokens and surfaced separately under output_tokens_details.reasoning_tokens for visibility. Likewise, input_tokens_details.cached_tokens is a subset of input_tokens (cached prompt prefixes, usually billed at a discount). Don't add these subset fields back into the totals or you'll double-count.

Per-request breakdowns live in request_usage_entries. The aggregate hides individual calls; this list preserves each request's tokens, which is what you want for accurate per-call cost math or context-window monitoring.

for i, req in enumerate(usage.request_usage_entries, start=1):
    print(f"Request {i}: {req.input_tokens} in / {req.output_tokens} out "
          f"({req.output_tokens_details.reasoning_tokens} reasoning)")

for i, req in enumerate(usage.request_usage_entries, start=1):
    print(f"Request {i}: {req.input_tokens} in / {req.output_tokens} out "
          f"({req.output_tokens_details.reasoning_tokens} reasoning)")

for i, req in enumerate(usage.request_usage_entries, start=1):
    print(f"Request {i}: {req.input_tokens} in / {req.output_tokens} out "
          f"({req.output_tokens_details.reasoning_tokens} reasoning)")

You can also read usage live inside lifecycle hooks — context.usage is available in RunHooks / AgentHooks, so you can log per-agent token spend at on_agent_end:

from agents import RunHooks

class UsageHooks(RunHooks):
    async def on_agent_end(self, context, agent, output):
        u = context.usage
        print(f"{agent.name}: {u.requests} requests, {u.total_tokens} tokens "
              f"({u.output_tokens_details.reasoning_tokens} reasoning)")

from agents import RunHooks

class UsageHooks(RunHooks):
    async def on_agent_end(self, context, agent, output):
        u = context.usage
        print(f"{agent.name}: {u.requests} requests, {u.total_tokens} tokens "
              f"({u.output_tokens_details.reasoning_tokens} reasoning)")

from agents import RunHooks

class UsageHooks(RunHooks):
    async def on_agent_end(self, context, agent, output):
        u = context.usage
        print(f"{agent.name}: {u.requests} requests, {u.total_tokens} tokens "
              f"({u.output_tokens_details.reasoning_tokens} reasoning)")

With sessions, each Runner.run() returns usage for that run only — and because session history is re-fed as input on each turn, your input_tokens naturally grow turn over turn (a good reason to cap retrieval with SessionSettings(limit=...) or compaction). Finally, a production gotcha: with third-party adapters (LiteLLM, Any-LLM) some provider backends don't report usage unless you set ModelSettings(include_usage=True), especially on streamed Chat Completions backends. If total_tokens comes back as 0 on a non-OpenAI provider, that flag is the first thing to check.

Part 3 — Tools: how agents take action

The SDK supports five categories of tools.

1. Function tools

Decorate any Python function. The SDK extracts the JSON schema from the signature (via inspect + Pydantic) and the descriptions from the docstring (via griffe).

from typing import Annotated
from agents import function_tool


@function_tool
async def fetch_invoice(
    invoice_id: Annotated[str, "The invoice ID to fetch."],
) -> str:
    """Fetch an invoice by ID."""
    return await billing_api.get(invoice_id)

from typing import Annotated
from agents import function_tool


@function_tool
async def fetch_invoice(
    invoice_id: Annotated[str, "The invoice ID to fetch."],
) -> str:
    """Fetch an invoice by ID."""
    return await billing_api.get(invoice_id)

from typing import Annotated
from agents import function_tool


@function_tool
async def fetch_invoice(
    invoice_id: Annotated[str, "The invoice ID to fetch."],
) -> str:
    """Fetch an invoice by ID."""
    return await billing_api.get(invoice_id)

Function tools have a rich production surface

Timeouts

@function_tool(timeout=2.0) with timeout_behavior="error_as_result" (default, returns a model-visible message) or "raise_exception" (raises ToolTimeoutError).

Error handling

Pass failure_error_function=... to control the message the model sees on a crash, or None to re-raise.

Validation

Use Pydantic Field(..., ge=0, le=100) for constraints that end up in the JSON schema.

Rich returns

Return text, images (ToolOutputImage), or files (ToolOutputFileContent).

2. Hosted tools (OpenAI Responses)

Run alongside the model on OpenAI's servers: WebSearchTool, FileSearchTool (your vector stores), CodeInterpreterTool, ImageGenerationTool, HostedMCPTool, and ToolSearchTool (deferred tool loading for large tool surfaces).

from agents import Agent, WebSearchTool, FileSearchTool

agent = Agent(
    name="Researcher",
    tools=[
        WebSearchTool(),
        FileSearchTool(max_num_results=3, vector_store_ids=["vs_..."]),
    ],
)

from agents import Agent, WebSearchTool, FileSearchTool

agent = Agent(
    name="Researcher",
    tools=[
        WebSearchTool(),
        FileSearchTool(max_num_results=3, vector_store_ids=["vs_..."]),
    ],
)

from agents import Agent, WebSearchTool, FileSearchTool

agent = Agent(
    name="Researcher",
    tools=[
        WebSearchTool(),
        FileSearchTool(max_num_results=3, vector_store_ids=["vs_..."]),
    ],
)

Hosted tools require OpenAI Responses models. They are not available on Chat Completions or non-OpenAI backends.

3. Local / runtime execution tools

ComputerTool (GUI/browser automation against a Computer you implement), ShellTool (local or hosted-container execution), and ApplyPatchTool (apply diffs through your editor). These let an agent operate a real environment.

4. Agents as tools

Expose an agent as a callable tool without transferring control — the manager pattern. The orchestrator keeps the conversation and calls specialists like functions.

orchestrator = Agent(
    name="Orchestrator",
    tools=[
        spanish_agent.as_tool(tool_name="to_spanish", tool_description="Translate to Spanish"),
        french_agent.as_tool(tool_name="to_french", tool_description="Translate to French"),
    ],
)

orchestrator = Agent(
    name="Orchestrator",
    tools=[
        spanish_agent.as_tool(tool_name="to_spanish", tool_description="Translate to Spanish"),
        french_agent.as_tool(tool_name="to_french", tool_description="Translate to French"),
    ],
)

orchestrator = Agent(
    name="Orchestrator",
    tools=[
        spanish_agent.as_tool(tool_name="to_spanish", tool_description="Translate to Spanish"),
        french_agent.as_tool(tool_name="to_french", tool_description="Translate to French"),
    ],
)

as_tool() supports structured input (parameters=), output extraction (custom_output_extractor=), streaming (on_stream=), conditional enabling (is_enabled=), and approval gates (needs_approval=).

5. MCP (Model Context Protocol)

MCP is "USB-C for AI tools." The SDK supports four integration styles:

Hosted MCP

HostedMCPTool — OpenAI's Responses API calls a public MCP server for you.

Streamable HTTP

MCPServerStreamableHttp — you run/manage the server.

HTTP+SSE

MCPServerSse — legacy transport.

stdio

MCPServerStdio — spawn a local subprocess.

MCP servers support tool filtering (static allow/block lists or dynamic callables), list_tools() caching, per-call _meta injection, approval policies, and prompts. Use MCPServerManager to connect many servers up front with reconnect handling.

Part 4 — Multi-agent design: handoffs vs. the manager pattern

There are two broadly applicable architectures, and the SDK supports both natively.

Handoffs

Handoffs are decentralized: a peer agent delegates the conversation to a specialist, which then takes over. Handoffs are represented to the model as tools (transfer_to_<agent_name>).

from agents import Agent, handoff

billing = Agent(name="Billing agent")
refund = Agent(name="Refund agent")

triage = Agent(
    name="Triage agent",
    instructions="Route billing questions to billing, refunds to refunds.",
    handoffs=[billing, handoff(refund)],
)

from agents import Agent, handoff

billing = Agent(name="Billing agent")
refund = Agent(name="Refund agent")

triage = Agent(
    name="Triage agent",
    instructions="Route billing questions to billing, refunds to refunds.",
    handoffs=[billing, handoff(refund)],
)

from agents import Agent, handoff

billing = Agent(name="Billing agent")
refund = Agent(name="Refund agent")

triage = Agent(
    name="Triage agent",
    instructions="Route billing questions to billing, refunds to refunds.",
    handoffs=[billing, handoff(refund)],
)

handoff() lets you customize the tool name and description, attach an on_handoff callback, require structured input_type metadata (e.g. a reason), filter the history the next agent sees with input_filter, and dynamically enable/disable via is_enabled. There's also an opt-in beta, nest_handoff_history, that collapses the prior transcript into a single summary message.

The manager pattern

The manager pattern (agents-as-tools, above) is centralized: the orchestrator never gives up control. Use handoffs when a specialist should own the rest of the conversation; use the manager pattern when you need to compose several specialists and synthesize their outputs.

For better routing, prepend RECOMMENDED_PROMPT_PREFIX (from agents.extensions.handoff_prompt) to your instructions.

Part 5 — Guardrails: cheap checks that protect expensive models

Guardrails run validations in parallel with or before your agent and can halt execution via a tripwire.

Input guardrails

Input guardrails run on the first agent's input. They support run_in_parallel=True (lowest latency, default) or run_in_parallel=False (blocking — the agent never starts if the tripwire trips, which saves tokens and avoids side effects).

Output guardrails

Output guardrails run on the final output of the last agent.

Tool guardrails

Tool guardrails wrap individual function tools — input guardrails before execution (can skip, replace output, or trip) and output guardrails after.

from pydantic import BaseModel
from agents import Agent, GuardrailFunctionOutput, Runner, input_guardrail


class Check(BaseModel):
    is_abuse: bool


guard_agent = Agent(name="Guard", output_type=Check, model="gpt-5-nano")


@input_guardrail
async def abuse_guard(ctx, agent, user_input) -> GuardrailFunctionOutput:
    result = await Runner.run(guard_agent, user_input, context=ctx.context)
    return GuardrailFunctionOutput(
        output_info=result.final_output,
        tripwire_triggered=result.final_output.is_abuse,
    )

from pydantic import BaseModel
from agents import Agent, GuardrailFunctionOutput, Runner, input_guardrail


class Check(BaseModel):
    is_abuse: bool


guard_agent = Agent(name="Guard", output_type=Check, model="gpt-5-nano")


@input_guardrail
async def abuse_guard(ctx, agent, user_input) -> GuardrailFunctionOutput:
    result = await Runner.run(guard_agent, user_input, context=ctx.context)
    return GuardrailFunctionOutput(
        output_info=result.final_output,
        tripwire_triggered=result.final_output.is_abuse,
    )

from pydantic import BaseModel
from agents import Agent, GuardrailFunctionOutput, Runner, input_guardrail


class Check(BaseModel):
    is_abuse: bool


guard_agent = Agent(name="Guard", output_type=Check, model="gpt-5-nano")


@input_guardrail
async def abuse_guard(ctx, agent, user_input) -> GuardrailFunctionOutput:
    result = await Runner.run(guard_agent, user_input, context=ctx.context)
    return GuardrailFunctionOutput(
        output_info=result.final_output,
        tripwire_triggered=result.final_output.is_abuse,
    )

When a tripwire fires, the SDK raises InputGuardrailTripwireTriggered / OutputGuardrailTripwireTriggered, which you catch and turn into a graceful response. The classic pattern: a fast, cheap model (gpt-5-nano) screens input before your slow, expensive model ever runs.

Part 6 — Memory: sessions, context, and conversation state

There are four ways to carry state between turns, and choosing correctly is a production decision:

The last two are OpenAI-managed and only apply to the Responses API. Pick one strategy per conversation — mixing client-managed history with server-managed state duplicates context.

Sessions

Sessions make the SDK retrieve history before each run and persist new items after. Backends include SQLiteSession (memory or file), AsyncSQLiteSession, RedisSession, SQLAlchemySession, MongoDBSession, DaprSession, OpenAIConversationsSession, AdvancedSQLiteSession (branching + analytics), EncryptedSession (transparent encryption + TTL), and OpenAIResponsesCompactionSession (auto-compaction for very long chats).

from agents import Agent, Runner, SQLiteSession

agent = Agent(name="Assistant", instructions="Reply concisely.")
session = SQLiteSession("user_123", "conversations.db")

await Runner.run(agent, "What city is the Golden Gate Bridge in?", session=session)
await Runner.run(agent, "What state is it in?", session=session)  # remembers context

from agents import Agent, Runner, SQLiteSession

agent = Agent(name="Assistant", instructions="Reply concisely.")
session = SQLiteSession("user_123", "conversations.db")

await Runner.run(agent, "What city is the Golden Gate Bridge in?", session=session)
await Runner.run(agent, "What state is it in?", session=session)  # remembers context

from agents import Agent, Runner, SQLiteSession

agent = Agent(name="Assistant", instructions="Reply concisely.")
session = SQLiteSession("user_123", "conversations.db")

await Runner.run(agent, "What city is the Golden Gate Bridge in?", session=session)
await Runner.run(agent, "What state is it in?", session=session)  # remembers context

You can control retrieval with SessionSettings(limit=N), customize the history/new-input merge with RunConfig.session_input_callback, and implement your own backend by following the Session protocol.

Local context (dependency injection)

RunContextWrapper[T] carries a local object — your user ID, permissions, DB handles, logger — to every tool, hook, and guardrail. It is never sent to the LLM. It also exposes aggregated usage and, inside as_tool() runs, tool_input.

from dataclasses import dataclass
from agents import Agent, RunContextWrapper, function_tool


@dataclass
class UserInfo:
    name: str
    uid: int


@function_tool
async def greet(ctx: RunContextWrapper[UserInfo]) -> str:
    return f"Hello, {ctx.context.name}"


agent = Agent[UserInfo](name="Assistant", tools=[greet])

from dataclasses import dataclass
from agents import Agent, RunContextWrapper, function_tool


@dataclass
class UserInfo:
    name: str
    uid: int


@function_tool
async def greet(ctx: RunContextWrapper[UserInfo]) -> str:
    return f"Hello, {ctx.context.name}"


agent = Agent[UserInfo](name="Assistant", tools=[greet])

from dataclasses import dataclass
from agents import Agent, RunContextWrapper, function_tool


@dataclass
class UserInfo:
    name: str
    uid: int


@function_tool
async def greet(ctx: RunContextWrapper[UserInfo]) -> str:
    return f"Hello, {ctx.context.name}"


agent = Agent[UserInfo](name="Assistant", tools=[greet])

If you serialize RunState for human-in-the-loop or durable jobs, the context travels with it — never put secrets in it.

Part 7 — Tracing and observability

Tracing is on by default. Every Runner.run() is wrapped in a trace, and each agent run, LLM generation, tool call, guardrail, and handoff becomes a span.

from agents import Runner, trace

with trace("Customer support", group_id=thread_id):
    first = await Runner.run(agent, "Hello")
    second = await Runner.run(agent, "Follow-up question")

from agents import Runner, trace

with trace("Customer support", group_id=thread_id):
    first = await Runner.run(agent, "Hello")
    second = await Runner.run(agent, "Follow-up question")

from agents import Runner, trace

with trace("Customer support", group_id=thread_id):
    first = await Runner.run(agent, "Hello")
    second = await Runner.run(agent, "Follow-up question")

Production knobs

RunConfig(workflow_name=..., trace_id=..., group_id=..., trace_metadata=...) for naming and correlation.

RunConfig(trace_include_sensitive_data=False) (or OPENAI_AGENTS_TRACE_INCLUDE_SENSITIVE_DATA=0) to exclude inputs/outputs from trace payloads.

flush_traces() to force an immediate export in workers (Celery, RQ, FastAPI background tasks) where the process may not flush in time.

add_trace_processor() to also ship traces somewhere else, or set_trace_processors() to replace the default exporter.

The ecosystem is broad: Logfire, Langfuse, Braintrust, Datadog, W&B Weave, MLflow, Arize-Phoenix, LangSmith, Comet Opik, AgentOps, and many more support the SDK's tracing surface out of the box.

Under a Zero Data Retention (ZDR) policy, OpenAI tracing is unavailable — disable it or use a custom processor.

Part 8 — Models and providers: the part that future-proofs your project

This is the heart of the SDK's "no one-way doors" philosophy. There are two OpenAI model shapes:

OpenAI model types

OpenAIResponsesModel (recommended)

The Responses API. Required for hosted tools and tool search.

OpenAIChatCompletionsModel

The Chat Completions API. The lowest common denominator that almost every provider supports.

Four ways to point the SDK somewhere other than api.openai.com

Plus the global switches: set_default_openai_api("chat_completions"), set_default_openai_key(...), and the OPENAI_BASE_URL / OPENAI_API_KEY environment variables.

The global client approach (the one you'll use most)

from openai import AsyncOpenAI
from agents import set_default_openai_client, set_default_openai_api, set_tracing_disabled

client = AsyncOpenAI(base_url="https://your-endpoint/v1", api_key="...")
set_default_openai_client(client, use_for_tracing=False)
set_default_openai_api("chat_completions")  # most non-OpenAI backends don't support Responses
set_tracing_disabled(True)                  # or set_tracing_export_api_key(...) for free OpenAI traces

from openai import AsyncOpenAI
from agents import set_default_openai_client, set_default_openai_api, set_tracing_disabled

client = AsyncOpenAI(base_url="https://your-endpoint/v1", api_key="...")
set_default_openai_client(client, use_for_tracing=False)
set_default_openai_api("chat_completions")  # most non-OpenAI backends don't support Responses
set_tracing_disabled(True)                  # or set_tracing_export_api_key(...) for free OpenAI traces

from openai import AsyncOpenAI
from agents import set_default_openai_client, set_default_openai_api, set_tracing_disabled

client = AsyncOpenAI(base_url="https://your-endpoint/v1", api_key="...")
set_default_openai_client(client, use_for_tracing=False)
set_default_openai_api("chat_completions")  # most non-OpenAI backends don't support Responses
set_tracing_disabled(True)                  # or set_tracing_export_api_key(...) for free OpenAI traces

That's it. Your agents, tools, handoffs, and guardrails don't change at all.

Part 9 — The production scenario, end to end: OpenAI → vLLM/Triton → Together AI → Azure

This is the journey that motivates most adoptions, so let's make it concrete.

Stage 1 — MVP on OpenAI

You start fast. Set OPENAI_API_KEY, pick gpt-5.5, ship. Responses API, hosted tools, full tracing — everything works.

Stage 2 — Move the model on-prem to your own GPUs (vLLM)

Cost, latency, data residency, or compliance pushes you to host the model yourself. You serve, say, Llama or Qwen with vLLM, which exposes an OpenAI-compatible /v1 endpoint:

vllm serve meta-llama/Llama-3.1-8B-Instruct --port 8000

vllm serve meta-llama/Llama-3.1-8B-Instruct --port 8000

vllm serve meta-llama/Llama-3.1-8B-Instruct --port 8000

Now you point the SDK at it — and nothing else changes:

from openai import AsyncOpenAI
from agents import Agent, Runner, set_default_openai_client, set_default_openai_api, set_tracing_disabled

client = AsyncOpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
set_default_openai_client(client, use_for_tracing=False)
set_default_openai_api("chat_completions")
set_tracing_disabled(True)

agent = Agent(name="Assistant", model="meta-llama/Llama-3.1-8B-Instruct")
print((await Runner.run(agent, "Hello")).final_output)

from openai import AsyncOpenAI
from agents import Agent, Runner, set_default_openai_client, set_default_openai_api, set_tracing_disabled

client = AsyncOpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
set_default_openai_client(client, use_for_tracing=False)
set_default_openai_api("chat_completions")
set_tracing_disabled(True)

agent = Agent(name="Assistant", model="meta-llama/Llama-3.1-8B-Instruct")
print((await Runner.run(agent, "Hello")).final_output)

from openai import AsyncOpenAI
from agents import Agent, Runner, set_default_openai_client, set_default_openai_api, set_tracing_disabled

client = AsyncOpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
set_default_openai_client(client, use_for_tracing=False)
set_default_openai_api("chat_completions")
set_tracing_disabled(True)

agent = Agent(name="Assistant", model="meta-llama/Llama-3.1-8B-Instruct")
print((await Runner.run(agent, "Hello")).final_output)

Stage 3 — Scale the serving layer with NVIDIA Triton (vLLM backend)

As traffic grows you move to NVIDIA Triton Inference Server with the vLLM backend for batching, multi-model hosting, and GPU scheduling. Triton can sit behind an OpenAI-compatible gateway, so from the SDK's perspective it's still just a base_url:

client = AsyncOpenAI(base_url="http://triton-gateway:9000/v1", api_key="EMPTY")
set_default_openai_client(client, use_for_tracing=False)

client = AsyncOpenAI(base_url="http://triton-gateway:9000/v1", api_key="EMPTY")
set_default_openai_client(client, use_for_tracing=False)

client = AsyncOpenAI(base_url="http://triton-gateway:9000/v1", api_key="EMPTY")
set_default_openai_client(client, use_for_tracing=False)

The agent code written in Stage 1 runs unchanged on infrastructure that didn't exist when you wrote it. That's the whole point.

Alternative — a managed open-model provider (Together AI)

Don't want to run GPUs at all? Together AI (and OpenRouter, Fireworks, Groq, etc.) expose OpenAI-compatible endpoints:

import os
client = AsyncOpenAI(base_url="https://api.together.xyz/v1", api_key=os.environ["TOGETHER_API_KEY"])
set_default_openai_client(client, use_for_tracing=False)
set_default_openai_api("chat_completions")

import os
client = AsyncOpenAI(base_url="https://api.together.xyz/v1", api_key=os.environ["TOGETHER_API_KEY"])
set_default_openai_client(client, use_for_tracing=False)
set_default_openai_api("chat_completions")

import os
client = AsyncOpenAI(base_url="https://api.together.xyz/v1", api_key=os.environ["TOGETHER_API_KEY"])
set_default_openai_client(client, use_for_tracing=False)
set_default_openai_api("chat_completions")

Azure OpenAI / Azure AI Foundry

Enterprises on Azure use a dedicated client, AsyncAzureOpenAI. The key difference: on Azure you reference a deployment name, not a model name.

from openai import AsyncAzureOpenAI
from agents import Agent, set_default_openai_client, set_default_openai_api

azure_client = AsyncAzureOpenAI(
    api_key="...",
    azure_endpoint="https://your-resource.openai.azure.com/",
    api_version="2024-10-21",
)
set_default_openai_client(azure_client, use_for_tracing=False)
set_default_openai_api("chat_completions")

# `model` is the Azure *deployment* name.
agent = Agent(name="Assistant", model="gpt-4o")

from openai import AsyncAzureOpenAI
from agents import Agent, set_default_openai_client, set_default_openai_api

azure_client = AsyncAzureOpenAI(
    api_key="...",
    azure_endpoint="https://your-resource.openai.azure.com/",
    api_version="2024-10-21",
)
set_default_openai_client(azure_client, use_for_tracing=False)
set_default_openai_api("chat_completions")

# `model` is the Azure *deployment* name.
agent = Agent(name="Assistant", model="gpt-4o")

from openai import AsyncAzureOpenAI
from agents import Agent, set_default_openai_client, set_default_openai_api

azure_client = AsyncAzureOpenAI(
    api_key="...",
    azure_endpoint="https://your-resource.openai.azure.com/",
    api_version="2024-10-21",
)
set_default_openai_client(azure_client, use_for_tracing=False)
set_default_openai_api("chat_completions")

# `model` is the Azure *deployment* name.
agent = Agent(name="Assistant", model="gpt-4o")

Azure AI Foundry exposes a catalog of models (OpenAI and open models) the same way; mix in LiteLLM (azure/..., azure_ai/... model strings) when you need broader Foundry coverage or routing.

Mixing providers in one workflow

You can even run a small/fast model for triage and a large model for hard tasks — by setting model per agent, or by routing prefixes with MultiProvider (openai/..., litellm/..., any-llm/...). Just remember: feature support differs across providers (structured outputs, multimodal input, hosted tools), so validate the exact backend you ship.

Third-party adapters (LiteLLM / Any-LLM)

When the built-in paths aren't enough — you need adapter-managed coverage of 100+ providers or specific routing — install openai-agents[litellm] or openai-agents[any-llm] and use litellm/... or any-llm/... model names (or LitellmModel / AnyLLMModel directly). Note that some adapter backends need ModelSettings(include_usage=True) to report token usage.

Part 10 — The production playbook

Capabilities are easy; running them reliably is the real work. Here's the checklist that separates a demo from a service.

Retries — explicit and opt-in

The SDK does not retry general model calls unless you ask. Opt in with ModelSettings(retry=...):

from agents import Agent, ModelRetrySettings, ModelSettings, retry_policies

agent = Agent(
    name="Assistant",
    model="gpt-5.5",
    model_settings=ModelSettings(
        retry=ModelRetrySettings(
            max_retries=4,
            backoff={"initial_delay": 0.5, "max_delay": 5.0, "multiplier": 2.0, "jitter": True},
            policy=retry_policies.any(
                retry_policies.provider_suggested(),
                retry_policies.retry_after(),
                retry_policies.network_error(),
                retry_policies.http_status([408, 429, 500, 502, 503, 504]),
            ),
        )
    ),
)

from agents import Agent, ModelRetrySettings, ModelSettings, retry_policies

agent = Agent(
    name="Assistant",
    model="gpt-5.5",
    model_settings=ModelSettings(
        retry=ModelRetrySettings(
            max_retries=4,
            backoff={"initial_delay": 0.5, "max_delay": 5.0, "multiplier": 2.0, "jitter": True},
            policy=retry_policies.any(
                retry_policies.provider_suggested(),
                retry_policies.retry_after(),
                retry_policies.network_error(),
                retry_policies.http_status([408, 429, 500, 502, 503, 504]),
            ),
        )
    ),
)

from agents import Agent, ModelRetrySettings, ModelSettings, retry_policies

agent = Agent(
    name="Assistant",
    model="gpt-5.5",
    model_settings=ModelSettings(
        retry=ModelRetrySettings(
            max_retries=4,
            backoff={"initial_delay": 0.5, "max_delay": 5.0, "multiplier": 2.0, "jitter": True},
            policy=retry_policies.any(
                retry_policies.provider_suggested(),
                retry_policies.retry_after(),
                retry_policies.network_error(),
                retry_policies.http_status([408, 429, 500, 502, 503, 504]),
            ),
        )
    ),
)

Lead with provider_suggested() — it preserves provider vetoes and replay-safety approvals, which matters for stateful previous_response_id / conversation_id follow-ups.

Timeouts and tool failure isolation

Give every external-call tool a timeout, and decide whether a timeout should recover (error_as_result) or fail the run (raise_exception). Use failure_error_function so a flaky API returns a clean message to the model instead of crashing the turn.

Bounding the loop

Always set a max_turns in production and provide an error_handlers={"max_turns": ...} fallback so a runaway agent returns a controlled message instead of raising. The same mechanism handles "model_refusal".

Human-in-the-loop (HITL)

Mark sensitive tools with needs_approval=True (or a per-call callable). When the model calls one, the run pauses and surfaces result.interruptions. Serialize with result.to_state() → state.to_json(), store it in a queue or DB, then later RunState.from_json(...), state.approve(...) / state.reject(...), and resume with Runner.run(agent, state). This works across handoffs and nested as_tool() calls, and survives process restarts — the foundation for durable approvals.

Durable execution

For workflows that span long waits, retries, or restarts, the SDK integrates with Temporal, Dapr, Restate, and DBOS. These wrap the runner so progress survives failures — essential for HITL approvals that might sit for hours.

Security and data governance

OPENAI_AGENTS_DONT_LOG_MODEL_DATA=1 and OPENAI_AGENTS_DONT_LOG_TOOL_DATA=1 keep prompts/outputs out of logs (on by default).

trace_include_sensitive_data=False keeps them out of traces.

Don't put secrets in RunContextWrapper.context if you serialize RunState.

Use EncryptedSession for conversation history at rest, with TTL.

Tool guardrails can redact secrets (API keys, PII) before and after tool execution.

Concurrency and cost control

ModelSettings(parallel_tool_calls=...) controls whether the model may emit multiple tool calls; RunConfig(tool_execution=ToolExecutionConfig(max_function_tool_concurrency=N)) controls how many the SDK runs at once.

Use blocking input guardrails (run_in_parallel=False) to stop spend before the expensive model runs.

For long contexts, lean on truncation="auto", server-side context_management compaction, or OpenAIResponsesCompactionSession.

prompt_cache_retention="24h" keeps cached prefixes warm to cut cost and latency.

Observability discipline

Name every workflow (workflow_name), correlate multi-run conversations with group_id, attach trace_metadata, and flush_traces() in background workers. Wire a real backend (Langfuse/Datadog/Logfire) from day one — debugging agents blind is the most expensive mistake in this space.

Streaming and latency

Use Runner.run_streamed() and forward raw text deltas (ResponseTextDeltaEvent) for snappy UIs. For OpenAI Responses you can opt into websocket transport (set_default_openai_responses_transport("websocket")) and reuse connections across turns with responses_websocket_session().

Part 11 — A reference architecture you can copy

To make all of this concrete, the companion agent-boilerplate project packages the patterns above into a plug-and-play starter:

agent-boilerplate/
├── app/
│   ├── config.py            # Typed settings (.env -> Settings)
│   ├── server.py            # FastAPI: /chat and /chat/stream (SSE)
│   ├── cli.py               # Interactive terminal demo
│   └── ai/                  # Everything AI lives here
│       ├── provider.py      # set_default_openai_client: OpenAI / Azure / compatible
│       ├── agents.py        # Triage agent + handoffs to specialists
│       ├── tools.py         # @function_tool definitions
│       ├── guardrails.py    # Input guardrail (relevance check)
│       ├── sessions.py      # Session factory (memory/sqlite/redis)
│       ├── context.py       # Local DI context
│       ├── service.py       # Thin Runner wrapper with error handling
│       └── instructions/    # System prompts as versioned .md files

agent-boilerplate/
├── app/
│   ├── config.py            # Typed settings (.env -> Settings)
│   ├── server.py            # FastAPI: /chat and /chat/stream (SSE)
│   ├── cli.py               # Interactive terminal demo
│   └── ai/                  # Everything AI lives here
│       ├── provider.py      # set_default_openai_client: OpenAI / Azure / compatible
│       ├── agents.py        # Triage agent + handoffs to specialists
│       ├── tools.py         # @function_tool definitions
│       ├── guardrails.py    # Input guardrail (relevance check)
│       ├── sessions.py      # Session factory (memory/sqlite/redis)
│       ├── context.py       # Local DI context
│       ├── service.py       # Thin Runner wrapper with error handling
│       └── instructions/    # System prompts as versioned .md files

agent-boilerplate/
├── app/
│   ├── config.py            # Typed settings (.env -> Settings)
│   ├── server.py            # FastAPI: /chat and /chat/stream (SSE)
│   ├── cli.py               # Interactive terminal demo
│   └── ai/                  # Everything AI lives here
│       ├── provider.py      # set_default_openai_client: OpenAI / Azure / compatible
│       ├── agents.py        # Triage agent + handoffs to specialists
│       ├── tools.py         # @function_tool definitions
│       ├── guardrails.py    # Input guardrail (relevance check)
│       ├── sessions.py      # Session factory (memory/sqlite/redis)
│       ├── context.py       # Local DI context
│       ├── service.py       # Thin Runner wrapper with error handling
│       └── instructions/    # System prompts as versioned .md files

The single most important file is provider.py. It implements exactly the migration story from Part 9: one function reads MODEL_PROVIDER and configures the global client for OpenAI, Azure, or any compatible backend. Flip one environment variable and the same agents run on gpt-5.5, on a vLLM box, behind Triton, on Together AI, or on Azure AI Foundry — with zero code changes.

That property — the model is a deployment detail, not an architectural commitment — is why so many teams reach for the OpenAI Agents SDK first, and why they're still using it when their MVP turns into a platform.

Closing

The OpenAI Agents SDK earns its default-choice status by being honest about its scope: it orchestrates agents, and it refuses to lock you into a model, a provider, or an infrastructure.

Learn the five primitives:

• agents

• tools

• handoffs

• guardrails

• sessions

Wire in tracing and provider configuration, then layer on the production playbook of retries, timeouts, HITL, and data governance.

Do that, and the gap between "impressive demo" and "service you can run at 3 a.m." mostly disappears.

Start on OpenAI.

Move to your own GPUs when it makes sense.

Keep your agents exactly as they are.

That's the promise — and, as this guide showed, it's one the SDK actually keeps.