Agent Loop Notes
entry
No. 6
date
filed under
tool use, structured outputs, agents
checked against
5 sources

Structured Outputs vs Function Calling: Which to Use When

Use function calling when the model needs to do something: look up an order, run a query, send a message. Use structured outputs when you need the model's answer in a fixed shape that your code will parse. In an agent you usually want both, and on current APIs you should turn on strict mode for each, so tool arguments and final answers are checked against your schema while the model writes them.

The confusion is understandable. Both features take a JSON Schema, both return JSON, and for a while a common trick for getting JSON out of a model was to force it to call a fake tool. That trick is now the wrong default, and on some of Anthropic's newest models it returns an error. This post covers what each feature does on the OpenAI and Claude APIs, where the guarantees stop, and code for both.

What each feature does

Function calling (tool use) gives the model a list of tools, each with a name, a description and a JSON Schema for its inputs. The model decides whether to call one, and your code runs it and sends the result back. The output is an intent to act: a tool name plus arguments.

Structured outputs constrain the model's reply itself. You pass a schema, and the response text is JSON that follows it. Nothing gets called. The output is the answer.

OpenAI's guide draws the line the same way. Function calling is for connecting the model to tools, functions or data in your system; its example is an assistant that looks up a customer's orders in a database. A structured text.format is for shaping the model's reply to the user; its example is a math tutoring app that wants each part of a solution in a separate field so the UI can display them differently.

Anthropic's documentation splits it into two features that share one engine. JSON outputs, set with output_config.format, control what Claude says. Strict tool use, set with strict: true on a tool, controls how Claude calls your functions. Both use grammar-constrained sampling, which means the model's token choices are limited to ones that keep the output valid against the schema.

A quick decision rule

Ask what happens to the output next.

  1. Your code runs something with it and feeds the result back. That is a tool call. Define a tool.
  2. Your code stores it, displays it or passes it to the next step, and the turn is over. That is a structured answer. Use structured outputs.
  3. Both. The agent calls tools along the way and ends with a structured answer. Use both in the same request.

A classifier that labels support tickets is case 2, even though "label" sounds like an action. A research agent that searches the web and then returns a report with fixed fields is case 3.

Why forcing a tool for JSON is no longer the default

Before native structured outputs, a common pattern was to define a tool like record_answer, force the model to call it with tool_choice, and read the arguments as your JSON. It worked, but it was a workaround, and current Claude documentation steers away from it in two ways.

First, not every model accepts forced tool use. Anthropic's tool use docs say that where forced tool use isn't supported, tool_choice of type any or tool fails, while auto and none still work. For several of its newest models, any and tool return a 400 error. The recommended alternative for those models is auto with strict tool use to guarantee schema-valid tool inputs, or structured outputs when you need a response in a fixed JSON shape.

Second, forcing a tool skips thinking. The same page says that on every model that accepts forced tool use, a forced call starts with the tool call and has no thinking block. If you want the model to reason before it answers, structured outputs or auto are the better fit.

On the OpenAI side, forcing still works: tool_choice accepts auto, required, a specific function, an allowed subset, or none. But the guide is clear that a schema for the reply belongs in text.format, not in a function nobody runs.

Structured outputs in code

On Claude, the raw API takes the schema under output_config.format. The older output_format parameter and its beta header are deprecated. This is the shape from Anthropic's docs:

import json
import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model=MODEL,  # any current Claude model
    max_tokens=1024,
    messages=[{"role": "user", "content": "Extract the key information from this email: ..."}],
    output_config={
        "format": {
            "type": "json_schema",
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "email": {"type": "string"},
                    "plan_interest": {"type": "string"},
                    "demo_requested": {"type": "boolean"},
                },
                "required": ["name", "email", "plan_interest", "demo_requested"],
                "additionalProperties": False,
            },
        }
    },
)
text = next(block.text for block in response.content if block.type == "text")
contact = json.loads(text)

With Pydantic, the Python SDK's client.messages.parse() takes the model class and returns a typed object in response.parsed_output.

On OpenAI's Responses API, client.responses.parse() does the same job with text_format:

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI()


class CalendarEvent(BaseModel):
    name: str
    date: str
    participants: list[str]


response = client.responses.parse(
    model=MODEL,  # any model that supports Structured Outputs
    input=[
        {"role": "system", "content": "Extract the event information."},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."},
    ],
    text_format=CalendarEvent,
)
event = response.output_parsed

OpenAI also still offers JSON mode, which only guarantees valid JSON. Its guide says only Structured Outputs ensure schema adherence, and recommends using them instead of JSON mode when possible.

Strict mode for tool calls

Without strict mode, tool arguments are the model's best effort. Anthropic's example is a booking tool that needs passengers as an integer: without strict mode the model might send "two" or "2", and with strict: true it always sends 2. Strict mode also guarantees the tool name is one you defined.

On Claude, add "strict": True to the tool and "additionalProperties": False to its schema:

tools = [{
    "name": "get_weather",
    "description": "Get the current weather in a given location",
    "strict": True,
    "input_schema": {
        "type": "object",
        "properties": {
            "location": {"type": "string", "description": "The city and state, e.g. San Francisco, CA"},
            "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
        },
        "required": ["location"],
        "additionalProperties": False,
    },
}]

OpenAI's function calling guide recommends always enabling strict mode, and its rules are tighter: additionalProperties must be false on every object, and every field must be listed in required. To make a field optional, give it a type that includes null, such as ["string", "null"]. Chat Completions requests are non-strict by default, while the Responses API tries to normalize your schema into strict mode when it can.

Strict mode fixes malformed arguments. It does not fix the model picking the wrong tool. For that, the problem is usually the tool set itself, which why your agent calls the wrong tool goes through cause by cause.

Using both in one agent

Anthropic's docs say the two features can be used independently or together in the same request, and describe the combination as useful for agentic workflows that need reliable tool calls and a structured final output. The request simply carries both: a tools list with strict: true on the tools that matter, and an output_config.format for the final answer.

Anthropic suggests marking only critical tools as strict and relying on the model's normal adherence for simple ones. There is a practical reason too: a request can have at most 20 strict tools, 24 optional parameters and 16 parameters with union types. Past internal limits, the API returns a 400 error saying the schema is too complex.

Where the guarantees stop

"Guaranteed schema" has exceptions, and your parsing code should handle them.

  • Refusals. If Claude refuses for safety reasons, the response has stop_reason: "refusal", a 200 status, and output that might not match your schema. OpenAI's API reports refusals in a separate refusal field, so check for it before reading the parsed result.
  • Truncation. If the response hits max_tokens, it has stop_reason: "max_tokens" and the JSON may be incomplete. Anthropic's advice is to retry with a higher limit.
  • Enum casing. Claude's structured outputs don't guarantee the capitalization of enum and const values. Anthropic recommends comparing them case-insensitively and avoiding values that differ only in case.
  • Unrelated input. OpenAI's guide warns that the model will always try to fit the schema, which can produce hallucinations if the input has nothing to do with it. Give the schema a way to say "not found", such as a nullable field or a status enum.
  • Reasoning fields. Anthropic notes that a property asking for the model's step-by-step reasoning might trigger a refusal, and suggests asking for a short explanation instead. That matters if you move a grader like the one in the LLM-as-a-judge post onto structured outputs.

A small wrapper covers the first three on Claude:

import json


class StructuredOutputError(Exception):
    pass


def read_json_output(response) -> dict:
    """Return the JSON object from a Claude response that used output_config.format."""
    if response.stop_reason == "refusal":
        raise StructuredOutputError("refused: the output may not match the schema")
    if response.stop_reason == "max_tokens":
        raise StructuredOutputError("cut off at max_tokens: retry with a higher limit")
    text = next(block.text for block in response.content if block.type == "text")
    return json.loads(text)


def match_enum(value: str, allowed: list[str]) -> str:
    """Map a returned enum value back to the schema's spelling, ignoring case."""
    by_lower = {a.lower(): a for a in allowed}
    try:
        return by_lower[value.lower()]
    except KeyError:
        raise StructuredOutputError(f"{value!r} is not one of {allowed}") from None

Given "Needs Review" and an allowed list of ["needs review", "done"], match_enum returns "needs review". Count these errors in your agent evals: a rising refusal or truncation rate is a regression like any other.

Schemas have a first-request cost

Both providers process your schema before its first use. OpenAI says the first request with any schema has extra latency, and later requests with the same schema do not. Anthropic says compiled grammars are cached for 24 hours from last use. Changing the schema's structure, or the set of tools when you use both features, invalidates that cache; changing only a name or description does not.

So keep schemas stable, the same advice that applies to prompt caching. An agent that builds a fresh schema on every request pays the compile cost every time.

Checklist

  • Tools for actions, structured outputs for answers, both when the agent does both.
  • Stop forcing a fake tool to get JSON; use output_config.format or text.format.
  • Turn on strict mode for tools whose arguments must be exact.
  • Set additionalProperties: false on every object.
  • Handle refusals and max_tokens before parsing.
  • Compare enum values case-insensitively on Claude.
  • Give the schema a way to say "not found".
  • Keep schemas stable so the compiled version is reused.

FAQ

What is the difference between structured outputs and function calling?

Function calling lets the model ask your code to run a tool with arguments that follow a schema. Structured outputs make the model's own reply follow a schema. One is an action, the other is an answer.

Is JSON mode the same as structured outputs?

No. JSON mode only guarantees valid JSON. Structured outputs also guarantee the JSON matches your schema, and OpenAI recommends them over JSON mode when possible.

Can I still force a tool call to get JSON from Claude?

On some models, but not all. Anthropic's docs list several of its newest models where tool_choice of type any or tool returns a 400 error, and recommend structured outputs instead when you need a fixed JSON shape.

Can I use tools and structured outputs in the same request?

Yes. Anthropic's docs say both features can be used together, so an agent can call strict tools during the turn and return a final answer that matches a JSON schema.

Does strict mode guarantee the output is always valid?

It guarantees the shape in normal responses. Refusals and responses cut off at the token limit can still break the schema, so check the stop reason before parsing.

Sources

Next in the notebook

  1. No. 2Why Your Agent Calls the Wrong Tool (and How to Fix It)

    A debugging checklist for tool-calling agents: overlapping tools, vague descriptions, ambiguous parameters, noisy output and bad errors, with a fix for each.

  2. No. 3How to Write Evals for an AI Agent: A Practical Starter Guide

    Build your first agent eval suite step by step: pick real tasks, grade outcomes instead of paths, measure consistency with pass^k and read the transcripts.

  3. No. 1Prompt Caching for AI Agents: How to Cut Input Costs

    How prompt caching works on Claude and OpenAI, how to order an agent's prompt so the cache hits, and how to measure your hit rate from the usage fields.