Agent Frameworks Compared: Tool-Calling Schema Handling

extract

Agent Frameworks Compared: Tool-Calling Schema Handling OpenRouter · You build an agent with working tool calling, then change the model behind it and the tool calls start failing. Your tool definition did not change. The request and response format the model expects did, because each provider uses its own shape for tool definitions, tool call responses, argument encoding, and tool results. If nothing in your stack translates between those shapes, a model swap turns into new parsing code and new failure modes. This article compares how six agent frameworks define tool schemas and translate them across providers, and where the translation runs in each case. It then shows how we normalize tool calling at the API layer so the shape your application sends and receives stays the same for every tool-capable model on OpenRouter. Tl;dr - OpenAI, Anthropic, and Google each define tools and return tool calls with a different wire format. A tool definition written for one does not work unchanged on another. - LangChain and LangGraph translate one tool definition per provider. CrewAI delegates translation to the client it routes to. The OpenAI Agents SDK, the Claude Agent SDK, and Google ADK are built around one provider’s format. Microsoft Agent Framework delegates to the configured model connector. - OpenRouter accepts an OpenAI-style tools array and returns a standard tool_calls response for every model that supports tool calling, so switching models is a change to the model string. - On requests that include tools, Auto Exacto reorders providers using throughput, tool-calling success rate, and benchmark data by default, and the Tool Call Error Rate metric behind it is visible on each model’s Performance tab. Why tool-calling schemas differ across providers The providers agree on what a tool is. A tool has a name, a description, and a parameter schema that says which arguments it takes. The model reads those three pieces, decides whether to call the tool, and produces the arguments. The agreement ends there. Each provider wraps those pieces in its own request and response format, and the formats don’t interchange. In OpenAI’s Chat Completions API, you pass a tools array where each entry has type: "function" and a function object holding the name, description, and JSON Schema parameters. When the model wants a tool, the assistant message carries a tool_calls array, and each call’s arguments field is a JSON-encoded string. { "tools": [ { "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city", "parameters": { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] } } } ] } Anthropic’s Messages API uses a flatter definition with the schema under input_schema. The model’s request comes back as a tool_use content block inside the assistant message, with the arguments already parsed into an input object, and the turn ends with stop_reason: "tool_use". { "tools": [ { "name": "get_weather", "description": "Get the current weather for a city", "input_schema": { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] } } ] } Google’s Gemini API nests tool definitions under a function_declarations array in the generateContent request, and the model’s request comes back as a functionCall part inside the response content rather than as a separate top-level field. Google’s newer Interactions API accepts a different tool entry shape again, with type: "function" at the top level of each tool. { "tools": [ { "function_declarations": [ { "name": "get_weather", "description": "Get the current weather for a city", "parameters": { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] } } ] } ] } That is one tool, three request shapes, and three response shapes. Open-weight models add a further case. A model that was not trained to emit a tool-calling format can only produce a tool call as text. Any tool-calling support for such a model depends on the serving layer placing the tool definitions in the prompt and parsing the model’s output back into a call. That parsing is not part of a provider wire format, and it fails differently from a native format. Your application defines one tool, but the exact request shape it has to produce depends on the model behind the request. Either something in your stack rewrites that one definition into the right per-model format, or you write and maintain that translation yourself. Where schema translation can live There are three places the translation can run. - In your application. You write the per-provider request and response mapping yourself and maintain it as providers change. - In the framework. The agent framework, or the provider client it delegates to, takes one tool definition and produces each provider’s format. - In the API layer. A gateway in front of every model accepts one format and translates on the way to each provider. Our tool calling and structured outputs documentation shows the normalized shape in full. The frameworks below differ mainly in how much of the second option they take on, and which provider they take as their native format. How each framework handles tool schemas Every framework here supports tool calling. What differs is where the translation lives and how much of it the framework owns. LangChain and LangGraph You define a tool once as a Python function with type hints using the @tool decorator, and the type hints define the tool’s input schema. You attach tools to a model with bind_tools(), and create_agent runs the tool-calling loop. LangChain’s chat model integrations translate the definition into each provider’s format, so the same agent code runs on OpenAI, Anthropic, and Google models through init_chat_model. The abstraction hides the wire-format differences from your code. The quality of any given translation depends on the specific provider integration, so test the exact model you plan to ship on. LangChain also documents an OpenRouter chat model integration in the langchain-openrouter package, so you can select any OpenRouter model through init_chat_model and leave the per-provider translation to our API. For MCP, LangChain documents MCPAdapter, built on FastMCP, which discovers a server’s tools and adapts them into LangChain tools. The langchain.mcp namespace requires langchain[mcp]>=1.4.0 and is documented as beta. CrewAI CrewAI is organized around roles and tasks. You assign tools to an agent, and the crew coordinates the work. CrewAI does not implement provider-specific tool formatting itself. CrewAI’s LLM documentation describes native SDK integrations for OpenAI, Anthropic, Google’s Gemini API, Azure, AWS Bedrock, and Snowflake Cortex, selected by the provider/model-id string you configure. All other providers run through LiteLLM. In either case, schema handling belongs to the client CrewAI routes to, not to CrewAI. For MCP, the crewai-tools package provides MCPServerAdapter and transport classes for stdio, SSE, and streamable HTTP servers. OpenAI Agents SDK The OpenAI Agents SDK is built around OpenAI’s tool format. You define function tools with the @function_tool decorator, and the SDK generates the JSON Schema for the arguments from the function signature and docstring. It gives you the most complete support on OpenAI models, including hosted tools that run on OpenAI’s side. It is not limited to OpenAI. The SDK’s model documentation describes three built-in paths to other providers. set_default_openai_client points the SDK at an OpenAI-compatible endpoint by setting base_url and api_key, ModelProvider applies a custom provider to one run, and Agent.model sets the model for one agent. Any-LLM and LiteLLM are documented as best-effort, beta third-party adapters for cases the built-in paths do not cover. When you route through Chat Completions rather than the Responses API, the SDK drops Responses-only fields, and the documentation warns that some providers don’t support JSON Schema structured outputs. The further you move from OpenAI’s models, the more you rely on a compatibility layer rather than first-party support. MCP support is built in, covering hosted MCP server tools through OpenAI’s Responses API and direct connections to MCP servers over stdio and HTTP transports. Claude Agent SDK The Claude Agent SDK is the agent harness behind Claude Code, packaged as a Python and TypeScript library. Unlike the Anthropic Messages API, it runs the tool-execution loop for you, with built-in tools, context management, permissions, hooks, and subagents. You define custom tools with the @tool decorator in Python or tool() in TypeScript, wrap them in an in-process MCP server with create_sdk_mcp_server, and pass that server to the query. It targets Claude models, so tool calls use Anthropic’s native tool_use format directly. There is no cross-provider translation layer. Pointing it at a non-Anthropic model is not what it is built for. Microsoft Agent Framework Microsoft describes Agent Framework as the direct successor to AutoGen and Semantic Kernel, combining AutoGen’s agent abstractions with Semantic Kernel’s enterprise features and adding explicit workflows. Its overview lists Microsoft Foundry, Anthropic, Azure OpenAI, OpenAI, and Ollama among the supported model providers, with tool calls and MCP servers handled through the agent abstraction. Tools attach to an agent as typed functions the framework turns into function tools with JSON Schemas. Tool-calling translation belongs to whichever model connector you configure, so changing the provider changes the schema handling with it. Microsoft publishes a migration guide from AutoGen for teams moving from the older framework. Google ADK Google’s Agent Development Kit is built for Gemini and its native function_declarations format. When you pass a Python function to an agent’s tool list, ADK wraps it as a FunctionTool and generates the schema from the function’s signature and docstring. For models outside Gemini, ADK documents connector pages including a LiteLLM connector. On that path, a call to a Claude or GPT model runs through LiteLLM’s translation rather than through anything native to ADK, so schema behavior on those models belongs to that integration. Test the exact model you plan to use. For MCP, ADK provides McpToolset, which connects to an MCP server and exposes its tools to an agent. Framework comparison Before you pick a framework, filter our model catalog by tool-calling support. A framework listing a provider adapter and a model supporting native tool calling are two different things, and the gap between them is where the surprises come from. Treat this table as a starting point and check the current documentation before you commit. These frameworks release often, and tool-calling behavior is one of the parts that changes. What breaks when you switch models The failures take a few shapes, and none of the frameworks above remove them. A schema that one provider accepts is rejected by a stricter one. A deeply nested parameter object, an unusual type, or a constraint that one provider ignores and another enforces is enough to produce a request error on the new model. A model without native tool calling returns the call as plain text. Nothing in the pipeline recognizes it as a tool_call, so there is no error to catch, only a response that isn’t what your loop expected. Unless the framework or your own code adds a text-parsing fallback, the agent stops making progress instead of raising an error. Repair behavior varies. Some code paths raise a catchable error when a call comes back malformed. Others leave detection and recovery to you. If your agent loop assumes one behavior and the framework provides the other, a model swap can make an error go unnoticed. A subtler version appears without switching model families. The same model served by two different providers can return valid tool calls at different rates. We have scored every tool call response across OpenRouter since August 2025, and in our Auto Exacto announcement we reported that the tool call error rate for GLM-5 and GLM-4.7 on the affected providers fell from approximately 8% to closer to 1% after we began routing tool-calling traffic away from weaker endpoints. We classify each failed tool call into three categories, and the same three checks work for your own logging. InvalidJson means the arguments don’t parse as JSON. UnknownName means the called function name isn’t in the request’s tool list. SchemaMismatch means the arguments don’t validate against the tool’s parameter schema. For the loop mechanics, including retries, stop conditions, and turn-by-turn control, see our guide to building a tool-calling agent loop. Normalizing tool calling at the API layer The third place the translation can live is the API layer, below the framework. We normalize tool calling once, in front of every model, so nothing above it sees the per-provider differences. You send an OpenAI-style request with a tools array. We transform it into the target provider’s format, run it, and return a standard tool_calls response. For every model that supports tool calling, the shape you send and receive is the same, so switching models is a change to the model string. The example below defines the tool once in OpenAI format and reads the result from message.tool_calls regardless of which model ran the request. import json import os from openai import OpenAI client = OpenAI( base_url="https://openrouter.ai/api/v1", api_key=os.environ["OPENROUTER_API_KEY"], ) def get_weather(city: str) -> dict: return {"city": city, "forecast": "sunny", "temperature_c": 24} tools = [{ "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city", "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], }, }, }] messages = [{"role": "user", "content": "What's the weather in Lisbon?"}] # Change this string to anthropic/claude-sonnet-4.6 or google/gemini-3.5-flash # and nothing else in this file changes. response = client.chat.completions.create( model="openai/gpt-5.5", messages=messages, tools=tools, ) choice = response.choices[0] messages.append(choice.message) # The same tool_calls shape comes back from every tool-capable model. for call in choice.message.tool_calls or []: args = json.loads(call.function.arguments) # arguments is a JSON string result = get_weather(**args) messages.append({ "role": "tool", "tool_call_id": call.id, "content": json.dumps(result), }) Two things carry across every model. The tool definition and the response you parse stay the same. tool_calls is always an array and each arguments value is a JSON string, so loop over the array and parse each entry rather than assuming a single call. This isn’t a replacement for an agent framework. You still need something to run the loop, hold state, and decide when to stop. What it removes is the per-model tool-calling adapter code from your application. The schema translation moves out of the layer you maintain and into one you call. Provider routing for tool-calling requests Normalizing the format is one half of the reliability question. The other half is which provider runs the call. Because we score every tool call response, we can route tool-calling requests toward the endpoints that return valid tool calls most often. Auto Exacto runs by default on every request that includes tools and requires no configuration. It reorders the available providers for your chosen model using three signals. Throughput is the real-time tokens-per-second measurement for each endpoint. Tool-calling success rate is derived from the Tool Call Error Rate metric. Benchmark data comes from our own benchmark harness, which runs GPQA Diamond and Tau2-Bench Airline against enrolled provider endpoints on a recurring schedule. The Tool Call Error Rate metric is on the Performance tab of each model page. For each request that includes tools, we inspect every tool call the model returned and validate its arguments against the tools[].function.parameters schema you supplied, using JSON Schema Draft 7. A tool whose parameters schema is absent or fails to compile counts as valid, so the metric stays conservative when a caller’s schema is malformed. The metric is aggregated per request rather than per tool call, so one request with several bad calls counts once. Auto Exacto can conflict with the sticky provider routing that keeps prompt caches warm, because it may move a request to a different endpoint mid-session. If cache hit rate matters more than provider reordering for your agent loop, the documentation describes how to disable Auto Exacto with sort: "price" in the provider object or the :floor model variant. Two checks apply before you rely on either mechanism. Tool calling isn’t universal, so confirm that each model you route to supports it through the catalog filter above or the model’s page. And if you run the same agent across several models to compare quality, compare cost at the same time. Current per-model rates are on the pricing page. Conclusion Tool calling isn’t one format. Some frameworks are translators, some are native to a single provider, and some delegate the question to a connector or client underneath them. If you need the deepest support on one provider, a native SDK such as the OpenAI Agents SDK or the Claude Agent SDK is the closest fit. If you need to move across providers, LangChain and LangGraph carry the most translation in the framework layer, or you can take the translation out of the framework by normalizing tool calling at the API layer. Decide where the schema translation lives before you build, and a model swap stays a one-line change. Frequently asked questions How do the major agent frameworks differ in handling tool-calling schemas? They differ in where schema translation runs. LangChain and LangGraph translate one tool definition into each provider’s format through per-provider chat model integrations. CrewAI delegates to the provider SDK or LiteLLM client it routes the call to. The OpenAI Agents SDK, the Claude Agent SDK, and Google ADK are each built around one provider’s format and reach other providers through compatibility endpoints, adapters, or connectors. Microsoft Agent Framework hands translation to the model connector you configure. Normalizing tool calling at the API layer moves the translation below all of them. Which agent SDK should you use for tool calling? Match the SDK to the set of models you need. A provider-native SDK such as the OpenAI Agents SDK or the Claude Agent SDK is the closest fit when you plan to stay on that provider’s models. LangChain and LangGraph carry the most translation for moving between providers. CrewAI delegates translation to the client it routes to. Routing through OpenRouter keeps the tool definition and response shape the same across models, so the SDK choice no longer constrains the model choice. Which agent SDKs support MCP? The OpenAI Agents SDK, the Claude Agent SDK, Microsoft Agent Framework, Google ADK, LangChain, and CrewAI all document support for Model Context Protocol servers as a tool source. The integration depth varies, and LangChain’s current MCP namespace is documented as beta. MCP standardizes how tools are discovered and described. It does not standardize the model’s tool-calling wire format, which is the separate problem this article covers. How do you keep a long tool-calling loop working across models? Call the model, check the finish reason, run each requested tool, append the results by tool call ID, and repeat until the model stops requesting tools or you reach a step cap. Validate tool arguments before executing them, because a model can return arguments that do not match your schema. Normalize the tool-calling format so the loop has no per-model branches, and route to providers with a low tool call error rate. References - Tool Calling, OpenRouter - Structured Outputs, OpenRouter - Auto Exacto, OpenRouter - Auto Exacto: Adaptive Quality Routing, On by Default, OpenRouter - Build a Reliable Tool-Calling Agent Loop on OpenRouter, OpenRouter - Function calling, OpenAI - Tool use overview, Anthropic - Function calling with the Gemini API, Google - Models, Tools, and MCP, LangChain - LLMs and MCP servers as tools, CrewAI - Models, Tools, and MCP, OpenAI Agents SDK - Agent SDK overview and Custom tools, Anthropic - Agent Framework overview and Migration guide from AutoGen, Microsoft - Models, LiteLLM connector, Function tools, and MCP tools, Google ADK