On its own, a large language model can't do anything except predict text. It has no live data, no ability to hit an API, no way to write to a database. Everything that makes an "AI agent" feel capable — checking the weather, querying your orders table, sending an email — runs through one mechanism: tool calling (also called function calling). It is the bridge between a text predictor and the real world, and it is far simpler than the agent frameworks built on top of it suggest. The single most important thing to understand about it is also the most commonly misunderstood: the model never runs your code.

What tool calling actually is

You give the model a menu. Alongside the user's prompt, you pass a tools array — a list of functions your application is willing to run, each described with a JSON Schema that specifies its name, what it does, and its typed parameters. Per OpenAI's documentation, a function definition carries a type, name, description, and parameters (the JSON schema for its arguments).

The model reads that menu and the conversation, and decides whether a tool would help. If it does, it does not execute anything. It emits a structured request: a tool_calls array where each entry names a function and supplies JSON-encoded arguments it wants you to run. As the docs put it, "the model can pass data to your application, where your code can access data or take actions suggested by the model." The model's entire contribution is producing a well-typed JSON object that says what to call with which arguments.

The loop, step by step

Everything agentic is built from this cycle:

  1. Define the tools (JSON Schema) and send them with the user message.
  2. The model responds — either with a normal answer, or with a tool_calls request.
  3. Your code executes the named function with the model's arguments.
  4. You send the result back in a follow-up message so the model can use it.
  5. The model produces a final answer — or asks for another tool call, and the loop repeats.

In OpenAI's API you append the result with role: "tool", a tool_call_id linking it to the request, and the output as content, then make a second request. Anthropic's Claude expresses the same idea with different names — tools use an input_schema, the model replies with tool_use content blocks, and you return a tool_result block — but the flow is identical. This request–execute–return cycle, run until the model stops asking for tools, is the agent loop. Frameworks like LangChain or the OpenAI Agents SDK are mostly ergonomics wrapped around exactly these steps.

python
# Illustrative (OpenAI-style) — the loop is the whole idea
tools = [{
    "type": "function",
    "name": "get_weather",
    "description": "Get current weather for a city",
    "parameters": {                     # JSON Schema
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"],
    },
}]

# The model returns tool_calls instead of a final answer:
for call in response.tool_calls:
    args = json.loads(call.function.arguments)   # model-generated — VALIDATE it
    result = get_weather(**args)                 # YOUR code runs the function
    messages.append({
        "role": "tool",
        "tool_call_id": call.id,
        "content": result,
    })
# Second request with the tool result -> final answer (or more tool calls)

The model never executes anything — and why that matters

Because the model only proposes a call, the boundary of trust sits squarely in your application. The arguments in a tool_calls object are model-generated text. The model can get them wrong, invent a value, or — if a user's input manipulated it — request something it shouldn't. Treat every tool argument as untrusted input: validate types and ranges, scope what each tool is allowed to touch, and never blindly pass model output into a shell, an eval, or an unparameterized SQL query.

This is not a theoretical concern. The entire class of agent security problems comes from applications that execute the model's requested actions without a gate in between. The safe mental model is that tool calling gives the LLM a suggestion box, not a set of car keys. Your code reads the suggestion and decides — with real authorization checks — whether and how to act on it.

Controlling and scaling it

You are not at the mercy of the model's judgment about when to call tools. OpenAI's tool_choice lets you steer: auto (the default — call zero, one, or many), required (it must call at least one), none (behave as if no tools exist), or a named function to force one specific call. This is invaluable for reliability — forcing a tool when you know one is needed, or disabling tools for a purely conversational turn.

Modern models can also request multiple tool calls in a single response, letting your application execute independent calls in parallel and return all the results together — much faster than a strict one-at-a-time chain when the calls don't depend on each other.

Key Takeaways

  • Tool calling is how an LLM reaches the real world: you expose functions as JSON Schema, the model requests them, your app runs them.
  • The model never executes anything — it emits a structured tool_calls (or tool_use) request; your code does the running and feeds results back.
  • The request–execute–return cycle repeated until the model stops is the agent loop; frameworks just wrap it.
  • Treat tool arguments as untrusted input — validate and authorize, because that boundary is where agent security lives.
  • Use tool_choice (auto/required/none/named) and parallel calls to control reliability and speed.

References

AI AgentsLLMFunction CallingTool UseAPI