Tool Calls (Function Calling)

Overview

Tool calls (also known as function calling) allow LLMs to invoke external tools and functions. This enables models to perform actions like fetching data, executing calculations, or interacting with APIs.

TokenRouter supports tool calling across all major providers with automatic transformation to each provider’s native format.

Basic Tool Call

TypeScript

import Tokenrouter from 'tokenrouter';

const client = new Tokenrouter({
  apiKey: process.env.TOKENROUTER_API_KEY
});

const response = await client.responses.create({
  model: 'auto:balance',
  input: 'What is the weather in San Francisco?',
  tools: [
    {
      type: 'function',
      function: {
        name: 'get_weather',
        description: 'Get the current weather for a location',
        parameters: {
          type: 'object',
          properties: {
            location: {
              type: 'string',
              description: 'City name, e.g. San Francisco'
            },
            unit: {
              type: 'string',
              enum: ['celsius', 'fahrenheit'],
              description: 'Temperature unit'
            }
          },
          required: ['location']
        }
      }
    }
  ]
});

// Check if model wants to call a function
if (response.output[0].content[0].type === 'tool_use') {
  const toolCall = response.output[0].content[0];
  console.log('Function:', toolCall.name);
  console.log('Arguments:', toolCall.input);
}

Python

from tokenrouter import Tokenrouter

client = Tokenrouter(
    api_key=os.getenv("TOKENROUTER_API_KEY")
)

response = client.responses.create(
    model="auto:balance",
    input="What is the weather in San Francisco?",
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get the current weather for a location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "City name, e.g. San Francisco"
                        },
                        "unit": {
                            "type": "string",
                            "enum": ["celsius", "fahrenheit"],
                            "description": "Temperature unit"
                        }
                    },
                    "required": ["location"]
                }
            }
        }
    ]
)

# Check if model wants to call a function
if response.output[0].content[0].type == "tool_use":
    tool_call = response.output[0].content[0]
    print(f"Function: {tool_call.name}")
    print(f"Arguments: {tool_call.input}")

cURL

curl https://api.tokenrouter.io/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer tr_..." \
  -d '{
    "model": "auto:balance",
    "input": "What is the weather in San Francisco?",
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get the current weather for a location",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {
                "type": "string",
                "description": "City name"
              },
              "unit": {
                "type": "string",
                "enum": ["celsius", "fahrenheit"]
              }
            },
            "required": ["location"]
          }
        }
      }
    ]
  }'

Tool Response Format

When a model decides to call a tool, the response includes:

{
  "id": "resp_...",
  "model": "gpt-4o",
  "output": [
    {
      "role": "assistant",
      "content": [
        {
          "type": "tool_use",
          "id": "call_abc123",
          "name": "get_weather",
          "input": {
            "location": "San Francisco",
            "unit": "fahrenheit"
          }
        }
      ]
    }
  ],
  "metadata": {
    "provider": "openai",
    "stop_reason": "tool_use"
  }
}

Complete Tool Call Flow

TypeScript

// Step 1: Define tools
const tools = [
  {
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Get weather for a location',
      parameters: {
        type: 'object',
        properties: {
          location: { type: 'string' },
          unit: { type: 'string', enum: ['celsius', 'fahrenheit'] }
        },
        required: ['location']
      }
    }
  }
];

// Step 2: Send initial request
const response = await client.responses.create({
  model: 'auto:balance',
  input: 'What is the weather in San Francisco?',
  tools
});

// Step 3: Execute tool if requested
const toolUse = response.output[0].content[0];

if (toolUse.type === 'tool_use') {
  // Execute the actual function
  const weatherData = await getWeather(
    toolUse.input.location,
    toolUse.input.unit
  );

  // Step 4: Send tool result back to model
  const finalResponse = await client.responses.create({
    model: 'auto:balance',
    input: [
      {
        role: 'user',
        content: 'What is the weather in San Francisco?'
      },
      {
        role: 'assistant',
        content: [
          {
            type: 'tool_use',
            id: toolUse.id,
            name: toolUse.name,
            input: toolUse.input
          }
        ]
      },
      {
        role: 'user',
        content: [
          {
            type: 'tool_result',
            tool_use_id: toolUse.id,
            content: JSON.stringify(weatherData)
          }
        ]
      }
    ],
    tools
  });

  console.log(finalResponse.output[0].content[0].text);
  // "The weather in San Francisco is currently 68°F and sunny."
}

// Helper function
async function getWeather(location: string, unit: string) {
  // Call actual weather API
  return {
    temperature: 68,
    unit: 'fahrenheit',
    condition: 'sunny',
    location: 'San Francisco'
  };
}

Python

import json

# Step 1: Define tools
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"},
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"]
                    }
                },
                "required": ["location"]
            }
        }
    }
]

# Step 2: Send initial request
response = client.responses.create(
    model="auto:balance",
    input="What is the weather in San Francisco?",
    tools=tools
)

# Step 3: Execute tool if requested
tool_use = response.output[0].content[0]

if tool_use.type == "tool_use":
    # Execute the actual function
    weather_data = get_weather(
        tool_use.input["location"],
        tool_use.input.get("unit", "fahrenheit")
    )

    # Step 4: Send tool result back to model
    final_response = client.responses.create(
        model="auto:balance",
        input=[
            {
                "role": "user",
                "content": "What is the weather in San Francisco?"
            },
            {
                "role": "assistant",
                "content": [
                    {
                        "type": "tool_use",
                        "id": tool_use.id,
                        "name": tool_use.name,
                        "input": tool_use.input
                    }
                ]
            },
            {
                "role": "user",
                "content": [
                    {
                        "type": "tool_result",
                        "tool_use_id": tool_use.id,
                        "content": json.dumps(weather_data)
                    }
                ]
            }
        ],
        tools=tools
    )

    print(final_response.output[0].content[0].text)
    # "The weather in San Francisco is currently 68°F and sunny."

# Helper function
def get_weather(location, unit):
    # Call actual weather API
    return {
        "temperature": 68,
        "unit": "fahrenheit",
        "condition": "sunny",
        "location": "San Francisco"
    }

Multiple Tools

Define multiple tools for the model to choose from:

TypeScript

const tools = [
  {
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Get current weather',
      parameters: {
        type: 'object',
        properties: {
          location: { type: 'string' }
        },
        required: ['location']
      }
    }
  },
  {
    type: 'function',
    function: {
      name: 'search_web',
      description: 'Search the internet',
      parameters: {
        type: 'object',
        properties: {
          query: { type: 'string' }
        },
        required: ['query']
      }
    }
  },
  {
    type: 'function',
    function: {
      name: 'calculate',
      description: 'Perform mathematical calculations',
      parameters: {
        type: 'object',
        properties: {
          expression: { type: 'string' }
        },
        required: ['expression']
      }
    }
  }
];

const response = await client.responses.create({
  model: 'auto:balance',
  input: 'What is 25 * 48?',
  tools
});

// Model will choose 'calculate' tool

Python

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "search_web",
            "description": "Search the internet",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {"type": "string"}
                },
                "required": ["query"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate",
            "description": "Perform mathematical calculations",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {"type": "string"}
                },
                "required": ["expression"]
            }
        }
    }
]

response = client.responses.create(
    model="auto:balance",
    input="What is 25 * 48?",
    tools=tools
)

# Model will choose 'calculate' tool

Tool Choice Control

Control which tool the model uses:

TypeScript

// Auto (default) - Model decides
const response1 = await client.responses.create({
  model: 'auto:balance',
  input: 'What is the weather?',
  tools,
  tool_choice: 'auto'
});

// Force specific tool
const response2 = await client.responses.create({
  model: 'auto:balance',
  input: 'Tell me about the weather',
  tools,
  tool_choice: {
    type: 'function',
    function: { name: 'get_weather' }
  }
});

// Require any tool
const response3 = await client.responses.create({
  model: 'auto:balance',
  input: 'Help me with this task',
  tools,
  tool_choice: 'required'
});

// Disable tools
const response4 = await client.responses.create({
  model: 'auto:balance',
  input: 'Just chat, no tools',
  tools,
  tool_choice: 'none'
});

Python

# Auto (default) - Model decides
response1 = client.responses.create(
    model="auto:balance",
    input="What is the weather?",
    tools=tools,
    tool_choice="auto"
)

# Force specific tool
response2 = client.responses.create(
    model="auto:balance",
    input="Tell me about the weather",
    tools=tools,
    tool_choice={
        "type": "function",
        "function": {"name": "get_weather"}
    }
)

# Require any tool
response3 = client.responses.create(
    model="auto:balance",
    input="Help me with this task",
    tools=tools,
    tool_choice="required"
)

# Disable tools
response4 = client.responses.create(
    model="auto:balance",
    input="Just chat, no tools",
    tools=tools,
    tool_choice="none"
)

Parallel Tool Calls

Some providers support calling multiple tools in one response:

{
  "output": [
    {
      "role": "assistant",
      "content": [
        {
          "type": "tool_use",
          "id": "call_1",
          "name": "get_weather",
          "input": {"location": "San Francisco"}
        },
        {
          "type": "tool_use",
          "id": "call_2",
          "name": "get_weather",
          "input": {"location": "New York"}
        }
      ]
    }
  ]
}

Handle multiple calls:

TypeScript

const content = response.output[0].content;

const toolCalls = content.filter(item => item.type === 'tool_use');

// Execute all tools in parallel
const results = await Promise.all(
  toolCalls.map(async (call) => {
    const result = await executeT ool(call.name, call.input);
    return {
      type: 'tool_result',
      tool_use_id: call.id,
      content: JSON.stringify(result)
    };
  })
);

// Send all results back
const finalResponse = await client.responses.create({
  model: 'auto:balance',
  input: [
    originalUserMessage,
    {
      role: 'assistant',
      content: toolCalls
    },
    {
      role: 'user',
      content: results
    }
  ],
  tools
});

Python

import asyncio

content = response.output[0].content

tool_calls = [item for item in content if item.type == "tool_use"]

# Execute all tools in parallel
async def execute_all():
    tasks = [
        execute_tool(call.name, call.input)
        for call in tool_calls
    ]
    return await asyncio.gather(*tasks)

results_data = asyncio.run(execute_all())

results = [
    {
        "type": "tool_result",
        "tool_use_id": call.id,
        "content": json.dumps(result)
    }
    for call, result in zip(tool_calls, results_data)
]

# Send all results back
final_response = client.responses.create(
    model="auto:balance",
    input=[
        original_user_message,
        {
            "role": "assistant",
            "content": tool_calls
        },
        {
            "role": "user",
            "content": results
        }
    ],
    tools=tools
)

Provider Compatibility

Provider	Tool Support	Parallel Calls	Notes
OpenAI	✅ Full	✅ Yes	Native support
Anthropic	✅ Full	✅ Yes	Native support
Mistral	✅ Full	✅ Yes	Native support
Google Gemini	✅ Full	⚠️ Limited	Single tool per turn
DeepSeek	✅ Full	✅ Yes	OpenAI-compatible

Streaming with Tools

Tool calls work with streaming:

TypeScript

const stream = await client.responses.create({
  model: 'auto:balance',
  input: 'What is the weather?',
  tools,
  stream: true
});

let toolCalls: any[] = [];

for await (const chunk of stream) {
  if (chunk.event === 'content.delta') {
    if (chunk.delta.type === 'tool_use') {
      toolCalls.push(chunk.delta);
    }
  }

  if (chunk.event === 'done') {
    // Execute tools
    for (const call of toolCalls) {
      const result = await executeTools(call);
      // Continue conversation...
    }
  }
}

Python

stream = client.responses.create(
    model="auto:balance",
    input="What is the weather?",
    tools=tools,
    stream=True
)

tool_calls = []

for chunk in stream:
    if chunk.event == "content.delta":
        if chunk.delta.type == "tool_use":
            tool_calls.append(chunk.delta)

    if chunk.event == "done":
        # Execute tools
        for call in tool_calls:
            result = execute_tool(call)
            # Continue conversation...

Best Practices

1. Clear descriptions - Be specific about what each tool does
2. Required fields - Always specify required parameters
3. Validate input - Check tool arguments before execution
4. Handle errors - Gracefully handle tool execution failures
5. Return structured data - Use JSON for tool results
6. Security - Validate and sanitize all tool inputs
7. Rate limiting - Control expensive tool calls

Common Patterns

Database Query Tool

{
  type: 'function',
  function: {
    name: 'query_database',
    description: 'Execute SQL query on database',
    parameters: {
      type: 'object',
      properties: {
        query: {
          type: 'string',
          description: 'SQL SELECT query'
        },
        limit: {
          type: 'number',
          description: 'Max rows to return',
          default: 100
        }
      },
      required: ['query']
    }
  }
}

API Call Tool

{
  type: 'function',
  function: {
    name: 'fetch_api',
    description: 'Make HTTP request to API',
    parameters: {
      type: 'object',
      properties: {
        url: { type: 'string' },
        method: {
          type: 'string',
          enum: ['GET', 'POST', 'PUT', 'DELETE']
        },
        body: { type: 'object' }
      },
      required: ['url', 'method']
    }
  }
}

File Operations Tool

{
  type: 'function',
  function: {
    name: 'read_file',
    description: 'Read contents of a file',
    parameters: {
      type: 'object',
      properties: {
        path: {
          type: 'string',
          description: 'File path'
        },
        encoding: {
          type: 'string',
          enum: ['utf-8', 'ascii', 'binary'],
          default: 'utf-8'
        }
      },
      required: ['path']
    }
  }
}

Error Handling

Handle tool execution errors gracefully:

TypeScript

try {
  const result = await executeToolusing(call.name, call.input);

  return {
    type: 'tool_result',
    tool_use_id: call.id,
    content: JSON.stringify(result)
  };
} catch (error) {
  // Return error as tool result
  return {
    type: 'tool_result',
    tool_use_id: call.id,
    content: JSON.stringify({
      error: error.message,
      type: 'execution_error'
    }),
    is_error: true
  };
}

Python

try:
    result = execute_tool(call.name, call.input)

    return {
        "type": "tool_result",
        "tool_use_id": call.id,
        "content": json.dumps(result)
    }
except Exception as error:
    # Return error as tool result
    return {
        "type": "tool_result",
        "tool_use_id": call.id,
        "content": json.dumps({
            "error": str(error),
            "type": "execution_error"
        }),
        "is_error": True
    }

Next Steps

• Response Format - Structure tool responses
• Streaming - Use tools with streaming
• Errors - Handle tool call errors