Skip to content
TokenRouter - Intelligent AI Routing Platform TokenRouter - Intelligent AI Routing Platform TokenRouter

Response Format

Response Format

Section titled “Response Format”

The response_format parameter controls how the LLM structures its output. TokenRouter supports three response formats:

  • text (default) - Natural language text responses
  • json_object - Valid JSON object responses
  • json_schema - Structured outputs conforming to a JSON schema

Format Types

Section titled “Format Types”

Text Format (Default)

Section titled “Text Format (Default)”

Natural language responses without any structure constraints. This is the default format when response_format is not specified.

import { TokenRouter } from 'tokenrouter';


const client = new TokenRouter({
  apiKey: process.env.TOKENROUTER_API_KEY!
});


const response = await client.responses.create({
  model: 'gpt-4o',
  input: 'Explain quantum computing in simple terms'
  // response_format defaults to text
});


console.log(response.output_text);
// "Quantum computing is a type of computing that uses..."

JSON Object Format

Section titled “JSON Object Format”

Forces the model to return a valid JSON object. The model will structure its response as JSON, but the exact schema is not enforced.

const response = await client.responses.create({
  model: 'gpt-4o',
  input: 'List three programming languages with their use cases',
  response_format: {
    type: 'json_object'
  }
});


const data = JSON.parse(response.output_text);
console.log(data);
// {
//   "languages": [
//     {"name": "Python", "use_case": "Data science and AI"},
//     {"name": "JavaScript", "use_case": "Web development"},
//     {"name": "Rust", "use_case": "Systems programming"}
//   ]
// }

JSON Schema Format

Section titled “JSON Schema Format”

Enforces a specific structure by providing a JSON schema. The model’s output will strictly conform to the provided schema.

const response = await client.responses.create({
  model: 'gpt-4o-mini',
  input: 'Generate a user profile for John Doe, age 30, from New York',
  response_format: {
    type: 'json_schema',
    json_schema: {
      name: 'user_profile',
      strict: true,
      schema: {
        type: 'object',
        properties: {
          name: {
            type: 'string',
            description: 'Full name'
          },
          age: {
            type: 'integer',
            description: 'Age in years'
          },
          location: {
            type: 'string',
            description: 'City and country'
          },
          interests: {
            type: 'array',
            items: { type: 'string' },
            description: 'List of interests'
          }
        },
        required: ['name', 'age', 'location'],
        additionalProperties: false
      }
    }
  }
});


const profile = JSON.parse(response.output_text);
console.log(profile);
// {
//   "name": "John Doe",
//   "age": 30,
//   "location": "New York, USA",
//   "interests": ["technology", "reading", "travel"]
// }

Provider Compatibility

Section titled “Provider Compatibility”

Different providers support different response format capabilities:

ProviderTextJSON ObjectJSON SchemaModels
OpenAI✅✅✅gpt-4o, gpt-4o-mini, gpt-4-turbo
DeepSeek✅✅❌deepseek-chat, deepseek-reasoner
Anthropic✅❌❌All models
Gemini✅❌❌All models
Mistral✅❌❌All models

Common Use Cases

Section titled “Common Use Cases”

Data Extraction

Section titled “Data Extraction”

Extract structured data from unstructured text:

const response = await client.responses.create({
  model: 'gpt-4o',
  input: `Extract information from this text:
    "John Smith works at Acme Corp as a Software Engineer.
    He can be reached at john@example.com or 555-1234."`,
  response_format: {
    type: 'json_schema',
    json_schema: {
      name: 'contact_extraction',
      strict: true,
      schema: {
        type: 'object',
        properties: {
          name: { type: 'string' },
          company: { type: 'string' },
          title: { type: 'string' },
          email: { type: 'string' },
          phone: { type: 'string' }
        },
        required: ['name'],
        additionalProperties: false
      }
    }
  }
});


const contact = JSON.parse(response.output_text);
// {
//   "name": "John Smith",
//   "company": "Acme Corp",
//   "title": "Software Engineer",
//   "email": "john@example.com",
//   "phone": "555-1234"
// }

Sentiment Analysis

Section titled “Sentiment Analysis”

Analyze sentiment with structured output:

const response = await client.responses.create({
  model: 'gpt-4o',
  input: 'Analyze the sentiment of: "This product exceeded my expectations!"',
  response_format: {
    type: 'json_schema',
    json_schema: {
      name: 'sentiment_analysis',
      strict: true,
      schema: {
        type: 'object',
        properties: {
          sentiment: {
            type: 'string',
            enum: ['positive', 'negative', 'neutral']
          },
          confidence: {
            type: 'number',
            minimum: 0,
            maximum: 1
          },
          keywords: {
            type: 'array',
            items: { type: 'string' }
          }
        },
        required: ['sentiment', 'confidence'],
        additionalProperties: false
      }
    }
  }
});


const analysis = JSON.parse(response.output_text);
// {
//   "sentiment": "positive",
//   "confidence": 0.95,
//   "keywords": ["exceeded", "expectations"]
// }

Classification

Section titled “Classification”

Classify content into predefined categories:

const response = await client.responses.create({
  model: 'gpt-4o',
  input: 'Classify this support ticket: "My password reset email never arrived"',
  response_format: {
    type: 'json_schema',
    json_schema: {
      name: 'ticket_classification',
      strict: true,
      schema: {
        type: 'object',
        properties: {
          category: {
            type: 'string',
            enum: ['account', 'billing', 'technical', 'feature_request']
          },
          priority: {
            type: 'string',
            enum: ['low', 'medium', 'high', 'urgent']
          },
          tags: {
            type: 'array',
            items: { type: 'string' }
          }
        },
        required: ['category', 'priority'],
        additionalProperties: false
      }
    }
  }
});


const classification = JSON.parse(response.output_text);
// {
//   "category": "account",
//   "priority": "medium",
//   "tags": ["password", "email", "authentication"]
// }

Streaming with JSON

Section titled “Streaming with JSON”

When using streaming with JSON formats, the response will be delivered in chunks. You need to accumulate the chunks and parse the complete JSON at the end:

const stream = await client.responses.create({
  model: 'gpt-4o',
  input: 'Generate a list of 5 programming tips',
  response_format: { type: 'json_object' },
  stream: true
});


let jsonText = '';


for await (const chunk of stream) {
  if (chunk.event === 'content.delta') {
    jsonText += chunk.delta.text;
  }
}


const data = JSON.parse(jsonText);
console.log(data);

Parse After Completion

When streaming JSON responses, always accumulate the full text before parsing. Partial JSON strings will cause parse errors.

Schema Design Best Practices

Section titled “Schema Design Best Practices”

Use Descriptive Field Names

Section titled “Use Descriptive Field Names”
{
  "type": "object",
  "properties": {
    "user_email": { "type": "string" },  // Good: Clear purpose
    "email": { "type": "string" }         // Avoid: Ambiguous
  }
}

Include Descriptions

Section titled “Include Descriptions”
{
  "type": "object",
  "properties": {
    "temperature": {
      "type": "number",
      "description": "Temperature in Celsius",  // Helps model understand
      "minimum": -273.15,
      "maximum": 1000
    }
  }
}

Use Enums for Fixed Values

Section titled “Use Enums for Fixed Values”
{
  "type": "object",
  "properties": {
    "status": {
      "type": "string",
      "enum": ["pending", "approved", "rejected"],  // Constrains output
      "description": "Current status"
    }
  }
}

Set additionalProperties: false

Section titled “Set additionalProperties: false”
{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "age": { "type": "integer" }
  },
  "additionalProperties": false  // Prevents unexpected fields
}

Mark Required Fields

Section titled “Mark Required Fields”
{
  "type": "object",
  "properties": {
    "id": { "type": "string" },
    "name": { "type": "string" },
    "email": { "type": "string" }
  },
  "required": ["id", "name"]  // Only id and name are mandatory
}

Error Handling

Section titled “Error Handling”

Invalid JSON Output

Section titled “Invalid JSON Output”

If the model fails to produce valid JSON:

try {
  const response = await client.responses.create({
    model: 'gpt-4o',
    input: 'Your prompt here',
    response_format: { type: 'json_object' }
  });


  const data = JSON.parse(response.output_text);
  console.log(data);
} catch (error) {
  if (error instanceof SyntaxError) {
    console.error('Model returned invalid JSON:', error.message);
    console.error('Raw output:', response.output_text);
  } else {
    throw error;
  }
}

Schema Validation Errors

Section titled “Schema Validation Errors”

When using json_schema with strict: true, the model should always conform to the schema. If validation fails:

import Ajv from 'ajv';


const ajv = new Ajv();
const schema = {
  type: 'object',
  properties: {
    name: { type: 'string' },
    age: { type: 'integer' }
  },
  required: ['name', 'age']
};


const validate = ajv.compile(schema);


const response = await client.responses.create({
  model: 'gpt-4o-mini',
  input: 'Generate a user profile',
  response_format: {
    type: 'json_schema',
    json_schema: { name: 'user', strict: true, schema }
  }
});


const data = JSON.parse(response.output_text);


if (!validate(data)) {
  console.error('Schema validation failed:', validate.errors);
}

Advanced Patterns

Section titled “Advanced Patterns”

Nested Objects

Section titled “Nested Objects”

Complex schemas with nested structures:

{
  "type": "object",
  "properties": {
    "user": {
      "type": "object",
      "properties": {
        "name": { "type": "string" },
        "contact": {
          "type": "object",
          "properties": {
            "email": { "type": "string" },
            "phone": { "type": "string" }
          },
          "required": ["email"]
        }
      },
      "required": ["name", "contact"]
    }
  },
  "required": ["user"]
}

Arrays of Objects

Section titled “Arrays of Objects”

Lists with consistent structure:

{
  "type": "object",
  "properties": {
    "products": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "id": { "type": "string" },
          "name": { "type": "string" },
          "price": { "type": "number", "minimum": 0 }
        },
        "required": ["id", "name", "price"]
      },
      "minItems": 1
    }
  },
  "required": ["products"]
}

Conditional Fields

Section titled “Conditional Fields”

Use oneOf for alternative structures:

{
  "type": "object",
  "oneOf": [
    {
      "properties": {
        "type": { "const": "user" },
        "username": { "type": "string" }
      },
      "required": ["type", "username"]
    },
    {
      "properties": {
        "type": { "const": "system" },
        "process_id": { "type": "integer" }
      },
      "required": ["type", "process_id"]
    }
  ]
}

Troubleshooting

Section titled “Troubleshooting”

Model Not Following Schema

Section titled “Model Not Following Schema”

Problem: Model output doesn’t match your schema

Solutions:

  1. Use strict: true in json_schema (OpenAI only)
  2. Add clear descriptions to each field
  3. Simplify complex schemas
  4. Include schema structure hints in your prompt

Parsing Errors with Streaming

Section titled “Parsing Errors with Streaming”

Problem: JSON.parse() fails with streamed responses

Solution: Accumulate full response before parsing:

let fullText = '';
for await (const chunk of stream) {
  if (chunk.event === 'content.delta') {
    fullText += chunk.delta.text;
  }
}
const data = JSON.parse(fullText);  // Parse after stream completes

Provider Doesn’t Support Format

Section titled “Provider Doesn’t Support Format”

Problem: Error when using json_schema with unsupported provider

Solutions:

  1. Use OpenAI or DeepSeek for JSON support
  2. Manually specify provider: model: 'openai:gpt-4o'
  3. Fall back to json_object or text format
  4. Use routing rules to force compatible providers

Inconsistent Output Structure

Section titled “Inconsistent Output Structure”

Problem: JSON structure varies between requests

Solutions:

  1. Use json_schema instead of json_object
  2. Add more explicit structure requirements in prompt
  3. Set strict: true for enforcement
  4. Validate output and retry if needed

Best Practices

Section titled “Best Practices”
  1. Start Simple - Begin with json_object before moving to json_schema
  2. Test Thoroughly - Validate schema compatibility with your models
  3. Handle Errors - Always wrap JSON.parse() in try-catch
  4. Use Descriptions - Add descriptions to help models understand intent
  5. Validate Output - Don’t assume strict compliance, validate responses
  6. Consider Tokens - Complex schemas increase token usage
  7. Stream Carefully - Accumulate full response before parsing JSON
  8. Monitor Costs - Structured outputs may use more tokens

Next Steps

Section titled “Next Steps”