Routing Rules - Custom AI Model Routing Logic

Overview

Routing rules allow you to create deterministic routing logic based on request properties.

Creating Rules

1. Go to Console → Routing Rules
2. Click Create Rule
3. Define conditions and actions
4. Save and activate

Rule Types

Content-Based Routing

Route based on input content:

IF input contains "code"
THEN route to provider: anthropic, model: claude-3-7-sonnet-latest

Metadata-Based Routing

Route based on request metadata:

IF metadata.task = "summarize"
THEN route to mode: cost

Model-Specific Routing

Force specific models for certain requests:

IF input contains "legal" OR input contains "compliance"
THEN route to provider: openai, model: gpt-4o

Rule Priority

Rules are evaluated in order:

1. Highest priority first (descending order)
2. For rules with the same priority, older rules (lower ID) are evaluated first
3. Once a rule matches and applies an action (e.g., set_provider), subsequent rules cannot override that specific action
4. Multiple rules can match and apply different non-conflicting actions
5. If no rules match, use default routing mode

Best Practices

• Start with broad rules, refine over time
• Use descriptive rule names
• Test rules before activating
• Monitor rule performance in analytics

Examples

Route Code Questions to Anthropic

Name: Code Questions → Claude
Condition: input contains "code" OR "function" OR "class"
Action: Route to Anthropic Claude 3.7 Sonnet
Priority: High

Cost Optimization for Summaries

Name: Summaries → Cost Mode
Condition: metadata.task = "summarize"
Action: Route to mode: cost
Priority: Medium

Compliance Requirements

Name: Legal/Compliance → GPT-4
Condition: input contains "legal" OR "gdpr" OR "compliance"
Action: Route to OpenAI GPT-4o
Priority: High

Next Steps

• Firewall
• Auto Routing