Why This Matters

Different models excel at different tasks. Orchestration routes prompts to the right model per task, optimizing cost and quality simultaneously.

Concept Explained

Multi-model orchestration: route prompts to different models based on task type, complexity, and cost. Build a router that picks the best model per request.

How It Works

Build a router: input task type → pick model from config → call Copilot with that model. On failure, fallback to next model in matrix. Track success rate per model per task.

Step-by-Step Tutorial

1. Define task types

2. Map to models

3. Build router

4. Add fallback

5. Track metrics

Real-World Example

A team built a multi-model router: refactors → Claude, tests → GPT-5, quick questions → GPT-5 mini. Cost dropped 30% (cheaper models for routine), quality improved 15% (right model per task).

Example Prompts / Commands / Code

# router.py
TASK_TO_MODEL = {
    'refactor': 'claude',
    'debug': 'claude',
    'tests': 'gpt-5',
    'docs': 'gpt-5',
    'quick': 'gpt-5-mini',
    'long_context': 'gemini',
}

FALLBACK = {
    'claude': 'gpt-5',
    'gpt-5': 'gpt-5-mini',
    'gpt-5-mini': 'gemini',
    'gemini': 'gpt-5',
}

def pick_model(task_type: str) -> str:
    return TASK_TO_MODEL.get(task_type, 'gpt-5')

def route_with_fallback(task_type: str, prompt: str) -> str:
    model = pick_model(task_type)
    while model:
        try:
            return copilot.run(prompt, model=model)
        except CopilotError:
            model = FALLBACK.get(model)
    raise RuntimeError('All models failed')

Track per (model, task_type):
- success rate (returns valid output)
- latency (p50, p95)
- cost (credits used)
- quality (human-rated 0-5)

Adjust routing based on data:
- If Claude has 95% success on refactors, keep.
- If GPT-5 has 60% success on debugs, switch to Claude.

Common Mistakes

• Routing by gut, not data.
• No fallback — single model failure breaks system.
• Not tracking metrics — can't improve routing.
• Routing too fine-grained — complexity without benefit.

Best Practices

• Route by task type; 5–7 categories is enough.
• Build fallback into router.
• Track per-(model, task) metrics.
• Adjust routing based on data.
• Start simple; add complexity as data justifies.

Troubleshooting

Practical Exercise

Professional Challenge

Frequently Asked Questions

Further Reading

• Model comparison

Official References

• Model comparison

Related lessons: IN-23, PR-05