Why This Matters

Production systems can't depend on one model. Multi-model fallback ensures availability even when one model has issues.

Concept Explained

Production fallback: health checks per model, fallback matrix (if A down, use B), traffic shifting (gradually move users).

How It Works

Health check each model periodically. If model fails health check, mark unavailable. Route to fallback. When recovered, gradually shift traffic back.

Step-by-Step Tutorial

1. Health checks

2. Fallback matrix

3. Traffic shifting

4. Alerting

5. Measure

Real-World Example

A team's production Copilot integration had Claude as primary, GPT-5 as fallback. Claude had an outage; system automatically fell back to GPT-5. Users barely noticed. After Claude recovered, traffic shifted back over 1 hour.

Example Prompts / Commands / Code

"""[User request]
       |
       v
[Router]
   |           |
   v           v
[Model A]   [Model B]
(Claude)    (GPT-5)
   |           |
   v           v
[Health Check] [Health Check]
   |           |
   +-----+-----+
         |
         v
   [Status: A=UP, B=UP]
         |
         v
   [Route to A]
   
If A fails health check:
   [Status: A=DOWN, B=UP]
   [Route to B]
   [Alert: A is down, fallback to B]
   
When A recovers:
   [Status: A=RECOVERING, B=UP]
   [Shift 10% traffic to A]
   [Monitor]
   [Shift 50%]
   [Monitor]
   [Shift 100%]
"""

"""import asyncio

async def health_check_model(model: str) -> bool:
    try:
        response = await copilot.run('health check', model=model, timeout=5)
        return True
    except:
        return False

async def monitor_health():
    while True:
        for model in ['claude', 'gpt-5', 'gemini']:
            healthy = await health_check_model(model)
            if not healthy and model_status[model] == 'up':
                model_status[model] = 'down'
                alert(f'{model} is DOWN')
                activate_fallback(model)
            elif healthy and model_status[model] == 'down':
                model_status[model] = 'recovering'
                shift_traffic_gradually(model)
        await asyncio.sleep(60)  # check every minute
"""

Common Mistakes

• No health checks — silent failures.
• No fallback matrix — single model failure breaks system.
• Traffic shifting too fast — overwhelm recovering model.
• No alerting — fallbacks active without team knowing.

Best Practices

• Health check each model every minute.
• Define fallback matrix per task type.
• Shift traffic gradually on recovery (10% → 50% → 100%).
• Alert when model down or fallback active.
• Measure: uptime, fallback frequency, user impact.

Troubleshooting

Practical Exercise

Professional Challenge

Frequently Asked Questions

Further Reading

• Health check patterns

Official References

• Circuit breaker

Related lessons: PR-05, PR-45