Cost-Aware Model Routing Gateway for LLM Applications
LLMRouter
A routing gateway that scores query complexity with local embeddings and dispatches each request to the cheapest capable model in a tiered pool, escalating only ambiguous queries to an LLM-based judge so classification stays off the hot path. Hardened for production with per-request budget ceilings, automatic failover on provider timeouts and rate limits, and structured logging of every routing decision.
- Python
- LLM Routing
- Embeddings
- FAISS
- Evaluation Harness
- Cost Benchmarking
- Structured Logging
- Failover

