glide

Glide is an open simple blazing-fast model gateway for rapid development of resilient GenAI applications across multiple modalities (text, speech, vision).

The current features:
- The routing functionality with four types of routing strategies (including a tricky one like the least latency routing)
- The first-class adaptive resiliency & fallbacking across all routing strategies
- Unified Chat API that supports popular model providers like OpenAI, Azure OpenAI (on-prem models), Cohere, OctoML, Anthropic
- The ability to have model-specific prompts
- Installation via Docker & Snapcraft & Homebrew

Roadmap: https://github.com/EinStack/glide/blob/develop/ROADMAP.md