Overview

This repository provides a high-performance model router designed for agentic systems. It intelligently routes every prompt to the most suitable model, achieving this in under 50 milliseconds. By simply changing your endpoint, you can expect to cut costs by 40-70%.

Key Features

• Action-Based Routing: Utilizes a sophisticated cluster scorer derived from Avengers-Pro to determine the optimal model for each upstream API request, rather than relying on simple turn-based logic.
• Universal API Compatibility: Seamlessly integrates with Anthropic Messages, OpenAI Chat Completions, and Gemini native APIs, supporting streaming, tools, and vision capabilities.
• Open Source Model Support: Empowers you to leverage open-source models like DeepSeek, Kimi, GLM, Qwen, Llama, and Mistral, either directly or through OpenRouter, or any OpenAI-compatible endpoint.
• Enhanced Security: By default, provider API keys remain on your local machine, encrypted at rest, ensuring greater control and security.
• Observability: Out-of-the-box OTLP traces are provided, allowing integration with popular monitoring platforms like Weave dashboard, Honeycomb, Datadog, and Grafana.
• Effortless Deployment: Offers a quickstart via npx @workweave/router for instant setup with popular tools like Claude Code, Codex, opencode, and pi, with options for project-level scoping and custom base URLs.
• Self-Hosting Option: Provides a straightforward make full-setup command to run the router and dashboard locally, with Postgres integration.

Typical Use Cases

• Cost Optimization for LLM Applications: Significantly reduce operational expenses for applications relying on large language models by dynamically selecting the most cost-effective model for each task.
• Building Sophisticated AI Agents: Create intelligent agents that can reliably leverage a diverse range of LLMs for specific functionalities, from complex reasoning to creative text generation.
• Streamlining LLM Development Workflows: Simplify the integration of various LLM providers and models into development pipelines, offering a unified API endpoint for all model interactions.
• Enhancing Real-time AI Services: Ensure low latency and responsiveness in applications requiring rapid model inference, such as chatbots, code assistants, and real-time content generation tools.
• Securing API Key Management: Maintain strict control over sensitive API credentials by keeping them local and encrypted, ideal for enterprise environments and security-conscious developers.