Otari: Own Your AI Stack
5 hours ago
- #LLM Gateway
- #AI Platform
- #Open Source
- Otari is an open-source LLM gateway that works with any model, closing the gap between frontier and open-weights models by providing features like usage tracking, budget controls, and tools.
- Otari.ai is the hosted platform built on the same open-source foundation, offering managed services, team management, and observability without compromising user privacy.
- The gateway includes server-side, model-agnostic tools such as sandboxed code execution, web search, audio transcription, image generation, reranking, and batch processing to enhance open-weights models.
- Operational features include virtual API keys, user management with budgets, real-time usage tracking, rate limiting, health metrics, and platform mode for multi-tenancy.
- Otari.ai provides identity and teams, routing policies, a secure vault, managed providers including Mozilla.ai for open-weights models, multi-level budgets, declarative YAML configuration, and OpenTelemetry-based observability.
- The design prioritizes open core, with Otari.ai as a business layer on top, ensuring privacy by not accessing user prompts, completions, or traces, and treating open-weights models as first-class citizens.