Rubberduck: Emulate OpenAI/Anthropic locally with caching and failure injection
11 days ago
- #LLM
- #Development
- #Proxy
- Rubberduck is a local LLM caching reverse proxy server that emulates major LLM providers.
- Supports OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, and Google Vertex AI.
- Features include caching, failure simulation, rate limiting, per-user proxy instances, and detailed logging.
- Perfect request/response compatibility with official SDKs and transparent header/authentication passthrough.
- SHA-256 cache keys based on normalized request bodies; only successful responses (2xx) are cached.
- Manual cache invalidation per proxy instance and respects upstream provider caching headers.
- Failure simulation includes timeouts, error injection (429, 500, 400), IP filtering, and rate limiting.
- Real-time request logging with metadata, exportable logs (CSV/JSON), and rolling metrics aggregation.
- Dashboard for live system stats, proxy monitoring, and cost tracking with token usage.
- Stripe-inspired UI with clean, modern, responsive design and JWT-based authentication.
- Supports email/password + social login (Google/GitHub), email verification, and password reset.
- Backend built with FastAPI (Python 3.11+), frontend with React + TypeScript (Node.js 18+).
- Easy setup with git clone, virtual environment, and npm install for frontend dependencies.
- Proxy management via web interface or SDK configuration with base URL changes.
- Testing includes unit, integration, and coverage tests for both backend and frontend.
- Development tools include Black, isort, mypy for Python, and linting for TypeScript.
- Open-source under MIT License with contributions welcome via fork, feature branch, and PR.