Show HN: Gerbil – an open source desktop app for running LLMs locally
12 days ago
- #Local AI
- #Desktop App
- #LLM
- Gerbil is a desktop app for running Large Language Models (LLMs) locally using KoboldCpp.
- It is cross-platform with native support for Windows, macOS, and Linux (including Wayland).
- Features include automatic updates, smart process management, and optimized performance.
- Supports image generation with built-in presets for Flux and Chroma workflows.
- Integrates with SillyTavern for character interactions and OpenWebUI for a web-based chat interface.
- Privacy-focused, with all operations running locally on the user's machine.
- Available for download on GitHub with specific executables for each platform.
- Arch Linux users can install via AUR with commands like `yay -S gerbil` or manual installation.
- CLI mode allows running KoboldCpp with custom arguments, ideal for custom frontends.
- Windows CLI mode requires installation via Setup.exe for proper functionality.
- Includes support for ROCm on Linux, though Windows ROCm support is problematic.
- Licensed under AGPL v3.