MDST Engine: run GGUF models in the browser with WebGPU/WASM
3 months ago
- #LLM
- #WebGPU
- #LocalInference
- MDST introduces GGUF to WebGPU, enabling local LLM inference in browsers without cloud dependencies.
- The GGUF format is popular for its support of various quantizations and ease of use on consumer devices.
- MDST is a free, secure, collaborative IDE with integrated cloud and local agentic inference.
- Features include real-time project syncing, end-to-end encryption, and a public WebGPU leaderboard for benchmarking.
- Supports multiple cloud and local LLM families, with plans to expand.
- WebGPU's performance and GGUF's simplicity make local browser inference practical on modest hardware.
- Encourages community contributions and feedback to guide future optimizations.