Vitalik Buterin – "My self-sovereign / local / private / secure LLM setup"
13 hours ago
- #self-sovereign AI
- #local LLM
- #privacy security
- Author explores a self-sovereign, local, private, and secure LLM setup as of April 2026.
- Highlights privacy and security concerns, including data leakage, jailbreaks, accidents, and backdoors.
- Describes hardware tested (NVIDIA 5090, AMD Ryzen AI Max Pro, DGX Spark), with NVIDIA preferred for performance.
- Uses Qwen3.5:35B model via llama-server for local inference, citing issues with ollama.
- Employs pi as an agent tool, with skills for tasks like using SearXNG and accessing a messaging daemon.
- Implements sandboxing with bubblewrap to restrict LLM access to files and network ports.
- Details a messaging daemon with strict firewalling, requiring human confirmation for external communications.
- Advocates for similar security measures when connecting LLMs to Ethereum wallets.
- Discusses limitations of local AI and proposes multi-layer defense for remote AI usage, including ZK APIs, mixnets, and TEEs.
- Envisions a future where AI enhances privacy through local-first approaches and user empowerment.