I Built an AI Slop Machine

14 days ago

Self-hosted LLMs offer privacy, cost savings, and no reliance on external companies, with local data and no subscription fees.
The author built an AI Slop Machine using a Radeon AI Pro R9700 GPU in an eGPU dock connected to an old Thinkpad via Thunderbolt 3 for running Qwen 3.6 27B.
Performance with llama.cpp includes prefilling at 507-814 TPS and decoding at 28-33 TPS, supporting large context windows up to 131k tokens.
The system is used as a server accessible remotely via SSH, keeping the client laptop cool and portable, though it requires a stable internet connection.
An AI agent collaborates on tuning and configuration, managing tasks in a shared workspace with ongoing improvements and experimentation.
OpenRouter data privacy issues highlight that some providers may retain and sell user data, prompting the author to use more expensive 'Zero Data Retention' options.

Hasty Briefsbeta