I wanted to run my own AI. My laptop says not yet
7 hours ago
- User wanted a private, offline AI assistant but faced hardware limitations with 24GB memory insufficient for capable models.
- System overhead reduces usable memory; must choose between weak models or smarter ones that strain the machine.
- Memory and storage prices have surged due to AI datacenter demand, making hardware upgrades costly.
- Tried various local AI software (Pi, OpenCode, Hermes) but the bottleneck remained the model and memory.
- Local models are not trusted for important tasks; user still relies on cloud for critical judgment.
- One successful local use case: analyzing job ads using a mixture-of-experts model (Gemma) quickly and privately.
- Concludes that full local AI is not yet achievable but hopes for future improvements; currently has one local workflow.