Cloning your Discord friends with Large Language Models
2 days ago
- The project uses an RTX 3090 to train a Mistral 7b model on Discord chat data, achieving humorous but not practically useful results.
- Data was cleaned by filtering out low-quality messages, single pings, non-responses, and conversation-enders, then formatted into JSONL using the sharegpt format for Axolotl training.
- Training took 6 hours with 10 epochs, producing a LoRA that was imported into oobabooga to create a Discord bot mimicking friends' conversational styles.