What GPT-OSS leaks about OpenAI's training data
15 hours ago
- #glitch-tokens
- #OpenAI
- #GPT-5
- OpenAI released an open-weights model, revealing insights into their training data.
- GPT-5 was found to have been trained on phrases from adult websites, identified through glitch tokens.
- The o200k tokenizer used in GPT-5 includes unusual and junk tokens, some with political and adult content.
- Membership inference techniques showed that certain sensitive strings were part of GPT-5's training corpus.
- Evidence suggests parts of the training data may have been scraped from GitHub, indicated by correlation with search hits.
- Glitch tokens can be used to probe model behavior, revealing unexpected responses and hallucinations.
- Recommendations include excluding uncommon strings from tokenizer vocabularies to mitigate privacy and content issues.