We Politely Insist: Your LLM Must Learn the Persian Art of Taarof
9 hours ago
- #Taarof
- #LLMs
- #Cultural Competence
- Large language models (LLMs) struggle with culturally specific communication norms like Persian taarof.
- Taarof is a Persian social norm emphasizing ritual politeness, deference, modesty, and indirectness.
- TaarofBench is introduced as the first benchmark to evaluate LLM understanding of taarof, with 450 role-play scenarios across 12 social topics.
- Evaluation of five frontier LLMs shows significant gaps in cultural competence, with accuracy 40-48% below native speakers.
- Performance varies by interaction topic, improves with Persian-language prompts, and shows gender-based asymmetries.
- Standard politeness metrics often misalign with taarof norms, highlighting limitations of Western frameworks.
- Supervised fine-tuning and Direct Preference Optimization improve model alignment by 21.8% and 42.3%.
- Human study with 33 participants (native, heritage, and non-Iranian speakers) establishes baselines for cultural familiarity.
- This work aims to develop culturally aware LLMs for better global social interaction handling.