Beyond Semantics: Measuring Fine-Grained Emotion Preservation in Small Language Model-Based Machine Translation
TLDR
This paper assesses how Small Language Models (SLMs) preserve fine-grained emotions in machine translation across five European languages.
Key contributions
- Evaluates EuroLLM, Aya Expanse, and Gemma for fine-grained emotion preservation in MT.
- Assesses emotion retention across 5 European languages using the GoEmotions dataset.
- Examines the impact of emotion-aware prompting on SLM translation fidelity.
- Compares ModernBERT and BERT for emotion classification in MT evaluation.
Why it matters
Current MT often prioritizes semantics over emotional nuance. This paper addresses that gap by rigorously evaluating SLMs for fine-grained emotion preservation. Its findings can guide the development of more emotionally intelligent translation systems.
Original Abstract
Preserving affective nuance remains a challenge in Machine Translation (MT), where semantic equivalence often takes precedence over emotional fidelity. This paper evaluates the performance of three state-of-the-art Small Language Models (SLMs) -- EuroLLM, Aya Expanse, and Gemma -- in maintaining fine-grained emotions during backtranslation. Using the GoEmotions dataset, which comprises Reddit comments across 28 distinct categories, we assess emotional preservation across five European languages: German, French, Spanish, Italian, and Polish. Specifically, we investigate (i) the inherent capability of these SLMs to retain emotional sentiment, (ii) the efficacy of emotion-aware prompting in improving preservation, and (iii) the performance of ModernBERT as a contemporary alternative to BERT for emotion classification in MT evaluation.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.