Aligning Language Models for Lyric-to-Melody Generation with Rule-Based Musical Constraints
Hao Meng, Siyuan Zheng, Shuran Zhou, Qiangqiang Wang, Yang Song
TLDR
This paper introduces a novel alignment framework using rule-based musical constraints and DPO/KTO to improve LLM-generated melodies, reducing musical implausibility.
Key contributions
- Addresses "constraint violation" in LLM-generated melodies, where SFT models produce musically implausible outputs.
- Proposes a novel alignment framework that instills musical knowledge using rule-based musical constraints.
- Employs a sequential DPO and KTO optimization process on automatically generated preference data.
- Substantially reduces rule violations and improves musicality and coherence in generated melodies.
Why it matters
This paper significantly advances lyric-to-melody generation by tackling the critical issue of musical implausibility in LLM outputs. It introduces an innovative, annotation-free alignment method, making LLMs more practical for creative music generation and enhancing their real-world applicability.
Original Abstract
Large Language Models (LLMs) show promise in lyric-to-melody generation, but models trained with Supervised Fine-Tuning (SFT) often produce musically implausible melodies with issues like poor rhythm and unsuitable vocal ranges, a phenomenon we term "constraint violation". To address this, we propose a novel alignment framework that instills musical knowledge without human annotation. We define rule-based musical constraints to automatically generate a preference dataset from an SFT model's outputs. The model is then aligned through a sequential process, first using Direct Preference Optimization (DPO) on paired preference data, followed by Kahneman-Tversky Optimization (KTO) on unpaired negative samples. Experimental results demonstrate that our aligned model substantially reduces rule violations and outperforms strong baselines in both objective and subjective evaluations, generating melodies with substantially improved musicality and coherence. An interactive demo with audio comparisons is available at https://arain233.github.io/AligningMelody-demo.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.