PLMGH: What Matters in PLM-GNN Hybrids for Code Classification and Vulnerability Detection
Mohamed Taoufik Kaouthar El Idrissi, Edward Zulkoski, Mohammad Hamdaqa
TLDR
This paper empirically studies PLM-GNN hybrids for code classification and vulnerability detection, finding PLM choice is more critical than GNN.
Key contributions
- Empirically compares PLM-GNN hybrids against PLM-only and GNN-only baselines for code tasks.
- Hybrids consistently outperform GNN-only models and often improve ranking over frozen PLMs.
- PLM choice and feature source are more impactful than GNN backbone or larger PLM size.
- Distills findings into practical guidelines for designing effective PLM-GNN hybrids for code understanding.
Why it matters
This paper provides crucial insights into designing effective PLM-GNN hybrids for code analysis. Its findings challenge assumptions about larger PLMs and highlight the importance of PLM feature choice over GNN architecture, offering practical guidance for researchers and practitioners.
Original Abstract
Code understanding models increasingly rely on pretrained language models (PLMs) and graph neural networks (GNNs), which capture complementary semantic and structural information. We conduct a controlled empirical study of PLM-GNN hybrids for code classification and vulnerability detection tasks by systematically pairing three code-specialized PLMs with three foundational GNN architectures. We compare these hybrids against PLM-only and GNN-only baselines on Java250 and Devign, including an identifier-obfuscation setting. Across both tasks, hybrids consistently outperform GNN-only baselines and often improve ranking quality over frozen PLMs. On Devign, performance and robustness are more sensitive to the PLM feature source than to the GNN backbone. We also find that larger PLMs are not necessarily better feature extractors in this pipeline, and that the PLM choice has more impact than the GNN choice. Finally, we distill these findings into practical guidelines for PLM-GNN design choices in code classification and vulnerability detection.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.