UFAL-CUNI at SemEval-2026 Task 11: An Efficient Modular Neuro-symbolic Method for Syllogistic Reasoning
Ivan Kartáč, Kristýna Onderková, Jan Bronec, Zdeněk Kasner, Mateusz Lango + 1 more
TLDR
This paper presents an efficient modular neuro-symbolic system for syllogistic reasoning, combining small LLMs with a symbolic prover.
Key contributions
- Introduces an efficient modular neuro-symbolic system for syllogistic reasoning, using small 4B-parameter LLMs.
- Integrates an LLM-based parser for natural language to FOL translation with an automated theorem prover.
- Achieves competitive accuracy and low content effect, outperforming zero-shot LLM baselines in its size range.
Why it matters
This paper offers a practical approach to formal reasoning with smaller LLMs, demonstrating that neuro-symbolic methods can achieve strong performance without massive models. It highlights the potential of combining symbolic logic with neural networks for precise logical inference, outperforming zero-shot baselines.
Original Abstract
This paper describes our system submitted to SemEval-2026 Task 11: Disentangling Content and Formal Reasoning in Large Language Models. We present an efficient modular neuro-symbolic approach, combining a symbolic prover with small reasoning LLMs (4B parameters). The system consists of an LLM-based parser that translates natural language syllogisms to a first-order logic (FOL) representation, an automated theorem prover, and two optional modules: machine translation for multilingual inputs and a symbolic retrieval component for the identification of relevant premises. The system achieves competitive accuracy and relatively low content effect on most subtasks. Our ablations show that this approach outperforms LLM-based zero-shot baselines in this parameter size range, but also reveal limited multilingual capabilities of small LLMs. Finally, we include a discussion of the task's main ranking metric and analyze its limitations.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.