Top AI Papers This Week

Pseudo2Code is a new benchmark and agentic framework demonstrating that structured pseudocode significantly improves LLM code generation and algorithmic reasoning.

• Introduces Pseudo2Code, a benchmark with 300 validated programming tasks to evaluate LLM structured algorithmic reasoning.
• Proposes Pseudo2Code Agentic Framework, a pipeline leveraging pseudocode for improved code generation.
• Shows the Agentic Pipeline outperforms baselines, achieving higher scores in correctness and algorithmic faithfulness.

ToolHazard is a scalable framework for synthesizing adversarial environments to evaluate and align LLM agents against indirect prompt injections.

• Proposes ToolHazard, a scalable framework for synthesizing adversarial environments.
• Synthesizes executable stateful environments, finds injection points, and generates payloads.
• Constructs ToolHazard-Bench for stress-testing LLM agents under complex attacks.

This paper uncovers mechanistic safety vulnerabilities in Diffusion LLMs, showing inherited safety footprints and introducing a low-cost, highly effective black-box jailbreak method.

• Reveals DLLMs inherit safety mechanisms from autoregressive predecessors, enabling transfer attacks.
• Demonstrates self-pruning and transfer pruning significantly increase attack success rates on DLLMs.
• Introduces SN-Guided Diffusion, an offline black-box jailbreak framework with near-perfect prompt separability.

ActBench is a self-evolving benchmark that evaluates behavioral safety risks in cowork agents by analyzing their execution trajectories.

• Introduces ActBench, a self-evolving benchmark for behavioral safety in cowork agents.
• Features 600 cases across 213 scenarios, covering 15 risk behaviors and 48 web APIs.
• Proposes a reward-guided beam search for dynamic payload generation and revision.

V-FiLLM is a new benchmark for evaluating LLM financial reasoning over structured data, generating verified questions at arbitrary scale.

• Introduces V-FiLLM, a framework generating financial reasoning benchmarks from real tables.
• Ensures correct-by-construction answers by evaluating computation trees symbolically.
• Provides four controllable difficulty axes: depth, breadth, concept complexity, and context size.

PRMU introduces a corpus-free benchmark and a new baseline (SGPE) for evaluating person-centric knowledge unlearning in Multimodal Large Language Models.

• Introduces PRMU, a novel corpus-free benchmark for person-centric knowledge unlearning in MLLMs.
• Evaluates unlearning by removing target knowledge while preserving related knowledge using diverse textual/visual probes.
• Proposes SGPE, a lightweight corpus-free unlearning baseline with knowledge displacement and locality control.

This paper introduces CoDyControlBench to evaluate LLMs for complex feedback controller design and develops a reasoning-distilled model for edge deployment.

• Introduced CoDyControlBench, a new benchmark for LLMs in complex feedback controller design across 132 configurations.
• Evaluated 6 LLMs, with GPT achieving 94.8% success and Qwen 50%, highlighting performance gaps.
• Developed a 1.5B-parameter reasoning-distilled model for edge deployment, outperforming other distilled models.

SWE-RPG is a new benchmark for coding agents, evaluating requirement clarification and planning to diagnose performance and identify implicit requirement recovery as a bottleneck.

• Introduces SWE-RPG, a repository-level benchmark for coding agents, evaluating beyond final patch success.
• Includes ground-truth references for Requirement Clarification and Implementation Planning for diagnostic analysis.
• Comprises 163 tasks from 31 Python/Java repositories, including bug fixes and feature additions.

SHE is a framework that evolves LLM agent safety harnesses by learning from trajectories, decomposing components, and refining boundaries to reduce risks.

• Proposes SHE, a framework for evolving LLM agent safety harnesses from rollout trajectories.
• Decomposes the harness into System Prompt, Rule Bank, Safety Memory, and Tool Policy for localized evolution.
• Uses an attribution-guided loop to refine artifact-specific safety boundaries based on observed failures.

This paper explores using security tests as executable specifications to improve LLM-generated code security, finding benefits depend heavily on test coverage.

• Introduces SecTDD, a scaffold for evaluating security tests as executable specifications for LLM code generation.
• Showing all visible security tests upfront boosts hidden security success by 19.3% on average, though benefits vary.
• Structured feedback effectively repairs 80 insecure code candidates without regressions, slightly better than raw feedback.