ArXiv TLDR

The A-R Behavioral Space: Execution-Level Profiling of Tool-Using Language Model Agents in Organizational Deployment

🐦 Tweet
2604.12116

Shasha Yu, Fiona Carroll, Barry L. Bentley

cs.AIcs.SE

TLDR

This paper introduces the A-R behavioral space to profile tool-using LLM agents, analyzing execution and refusal across contexts and autonomy levels.

Key contributions

  • Introduces the A-R behavioral space (Action Rate, Refusal Signal) for execution-level profiling of LLM agents.
  • Evaluates agents across four normative regimes and three autonomy configurations (direct, planning, reflection).
  • Shows execution and refusal are separable dimensions, varying systematically with context and autonomy levels.
  • Demonstrates reflection-based scaffolding can increase refusal in risky situations, with model-specific patterns.

Why it matters

This work offers a novel, deployment-oriented framework for understanding how tool-using LLM agents behave at the execution layer. By characterizing execution and refusal, it moves beyond aggregate safety scores, enabling organizations to make informed decisions when selecting and deploying agents with varying risk tolerances and execution privileges.

Original Abstract

Large language models (LLMs) are increasingly deployed as tool-augmented agents capable of executing system-level operations. While existing benchmarks primarily assess textual alignment or task success, less attention has been paid to the structural relationship between linguistic signaling and executable behavior under varying autonomy scaffolds. This study introduces an execution-layer be-havioral measurement approach based on a two-dimensional A-R space defined by Action Rate (A) and Refusal Signal (R), with Divergence (D) capturing coor-dination between the two. Models are evaluated across four normative regimes (Control, Gray, Dilemma, and Malicious) and three autonomy configurations (di-rect execution, planning, and reflection). Rather than assigning aggregate safety scores, the method characterizes how execution and refusal redistribute across contextual framing and scaffold depth. Empirical results show that execution and refusal constitute separable behavioral dimensions whose joint distribution varies systematically across regimes and autonomy levels. Reflection-based scaffolding often shifts configurations toward higher refusal in risk-laden contexts, but redis-tribution patterns differ structurally across models. The A-R representation makes cross-sectional behavioral profiles, scaffold-induced transitions, and coordination variability directly observable. By foregrounding execution-layer characterization over scalar ranking, this work provides a deployment-oriented lens for analyzing and selecting tool-enabled LLM agents in organizational settings where execution privileges and risk tolerance vary.

📬 Weekly AI Paper Digest

Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.