Ananya Joshi, Michael Rudow
February 2, 2026
The paper introduces a multi-agent framework for running complex, regulated workflows (like compliance review) using LLM-based agents, formalized as a bounded-horizon Markov Decision Process (MDP) constrained by a directed acyclic graph (DAG). Instead of relying on a single LLM agent with a long prompt to handle an entire compliance process, the system maps each step of an existing Standard Operating Procedure (SOP) to a specialized agent node — for example, a content reviewer, a triage router, a risk assessor, and a legal compliance checker. Uncertain cases are escalated along predefined paths, just like they would be in a real compliance team, and the system uses Monte Carlo sampling to quantify each agent’s confidence without needing access to the LLM’s internal probabilities.
Tested on NVIDIA’s AEGIS 2.0 AI Safety Benchmark for self-harm detection, the multi-agent framework achieved up to a 19% accuracy improvement over a single-agent baseline (88% vs. 70%), reduced the number of cases requiring human review by up to 85x (from ~17 to ~0.2 per run), and in the fastest configuration ran 30% quicker. The framework also caught annotation errors in the benchmark dataset itself, demonstrating its practical value for auditable, high-stakes AI deployments.
Markov Decision Process (MDP) is a mathematical framework for modeling sequential decision-making where outcomes are partly random and partly under the control of a decision-maker. An MDP consists of: states (where you are), actions (what you can do), transition probabilities (how likely each next state is given current state and action), and rewards (feedback on how good each action was). In the paper, each state is an agent in the compliance workflow, actions are classification labels (safe/unsafe/uncertain), and transitions follow the escalation paths in the process map. The “bounded-horizon” part means every case must resolve within a fixed maximum number of steps.
Directed Acyclic Graph (DAG) is a graph structure where edges have direction (A -> B means A comes before B) and there are no cycles (you can never loop back to a previous node). In the paper, the DAG represents the compliance process map: Worker -> Triage -> Risk/Legal -> Labeled Data or Human Review. The DAG constraint guarantees that every case terminates — no infinite loops are possible, which is a known problem in some multi-agent LLM architectures like LangChain.
AEGIS 2.0 AI Safety Benchmark is a dataset released by NVIDIA (August 2025) for evaluating AI safety guardrails. The subset used in the paper covers suicide and self-harm topics, with 112 examples (68 safe, 44 unsafe) derived from the Suicide Detection corpus. Each example consists of a user prompt and an LLM response that needs to be classified as safe or unsafe.