Study reveals AI compliance guardrails rely on superficial patterns rather than legal rules

A new auditing study published by researchers on arXiv has revealed a structural flaw in regulatory compliance detection systems across language models, where these tools rely on superficial contextual features rather than reading and analyzing governing legal rules. The research paper, authored by Saisab Sadhu and his research team, showed that guardrails and activation probes used as legal and auditing controls suffer from a systematic failure the researchers term 'rule blindness,' rendering compliance monitoring ineffective when examining outputs in sensitive sectors such as data protection, healthcare, financial regulation, and platform policies.

Altering regulatory text or deleting it entirely does not change guardrail detection accuracyExperiments demonstrated that removing the governing rule, reordering its clauses, or replacing it with another rule left detection accuracy virtually unchanged across all tested guardrails and activation probes. This limitation extended to policy-conditioned guardrails that cite the correct regulatory clause in their report, yet barely changed their final verdict when the restrictive clause was replaced with a regulatory text granting explicit permission in the exact same scenario.

To verify the depth of this limitation, the team designed a custom evaluation benchmark crossing two regulatory provisions with two different scenarios, such that neither alone could predict the correct outcome without understanding the mutual relationship between text and context. Benchmark results confirmed that all fast detectors and activation probes failed this test, whereas step-by-step reasoning proved to be the only approach capable of escaping the superficiality trap and handling the required regulatory logic.

Building a low-cost activation score enables large-scale guardrail auditingTo enable scaled auditing without retraining models, the researchers introduced the Internal Compliance Score (ICS), a training-free mechanism for reading internal activations that relies on calibration from just ten labeled pairs and computes scores via a single arithmetic projection. The team subjected this score to the same rigorous evaluation benchmarks, finding that it failed to meet the preregistered threshold for outperforming simple baselines, as its fully generalized performance matched that of a basic bag-of-words model.

Despite these limitations, the Internal Compliance Score retained practical utility due to its low operational cost, allowing the team to use it in auditing four deployed guardrails, a zero-shot judge model with eight billion parameters, and thirteen evaluation benchmarks. Tests showed that using the score to rank candidate model responses increased the automated verification success rate, though this gain vanished entirely under an adaptive white-box attack. The research team has released the adversarial evaluation protocol and crossed benchmark data to test for rule blindness in future guardrails and activation probes.