AI Auditor Captures Most Agent Misconduct, But Issues a Large Number of False Positives, New Research Finds
A new study finds that AI auditors perform well in capturing the improper behavior of agents, but the pursuit of accuracy can lead to an increase in false positives. The study compares models such as GPT-5, GPT-6 Astra, and finds that they are effective in capturing reward hacking and sandbagging behaviors, but also have a high false positive rate. The study points out that more expensive and advanced models are not necessarily more effective, and multi-model combinations do not significantly improve the results.
