AI Coding Agent Security Benchmark | Endor Labs
AI Coding Agent Security Benchmark
The Agent Security League is an open benchmark tracking the functional and security correctness of popular AI coding agents across real-world tasks.
87.2% highest functional correctness score (Claude Code with Claude Fable 5.1)
32 agent + model combos independently benchmarked
37.4% highest security correctness score (Claude Code with Claude Fable 5.1)
Agent Security League
Updated as new commercial agents and models ship. Sorted by security correctness.
Harness:
AllClaude CodeCodexCursorSWE-Agent
Model:
All ModelsClaude Fable 5.1GPT-6 AstraGPT-6.1 SolClaude Opus 5.5Claude Opus 5Claude Fable 5GPT-6 SolGPT-5.5GPT 5.6 SolGPT-5.4Claude Opus 4.8Claude Sonnet 5Claude Opus 4.7Gemini 3.5 FlashGemini 3.1 ProGPT-5.3Composer 2.5Claude Sonnet 4.6GLM 5.2Claude Opus 4.6Gemini 3 ProClaude Opus 4.5Claude Sonnet 4Gemini 2.5 Pro
| Harness | Model | Functional (%)* | Secure (%)† | Date |
|---|---|---|---|---|
| 01 | Claude Code | 87.2 | 37.4 | 2026-09-04 |
| 02 | Codex | GPT-6 Astra | 82.1 | 34.6 |
| 03 | Codex | GPT-6.1 Sol | 77.7 | 34.1 |
| 04 | Claude Code | Claude Opus 5.5 | 68.7 | 33.5 |
| 05 | Claude Code | Claude Opus 5 | 73.7 | 32.4 |
| 06 | Cursor | Claude Fable 5 | 72.6 | 29.0 |
| 07 | Codex | GPT-6 Sol | 72.1 | 25.1 |
| 08 | Cursor | GPT-5.5 | 84.9 | 24.0 |
| 09 | Codex | GPT 5.6 Sol | 70.9 | 23.5 |
| 10 | Codex | GPT-5.5 | 62.6 | 22.4 |
| 11 | Codex | GPT-5.4 | 63.1 | 21.8 |
| 12 | Cursor | Claude Opus 4.8 | 75.4 | 20.7 |
| 13 | Claude Code | Claude Sonnet 5 | 83.2 | 19.6 |
| 14 | Claude Code | Claude Fable 5 | 59.8 | 19.0 |
| 15 | Cursor | Claude Opus 4.7 | 79.9 | 18.4 |
| 16 | Cursor | Gemini 3.5 Flash | 79.3 | 17.9 |
| 17 | Claude Code | Claude Opus 4.7 | 76.5 | 16.8 |
| 18 | Cursor | Gemini 3.1 Pro | 73.2 | 15.1 |
| 19 | Claude Code | Claude Opus 4.8 | 73.7 | 14.5 |
| 20 | Cursor | GPT-5.3 | 48.0 | 14.5 |
| 21 | Cursor | Composer 2.5 | 75.4 | 14.0 |
| 22 | Claude Code | Claude Sonnet 4.6 | 69.8 | 12.3 |
| 23 | Cursor | GLM 5.2 | 82.5 | 12.0 |
| 24 | Cursor | Claude Opus 4.6 | 84.9 | 11.2 |
| 25 | Claude Code | Claude Opus 4.6 | 78.2 | 10.6 |
| 26 | Claude Code | Gemini 3 Pro | 44.1 | 9.5 |
| 27 | Claude Code | Claude Opus 4.5 | 68.7 | 8.4 |
| 28 | Cursor | Gemini 3 Pro | 27.9 | 5.6 |
| 29 | Claude Code | Claude Sonnet 4 | 45.3 | 6.1 |
| 30 | Claude Code | Gemini 2.5 Pro | 19.6 | 5.0 |
| 31 | SWE-Agent | Claude Sonnet 4 | 55.3 | 7.8 |
| 32 | SWE-Agent | Gemini 2.5 Pro | 20.7 | 4.5 |
*Functional (FuncPass): The generated code passes the task's functional test suite — it works.
†Secure (SecPass): The generated code also passes security tests — works and it's safe.
Methodology
Peer-reviewed research
The Agent Security League extends SusVibes, a foundational benchmark developed at Carnegie Mellon University.
Real-world tasks
The benchmark consists of 200 tasks drawn from 108 open-source Python projects spanning 77 CWE vulnerability classes.
Anti-cheating safeguards
Our evaluation pipeline includes prompt hardening, workspace sanitization, and automated cheating detection.
Resources
.webp)
Research
How AI coding agents cheat benchmarks
Read More](/content/learn/recall-not-reasoning-how-ai-coding-agents-cheat-security-benchmarks/index.html)
.webp)
Whitepaper
Evaluating the Security of AI Coding Agents
Read More](/content/learn/agent-security-league-evaluating-the-security-of-ai-coded-software/index.html)
Blog Post
Introducing the Agent Code Security Leaderboard
Read More](/content/learn/is-ai-coding-safe-introducing-the-agent-security-league/index.html)
Security in every line, so you can code without compromise.
AI coding agents can write code, but they lack security context. AURI is the security harness your coding agent is missing.
Always free for developers.