The Rise of AI Agents in the SOC

The Cloud Security Alliance has published a report, “Beyond the Hype: A Benchmark Study of AI Agents in the SOC.” The study evaluated how AI-assisted SOC analysts performed compared to traditional, manual investigation teams.

Key Findings

The research involved 148 participants with varying experience levels in SOC and incident response, divided into two groups:

➡️ AI-Assisted analysts using Dropzone AI

➡️ Manual analysts using standard tools (AWS GuardDuty & Microsoft Sentinel)

Photo: https://pl.freepik.com/

Each participant handled two simulated security incidents:

➡️ An AWS S3 bucket compromise attempt

➡️ A Microsoft Entra failed login brute-force scenario

The results show that AI-assisted are:

✅ 23.9% more accurate in identifying correct actions than manual analysts

✅ 45–61% faster in completing investigations across scenarios

✅ Greater consistency & less fatigue — fewer declines in completeness and report detail across tasks

✅ 94% of analysts reported a more positive view of AI after hands-on experience

Speed of Investigation:

➡️ AI-assisted analysts completed tasks in 30–58 minutes, compared to 1–1.5 hours manually — up to 61% faster.

➡️ AssistedAI agents helped analysts make more correct determinations and maintain thoroughness.

➡️ Manual analysts’ performance dropped by 29%, while AI-assisted teams declined only 16%, showing resilience to cognitive fatigue.

➡️ Human-only reports lost 20–27% of their length and detail by the second task, while AI-assisted analysts maintained or improved their documentation depth.

➡️ 94% said the AI tools sped up their work, and 100% described the AI SOC agent as efficient.

I’ve always been a strong advocate for automation in cybersecurity, but always with caution — especially ensuring there’s still a human in the loop.

What’s particularly interesting in this report is that AI agents can be a practical aid to humans. Humans were still slightly faster in average investigation time, but AI matched or exceeded them in action-needed accuracy, scenario handling, and documentation completeness. This shows a massive opportunity for SOC modernization. In an environment where we face:

➡️ Growing attack volumes,

➡️ Limited cybersecurity workforce,

➡️ And escalating complexity of systems,

➡️ AI-driven SOC assistants could become indispensable.

➡️ The study also proves that AI can enhance morale and perception among analysts — 94% of participants felt more positive toward AI after real-world testing.

Link

Author: Sebastian Burgemejster