Google’s AI-powered bug hunter, Big Sleep, developed by DeepMind and Project Zero, has identified 20 security vulnerabilities in open-source software, marking its initial public reporting of flaws.
Today as part of our commitment to transparency in this space, we are proud to announce that we have reported the first 20 vulnerabilities discovered using our AI-based “Big Sleep” system powered by Gemini — https://t.co/0sgPlazqaq
— Heather Adkins – Ꜻ – Spes consilium non est (@argvee) August 4, 2025
Heather Adkins, Google’s vice president of security, disclosed on Monday that Big Sleep, an LLM-based vulnerability researcher, detected these flaws across various popular open-source software applications. The identified vulnerabilities primarily impact systems such as FFmpeg, an audio and video library, and ImageMagick, an image-editing suite. Details regarding the specific impact or severity of these vulnerabilities have not been released due to Google’s standard protocol of withholding information until fixes are implemented.
Kimberly Samra, a Google spokesperson, clarified the process involved in these discoveries. Samra stated, “To ensure high quality and actionable reports, we have a human expert in the loop before reporting, but each vulnerability was found and reproduced by the AI agent without human intervention.” This procedure confirms that while human verification occurs, the initial detection and reproduction of the vulnerability are performed autonomously by the AI agent.
Royal Hansen, Google’s vice president of engineering, characterized these findings as demonstrating “a new frontier in automated vulnerability discovery” in a post on X. Big Sleep is not the only LLM-powered tool designed for vulnerability detection; other notable examples include RunSybil and XBOW.
XBOW has gained attention for reaching the top position on a U.S. leaderboard within the bug bounty platform HackerOne. Similar to Big Sleep, many of these AI-powered bug hunters incorporate a human verification step to confirm the legitimacy of detected vulnerabilities. Vlad Ionescu, co-founder and chief technology officer at RunSybil, a startup specializing in AI-powered bug hunters, described Big Sleep as a “legit” project. He attributed this assessment to its “good design, people behind it know what they’re doing, Project Zero has the bug finding experience and DeepMind has the firepower and tokens to throw at it.”
While the potential of these AI tools for identifying security flaws is evident, there are also noted drawbacks. Several maintainers of various software projects have reported receiving bug reports that are later identified as hallucinations, drawing parallels to “AI slop” in the context of bug bounties. Ionescu previously commented on this issue, stating, “That’s the problem people are running into, is we’re getting a lot of stuff that looks like gold, but it’s actually just crap.”
