The rapid adoption of artificial intelligence in vulnerability research has created an unexpected crisis: a flood of low-quality security reports that threaten to drown the very maintainers these tools were meant to assist. Overworked open source volunteers and security teams are now spending hours sifting through duplicative, poorly validated submissions, delaying fixes for genuine threats.
Linus Torvalds, creator of the Linux kernel, recently described the situation as making the project's security mailing list "almost entirely unmanageable." In a note accompanying the latest kernel release candidate, Torvalds highlighted the enormous duplication caused by different researchers using the same AI tools to find the same bugs. "If you found a bug using AI tools, the chances are somebody else found it too," he wrote. His advice to researchers: add real value on top of what the AI did, such as creating a patch or understanding the code context, rather than submitting drive-by reports with no real understanding.
A Perfect Storm of Noise
The underlying issue is straightforward. AI-powered security scanning tools—from large language models trained on vulnerability databases to automated fuzzing frameworks—have dramatically lowered the barrier to entry for bug hunting. While this democratization was initially celebrated, the sheer volume of output has overwhelmed the limited human capacity to triage results. Each new AI tool, whether it's a static analysis plugin or a prompt-based vulnerability generator, can produce dozens of reports per hour. Many of these reports lack proof-of-concept code, describe theoretical attack scenarios that don't hold up under scrutiny, or cover issues already listed as out of scope by projects.
Jarom Brown, a senior product security engineer at GitHub, acknowledged the problem in a recent statement. He noted that while the lowering of barriers is a welcome development, his team is "being inundated by submissions that fail to demonstrate any real security impact." These include reports without a proof of concept, theoretical attack scenarios that don't hold up under scrutiny, and findings already covered by the company's published ineligible list. Brown emphasized that GitHub is not alone: "Programs across the industry are grappling with the same challenge, and some have shut down entirely."
To combat the flood, GitHub has updated its submission requirements. Going forward, submitters must validate AI-assisted findings before sending them in, include a working proof of concept that demonstrates exploitation potential and concrete security impact, and ensure submissions are concise—bloated, AI-padded reports will slow down triage and waste everyone's time. Reports covering known ineligible categories will be closed as "Not Applicable," which may affect the submitter's HackerOne Signal and reputation scores.
The cURL Project's Experiment
Perhaps the most illustrative case study comes from the cURL project, a widely used command-line tool and library for transferring data with URLs. Facing an overwhelming number of AI-generated junk reports, lead developer Daniel Stenberg made a bold move: the project temporarily stopped accepting submissions via HackerOne and eliminated monetary rewards for security reports entirely. Stenberg hoped the removal of financial incentives would stop the flow of automated submissions, reasoning that the most valuable security reporters would continue to report vulnerabilities out of goodwill.
Initially, cURL switched to accepting reports via GitHub issues and email. However, after a month, Stenberg found those channels less effective for managing vulnerability disclosures, and the project returned to using HackerOne. Importantly, the decision to eliminate bounties stuck. The results were striking: the nature of security report submissions changed dramatically. The "slop situation is not a problem anymore," Stenberg noted. The number of reports rose, their quality was higher (even if some were compiled with the help of AI), and the rate of confirmed vulnerabilities surpassed the 2024 pre-AI level.
Yet Stenberg cautions that this improvement brings its own challenge. The raised influx of legitimate vulnerability reports will exacerbate maintainer overload. "This avalanche is going to make maintainer overload even worse," he said. "Some projects will have a hard time handling this kind of backlog expansion without any added maintainers to help." The cURL experience shows that even when AI noise is reduced, the signal itself can become a burden if human resources don't scale.
Researcher Disillusionment
The collateral damage extends beyond the programs themselves. Shubham Shah, co-founder of Assetnote and a respected security researcher, says organizations are now taking far longer to review legitimate reports and act on real flaws, killing the feedback loop that keeps top researchers engaged. While bug bounty platforms like HackerOne and Bugcrowd are trying to fight AI-created spam with their own AI and added controls, Shah argues that "the joy of reporting vulnerabilities to bug bounties is quickly dissipating."
He warned that until platforms work this out, he cannot see himself continuing to report high-quality original research to certain programs where he has meaningfully contributed for a decade—especially when those programs fail to distinguish between his credible reports and those from uncredentialed researchers. In the near term, many experienced researchers may retreat to private vulnerability research and invite-only bounties, reducing the overall pool of eyes on public software.
Broader Industry Response
HackerOne, one of the largest bug bounty platforms, has publicly acknowledged the problem. In response to cURL's departure and return, the company advised customers to refine scope and submission guidelines to reduce noise, use AI-assisted triage tools, and pair automation with human oversight. Michiel Prins, co-founder and senior director of product management at HackerOne, stated: "As AI makes it easier to automate submissions, preserving signal quality becomes critical so open source maintainers can stay focused on fixing real issues." He emphasized the company's focus on helping programs manage the shift with workflows that filter noise early, surface credible reports, and keep vulnerability management sustainable.
Meanwhile, the Open Source Security Foundation's (OpenSSF) Vulnerability Disclosures Working Group is seeking community feedback as it works to help maintainers tackle AI-generated junk reports. Its goals include compiling best practices, creating policy templates, and developing guidance to help maintainers spot and handle AI-assisted submissions. This collaborative effort recognizes that open source projects, which rely on volunteer maintainers with limited time, bear the brunt of the AI firehose more than large organizations like Microsoft or Google that have dedicated security teams.
Historical Context and Future Risks
The current AI-driven crisis echoes earlier disruptions in the vulnerability research ecosystem. The rise of automated fuzzing tools in the 2010s similarly generated large volumes of crashes and findings, but those were often supported by a relatively small number of experts who could interpret results. AI tools today are far more accessible, enabling researchers with minimal experience to produce thousands of reports. The difference is that the human triage capacity has not increased proportionally.
Furthermore, the economic incentives of bug bounty platforms can encourage quantity over quality. While bounties reward confirmed vulnerabilities, many platforms also award reputation points for all submissions, inadvertently incentivizing spam. The industry now faces a critical inflection point: either develop robust automated filtering systems, or risk losing the trust of both maintainers and elite researchers who provide the most value.
For now, the consensus is clear. AI-assisted vulnerability research is here to stay, and its output will only grow. The challenge is not to stop AI from discovering bugs, but to ensure that the reports it generates are actionable, unique, and accompanied by real insight. As Torvalds, Brown, Stenberg, and others have argued, the burden is on researchers to add value, and on platforms to enforce quality standards—before the noise drowns out the signal entirely.
Source: Help Net Security News