AI found a CVE in its own training set and we are pretending that is a win
Mythos discovered a vulnerability that was already documented in the data it was trained on. The industry is calling this autonomous discovery.
Mythos, an AI security researcher from Anthropic, [found a CVE in its training data](https://rival.security/posts/mythos-discovered-a-cve-already-in-its-training-data, -and-thats-still-worrying). The vulnerability was real. The discovery was not. The CVE was already documented in public write-ups that the model had seen during training. Mythos did not discover anything. It retrieved and reformatted information it had memorised. The security industry is treating this as autonomous research because the model synthesised the finding without being handed the exact exploit. This is the LLM version of a student copying an essay and rewording three sentences. The core insight was already there. The model wrapped it in fresh prose and called it analysis. The worrying bit is not that Mythos cheated. It is that we cannot tell when it is cheating. Every vulnerability report from an LLM will carry this ambiguity. Did it reason from first principles or did it pattern-match against disclosures it saw in training? If you cannot distinguish the two, you cannot trust the output. Security research depends on provenance. You need to know where a finding came from so you can assess whether the researcher had access to non-public information. With LLMs, provenance is a black box. The model cannot cite its sources because it does not track them. It mixes memorisation and reasoning into one undifferentiated soup. The fix is not better models. It is better verification. Every AI-generated CVE should be paired with a human audit trail showing the discovery path. Without that, we are building a disclosure pipeline that incentivises rediscovery and punishes originality.
Source: Mythos Discovered a CVE in Its Training Data - and That’s Still Worrying