LLM code review found 41 security issues for three thousand dollars

A security consultancy ran an LLM-assisted code review of GlobaLeaks and found 41 security findings for 3,140 dollars total. That works out to about 76 dollars per finding. GlobaLeaks is an open-source whistleblowing platform, so the codebase and threat model are public. The firm used Claude 3.5 Sonnet with a structured prompt. They fed it chunks of code, asked for OWASP-style analysis, then manually triaged the output. The 41 findings were medium-severity issues like path traversal risks, session fixation, and SQL injection vectors. No critical issues, but enough to matter in a production deployment. What surprised me is the cost breakdown. They spent 2,640 dollars on engineering time and 500 dollars on API calls. The API cost is the interesting part. Five hundred dollars of Claude calls suggests they ran a lot of context through the model, not just a single pass. That tracks with how code review actually works. You ask follow-up questions, you re-run sections after refactoring, you chase down edge cases. The obvious question is whether a human pentester would have been faster. A senior AppSec engineer charging 200 dollars per hour could do this review in fifteen hours for the same price, but might miss some of the OWASP checklist items that an LLM will mechanically catch. The LLM also does not get tired on the hundredth SQL query. The less obvious question is what this means for junior security engineers. If an LLM can find medium-severity issues for seventy-six dollars each, the value of a human doing the same work drops unless that human is bringing context the model cannot learn from a codebase alone. Threat modelling, business logic flaws, understanding what the developers actually intended. Those still need a human. I would not trust an LLM to run a security review unsupervised, but as a first-pass filter before human triage, this seems like a reasonable tradeoff. The firm published their methodology, so anyone with a Claude API key can try the same approach.

---

Source: An LLM-assisted security review of GlobaLeaks: 41 findings for $3,140