Notes on measuring security tools honestly.

What Matters When Benchmarking Cyber Security in LLMs (And Why I Don't Trust Existing Approaches)

The dominant AI security benchmarks score the wrong things, ignore false positives entirely, and can rank a tool that emits 500 junk findings above one that emits two real ones. What an honest benchmark would have to do instead.