AutoPatchBench serves as a standardized benchmark tailored for assessing large language model (LLM) agents in the domain of automated security patching for C/C++ vulnerabilities. It derives its tests from ARVO, a comprehensive dataset featuring over 5,000 real-world vulnerabilities identified through fuzz testing. By focusing on the specific challenges associated with fuzzing-derived bugs, AutoPatchBench promotes transparency and reproducibility in research while supporting a better understanding of AI-driven bug repair capabilities. The benchmark offers distinct subsets for both comprehensive evaluation and focused testing of AI tools against real-world security issues.
AutoPatchBench is a benchmark designed to evaluate how effectively LLM agents patch security vulnerabilities in native code, providing a consistent assessment framework.
By offering a standardized set of tests on AI-driven approaches to security, AutoPatchBench enhances transparency and reproducibility in assessing vulnerability repair.
Collection
[
|
...
]