Keep Evaluation Fair: Detecting Data Leakage in Code Generation Benchmarks via Membership Inference Attacks
Experiments on eight code generation benchmarks show that CGMIA outperforms eight existing membership inference methods in most cases, and effectively detects known leaked APPS samples in StarCoder-7B's training data.
Dong-Dong Zhao, Jian Chen, Guan-Cheng Lin et al.
· 0 citations