Skip to content
Open access

Bitzkrieg at SemEval-2026 Task 13: Calibration-Aware Dual CodeBERT for Multilingual Machine-Generated Code Detection

2026 · SemEval@ACL · pp. 2233-2237 · 0 citations · 8 references
Computer Science

TL;DR

The submission to SemEval-2026 Task 13 (Orel et al., 2026), addressing binary detection, generator attribution, and hybrid/adversarial author-ship classification of machine-generated code of machine-generated code (MGC).

Abstract

We describe our submission to SemEval-2026 Task 13 (Orel et al., 2026), addressing binary detection (Subtask A), generator attribution (Subtask B), and hybrid/adversarial author-ship classification (Subtask C) of machine-generated code (MGC). For Subtask A, we fine-tune two CodeBERT (Feng et al., 2020) models with complementary sampling strategies and apply percentile-based post-hoc calibration, improving Macro-F1 from 0.47 to 0.56 without additional training. For Subtask B, we combine TF-IDF n-grams, frozen Code-BERT embeddings, and language features with XGBoost (Chen and Guestrin, 2016), us-ing synthetic augmentation and class weighting to handle an 11-class dataset skewed 88% toward the human class, achieving Macro-F1 of 0.289. For Subtask C, we fine-tune a Code-BERT classifier for four-way authorship classification, achieving Macro-F1 of 0.49. Our results highlight the importance of probability calibration for binary detection and class balancing for multi-class attribution.

Read PDF

Similar papers

Open access 2026

Response-Only Adversarial Detection of LLM Outputs: A Comparative Study of Lexical, Contrastive, and Contextual Representations

This study investigates response-only adversarial classification of LLM outputs through a comparative evaluation of lexical, contrastive, and contextual representations and provides a realistic benchmark for response-only adversarial detection and clarify the limits of contrastive representations when prompt information is absent.

A. Nasution, Ahmet Emre Ergün, Aytu˘g Onan et al. · 0 citations
Jul 2026

Adversarial Deepfake Generation and an Investigation of Purification-Based Adversarial Detection

A self-initiated investigation of purification-based adversarial detection, comparing three families of detection signals across six detectors that share a CLIP ViT-L/14 backbone finds that raw $|\Delta \text{logit}|$ under median-3 purification, applied through the EFFORT detector, separates adversarial inputs from clean inputs with AUROC 0.81-0.98.

Junghyun Kim, Seunghyun Kim, Ji-myung Woo · 0 citations
Open access Jul 2026

From Evaluation to Optimisation: Hierarchy-Aware Training Signals for CWE Prediction in Python

Testing three delivery mechanisms–supervised fine-tuning, a dual-head classification loss, and reinforcement learning with a dense reward derived from the normalised penalty finds that supervised approaches consistently regress below the zero-shot baseline under distribution shift, while GRPO succeeds and generalises across independent training seeds.

Muntasir Adnan, Manile Srun, Carlos C. N. Kuhn · 0 citations
Jul 2026

CoGate: Confidence-Gated Co-Decoding for Secure Code Generation

CoGate is proposed, a confidence-gated co-decoding approach that controls the expert's influence on the co-decoding process based on its confidence, and outperforms existing co-decoding methods (CoSec+) across multiple benchmarks.

Minghao Hu, Lannan Luo, Allen G. Roush et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.