Confidence-aware pseudo-label selection and verifier training for semi-supervised LLM reasoning with minimal labels

Abstract

This paper studies how to improve the reasoning ability of large language models (LLMs) with minimal supervision. Recent gains in LLM reasoning largely come from learning intermediate reasoning traces, and many methods reduce supervision cost by using traces whose final answers are correct. In realistic settings, however, obtaining even those answer labels can be costly, motivating methods that extend reasoning from a very small labeled set with abundant unlabeled questions. Verifier-based semi-supervised learning is a promising approach: a verifier trained on a small labeled set can score reasoning traces on unlabeled questions and identify candidates for pseudo-labeling. However, even with a verifier, it remains unclear how pseudo-labeled samples should be selected to support downstream training. In particular, pseudo-label selection must balance quality and quantity. To address this, we introduce an adaptive threshold selection policy that chooses thresholds on validation data using pseudo-label precision and sample count. We further combine this policy with confidence-aware verifier training to support confidence-based selection. Experiments on verifiable math reasoning benchmarks show that, under our training setup, this combination improves downstream reasoning accuracy over the tested baselines and selects pseudo-labeled subsets with a more favorable reliability–coverage trade-off. These results suggest a practical design direction for verifier-guided pseudo-label selection in answer-verifiable, minimal-label reasoning settings.