Subjective NLP tasks often exhibit systematic annotator disagreement, requiring models that represent uncertainty rather than collapse it. We introduce Ensemble Diversity Optimization (EDO), a prediction-space framework that jointly optimizes ensemble weights, effective cardinality, and calibration through a unified differentiable objective. EDO learns ensemble composition and size end-to-end via Gumbel-Softmax relaxation and incorporates a signed diversity regularizer, tuned on validation data, to steer optimization toward either preserving or suppressing disagreement. This regularization prevents ensemble collapse and enables controlled navigation of the utility-calibration trade-off. The framework integrates a soft F1 surrogate, class-weighted cross-entropy to address imbalance, and reliability-weighted diversity to regulate intra-ensemble variability. Experiments on four subjective text-classification benchmarks (ArMIS, ConvAbuse, HS-Brexit, MD-Agreement) show that EDO substantially improves probabilistic calibration, reducing cross-entropy (40-78% depending on baseline) and lowering Brier scores relative to Soft-CE, Soft-MD, Top-5 Voting, and WEL, while maintaining competitive F1 and better alignment with annotator distributions. These results demonstrate that jointly optimizing ensemble structure with a signed diversity regularizer provides an efficient, model-agnostic approach for modeling human subjectivity in supervised learning.
Xia Cui, Ziyi Huang, N. Abeynayake· arXiv.org· 0 citations
Node classification is a central graph data mining task, yet repeated message passing can over-smooth representations and degrade frozen graph neural network (GNN) predictions after deployment. We present SETTA (Spectral-Energy Test-Time Adaptation), a prediction-level graph test-time adaptation framework that refines frozen outputs without test labels, gradients, parameter updates, or learnable adaptation parameters. SETTA denoises features for semantic-neighbor construction, adds complementary semantic routes while preserving observed edges, monitors a smoothness-energy proxy during diffusion, and accepts refinements through entropy-based gating. Configurations are fixed by a dataset-level protocol or selected using validation data only. Across six mostly homophilic benchmarks with 2708–19,717 nodes, SETTA improved a frozen two-layer GCN on every dataset and achieved the highest mean accuracy among the evaluated methods on five, with gains of 4.61, 3.08, and 2.01 percentage points on Cora, CiteSeer, and PubMed, respectively. Positive mean gains were also observed across all 30 dataset–backbone settings. Ablations and transition analyses indicate that semantic injection is most beneficial on sparse citation graphs and that selective refinement limits harmful changes. The current dense implementation supports benchmark-scale, amortized refinement; scalability and robustness on heterophilic graphs remain open.
Dongyang Yu, Xia Cui, Rong Xiao· Big Data and Cognitive Compu...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
