Aug 2026· Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2· pp. 2225-2236· 0 citations· 13 references
Abstract
Dynamic Ensemble Selection (DES) is an adaptive ensemble learning paradigm that selects a subset of base classifiers specific to each test input, enabling more flexible predictions than static ensemble methods. Although successful in tabular settings, DES remains largely unexplored in robust vision applications. We introduce VisionDES, a novel DES framework for image classification that uses deep model embeddings to estimate classifier competence. VisionDES leverages pre-trained vision transformer models to embed inputs and employs efficient nearest neighbor search to define a local region of competence for each sample. It then dynamically selects and fuses the most reliable models, using a similarity-weighted combination that down-weights less reliable or adversarially-compromised classifiers. VisionDES is extensively evaluated on various benchmarks and under clean conditions, distribution shifts, and strong adversarial attacks. It consistently outperforms static ensembles and existing uncertainty-based DES methods, improving robust accuracy by up to 20% under strong attacks and 2-3% higher accuracy under distribution shifts, with modest inference overhead. VisionDES offers instance-level interpretability by revealing models' contributions to the final decision.
A collaborative adversarial fine-tuning framework that maintains multiple experts during training, enables knowledge exchange through embedding-space harmonization, and consolidates the learned knowledge into a single unified robust model.
Nguyen Duc Thai, Junhao Dong, Sua Qi Rong et al.· arXiv.org· 0 citations
This work frames the fusion of multiple imperfect ViT-based detectors as a consistency-based abduction problem solved at test time by an exact Integer Program (IP) and a polynomial-time heuristic and shows that this metacognitive layer can be learned without any domain knowledge by exploiting vector-space geometry.
Mario A. Leiva, Yue Ma, Qinru Qiu et al.· 0 citations
While Large Vision-Language Models (LVLMs), represented by LLaVA and GPT-4V, have demonstrated remarkable capabilities, their visual inputs remain vulnerable to adversarial attacks, posing significant security risks. Existing defense methods predominantly target single-task scenarios (e.g., zero-shot classification) and consequently lack generalizability across various multimodal tasks. To address this limitation, we propose a dual adversarial fine-tuning framework that jointly optimizes visual and semantic supervision signals from two modalities, enhancing model robustness while generalizing across multiple downstream tasks. The proposed framework comprises two core components, i.e., $\textbf{Visual}$ supervision branch and $\textbf{Semantic}$ supervision branch. The former branch leverages features from clean images, extracted via a frozen original vision encoder, to guide adversarial robustness while the latter incorporates caption-image alignment as a contextual signal to preserve semantic coherence under attack. Moreover, our method achieves cross-task robustness by simply replacing the CLIP vision encoder in the original model, with no need of separate task-specific retraining or architecture modifications.Extensive experiments demonstrate that our approach outperforms the state-of-the-art method in adversarial robustness evaluation across zero-shot classification, image captioning, and visual question answering (VQA) tasks.
Sibo Wang, Jie Zhang, Shiguang Shan et al.· arXiv.org· 0 citations
DeepFakeBuster is presented as a confidence-calibrated adaptive ensemble for deepfake image detection by fusing together heterogeneous deep learning models built around detecting complementary forensic cues e.g., spatial inconsistencies, boundary artifacts, noise residuals, semantic consistency, and frequency-domain features.
Rachana Patil, R. Shinde, S. Patil et al.· Scientific Reports· 0 citations
Deep neural networks often suffer significant accuracy degradation when exposed to real-world image corruptions and distribution shifts. To overcome the limitations of fixed, input-agnostic test-time augmentation (TTA), an adaptive framework is proposed that learns per-sample transformations via reinforcement learning. Augmentation selection is cast as a Markov decision process and proximal policy optimization (PPO) agents are trained to choose sample-specific transforms under a composite reward combining classifier confidence gains with a self-consistency KL-divergence penalty on the model's own softmax outputs, thereby preserving overall belief stability. On clean CIFAR-10 (1 000 samples), the adaptive ensemble raises accuracy from 88.5% (baseline) and 87.3% (static TTA) to 90.0% (+1.5 pp). On CIFAR-10-C (15 corruptions × 5 severities; 1 000 images per condition), pooled top-1 accuracy improves from 75.7% (baseline) and 74.3% (static TTA) to 76.4% (+0.7 pp), and exceeds a TENT entropy-minimization baseline (75.9%) while operating in a strictly label-free regime that updates no model weights. Per-corruption gains are consistently positive across noise, blur, weather, and compression distortions, with the adaptive policy outperforming TENT on texture and compression corruptions where input-space transforms are most effective. These findings demonstrate that learned, per-sample augmentation policies improve robustness and reliability of deep vision models under diverse image conditions, against a strong baseline classifier.
T. Mittal, A. Dubey, Dharmender Saini et al.· Scientific Reports· 0 citations
Visual classifiers are expected to generalize under data shifts, target shifts, and their combinations, yet most existing methods focus on domain invariance while failing to address intra-image predictive sufficiency. We investigate the structural hypothesis that each image contains a sample-adaptive oracle intra-image predictive subset sufficient for label prediction, while the remaining patches form non-essential complementary context that may correlate with the label. The theoretical analysis shows that restricting prediction to this oracle subset preserves the Bayes risk achievable by the full-patch representation while admitting a complexity bound that tightens with the oracle-subset size. Based on this view, we propose PatchGen, a text-free module that learns a sample-dependent soft predictive-subset mask as a task-driven proxy for the unobserved oracle subset mask. Specifically, histopathology visualizations suggest that PatchGen assigns higher scores to tumor-consistent regions than to some frequently co-occurring inflammatory context. Extensive experiments on natural and histopathological image benchmarks spanning all three shift settings show that PatchGen improves average performance over matched-backbone baselines in most evaluated configurations, enhances generalization to unknown classes, and remains competitive with vision-language methods without text supervision.
Zhaorui Tan, Weimiao Yu, Xi Yang· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
