Partial multi-label learning (PML) learns from over-complete candidate labels: All true labels are included in the candidate set, but irrelevant labels may also be annotated. Existing PML methods mainly reduce this ambiguity during classifier training by estimating label reliability, recovering latent label distributions, or modeling label correlations. This paper studies a complementary question: how can PML supervision be used for contrastive representation pretraining? The main difficulty is relation construction. Raw candidate label overlap is not a reliable contrastive signal, because two images can share only false candidate labels. We therefore formulate PML pretraining as a relation disambiguation problem and propose a confidence-aware relation contrastive learning framework. The method estimates candidate label confidence, uses it to construct semantic relation distributions over current and queued samples, and aligns the embedding distribution with these confidence-weighted relations. The same candidate label confidence semantics are used during downstream fine-tuning, keeping non-candidate labels as reliable negatives while softly disambiguating the candidate set. Across VOC2007, NUS-WIDE, and COCO2014, the method improves subset accuracy and F1-oriented measures under moderate and severe partial-label noise. For example, on VOC2007 at p=0.8, it improves subset accuracy from 0.3322 to 0.5269 and micro-F1 from 0.6710 to 0.7617 over PML-CD. However, the CUB200 results further identify a limitation: fine-grained attribute prediction is less consistently served by the present global representation pretraining design.
Shen-Huan Lyu, Ning Chen, Can Jiao et al.· Electronics· 0 citations
Split Learning (SL) enables collaborative model training by distributing computation between resource-constrained mobile edge devices and a central server. In this distributed framework, mobile edge devices compute the early layers of the model and transmit feature maps to the server. However, due to practical mobile wireless bandwidth limitations, these feature maps should be compressed before transmission. In this paper, we propose a lightweight Mask-Encoded Sparsification (MS) compressor that enhances Top-<inline-formula><tex-math notation="LaTeX">$k$</tex-math><alternatives><mml:math><mml:mi>k</mml:mi></mml:math><inline-graphic xlink:href="qu-ieq1-3696694.gif"/></alternatives></inline-formula> sparsification with a narrow-bit mask, significantly reducing compression error while improving computational and communication efficiency. Our theoretical analyses reveal that feature map compression generally induces estimator bias in the gradients. Furthermore, we establish the relationships between compression error and convergence, as well as between the choice of cutlayer and output error. The first relationship demonstrates that reducing compression error facilitates convergence, confirming the advantage of MS, whereas the second provides theoretical guidance for determining the cutlayer, indicating that earlier layers are more sensitive to compression errors. Extensive experiments on vision and language models demonstrate that our method outperforms existing compression techniques in terms of compression error and model performance, achieving up to a 21.38× speedup in communication efficiency under wireless environments.
Wen-Xuan Zhou, Zhi-Hao Qu, Baoliu Ye et al.· IEEE Transactions on Mobile...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
