From Category Strings to Category Scores: A Controlled Screening Study of Transformer Output Interfaces

Abstract

Extreme multi-label classification asks a model to identify a small set of relevant labels from a large catalog. A general-purpose language model can write label identifiers as text, but the same pretrained Transformer can instead score every label in one pass. We study this output-interface choice on a fixed 20,000-example training sample and 2,000-example held-out sample from AmazonCat-13K, with 13,330 labels. Both systems start from Qwen3-0.6B, use identical input tokens, and are fully fine-tuned for one epoch under three seeds. We nickname the direct- head configuration LabelScout-1 solely to distinguish the configurations; it is a standard multi-label head, not a proposed architecture. Direct scoring improves mean precision@5 from 0.3889 to 0.4635, recall@5 from 0.4947 to 0.5694, and nDCG@5 from 0.5755 to 0.6800. Median batch-one inference falls from 767.5 to 22.2 ms on the measured host. Conversely, text generation achieves 0.1367 exact-set match versus 0.0060 for the fixed-top-five direct output, a comparison strongly affected by output cardinality. We report per-seed outcomes, paired uncertainty intervals, cardinality diagnostics, compute, and limitations. This is an output-interface screening result on a sampled benchmark, not evidence of a novel Transformer architecture or state-of-the-art extreme classification.