CATF-Net: Content-Adaptive Multi-View Time–Frequency Fusion for Pretraining-Free Classification Across Heterogeneous Audio Tasks
Abstract
Audio classification across speech-command and environmental-sound tasks requires both fine temporal cues and structured spectral context, yet many high-performing systems rely on external pretraining. We propose CATF-Net, a supervised framework trained from random initialization using only the labeled training data of each target benchmark. CATF-Net first learns sample-dependent weights over Mel, log-magnitude, and phase-derived spectral views and then combines the resulting spectral embedding with a raw-waveform temporal embedding through branch-wise regularization and parameter-free addition. The shared spectral interface also permits alternative descriptors to be incorporated without changing the temporal branch. The dataset-specific configurations contain 15.61–21.23 M parameters, compared with 81–90 M for several pretrained baselines considered in our experiments. CATF-Net achieves 92.95%, 98.50%, 86.49%, and 98.37% accuracy on ESC-50, ESC-10, UrbanSound8K, and Speech Commands V2, respectively, under their corresponding benchmark protocols.