Jul 2026· International journal of computer information systems and industrial management applications· Vol 18, pp. 1247-1252· 0 citations
TL;DR
A Domain-Adaptive Neural Architecture Search (DA-NAS) framework that learns domain-aware architectural patterns while it maintains a shared search space and optimisation strategy, demonstrating that the proposed approach reduces search cost and improves cross-domain transferability.
Abstract
Neural Architecture Search (NAS) has emerged recently as a powerful paradigm for automating deep neural network design. However, most existing NAS methods are optimised for a single domain, limiting their generalisation to diverse application areas such as computer vision, natural language processing, healthcare, speech recognition, and edge intelligence. This paper proposes a Domain-Adaptive Neural Architecture Search (DA-NAS) framework that learns domain-aware architectural patterns while it maintains a shared search space and optimisation strategy. DA-NAS combines domain embeddings, multi-objective optimisation, and resource-awareness to generate architectures that adapt to heterogeneous data characteristics and deployment constraints. Extensive experiments across multiple domains demonstrate that the proposed approach reduces search cost and improves cross-domain transferability, consistently outperforming domain-specific handcrafted models and conventional NAS baselines.
Neural Architecture Search (NAS) offers a promising pathway to automate the design of deep neural networks, yet it faces a significant challenge in balancing computational efficiency with search effectiveness in a huge search space. Coarse-to-fine (C2F) NAS methods mitigate this by narrowing the architecture search to promising subspaces. However, by focusing only on a single region, they risk overlooking globally optimal architectures due to the multi-modal nature of the search space. This paper highlights the limitation of existing C2F approaches and motivates the need for more sophisticated search strategies capable of efficiently exploring multiple promising regions to achieve a better performance-efficiency trade-off. To achieve this goal, we propose a multi-stage NAS (MstageNAS) framework that progressively constructs multiple high-quality subspaces and implements an efficient exploration within them. MstageNAS initiates with a coarse search to identify promising architectures within the whole search space. Subsequently, the individual subspace is constructed around each promising architecture. To ensure the quality of this subspace, an architecture explanation method is devised to identify the core sub-structure of the promising architecture and use it to form the basis of the individual subspace. Finally, a Monte Carlo-based search strategy is developed to facilitate architecture search within these subspaces, with the goal of striking a good balance between exploration and exploitation. We evaluate the proposed MstageNAS framework across search spaces of various types and tasks. Extensive experiments demonstrate that MstageNAS can outperform state-of-the-art NAS methods or achieve comparable performance but with about 2× less search cost.
It is demonstrated that network architecture and its coeficients can be learned together by unifying concepts of evolutionary search within a population based traditional training process.
In recent years, artificial intelligence (AI)-based clinical image analysis has achieved remarkable diagnostic performance; however, when it is practical deployment remains limited by poor generalization across heterogeneous clinical environments. Variations in imaging devices, acquisition protocols, patient demographics, and institutional practices frequently introduce domain shifts that substantially reduce the reliability of deep learning models when evaluated on previously unseen hospitals. This study proposes a Cross-Domain Generalization Neural Architecture Search (CDG-NAS) framework that automatically discovers neural network architectures capable of maintaining high diagnostic accuracy under diverse clinical domain shifts.The proposed framework integrates domain generalization directly into the neural architecture search process by combining multi-domain architecture optimization, Domain Consistency Regularization (DCR), and a Robustness-Aware Architecture Scoring (RAS) strategy. During architecture search, candidate models are trained using multiple source domains while simultaneously being validated on unseen domains to encourage domain-invariant feature learning. Experiments were conducted using four publicly available clinical imaging datasets representing chest radiography, histopathology, brain magnetic resonance imaging, and retinal fundus imaging. The performance was evaluated using Accuracy, Precision, Recall, F1-score, Area Under the ROC Curve (AUC), Expected Calibration Error (ECE), Robustness Score, and paired statistical significance tests for the proposed framework.The proposed CDG-NAS framework achieved an average cross-domain classification accuracy of 94.6%, outperforming conventional architectures including ResNet50 (85.7%), DenseNet121 (87.3%), EfficientNet-B4 (89.4%), Vision Transformer (90.2%), and Differentiable Architecture Search (91.1%). The framework improved the average robustness score by more than 8% under unseen domain conditions while simultaneously reducing inter-domain performance variance. Ablation experiments demonstrated that removing Domain Consistency Regularization reduced accuracy from 94.6% to 91.8%, whereas eliminating the Robustness-Aware Architecture Score decreased accuracy to 92.4%, confirming the contribution of each proposed component. Statistical analysis further verified that the observed improvements were significant (p < 0.01).Unlike existing neural architecture search methods that optimize architectures primarily for source-domain accuracy, the proposed CDG-NAS framework incorporates domain robustness as a first-class optimization objective throughout the search process. The integration of multi-domain validation, Domain Consistency Regularization, and Robustness-Aware Architecture Scoring enables the automatic discovery of architectures that exhibit superior generalization across previously unseen clinical environments. This approach provides a practical and scalable solution for developing trustworthy clinical AI systems suitable for real-world deployment.
A. Sindhu Devi, L. Godlin Atlas· International journal of com...· 0 citations
Transformer-based models are widely used for clinical prediction from electronic health records (EHRs), yet their architectures require manual tuning, and the optimal configuration may vary across tasks and hospitals. Neural architecture search (NAS) automates architecture design, but conventional methods are computationally costly for Transformer-based EHR models. Recent large language model (LLM)-guided NAS methods reduce manual search design but conduct each search independently, without reusing architecture knowledge across hospitals. In this study, we propose ATHENA (Agentic Transfer across Hospitals for EHR Neural Architecture Search), a knowledge-guided agentic NAS framework for Transformer-based EHR modeling. ATHENA uses a weight-sharing supernet that is pretrained once per hospital, allowing candidate architectures to be instantiated as inherited subnetworks and evaluated through fine-tuning rather than independent pretraining. It incorporates a two-layer cross-hospital architecture prior. The first layer retrieves high-performing architecture examples from source sites based on task descriptors, while the second estimates the effects of architectural components using SHapley Additive exPlanations (SHAP)-based meta-regression. These priors guide a multi-agent LLM search using validation feedback from the target hospital. Across six clinical prediction tasks evaluated at one held-out OneFlorida+ site and one external MIMIC-IV site, ATHENA significantly outperforms all four baselines in 9 of 12 site-task evaluations under a strict equal-compute comparison. Using a common pretrained AutoFormer supernet for candidate evaluation, ATHENA ranks first in 9 of 12 evaluations at a search budget of 30. It also shows more consistent architecture selection across repeated searches. ATHENA provides a practical approach for reducing manual architecture tuning in Transformer-based EHR modeling.
Deep neural networks excel in various tasks but struggle to generalize across evolving data distributions, leading to significant performance degradation under domain shifts. Domain incremental learning (DIL) addresses this challenge by enabling models to continuously adapt while retaining prior knowledge. Among existing DIL approaches, the parameter-isolation paradigm achieves state-of-the-art performance. However, these methods often adopt a one-size-fits-all approach to adapt to new domains, resulting in either insufficient learning capacity or redundant parameters. In this work, we propose BPG, a unified framework that addresses both challenges through two complementary components: BPG-Adapter, which dynamically determines each domain's adapter hidden dimension based on domain-specific feature separability, and BPG-Inference, a soft domain mixture strategy that integrates multiple domain-specific models at test time, mitigating domain ID misselection. Experimental results on DomainNet, CDDB, and CORe50 demonstrate that BPG consistently outperforms uniform adapter-based approaches and hard domain selection strategies, achieving state-of-the-art average accuracy while reducing forgetting to as low as 0.22% on DomainNet.
Qiang Wang, Songlin Dong, Shaokun Wang et al.· 0 citations
Experimental findings show that pretrained models outperform those trained from scratch in terms of accuracy, convergence speed, and robustness, and a unified framework is proposed to integrate both processes in a deep learning pipeline.
Kwame Nkosi· International Journal of App...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
