Aug 2026· Journal of Intelligent Decision Making and Information Science· 0 citations· 30 references
TL;DR
AMF-CloudForge is presented, a unified machine learning-driven framework that integrates migration state analysis, intelligent scheduling, consistency preservation, and real-time adaptive management into a single end-to-end architecture and transforms cloud data migration from a static, tool-centric process into a reliable, adaptive, and continuously optimized cloud service.
Abstract
Cloud data migration has become an essential requirement for modern enterprises due to the rapid growth of cloud computing and data-intensive applications. However, achieving reliable and efficient migration remains challenging because of heterogeneous data types, fluctuating network bandwidth, diverse migration tools, and the need to preserve data consistency throughout the migration process. Most existing approaches evaluate migration tools independently, assume static network conditions, and provide limited support for adaptive optimization and verifiable migration outcomes. This paper presents AMF-CloudForge, a unified machine learning-driven framework that integrates migration state analysis, intelligent scheduling, consistency preservation, and real-time adaptive management into a single end-to-end architecture. The proposed framework begins with Polytype Migration State Encoding (PMSE), which performs comprehensive micro-benchmarking across different data types, file sizes, and network conditions to generate Migration State Tensors and Tool Efficiency Scores that accurately characterize migration behavior. These learned representations are then utilized by the Topology-Aware Diffusion Scheduler (TADS) to optimize chunking, routing, and migration tool selection by considering dynamic network topology, bandwidth variations, and transfer costs. To ensure data integrity and consistency, the Causal Delta with Erasure-coded Commitment (CDEC) module combines causal dependency tracking, adaptive erasure coding, and cryptographic quorum verification to guarantee reliable and verifiable data commits without data loss or metadata inconsistencies. Furthermore, the Uncertainty-Quantified Twin Supervisor (UQTS) continuously monitors the migration process through a probabilistic digital twin, enabling risk-aware scheduling adjustments and real-time Service Level Objective (SLO) verification under changing system conditions.Together, these components form a fully automated migration framework capable of policy-driven execution and machine-verifiable migration reporting. Experimental evaluation on heterogeneous cloud workloads demonstrates that the proposed framework reduces overall migration makespan by 35–45%, sustains effective bandwidth utilization above 85%, and limits unrecoverable data loss to approximately 10⁻¹¹ per 10 TB of migrated data. By integrating machine learning-based performance modeling with adaptive scheduling, cryptographic consistency verification, and intelligent runtime supervision, the proposed framework transforms cloud data migration from a static, tool-centric process into a reliable, adaptive, and continuously optimized cloud service.
Optimize techniques for cloud-based distributed systems to enhance the deployment and scaling of ML models in real-time applications and delves into the application of machine learning for cloud resource provisioning, emphasizing dynamic allocation based on real-time usage patterns.
Emma Roberts, William Hughes· International Journal of Art...· 0 citations
The rapid growth of large datasets and AI models has necessitated scalable and powerful computing resources, driving the extension of training workloads to the cloud to balance cost and performance. Among distributed learning strategies, synchronous data-parallel training remains the most widely adopted approach due to its simplicity and stable convergence. However, heterogeneity between worker nodes and unreliability of network bandwidth pose substantial challenges. Since workers must synchronize gradients at every update step, even a single straggler can significantly degrade overall performance. In this paper, we present ASTRA, an auto-adaptive synchronous training framework tailored for hybrid cloud environments. ASTRA addresses these issues with three coordinated mechanisms: i) a network-topology detection module that clusters workers into groups by pairwise connection cost to enable hierarchical aggregation, where gradients are first aggregated within each group and then across groups, mitigating slow-link bottlenecks; ii) a load balancing module to efficiently distribute workloads among heterogeneous GPUs over time, reducing resource imbalance and straggler effects; and iii) an adaptive gradient compression module that dynamically adjusts compression levels based on network conditions to minimize inter-group communication overhead. Through automated adaptation to diverse system environments, our unified approach alleviates the burden of manual configuration for AI practitioners. Experimental evaluations demonstrate the effectiveness of the framework ASTRA, which achieves lower time-to-accuracy than a resource-heterogeneity-aware baseline and several compression-based frameworks, while preserving convergence quality and robustness across heterogeneous hybrid cloud environments.
Tuan Anh Vuong, Thanh Loi Hoang, Huan Le et al.· IEEE Access· 0 citations
The rapid growth of enterprise systems and cloud computing has transformed data management across hybrid environments integrating on-premise databases, private clouds, and public cloud infrastructures. However, challenges such as data consistency, latency, conflict resolution, security, and fault tolerance remain critical in distributed heterogeneous systems. Traditional synchronization methods are often inadequate for dynamic real-time workloads. This study reviews intelligent data synchronization techniques for hybrid data platforms, emphasizing AI- and machine learning-based approaches that enhance synchronization efficiency, scalability, and reliability. The proposed framework includes four layers: Data Acquisition, Intelligent Synchronization Engine, Adaptive Conflict Management, and Distributed Analytics. Predictive learning algorithms optimize synchronization timing and resource allocation, while adaptive conflict resolution mechanisms minimize inconsistencies. Experimental results show that intelligent synchronization methods reduce delay, improve throughput, enhance scalability, and strengthen failure recovery compared to traditional approaches. The study concludes that AI-driven synchronization is essential for real-time analytics, distributed transactions, and scalable cloud-native applications in modern enterprise environments.
J. Arsac, Gérard Huet· International Journal of Dat...· 0 citations
With the rapid advancement of computing technologies, high-performance computing (HPC) systems have continued to expand in scale and have been widely applied across various domains. To improve resource utilization, existing studies commonly employ offline-trained models based on historical resource usage data to classify HPC jobs as compute-bound or memory-bound. However, such methods typically rely on static historical data and struggle to adapt to dynamically changing workloads over time, resulting in degraded prediction performance and reduced stability under concept drift. To address this issue, this paper proposes an online drift-adaptive framework for HPC job semantic representation and classification (DA-HJSC). The framework employs a hybrid mechanism that combines offline semantic learning with online dynamic adaptation. Specifically, an offline classification model is first trained using semantic representations of job descriptions together with user historical behavioral features (UHBF). During the inference stage, an exponentially weighted moving average (EWMA) mechanism is further introduced to dynamically fuse the output probabilities of the classification model, thereby producing the final classification results for HPC jobs. Experimental results on the publicly available F-DATA dataset covering the period from 1 May 2023 to 30 April 2024 demonstrate that DA-HJSC consistently improves HPC job classification performance across different pre-trained language models and base classifiers, achieving a maximum F1macro (overall) score of 0.9272. Compared with the corresponding baselines, DA-HJSC reduces the number of low- and medium-performance days by up to 11 and 35 days, respectively, across all experimental configurations. Overall, the DA-HJSC framework achieves an effective balance among classification accuracy, performance stability, and online adaptability, providing a solution with practical deployment potential for dynamic HPC workload classification.
Centralized data operations are often using in Small and medium-sized enterprises (SMEs) for easy data management, but there suffering from few limitations like cloud platform integration with bigdata yet many still face difficulties integrating cloud platforms with big-data capabilities in a scalable and governed manner. To address the problems, this communication presents an Adaptive Cloud -Big-Data Enablement Framework (ACBDEF), which is a realistic mechanism of SME digital data transformation. This framework consists of five main steps, which are technological infrastructure, data governance and compliance, organizational capability development, environmental alignment, and intelligence/value extraction into a common architecture. One of the key elements in this work is the Adaptive Migration Engine (AME) which is used to assess dynamically workloads, the parameters such as data characteristics, regulatory constraints, the cost-performance metrics are used to decide on the optimal deployment in cloud premises, or in hybrid environments. The adaptive decision process is beneficial in assisting SMEs in mitigating technical and organizational issues in enhancing the efficiency, security, and analytical responsiveness. The proposed mechanism brings into line theoretical adoption factors with actionable implementation which provides a structured model for supporting SMEs to achieve sustainable and data-driven cloud transformation.
B. Madhu Uthej, Dudde Lohith, Atluru Sai Charan Reddy et al.· 2026 7th International Confe...· 0 citations
Deploying machine learning models efficiently within cloud data platforms is critical for enabling real-time, data-driven decision making. Snowflake, a leading cloud data warehouse, provides powerful capabilities for data storage and processing but lacks native tools for hosting and serving complex ML models. This paper presents a novel approach for optimizing machine learning model deployment by integrating Snowflake’s external functions with AWS SageMaker, a managed machine learning service. We describe an architecture where Snowflake invokes SageMaker-hosted models via AWS Lambda-based external functions, enabling seamless, scalable, and secure model inference directly from within SQL queries. The implementation details and optimization techniques to minimize latency and cost are discussed. Experimental evaluations demonstrate significant performance improvements and cost savings, showcasing the feasibility and benefits of this integration. This approach empowers data teams to leverage their existing Snowflake infrastructure while harnessing advanced ML capabilities from SageMaker, facilitating faster and more efficient AI-driven insights.
Peter Wilson· International Journal of Dat...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
