2022· International Journal of Applied Data Science & Modern Computing· 0 citations
TL;DR
The analysis of performance has shown that the proposed solution is much better in terms of throughput, latency and fault isolation than the conventional architectures, proving that microservices architecture with proper design presents a solid base of scalable data-intensive systems.
Abstract
The modern software architectures have never had higher requirements than they do currently as the result of the rapid growth of data-intensive applications in the fields of cloud computing, Internet of Things (IoT), artificial intelligence, and large-scale web services. Amongst the problems that traditional monolithic systems cannot cope with are scalability, fault tolerance and on-going deployment demands using extensive data workloads. The consequence of this is that microservice architecture has become an architecture with potential to make systems that are scalable, resilient, and maintainable. The paper provides a detailed research of scalable microservices architecture specifically to the data intensive applications. The suggested architecture breaks the complex applications in loosely coupled, independently deployable services, which are specialized in the processing of specific data. Such fundamental architectural values as service granularity, data decentralization, containerization, orchestration, and asynchronous communication are discussed in detail. An overlay-based architectural design is presented to help scalability horizontally and provide high availability as well as efficient data management. Also, the analysis of current architectural methods is done in this study based on the comprehensive literature review that finds out the limitations in scalability, consistency, and operational complexity. An end-to-end process is suggested, where distributed data stores are combined with event-driven communication and automatic scaling processes. The analysis of performance has shown that the proposed solution is much better in terms of throughput, latency and fault isolation than the conventional architectures. These findings prove that microservices architecture with proper design presents a solid base of scalable data-intensive systems. The paper is useful to the researchers and practitioners who need to develop high-performance, cloud-native applications using micro services.
The paper addresses the transformation of enterprise application infrastructure out of on-premise legacy resource setting into service-based cloud environments properly configured to scale horizontally, and presents experimental evaluations of the response time, throughput, service resiliency, and infrastructure utilization in both traditional and cloud-native deployments.
Kanya Mohammed, Naree Thongchai· International Journal of Mod...· 0 citations
The explosive increase in digital information newest as a result of the social media sites, Internet of Things (IoT) devices, enterprise information systems, scientific simulations, and e-business programs has radically changed the computing needs of current-day data analytics. The conventional centralized designs of data processing architecture can not handle the volume, speed, and characteristics of such data, leading to scalability bottlenecks, high latency, and lower fault tolerance. The distributed computing setups have become a key facilitator of massive data processing through the harnessing of parallelism, data locality, and elasticity of resources on groupings of commodity hardware. The current paper is the in-depth study of the large-scale data processing in the context of distributed computing structures. It looks at the architectural concepts, programming models and mechanisms of execution used to implement contemporary distributed data processing systems. This paper critically evaluates leading systems like Hadoop MapReduce, Apache Spark, and Apache Flink systems and how they evolved to be based around batch processing rather than a hybrid batch/stream processing model. An extensive literature review brings together the previous studies carried out on scalability, fault tolerance, scheduling and performance optimization in a distributed environment. The suggested methodology comes up with a distributed data processing architecture that is layered and incorporates the resource intelligent resource management, parallel execution engines, and scalable storage. There are mathematical data partitioning, execution cost, and scalability mathematical formulations that are used to formalize system behavior. Experimental measurements based on the benchmark workloads show that there is a high increase in the throughput, execution time, and fault recovery against the traditional centralized systems. The trade-offs between frameworks analyzed in the discussion are based on latency, resource efficiency, and programming complexity. The paper ends by presenting the questionable opportunities to open research, such as scheduling of resources adaptively, data processing energy-efficiently, and applying artificial intelligence to autonomous optimization. The results are very helpful to researchers and practitioners who would have to create the next-generation of large-scale data analytics platforms.
Results indicate that organizations adopting microservices design on Kubernetes can achieve higher deployment frequency, better resiliency, improved resource utilization and greater agility to respond to evolving market needs.
Srichandra Boosa· American International Journ...· 0 citations
Cloud native data pipelines have become an enabling ingredient of the modern enterprise analytics to fulfill the ever-increasing demand of a scale-loving, resilient, and real-time processing of a wide range of data sources. Organizations currently produce large amounts of structured, semi-structured, and unstructured data in transactional systems, Internet of Things (IoT) platforms, digital channels, and data sources that are external (Bank of America 2017). Old monolithic data integration architectures are designed to provide batch-oriented processing and static infrastructure capabilities have challenges satisfying low latency, scale on demand, and 24/7 requirements. Reactively, cloud-native paradigms, including the foundations of microservices, container orchestration, event-driven architectures, and managed cloud services have caused a rethinking of the data pipeline design, deployment and operation. This article provides an in-depth analysis of cloud-native pipeline data to enterprise analytics along with their main architectural concepts and processing models as well as operational aspects that fall within the professional scope of IEEE publications. The research paper summarizes the literature and business methodologies to present a reference model which brings together data ingestion, stream processing, batch processing, storage, governance, and analytics consumption layers. Special concern is opened to the contributions of containerization, orchestration platforms, and serverless computing towards facilitation of elasticity and fault tolerance. The paper also examines design patterns like Lambda architecture and Kappa architecture, data mesh theory and metadata-based orchestration, with an emphasis on its application to the large enterprise environment. An organized approach to the design and deployment of cloud-native data pipelines with the inclusion of data quality management, security controls, observability, and cost optimization is suggested. Throughput, latency and scalability modeling mathematical formulations are proposed in order to facilitate capacity planning and performance measurement. Representative enterprise workloads as shown through experiment results exhibit evident increases in data processing latency, pipeline reliability and operational efficiency over traditional architectures. These findings are placed in context to the discussion of the broader transformation efforts at enterprises, whereas the conclusion provides recommendations on future research opportunities, such as autonomous pipeline optimization and AI-based orchestration.
Ethan Williams· International Journal of App...· 0 citations
WebAssembly (WASM) is emerging as a lightweight alternative to containers for Function-as-a-Service (FaaS) across the edge-cloud continuum. However, existing WASM-based serverless platforms are tightly coupled to specific execution engines and predominantly designed for stateless workloads. This clashes with the heterogeneity of edge deployments, which demand support for stateful applications under diverse hardware and workload constraints. We introduce WASP, a configurable framework that brings stateful serverless execution to the edge-cloud continuum. By abandoning monolithic architectures in favor of strictly decoupled, pluggable components, WASP lets system administrators swap the WASM runtime and the datastore to fit available resources and application requirements, without altering application code. Configurable lifecycle and caching policies further enable fine-tuning for diverse non-functional requirements. Our experimental evaluation demonstrates that WASP introduces negligible runtime overhead and, by swapping runtimes, datastores, and policies, exposes radically different memory and latency profiles, confirming its adaptability to the heterogeneous constraints of the edge-cloud continuum.
