Sarcouncil Journal of Multidisciplinary
Sarcouncil Journal of Multidisciplinary
An Open access peer reviewed international Journal
Publication Frequency- Monthly
Publisher Name-SARC Publisher
ISSN Online- 2945-3445
Country of origin- PHILIPPINES
Frequency- 3.6
Language- English
Keywords
- Social sciences, Medical sciences, Engineering, Biology
Editors

Dr Hazim Abdul-Rahman
Associate Editor
Sarcouncil Journal of Applied Sciences

Entessar Al Jbawi
Associate Editor
Sarcouncil Journal of Multidisciplinary

Rishabh Rajesh Shanbhag
Associate Editor
Sarcouncil Journal of Engineering and Computer Sciences

Dr Md. Rezowan ur Rahman
Associate Editor
Sarcouncil Journal of Biomedical Sciences

Dr Ifeoma Christy
Associate Editor
Sarcouncil Journal of Entrepreneurship And Business Management
Next-Generation Cloud ETL Pipelines: A Comparative Study of Serverless and Containerized Architectures
Keywords: Serverless computing, containerized architectures, cloud-native ETL, event-driven processing, microservices patterns, data pipeline orchestration.
Abstract: The transition of data processing architectures from traditional batch-processing systems to cloud-native paradigms constitutes a paradigm shift in enterprise data management approaches. Serverless computing paradigms have become feasible alternatives to traditional infrastructure-based models, with scaling features and event-driven processing patterns alleviating infrastructure management complexities. Event-driven architectures enable near-real-time response to data availability, along with cost-efficient resource usage through pay-per-execution pricing models. Containerized solutions offer enormous advantages through persistent stateful processing features, sophisticated orchestration patterns, and high-level integration patterns enabling complex data transformation workflows. The performance attributes are significantly different across architectural paradigms, with serverless platforms showing enhanced efficiency in variable workload patterns, while containerized solutions provide consistent performance statistics for continuous processing requirements. Cost optimization features differ significantly based on workload patterns, with serverless architectures being cost-effective for ad-hoc processing patterns, and containerized deployments being effective for high-volume sustained activities. Integration features encompass varied connectivity to data sources, sophisticated error handling processes, and comprehensive monitoring frameworks enabling reliable pipeline execution. This article discusses the architectural trade-offs, performance considerations, and operational implications that influence technology selection criteria for cloud-native data processing applications.
Author
- Krishna Chaitanya Batchu
- Horizon International Trd Inc. USA