Sarcouncil Journal of Engineering and Computer Sciences
Sarcouncil Journal of Engineering and Computer Sciences
An Open access peer reviewed international Journal
Publication Frequency- Monthly
Publisher Name-SARC Publisher
ISSN Online- 2945-3585
Country of origin-PHILIPPINES
Impact Factor- 3.7
Language- English
Keywords
- Engineering and Technologies like- Civil Engineering, Construction Engineering, Structural Engineering, Electrical Engineering, Mechanical Engineering, Computer Engineering, Software Engineering, Electromechanical Engineering, Telecommunication Engineering, Communication Engineering, Chemical Engineering
Editors

Dr Hazim Abdul-Rahman
Associate Editor
Sarcouncil Journal of Applied Sciences

Entessar Al Jbawi
Associate Editor
Sarcouncil Journal of Multidisciplinary

Rishabh Rajesh Shanbhag
Associate Editor
Sarcouncil Journal of Engineering and Computer Sciences

Dr Md. Rezowan ur Rahman
Associate Editor
Sarcouncil Journal of Biomedical Sciences

Dr Ifeoma Christy
Associate Editor
Sarcouncil Journal of Entrepreneurship And Business Management
Metadata‑Driven Frameworks for Configurable, Idempotent, and Scalable Financial Data Pipelines
Keywords: Metadata-driven Frameworks, Financial Data Governance, Idempotent Processing, Real-time Data Integration, Regulatory Compliance Automation.
Abstract: Financial institutions face exponential data growth exceeding 300% annually while navigating over 750 new regulatory requirements introduced since 2020, rendering traditional Extract-Transform-Load (ETL) approaches increasingly brittle and unmanageable. This article presents a metadata-driven framework that leverages knowledge graph technologies as a control plane for financial data pipelines, enabling declarative configuration, comprehensive lineage tracking, and cryptographically-verifiable audit mechanisms through a graph-based metadata repository. Implementation at a global financial institution demonstrated 73% faster recovery from failure scenarios, 68% reduced time for regulatory change implementation, and 41% improved resource utilization when evaluated against the Transaction Processing Performance Council's Data Integration benchmark. The approach establishes a future-proof foundation for both regulatory compliance and advanced analytics applications, enabling financial institutions to maintain strict governance while accelerating AI/ML adoption for fraud detection, risk assessment, and customer insights.
Author
- Jyoti Maheshwari
- Independent Researcher USA