TITLE:
Improving the Central Information System through Batch Processing Integration: A Scalable, Reliable Hybrid Architecture
AUTHORS:
Ahmed S. Al Mahmeed
KEYWORDS:
Central Information System, Batch Processing, System Integration, Enterprise Architecture, Middleware, Performance Evaluation, Message Queuing, Change Data Capture, Scalability
JOURNAL NAME:
Journal of Computer and Communications,
Vol.14 No.7,
July
27,
2026
ABSTRACT: Central Information Systems (CIS) are critical for organizational data management but exhibit fundamental scalability limits under large-scale workloads exceeding 10 M records/day. Traditional synchronous architectures suffer from latency amplification, with response times degrading from 50 ms to >180 ms when transaction rates exceed 800 TPS. Batch Information Systems (BIS) provide high-throughput automation through parallel ETL pipelines, yet lack the real-time responsiveness required for operational queries. This fundamental tension between throughput and latency creates an integration gap that current middleware solutions address inadequately. This study proposes and validates a hybrid CIS-BIS integration framework using event-driven middleware, transactional message queuing with Kafka 3.6, and Enterprise Service Bus (ESB) abstraction via Apache Camel. The architecture employs change-data-capture (CDC) triggers on PostgreSQL 16 to publish transactional deltas to partitioned topics, enabling exactly-once semantics. Evaluation on dual Intel Xeon 64-core servers with 256 GB RAM and NVMe SSDs demonstrates batch throughput of 5000 records/sec and real-time streaming at 1200 records/sec, with p99 latencies of 120 ms and 45 ms, respectively, and error rates below 0.2%. A 3-month pilot deployment in the PAAET finance department processing 12 M student financial records shows a 40% reduction in month-end closing time, data accuracy improvement from 97.1% to 99.8%, and zero downtime during BIS batch windows. Statistical analysis using paired t-tests confirms significance (p