TITLE:
Adaptive Temporal Chinese Restaurant Process for Dynamic Clustering of Evolving Data Streams
AUTHORS:
Lahiru Wickramasinghe
KEYWORDS:
Dynamic Clustering, Chinese Restaurant Process, Online Variational Inference, Nonparametric Bayesian
JOURNAL NAME:
Open Journal of Statistics,
Vol.16 No.6,
November
20,
2026
ABSTRACT: In the era of big data, the inherent temporal dynamics and evolving patterns of high-velocity streaming data present a significant challenge. This paper introduces the Adaptive Temporal Chinese Restaurant Process (AT-CRP), a novel nonparametric Bayesian approach for dynamic clustering of streaming data. The AT-CRP addresses the limitations of traditional clustering methods by modelling the full lifecycle of clusters, including birth, growth, and death, without requiring a fixed number of clusters. I derive an online variational inference algorithm for efficient real-time updates to cluster assignments, providing a structural approach for analyzing complex, evolving data streams. The method is validated on synthetic datasets and real-world NYC traffic data, including a month-long NYC traffic demand dataset and NYC traffic trip data. Experimental results demonstrate AT-CRP’s ability to extract domain-specific knowledge and capture meaningful temporal patterns in urban traffic dynamics. My approach outperforms existing methods in clustering accuracy and pattern detection, offering insights into evolving trends in traffic flow and demand. This work contributes to the field of data mining and pattern recognition by providing a flexible, scalable method for modelling and analyzing large-scale streaming data, with potential applications in urban planning, traffic management, and real-time decision support systems for smart cities.