TITLE:
A Data Analysis Framework for Earth System Simulation within an In-Situ Infrastructure
AUTHORS:
D. Wang, X. Luo, F. Yuan, N. Podhorszki
KEYWORDS:
In-Situ Data Analysis, Source Code Analysis, Data Staging, ADIOS, Earth System Model, Machine Learning, SciKit-Learn, E3SM
JOURNAL NAME:
Journal of Computer and Communications,
Vol.5 No.14,
December
29,
2017
ABSTRACT: This paper presents a generic procedure to implement a scalable and high performance data analysis framework for large-scale scientific simulation within an in-situ infrastructure. It demonstrates a unique capability for global Earth system simulations using advanced computing technologies (i.e., automated code analysis and instrumentation), in-situ infrastructure (i.e., ADIOS) and big data analysis engines (i.e., SciKit-learn). This paper also includes a useful case that analyzes a globe Earth System simulations with the integration of scalable in-situ infrastructure and advanced data processing package. The in-situ data analysis framework can provides new insights on scientific discoveries in multiscale modeling paradigms.