TITLE:
A Study of Performance Testing in Configurable Software Systems
AUTHORS:
Xue Han
KEYWORDS:
Configurable Software Systems, Performance Testing, Software Configuration, Performance Bug Study
JOURNAL NAME:
Journal of Software Engineering and Applications,
Vol.14 No.9,
September
13,
2021
ABSTRACT: Customizing applications through program configuration options has been
proved by many open-source and commercial projects as one of the best practices
in software engineering. However, traditional performance testing is not in
synch with this industrial practice. Traditional performance testing techniques
consider program inputs as the only external factor. It ignores the performance
influence of configuration options. This study aims to stimulate research
interest in performance testing in the context of configurable software systems
by answering three research questions. That is, why it is necessary to conduct
research in performance testing, what are the state-of-the-art techniques, and
how do we conduct performance testing research in configurable software
systems. In this study, we examine the unique characteristics and challenges of
performance testing research in configurable software systems. We review and
discuss research topics on the performance bug study, performance anti-patterns,
program analysis, and performance testing. We share the research findings from
the empirical study and outline the opening opportunities for new and advanced
researchers to contribute to the research community.