TITLE:
Entropy—A Universal Concept in Sciences
AUTHORS:
Vladimír Majerník
KEYWORDS:
Probability, Uncertainty, Shannon Entropy, Shannon-Like Entropies
JOURNAL NAME:
Natural Science,
Vol.6 No.7,
April
25,
2014
ABSTRACT: Entropy represents
a universal concept in science suitable for quantifying the uncertainty of a series of random events. We define and describe this notion in an appropriate
manner for physicists. We start with a brief recapitulation of the basic
concept of the theory probability being useful for the determination of the
concept of entropy. The history of how this concept came into its to-day exact
form is sketched. We show that the Shannon entropy represents the most adequate
measure of the probabilistic uncertainty of a random object. Though the notion
of entropy has been introduced in classical thermodynamics as a thermodynamic
state variable it relies on concepts studied in the theory of probability and
mathematical statistics. We point out that whole formalisms of statistical
mechanics can be rewritten in terms of Shannon entropy. The notion “entropy” is
differently understood in various science disciplines: in classical physics
it represents the thermodynamical state variable; in communication theory it
represents the efficiency of transmission of communication; in the theory of
general systems the magnitude of the configurational order; in ecology the
measure for bio-diversity; in statistics the degree of disorder, etc. All these
notions can be mapped on the general mathematical concept of entropy. By means
of entropy, the configurational order of complex systems can be exactly
quantified. Besides the Shannon entropy, there exists a class of Shannon-like
entropies which converge, under certain circumstances, toward Shannon
entropy. The Shannon-like entropy is sometimes easier to handle mathematically
then Shannon entropy. One of the important Shannon-like entropy is well-known
Tsallis entropy. The application of the Shannon and Shannon-like entropies in
science is really versatile. Besides the mentioned statistical physics, they
play a fundamental role in the quantum information, communication theory, in the
description of disorder, etc.