The foundational system-identification approach using least-squares regression is rooted in auto-regressive (AR) models which are used to represent a stochastic discrete time process. When a sampled discrete-time signal is represented by Equation (1).

A reduced representation having p parameters can be calculated by using the pseudo inverse employed in the matrix-algebra formulation of Gauss’s least-squares technique using the following matrix equation. (See Equation (2))

wherein its instantiation of the regressed form is realized using a time-shifted variant of Equation (1). This is a strange reformulation of the curve-fitting exercise for a line, quadratic or cubic. It is applicable principally to stochastic signals like the one in Figure 1Figure 1. While there are variants like Auto-Regressive Exogenous (ARX), Auto-Regressive Moving Average Exogenous (ARMX), to handle signals from systems driven by external/exogenous variables, the core of the technique

Figure1. A stochastic signal in time.

will have utility limited to stochastic applications. Works purporting to introduce the field of system identification [2] expose almost exclusively stochastic AR approaches. Even the work in [3] entitled with the words “System Identification: A Frequency Domain Approach” does this and omits the novelty claimed in this thesis.

Because there are practical physical systems heavily governed by a stochastic element (such as gear and bearing vibration), AR models are nevertheless an important starting point in creating diagnostic and prognostic techniques applied to real in-situ machinery operation. Augmentations of the techniques to account for how such systems are forced (with time-varying speed and load) are essential to their successful use in practice.

Most practical machines will be governed by a heavy deterministic aspect representable using a system of differential equations – possibly with such stochastic added on.

The field of artificial intelligence (AI) over the past 5 years has grown to include almost anything clever a computer programmer creates with software. Prior to that, AI research as it relates to the subject of diagnostics, was focused on machine learning techniques like neural networks (NN).

Figure2. A neural network.

“There is nothing particularly magical about multi-layer neural networks; they implement linear discriminants, but in a space that has been mapped non-linearly.” [4] The decision surface reflects this fact. (See Equation (3))

and the “back-propagation learning algorithm” is applied to it to minimize the classification error by setting the weights in this function appropriately for the training set.

Neural networks can be considered as a network of nodes similar to the brain's neurons as in Figure 2Figure 2. The transitions in the network are equivalent to the weights in the expression for the decision surface and the nodes are the non-linear mappings given by the function in Equation (3) which is selected by the user to provide good fitting of the data. This view on NNs allows for variances for forecasting through recursive NNs and other techniques like self-organizing maps and auto-encoders (See Figure 3Figure 3).

Figure 3. Novelty detection using support vectors instead of neural networks.

The notion of a neural network is an abstraction of a very concrete mathematical notion—fitting a curve to data. The network is the curve’s equation represented in graphical form and its coefficients are learned through optimization techniques in an abstract parallel to how the brain’s neurons strengthen over time.

Gauss’s foundational least-squares curve fitting technique already discussed likely started the field. Whereas it seeks to fit a curve to represent a set of data, a neural network seeks to fit a curve to separate data or, similarly, to classify it. Figure 4Figure 4 demonstrates how data from two classes (e.g. pike and walleye) might be separated with a decision boundary using the length and weight of the fish as classification parameters (in the figure: x,y). Neural networks will generally create much more sophisticated boundaries than just a line.

Using the length and weight as classification features for fish should seem quite dubious since the data spaces of each class will intersect heavily—making it difficult to find a curve that will classify/separate the two. The space of machine learning is challenged by this problem of feature selection with a great variety of techniques that range from computational mathematics to domain-specific analysis.

Figure 4. Neural Networks—Curve fits to separate 2D data from two classes for classification.

The biologist might add features like the number of spines on the fish’s fins to its length and weight—increasing the dimensionality of the feature space which will require a greater amount of data to create good classification results (a problem known as the curse of dimensionality); while the computer scientist might create techniques like principle component analysis to reduce that data’s dimensionality to a smaller one.

In the field of machine diagnostics, combinations of auto-regressive models and machine-learning techniques are often used to digest signals (e.g. vibration) measured from the physical machine into machine-learning algorithms for classification results. A special subset of machine learning, called one-class classification or novelty detection, is predominantly used to acknowledge that one cannot break production machinery in every way it might fail to gather data on how the machine “sounds” in those faulty states. Instead, behavior of the normal operating state is gathered and then fit to a decision boundary that circumscribes the data of the normal operating state. Figure 3Figure 3 demonstrates the notion of this approach using support vectors as the machine-learning technique rather than neural networks.

Once the boundary is learned, it can be used to classify the data from machinery signals operating in situ to generate a novelty score. The decision boundary in Figure 3Figure 3 can be seen in the plane of the graph in Figure 5Figure 5 which also includes 3-dimensional exposure of the consequences of a novelty score; healthy areas of operation are shown in red while unhealthy (i.e. novel) ones are shown in blue with increasing intensity. If this seems ideal, the reader should be cautioned that the result is only a magic number and has limited physical meaning. For the purposes of prognostics, attempting to measure the time to failure, the novelty score has some but limited merit. Nevertheless, it does offer some possibilities for this domain.

Figure 5. Novelty Score from classification boundary.

This type of application suffers from a double curse of dimensionality – since to properly characterize it one must gather both adequate data to represent the machine’s operation but also over all its operating modes (e.g. for changes in load and speed for a gearbox). Creating a diagnostic rule that does not account for these parameters could miss significant classification opportunities—ones that would distinguish a machine operating under heavy duty cycles vs one that is faulted.

Many machine signals are periodic with frequency components that are phase-locked to the various shaft speeds and their harmonics, like gear-mesh frequencies. Others, like bearing signals, are tied to shaft speed with some randomness to their signals due to slippage.

These issues require care in online monitoring of steady-state machinery. When varying speed and load are introduced these concerns are greatly exacerbated. Vibration in most mechanical systems involves the periodic oscillation of energy from potential to kinetic which arises when under-damped mechanical systems are displaced from their equilibrium position and released. Oscillations will occur at the damped natural frequency of the system which can be estimated by an amalgamation of spring-like elements representing the equipment. The system will move about its frequency response curve according to changes in speed and load; vibrations will thus move through regions of damping or resonance. As Stack notes in [5], “when the machine and its bearings are healthy, these deviations in vibration are less pronounced and usually go unnoticed. However, as bearing health degrades, these deviations in machine vibration due to speed become quite significant.” Figure 6Figure 6 shows a reproduction of his results—damping in undamaged machinery is largely insensitive to speed/load changes while damaged machinery is very sensitive to these parameters. A damaged bearing could be operated at maximum load and its vibratory levels could fall below the alarm threshold despite its damaged state while operation at a lower load could greatly exceed the same threshold.

Figure 6. A health threshold in the context of healthy and undamaged machinery as a function of duty (sourced from Stack 2003).

Successful attempts to solve these problems can be found in the references but there is still much room for solutions that solve a broader range of diagnostic and prognostic challenges. The potential of system identification to take a system’s inputs and outputs and infer its physical parameters (e.g. mass, spring stiffness, damping coefficient, capacitance, inductance, etc.) would be a silver bullet in this domain. A solution for prognostics follows almost immediately from the diagnostic work since the inferred parameters can be tracked to a threshold set by the system engineer.

This paper’s thesis is mirrored in the foundational work in frequency-domain analysis by Joseph Fourier [6] from 1878. Fourier provided the foundational solution to the problem that many researchers [7] had been exploring – trying to find a way to fit a series of sinusoids to some generic curve. The problem was not identified by him but his solution is the predominant one used today – along with the almost identical one created by Laplace (exposed in rectangular coordinates rather than polar).

The modern exposition of the Fourier transform uses imaginary numbers and is explained, in part, through complex notions of Gram-Schmidt orthogonalization [8] both of which were not used by Fourier himself. In section II of [6] entitled “First example of the use of trigonometric series in the theory of heat” at page 137, Fourier states the problem of fitting some generic function with a series of cosines (where the generic function is simplified to the value of one for exemplar purposes). (See Equation (4))

which is one equation with four unknowns (a, b, c and d) excluding a possible multitude of such sinusoids represented by &c. To create more equations with a commensurate number of unknowns, Fourier takes the derivative of the equation with respect to y three times. (See Equation (5))

Of course, the derivative of cos y is − sin y and successive applications of this approach will result in alternating negative signs which is analogous to the result when one multiples j (or the − 1 ) by itself successively. Fourier did not feel the need to use j in his exposition of the Fourier Series or Transform – although it appears in other parts of his work.

To continue the process started by Equation (5), Fourier then takes the further rudimentary step of reducing these equations by setting y = 0 which produces. (See Equation (6))

Some of the equations in Equation (5) disappeared when substituting in y = 0 so further equations need to be created by subsequent differentiation. Equation (6), having four equations and four unknowns, can now be solved.

Fourier’s exposition follows the pattern of using simple examples and then merging many complex steps to achieve his final result. Fourier’s collegial sharing of this simple intuition then later turns into sophisticated analysis to extend the technique from the trivial example in Equation (4) to a transform that can be applied to some generic function. In section VI of his work, entitled “Development of an arbitrary function in trigonometric series,” he uses Taylor series to expand this intuition to a generic function to create the Fourier Series of a generic function. Later in the same work, he creates an early version of the well-known Fourier Transform. These derivations are not repeated here because the intuition of the basic technique was already exposed with enough detail hopefully to convince the reader that the overly simplistic nature of this work’s thesis/novelty is plausible.

The intuition of Fourier’s groundbreaking work resonates in the Hartley transform [9] which is an equivalent to the Fourier and Laplace Transforms except without imaginary numbers using purely sinusoids. Hartley’s transform is scarcely touched in the research but it is well established theory with an equivalent Fast Hartley Transform for computational efficiency.