In practical problems, the datasets are usually nonlinearly separable, so a nonlinear mapping function $\Phi (\cdot )$ is needed to map the original dataset to a high- dimensional Hilbert space, i.e., $x\to \Phi \left(x\right)$. Thus, the nonlinear problem in the original space can be transformed into a linear classification problem in a high- dimensional Hilbert space, and then go on to find the optimal hyperplane. The kernel function is defined as follows:

$K\left(X\mathrm{,}Y\right)=\left(\Phi \left(X\right)\mathrm{,}\Phi \left(Y\right)\right)\mathrm{,}$ (1)

where $\left(\mathrm{,}\right)$ denotes the inner product of Hilbert spaces, each sample $X\mathrm{,}Y\in {ℝ}^{n_1}\otimes {ℝ}^{n_2}$.

Gao et al. [18] proposed a kernel function in which the input data are tensors. This function $\Phi \left(X\right)$ is a nonlinear mapping that maps X to a high-dimensional feature space. Similar to Gao et al. [18] , we define $\Phi \left(X\right)$ as:

$\Phi \left(X\right)=\left[\begin{array}{c}\varphi \left(x_1\right)\\ \varphi \left(x_2\right)\\ ⋮\\ \varphi \left(x_{n_1}\right)\end{array}\right]\mathrm{,}$ (2)

where $x_k$ is the kth row of X. Correspondingly, the following $y_k$ is the kth row of Y. Therefore, this kernel function can be defined as

$\begin{array}{c}K\left(X\mathrm{,}Y\right)=\Phi \left(X\right)\Phi {\left(Y\right)}^{\text{T}}=\left[\begin{array}{c}\varphi \left(x_1\right)\\ \varphi \left(x_2\right)\\ ⋮\\ \varphi \left(x_{n_1}\right)\end{array}\right]{\left[\begin{array}{c}\varphi \left(y_1\right)\\ \varphi \left(y_2\right)\\ ⋮\\ \varphi \left(y_{n_1}\right)\end{array}\right]}^{\text{T}}\\ =\left[\begin{array}{ccc}\varphi \left(x_1\right)\varphi {\left(y_1\right)}^{\text{T}}& \cdots & \varphi \mathrm{<mo\; stretchy="false">\; (\; </mo>}{x}_1\mathrm{<mo\; stretchy="false">\; )\; </mo>}\varphi {\left(y_{n_1}\right)}^{\text{T}}\\ ⋮& \ddots & ⋮\\ \varphi \left(x_{n_1}\right)\varphi {\left(y_1\right)}^{\text{T}}& \cdots & \varphi \left(x_{n_1}\right)\varphi {\left(y_{n_1}\right)}^{\text{T}}\end{array}\right]\mathrm{.}\end{array}$ (3)

Choi derived least square one-class support vector machine (LSOCSVM) [9] which is a least squares version of standard One-Class Support Vector Machine (OCSVM) [5] . Suppose the training samples are $\left\{x_i\mathrm{<mo>\; :\; </mo>}i=\mathrm{1,2,}\cdots \mathrm{,}l\right\}$, and $x_i\in {ℝ}^n$. LSOCSVM can be formulated as the following optimization problem:

$\begin{array}{l}\underset{\omega \mathrm{,}\rho \mathrm{,}\xi }{\min }\frac{1}{2}{\Vert \omega \Vert }^2-\rho +\frac{C}{2}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\xi }_i^2\mathrm{,}\\ \text{s}\text{.t}\text{.}\omega \cdot \varphi \left(x_i\right)=\rho -{\xi }_i\mathrm{,}\end{array}$ (4)

where $\omega$ is the normal vector of the hyperplane and $\rho$ is the intercept of the hyperplane. C is the penalty parameter. Unlike OCSVM [5] , the restriction on slack variable ${\xi }_i\ge 0$ no longer exists. Optimization problem (4) tries to find a

hyperplane which has the maximal distance $\frac{\rho }{{\omega }^2}$ from the origin, and with re-

spect to this distance, ${\xi }_i^2$ are minimized. To solve the optimization problem, Lagrangian multipliers ${\alpha }_i\mathrm{,}i=\mathrm{1,2,}\cdots \mathrm{,}l$ are introduced for these equality constrains. The Lagrangian function can be written as follows:

$L\left(\omega \mathrm{,}\rho \mathrm{,}\xi \mathrm{,}\alpha \right)=\frac{1}{2}{\Vert \omega \Vert }^2-\rho +\frac{C}{2}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\xi }_i^2-{\displaystyle \underset{i\mathrm{<mo>\; =\; </mo>1}}{\overset{l}{\sum }}}{\alpha }_i\left(\omega \cdot \varphi \left(x_i\right)-\rho +{\xi }_i\right)\mathrm{.}$ (5)

Setting the derivatives with respect to the primal variables $\omega \mathrm{,}\rho \mathrm{,}{\xi }_i\mathrm{,}\alpha$ equal to zero, and then simplifying, which gives

${\displaystyle \underset{j=1}{\overset{l}{\sum }}}{\alpha }_j\varphi \left(x_i\right)\varphi \left(x_j\right)-\rho +\frac{{\alpha }_i}{C}=0.$ (6)

Unlike OCSVM, LSOCSVM does not have a decision function. Instead, the hyperplane in (6) itself represents the optimal hyperplane in a regularized least squares sense, where most of training objects may reside. Least squares minimizes the sum of squared errors and establishes an equilibrium between the errors of the individual equations, thus preventing one extreme error from gaining dominance. The computational process is clear and convenient as only the partial derivatives are required to solve the system of linear equations. However, the LSOCSVM method is not directly adaptive to tensor data.

Chen developed the one-class support tensor machine (OCSTM) [22] , which separates most samples of interested class from the origin in the tensor space, with maximal margin. Suppose the training samples $\left\{X_i\mathrm{<mo>\; :\; </mo>}i=\mathrm{1,2,}\cdots \mathrm{,}l\right\}$, every sample $X_i\in {ℝ}^{n_1}\otimes {ℝ}^{n_2}$ which is the data point in 2nd-order tensor space. The decision function of an one-class classifier in the tensor space can be represented as follows:

$f\left(X\right)=sgn\left(u^{\text{T}}\Phi \left(X\right)v-\rho \right)\mathrm{,}u\in {ℝ}^{n_1}\mathrm{,}v\in {ℝ}^{n_2}\mathrm{,}$ (7)

where $\Phi$ is a nonlinear mapping function noted in Section 2.1. OCSTM can be denoted as the following optimization problem:

$\begin{array}{l}\underset{u\mathrm{,}v\mathrm{,}\rho \mathrm{,}\xi }{\min }\frac{1}{2}{\Vert u{v}^{\text{T}}\Vert }^2-\rho +\frac{C}{2}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\xi }_i\mathrm{,}\\ \text{s}\text{.t}\text{.}{u}^{\text{T}}\Phi \left(X_i\right)v\ge \rho -{\xi }_i\mathrm{,}\\ {\xi }_i\ge \mathrm{0,}i=\mathrm{1,2,}\cdots \mathrm{,}l\mathrm{,}\end{array}$ (8)

where $u$ and $v$ can be obtained by iteratively solving the optimization problems.

Although OCSTM is a one-class classification method for tensor, solving optimization problems requires solving quadratic programming problems iteratively, so it is very time-consuming.

Based on the two existing methods in Section 2, we utilize the idea of LSOCSVM to improve OCSTM and propose a new tensor one-class classification method called LSOCSTM. Specifically, we change the loss term in the objective function to a least squares loss, which allows us to replace the QP problem in the solution process with solving a system of linear equations. We use an alternating iteration algorithm, which greatly reduces the number of decision variables. These features make LSOCSTM particularly suitable when it comes to small sample size problems.

Suppose $\left\{X_i\mathrm{<mo>\; :\; </mo>}i=\mathrm{1,2,}\cdots \mathrm{,}l\right\}$ is the training samples, where $X_i\in {ℝ}^{n_1}\otimes {ℝ}^{n_2}$ which is the data point in 2nd-order tensor space. Based on this, LSOCSTM can be formulated as the following optimization problem:

$\begin{array}{l}\underset{u\mathrm{,}v\mathrm{,}\rho \mathrm{,}\xi }{\min }\frac{1}{2}{\Vert u{v}^{\text{T}}\Vert }^2-\rho +\frac{C}{2}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\xi }_i^2\mathrm{,}\\ \text{s}\text{.t}\text{.}{u}^{\text{T}}\Phi \left(X_i\right)v=\rho -{\xi }_i\mathrm{,}\end{array}$ (9)

where C is a trade-off parameter between the margin maximization and empirical error minimization, ${\xi }_i$ is a slack variable. Unlike the hinge loss function of the OCSTM, the loss in our optimization problem is the least squares loss, which is easier to solve. At the same time, like LSOCSVM, the restriction on slack variable ${\xi }_i\ge 0$ no longer exists. Optimization problem (9) tries to find a hyper-

plane which has the maximal distance $\frac{\rho }{{\Vert u{v}^{\text{T}}\Vert }^2}$ from the origin, and with respect to this distance, ${\xi }_i^2$ are minimized.

To solve the optimization problem (9), we introduce Lagrangian multipliers ${\alpha }_i\mathrm{,}i=\mathrm{1,2,}\cdots \mathrm{,}l$ for these equality constrains. The Lagrangian function can be written as follows:

$L\left(u\mathrm{,}v\mathrm{,}\rho \mathrm{,}\xi \mathrm{,}\alpha \right)=\frac{1}{2}{\Vert u{v}^{\text{T}}\Vert }^2-\rho +\frac{C}{2}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\xi }_i^2-{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\alpha }_i\left(u^{\text{T}}\Phi \left(X_i\right)v-\rho +{\xi }_i\right)\mathrm{.}$ (10)

Setting the derivatives with respect to the primal variables $u\mathrm{,}v\mathrm{,}\rho \mathrm{,}\xi$ equal to zero, we have

$\frac{\partial L}{\partial u}=0\Rightarrow u=\frac{1}{{\Vert v\Vert }^2}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\alpha }_i\Phi \left(X_i\right)v\mathrm{,}$ (11)

$\frac{\partial L}{\partial v}=0\Rightarrow v=\frac{1}{{\Vert u\Vert }^2}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\alpha }_i\Phi {\left(X_i\right)}^{\text{T}}u\mathrm{,}$ (12)

$\frac{\partial L}{\partial {\xi }_i}=0\Rightarrow {\xi }_i=\frac{{\alpha }_i}{C}\mathrm{,}$ (13)

$\frac{\partial L}{\partial \rho }=0\Rightarrow {\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\alpha }_i=1.$ (14)

It can be seen from (11) and (12), $u$ and $v$ are interdependent and cannot be solved independently of each other. We develop alternating projection algorithm to solve our problem, and the specific solution procedure is as follows:

First, we fix $u$ and then let ${\mu }_1={\Vert u\Vert }^2$. After that, we denote that $x_i=\Phi {\left(X_i\right)}^{\text{T}}u$. Optimization problem (9) can be rewritten in the following form:

$\begin{array}{l}\underset{v\mathrm{,}\rho \mathrm{,}\xi }{\min }\frac{1}{2}{\mu }_1{\Vert v\Vert }^2-\rho +\frac{C}{2}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\xi }_i^2\\ \text{s}\text{.t}\text{.}{v}^{\text{T}}{x}_i=\rho -{\xi }_i\end{array}$ (15)

The Lagrangian function can be rewritten as follows:

$L\left(v\mathrm{,}\rho \mathrm{,}\xi \mathrm{,}\alpha \right)=\frac{1}{2}{\mu }_1\left(v^{\text{T}}v\right)-\rho +\frac{C}{2}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\xi }_i^2-{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\alpha }_i\left(v^{\text{T}}{x}_i-\rho +{\xi }_i\right)\mathrm{.}$ (16)

By setting the derivatives with respect to the primal variables $v\mathrm{,}\rho \mathrm{,}\xi \mathrm{,}\alpha$ equal to zero, we have

$\frac{\partial L}{\partial v}=0\Rightarrow v=\frac{1}{{\Vert {\mu }_1\Vert }^2}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\alpha }_i\Phi {\left(X_i\right)}^{\text{T}}u\mathrm{,}$ (17)

$\frac{\partial L}{\partial {\xi }_i}=0\Rightarrow {\xi }_i=\frac{{\alpha }_i}{C},$ (18)

$\frac{\partial L}{\partial \rho }=0\Rightarrow {\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\alpha }_i=1,$ (19)

$\frac{\partial L}{\partial {\alpha }_i}=0\Rightarrow v^{\text{T}}{x}_i-\rho +{\xi }_i=0.$ (20)

Substituting (17) and (18) into (20), we get

$\frac{1}{{\mu }_1}{\displaystyle \underset{j=1}{\overset{l}{\sum }}}{\alpha }_j{u}^{\text{T}}K\left(X_j\mathrm{,}{X}_i\right)u-\rho +\frac{{\alpha }_i}{C}=0.$ (21)

Coupled with the constraint in (19), equations in (21) can be reduced to the following system of linear equations to solve:

$\left[\begin{array}{cc}0& e^{\text{T}}\\ e& \frac{M}{{\mu }_1}+\frac{I}{C}\end{array}\right]\left[\begin{array}{c}-\rho \\ a\end{array}\right]=\left[\begin{array}{c}1\\ 0\end{array}\right]\mathrm{,}$ (22)

where vectors $e$ and 0 represent all one and all zero column vectors of l dimension, I is the identity matrix, $a$ is the column vector of Lagrangian multipliers $\left[{\alpha }_1\mathrm{,}\cdots \mathrm{,}{\alpha }_l\right]$ and M denotes a new matrix which $M_{ij}=u^{\text{T}}K\left(X_j\mathrm{,}{X}_i\right)u$.

After simple algebraic operations, we can get

$a^{\mathrm{<mo>\; *\; </mo>}}=\frac{{\left(\frac{I}{C}+\frac{M}{{\mu }_1}\right)}^{-1}e}{e^{\text{T}}{\left(\frac{I}{C}+\frac{M}{{\mu }_1}\right)}^{-1}e}\mathrm{,}$ (23)

$v=\frac{1}{{\mu }_1}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\alpha }_i^{\mathrm{<mo>\; *\; </mo>}}\Phi {\left(X_i\right)}^{\text{T}}u\mathrm{,}$ (24)

${\Vert v\Vert }^2=\frac{1}{{\mu }_1^2}{\displaystyle \underset{i\mathrm{,}j=1}{\overset{l}{\sum }}}{\alpha }_i^{\mathrm{<mo>\; *\; </mo>}}{\alpha }_j^{\mathrm{<mo>\; *\; </mo>}}{u}^{\text{T}}K\left(X_i\mathrm{,}{X}_j\right)u\mathrm{.}$ (25)

Secondly, to calculate $u$, let ${\mu }_2={\Vert v\Vert }^2$ and ${x^{\prime }}_i=\Phi \left(X_i\right)v$. The optimal problem for $u$ can be constructed as follow:

$\begin{array}{l}\underset{u\mathrm{,}\rho \mathrm{,}\xi }{\min }\frac{1}{2}{\mu }_2{\Vert u\Vert }^2-\rho +\frac{C}{2}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\xi }_i^2\mathrm{,}\\ \text{s}\text{.t}\text{.}{u}^{\text{T}}{x^{\prime }}_i=\rho -{\xi }_i\mathrm{.}\end{array}$ (26)

As with the arithmetic procedure from (15) to (21), we can get

$\frac{1}{{\mu }_2}{\displaystyle \underset{j=1}{\overset{l}{\sum }}}{\alpha }_j{x^{\prime }}_j^{\text{T}}{x^{\prime }}_i-\rho +\frac{{\alpha }_i}{C}=0.$ (27)

Similar to the treatment of (21), equations in (27) can be reduced as follow linear equation:

$\left[\begin{array}{cc}0& e^{\text{T}}\\ e& \frac{Q}{{\mu }_2}+\frac{I}{C}\end{array}\right]\left[\begin{array}{c}-\rho \\ a\end{array}\right]=\left[\begin{array}{c}1\\ 0\end{array}\right]\mathrm{,}$ (28)

where vectors $e$ and $0$ represent all one and all zero column vectors of l dimension, and $Q$ denotes a new matrix which $Q_{ij}=u^{\text{T}}{\displaystyle {\sum }_{p=1}^l}K\left(X_p\mathrm{,}{X}_j\right){\displaystyle {\sum }_{q=1}^l}K\left(X_i\mathrm{,}{X}_q\right)u$.

By solving the linear equation in (28), we can get

$\hat{a}=\frac{{\left(\frac{I}{C}+\frac{Q}{{\mu }_2}\right)}^{-1}e}{e^{\text{T}}{\left(\frac{I}{C}+\frac{Q}{{\mu }_2}\right)}^{-1}e}\mathrm{,}$ (29)

and then we can update $u_{new}$ by

$u_{new}=\frac{1}{{\mu }_2}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\hat{\alpha }}_i\Phi \left(X_i\right)v=\frac{1}{{\mu }_1{\mu }_2}{\displaystyle \underset{i\mathrm{,}j=1}{\overset{l}{\sum }}}{\hat{\alpha }}_i{\alpha }_j^{\mathrm{<mo>\; *\; </mo>}}K\left(X_i\mathrm{,}{X}_j\right)u\mathrm{.}$ (30)

Thus, $u$ and $v$ can be obtained iteratively. We mention training samples with non-zero ${\alpha }_i^{\mathrm{<mo>\; *\; </mo>}}$ are support tensors, labeled as $X_{i\mathrm{<mo>\; \_\; </mo>}s}$. Therefore, $\rho$ and $f\left(X\right)$ can be calculated as follow:

$\rho =mea{n}_{i\mathrm{<mo>\; \_\; </mo>}s}\left(\frac{1}{{\mu }_1}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\alpha }_i^{\mathrm{<mo>\; *\; </mo>}}{u}^{\text{T}}K\left(X_i,X_{i\text{\_}s}\right)u\right)$ (31)

$f\left(X\right)=\frac{1}{{\mu }_1}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\alpha }_i^{\mathrm{<mo>\; *\; </mo>}}{u}^{\text{T}}K\left(X,X_i\right)u-\rho$ (32)

The maximum value of $f\left(X_{i\text{\_}s}\right)$ is the threshold for distinguishing whether it is a target class or not. Substitute the test sample X into $f$. If it is less than this maximum value, the test sample belongs to the target class.

Substitute $f$ for X in the training set and compare the minimum values of these $f$ with $f$ in the test set.

LSOCSTM is a new tensor based one-class classification method that has two advantages. One is that it can directly use tensor data as input to avoid structural information loss and overfitting problems caused by vectorization. Secondly, it replaces the loss function with the least squares loss, thereby transforming the optimization problem into solving a system of linear equations, greatly reducing the complexity and time consumption of operations.

Lemma 1. The iterative process of the optimization problems (15) and (26) will make $\frac{1}{2}{\Vert u{v}^{\text{T}}\Vert }^2-\rho +\frac{C}{2}{\displaystyle {\sum }_{i=1}^l}{\xi }_i^2$ in the optimization problem (9) monotonically decreasing, and thus the LSOCSTM algorithm converges.

Proof. Define the function as follows:

$f\left(u\mathrm{,}v\right)={\Vert u{v}^{\text{T}}\Vert }^2-\rho +\frac{C}{2}{\displaystyle \underset{i=1}{\overset{l}{\sum }}}{\xi }_i^2$ (33)

Since LSOCSVM is convex optimization, the optimization problems (15) and (26) are also convex optimization problems. Therefore, their solutions are globally optimal. Assume that the initial value of $u$ is $u_0$, and solve the optimization problem (15) to obtain the initial value of $v$ as $v_0$, and then solve the optimization problem (26) to obtain the solution $u_1$ for one iteration. Thus we can obtain:

$f\left(u_0\mathrm{,}{v}_0\right)\ge f\left(u_1\mathrm{,}{v}_0\right)$ (34)

Repeating the above procedure gives:

$f\left(u_0\mathrm{,}{v}_0\right)\ge f\left(u_1\mathrm{,}{v}_0\right)\ge f\left(u_1\mathrm{,}{v}_1\right)\ge f\left(u_2\mathrm{,}{v}_1\right)\ge \cdots$ (35)

And since f is constantly greater than or equal to 0, f converges.

In this section, we present our experimental results and compare the performance of least squares one-class support tensor machine (LSOCSTM) with three methods: OCSTM, LSOCSVM and OCSVM. All of our datasets are downloaded from UCI repository. Parameter m presents the total sample size, n presents the dimensionality of the vector sample and $n_1\times n_2$ shows the tensor size of our datasets. For each dateset, features are scaled to [−1, 1]. Since we consider one-class classification problem, our focus is only on the one target class. Table 1 summarizes the information of the datasets.

We choose Radial Basis Function (RBF) kernel function for the proposed method since it has been proven that RBF kernel function performs better than the other kernels [24] . For each classification problem, ten independent runs are performed and the average classification accuracy (ACC) and AUC [25] on the

sets can be calculated and then averaged. It is well known that tensor methods have obvious advantages when dealing with small sample problems [26] , thus, our training sample size l is set to two cases: $l=3$, $l=6$. All the algorithms have been implemented in MATLAB R2019b on Windows 11 running on a PC with system configuration AMD Ryzen 7 5800H (3.2 GHz) and 16 GB of RAM.

There is only one hyper-parameter in our work, so for each test, we train every machine by choosing C from {0.001, 0.01, 0.1, 1, 10, 100} for 3 times, and then the best C for every test could be found. In Table 2 , for example, in Iris Dataset Class 1, 0.01 is the optimal value.

In this subsection, we focus on the size selection of tensors. For example, for each sample $x\in {ℝ}^n$, a suitable tensor size needs to be found. In our work, we transform the vectors into second-order tensors $X\in {ℝ}^{n_1}\times {ℝ}^{n_2}$ ( $n_1\times n_2\approx n$). Cai [21] provided a method to establish $n_1$ and $n_2$, which was simplified as minimizing $n_1+n_2$ while $\left(n_1-1\right)\times n_2\le n\le n_1\times n_2$. Under such conditions, there are still many choices of $n_1$ and $n_2$. For example, in Ionosphere Dataset, there are 5 choices of sizes available. Generally all these types can be used. Therefore, it is worth finding out which one is the best. Table 3 below shows the performance of these 5 choices. These experiments suggest that, $n_1$ and $n_2$ should be as close as possible. In the meanwhile, $n_1\le n_2$ can be a good choice and it is particularly suitable for small sample size problem.

Figure 1 is the 2-norm of the differences of $u$ between two adjacent iterations from Sonar Dataset Class 1. It can be seen that after 20 iterations, the value converges to 0. This also verifies that Lemma 1 is valid.

In this subsection, we compared our method with the three methods mentioned in the first subsection. The performance of the four methods with training set sample sizes of 3 and 6 is shown in the following Table 4 and Table 5 .

From the two tables, we can find that our method has the best results in most cases. Meanwhile, the two tensor classification methods are significantly better than the vector methods. From the tendency of the averaged test accuracy (ACC) and AUC, we can conclude that tensor based classifiers LSOCSTM and OCSTM have significant advantages over all vector based classifiers in all datasets, especially for small training sample size cases. It also supports the conclusion that tensor-based methods are suitable for dealing with small sample problems.

Table 6 summarized the time required for each method. From Table 6 , it can be found that the proposed method has a significant advantage over the tensor-based method OCSTM in terms of time duration. Although the proposed method takes more time than the vector-based method due to iteration and computation involved kernel functions, the accuracy of classification is also significantly higher than theirs.