Speaker verification is a subtask of speaker recognition, whose purpose is to verify whether a segment of speech is spoken by a designated speaker [1] [2]. Total variability factor analysis has been widely used in speaker verification [3] [4] [5] [6]. In total variability factor analysis, the speaker and the channel variabilities are contained simultaneously in a low-dimensional space which is referred to as the total variability space. By the space mapping, the useful information can be obtained by reducing the dimensionality of the mean supervector of the Gaussian mixture model (GMM) and the latent variables can be estimated using limited data. The low-dimensional variable characteristic of the speaker’s identity is called the total variability factor vector, or i-vector. Support vector machine (SVM) can be used as a classifier for i-vector [7] [8].

As an application of probabilistic principal component analysis (PPCA), total variability factor analysis only analyzes the speech data from a global perspective [9] [10]. To compensate for the deficiency, we introduced locality preserving projection (LPP) [11], neighborhood preserving embedding (NPE) [12], and discriminant neighborhood embedding (DNE) [13] to speaker verification. By constructing a graph containing the neighborhood information of the speech data, the inherent local neighborhood relationship of the speech data is optimally preserved. Combined with total variability factor analysis, the performance of speaker verification is improved [14] [15]. Here, LPP is an unsupervised learning algorithm [11] [16] that is not concerned with the speaker label information in the dimensionality-reduction process and does not make use of the discriminative information between the speech data of different speakers. However, the speaker label information of the training data and the discriminative information of the speech data are of great importance in speaker verification.

In view of the above shortcomings of LPP, we apply the locality preserving discriminant projection (LPDP) algorithm in speaker verification. LPDP can bring in the speaker label information from the speech data and, through optimization, preserve the inherent local manifold structure of the speech data samples from the same speaker to reduce the distance between them. At the same time, the distance between the speech data samples from different speakers is enlarged to enhance the discriminative ability of the embedding space.

The remainder of this paper is organized as follows. The LPP algorithm based on i-vector is introduced in Section 2. The LPDP algorithm is proposed in Section 3. The experiment and results are presented in Section 4. The conclusion is given in Section 5.

LPDP is an effective manifold learning method that has been successfully applied in face recognition [19]. The basic idea of LPDP is to divide the nearest neighbor graph in the LPP algorithm into intra-class and out-of-class graphs. LPDP can maintain the local neighborhood relationship of the same speaker's speech data samples and reduce the distance between them. At the same time, LPDP emphasizes the discrimination information between speakers and expands the distance between their speech data. Combined with total variability factor analysis, the algorithm can globally and locally analyze the feature structure of speech data more comprehensively, and at the same time reflects the between-speaker difference and enhances the discriminatory ability of the embedding space.

The idea of applying LPDP to speaker verification is similar to that of LPP as shown in Figure 1. The corresponding i-vectors of given N items of training speech data with speaker labels constitute a vector set W = { w 1 , w 2 , ⋯ , w N } , where w i ∈ R D , i = 1 , 2 , ⋯ , N . The purpose of LPDP is to find an optimal locality preserving discriminant projection space matrix A = [ a 1 , a 2 , ⋯ , a K ] and embed the i-vector of the speech in space R^D in the feature-space R^K (K < D). In the R^K space, the speech data point x_i is transformed to y i = A T w i . The steps to train the locality preserving discriminant projection space matrix A are as follows.

Step 1: Determine the neighborhood of the i-vector w_i, which consists of all the i-vectors whose similarity with w_i is less than its average similarity, i.e.,

M S ( w i ) = 1 N ∑ j = 1 N w i T w j ‖ w i ‖ 2 ‖ w j ‖ 2 (2)

N B ( w i ) = { w j | w i T w j ‖ w i ‖ 2 ‖ w j ‖ 2 > M S ( w i ) } (3)

where MS (w_i) is the average similarity of all the N i-vectors for the training speech data with i-vector w_i, and NB (w_i) represents the neighborhood i-vectors of w_i.

Step 2: Construct two subgraphs of the neighborhood graph: the in-class graph Gⁱⁿ and out-of-class graph G^out. In both the in-class graph Gⁱⁿ and the out-of-class graph G^out, the i-th node corresponds to the i-vector w_i. For the in-class graph Gⁱⁿ, we put a directed edge from node i to j if i-vector w_j is in the neighborhood of i-vector w_i and is from the same class as i-vector w_i. For the out-class graph G^out, we put a directed edge from node i to j if i-vector w_j is in the neighborhood of i-vector w_i but is from the different class of w_i.

Step 3: Calculate the weights of the edges in Gⁱⁿ and G^out, and obtain their respective weight matrices, Wⁱⁿ and W^out.

1) Denote the weight of the edge between i-vector w_i and i-vector w_j in Gⁱⁿ as W i j in and choose its value as

W i j in = { exp ( − ‖ w i − w j ‖ 2 t ) s p k ( w i ) = s p k ( w j ) , w j ∈ N ( w i )     or     w i ∈ N ( w j ) 0 other (4)

2) Denote the weight of the edge between i-vector w_i and i-vector w_j in G^out as W i j out and choose its value as

W i j out = { 1 s p k ( w i ) ≠ s p k ( w j ) , w j ∈ N ( w i )     or     w i ∈ N ( w j ) 0 other (5)

Here, spk (w_i) represents the speaker label information of i-vector w_i, and t is the mean distance of all the i-vectors for the training speech data.

Step 4: Calculate the locality preserving discriminant projection matrix A. The idea of LPDP is that, in the embedding space, the i-vectors from the same speaker have the smallest in-class divergence after projection, i.e., the distance between the same speaker’s i-vectors is as small as possible. Conversely, the i-vectors from different speakers have the largest between-class divergence after projection, i.e., they are as far from each other as possible. To achieve these goals, they are integrated into the following two optimization problems [20]:

min ∑ i , j ‖ y i − y j ‖ 2 W i j in = min t r ( A T X L in X T A ) (6)

max ∑ i , j ‖ y i − y j ‖ 2 W i j out = max t r ( A T X L out X T A ) (7)

where L in = D in − W in is a Laplace operator for the in-class graph, Dⁱⁿ is a diagonal matrix, D i i in = ∑ j W i j in , L out = D out − W out is a Laplace operator for the out-of-class graph, D^out is a diagonal matrix, and D i i out = ∑ j W i j out .

Using the constraint condition A T X D out X T A = I , (6) and (7) can be integrated into one optimization problem,

min [ α t r ( A T X L in X T A ) − β t r ( A T X L out X T A ) ] H = α L in − β L out } ⇕ min t r ( A T X H X T A ) (8)

which can be further transformed to a generalized eigenvalue problem,

X H X T A = λ X D out X T A (9)

By solving Equation (9), the locality-preserving discriminant projection space matrix A = [ a 1 , a 2 , ⋯ , a K ] can be obtained, where a 1 , a 2 , ⋯ , a K are the eigenvectors corresponding to the largest K eigenvalues of the above problem.

On the basis of LPP, this paper introduced LPDP to speaker verification. LPDP makes full use of the speaker label information of the speech data to categorize and differentiate the neighborhood. It can overcome the shortcomings of the total variability factor analysis method and maintain the intrinsic local neighborhood relationship of in-class (same speaker) speech data and more comprehensively reflect the global and local structure of the speech data. It can also address the inadequacy of LPP and maximize the distance between out-of-class (different speakers) speech data to obtain the most discriminative feature vector and enhance the discriminative ability of the projection space, thereby improving the recognition performance of the system. Our future work will be devoted to enhance the discrimination of the embedding space and further improve the recognition performance of the system.

This work was supported by the National Natural Science Foundation of China (No.11704229).

The authors declare no conflicts of interest regarding the publication of this paper.

Liang, C.Y., Cao, W. and Cao, S.X. (2020) Locality Preserving Discriminant Projection for Speaker Verification. Journal of Computer and Communications, 8, 14-22. https://doi.org/10.4236/jcc.2020.811002