The network mainly consists of three modules: the shared teacher encoder, the feature bottleneck, and the student decoder.

The proposed framework operates through two core stages, as illustrated in Figure 2 . The forward distillation stage (yellow arrows) is executed within the shared teacher encoder, where knowledge propagation and hierarchical feature learning are achieved by jointly optimizing the fusion module and deformable convolution network (DCN) [17] [18] . In contrast, the reverse distillation stage (blue arrows) facilitates inverse performance optimization via coordinated training of the feature bottleneck module and the student decoder. Notably, the encoder utilizes a pre-trained model with frozen parameters, while all other modules dynamically update their parameters throughout the training process. The subsequent sections elaborate on the implementation mechanisms of these dual-stage distillation paradigms, followed by a comprehensive description of the loss function design and the anomaly localization methodology.

Forward distillation is carried out in the multi-teacher encoding part. Inspired by the way humans detect anomalies in images, a simple and effective strategy is to compare the samples to be detected with normal samples to find differences. Therefore, we randomly select a pair of images $\left(I_{query},I_{support}\right)$ from the given training set of normal samples in the same category, serving as the query set and the support set respectively, with the number of the support set denoted by k-shot.

During training, the teacher encoder uses the first four stages of WideResNet50 pretrained on ImageNet as the teacher network. First, a query—support image pair is input. The support set is randomly drawn from normal samples and undergoes data augmentation through translation, rotation, scale transformation, etc. Subsequently, these two groups of data are input into the teacher network sharing the same network weights.

We extract the features of the query set and the support set through the shared teacher network. Since the weights of the teacher network are frozen during training, its semantic information is consistent, only differing in spatial distribution. Based on this, we propose an information alignment distillation method under few shot registration to achieve semantic consistency between the support set and the query set. During testing, the semantics of normal regions are easy to align, while those of abnormal regions have large alignment errors. We can use this characteristic to locate abnormal regions. This design not only provides constraints on abnormal regions for subsequent distillation but also effectively suppresses the over generalization of the student network. The feature alignment network involves two key steps: feature fusion and feature alignment, which occur as indicated by the yellow lines in Figure 2 .

In the feature fusion part, the feature maps from the query set and the support set at the same level are collected, denoted as $Q^L$ and $S_k^L$ respectively. Here, $L$ represents the feature maps generated by the encoder at different stages, and $k$ is the number of samples in the support set. The fusion operation of the feature maps is as follows:

$F_{offset}^L=f\left(\left[Q^L,S_k^L\right]\right)=\text{Conv}\left(\text{Concat}\left(Q^L,S_k^L\text{}\right),W\right),L\in \left[1:3\right]$ (1)

In the formula, $\left[\cdot ,\cdot \right]$ represents the concatenation operation of feature maps, $f$ represents the convolution operation, and $W$ is the weight of the convolution kernel. The objective of the fusion stage is to obtain a deviation feature map containing offset information. This feature map is helpful for identifying and locating the deviations between features in subsequent processing.

In the design of the feature alignment network, STN [19] is often the core module. Yet, it has limitations in handling image edges and local details, and CNN also struggles with geometric transformations. In this study, DCN [17] [18] is used to replace STN. DCN introduces learnable offset parameters, which can dynamically adjust the positions of convolutional kernels to achieve adaptive feature alignment. This overcomes the limitations of traditional convolution, improves the ability to model geometric transformations, and enhances the accuracy and robustness of feature alignment.

More precisely, In this study, the support set $S_k^L$ is regarded as the dataset to be registered, and the query set $Q^L$ is taken as the reference dataset. The aim is to precisely align the features between the support set and the query set.

To achieve this goal, a modulated deformable convolution module is adopted and applied to each shot in the support set. Assume that the deformable convolutional kernel has $m$ sampling positions, and its weights and offsets are obtained through pre-learning. The $m$-th position is represented as ${\omega }_m$, $p_m=\left\{\left(-1,-1\right),\left(-1,0\right),\cdots ,\left(1,1\right)\right\}$. For a $3\times 3$ convolutional kernel, the total number of sampling points is defined as 9.

Based on this, the aligned feature ${\tilde{S}}_k^L$ at position $p_0$ in the feature map of layer $L$ can be calculated by the following formula:

${\tilde{S}}_k^L=\text{DConv}\left(S_k^L,\Delta p_m^L\right)={\displaystyle \underset{m=1}{\overset{\text{kernel}}{\sum }}{\omega }_m\cdot \left[S_k^L\otimes \left(p_0+p_m+\Delta p_m^L\right)\right]\cdot \Delta m_m^L}$ (2)

Among them, The symbol $\otimes$ represents the convolution operation between the convolutional kernel and the feature map. $\Delta m_m^L$ is the modulation factor, which is predicted together with $\Delta p_m^L$ by DCNv1 [17] . $\Delta p_m^L$ represents the learnable offset of the $m$-th sampling point in layer $L$, which is obtained by activating the feature map $F_{offset}^L$, that is,

$\Delta p_m^L=\text{softmax}\left(F_{offset}^L\right),L\in \left[1:3\right]$ (3)

For simplicity, only the learnable offset $\Delta p_m^L$ is considered, and the modulation factor $\Delta m_m^L$ is ignored. Since the position $\left(p_0+p_m+\Delta p_m^L\right)$ in the offset calculation may be a decimal, bilinear interpolation is used to obtain the actual offset.

In the alignment task, to tackle complex motion and large parallax, models with a large effective receptive field are found to perform better. Two improvements are proposed:

First, construct an $L$-level pyramid using three downsampling layers of the backbone network to capture long range dependencies and address large parallax.

Second, adopt a cascaded refinement strategy. In layer $L$, combine the ×2 upsampled offsets and features from the previous layer to predict the current layer offsets and features. The specific formulas are as follows:

$\Delta p_m^L=f\left(\left[Q^L,S_k^L\right]\right)\odot {\left(\Delta p_m^{L+1}\right)}^{\uparrow 2}$ (4)

${\tilde{S}}_k^L=\text{DConv}\left(S_k^L,\Delta p_m^L\right)$ (5)

Here, ${(\cdot )}^{\uparrow s}$ represents the upsampling scale factor $s$, and ×2 upsampling is achieved through bilinear interpolation in this case. DConv is the deformable convolution operation defined in Equation (2). subsequent offsets are cascaded to further refine the aligned features (as shown in the part of the green lines in the Figure 1 ).

As shown in the figure, reverse distillation consists of the feature bottleneck module and the student decoder module. The bottleneck module modulates the feature space using the memory-guided mechanism and fuses features with the OCBE (Orthogonal Channel Bottleneck Embedding) method. The student decoder recovers the features embedded in the bottleneck during training.

To increase the feature differences in abnormal modalities between the student and teacher networks, this study integrates a memory guided module into reverse distillation. The structure of the module is shown in Figure 3 .

Define the vector space as $ℝ$ and introduce a memory matrix $M\in {ℝ}^{I\times N}$. Here, $I$ is the number of row vectors in $M$. Each row vector is a memory unit, and its dimension is the same as that of the latent feature vector $z$ output by the encoder.

The memory processing module generates the attention coefficient $\omega$ by calculating the similarity between the query vector $z$ and each memory unit. The calculation formula is as follows:

${\omega }_i=\text{Softmax}\left(d\left(z,m_i\right)\right)$ (6)

$d\left(z,m_i\right)=\frac{z{m}_i^{\text{T}}}{\Vert z\Vert \cdot \Vert m_i\Vert }$ (7)

where $z\in {ℝ}^{1\times N}$ is the query vector, $m_i\in {ℝ}^{1\times N}$ is the $i$-th row vector of the memory matrix $M$, $d\left(z,m_i\right)$ represents the cosine similarity between them, and ${\omega }_i$ is the $i$-th element of the attention coefficient $\omega$.During the training process, only normal samples are used to update the memory matrix $M$.

To avoid the small and dense attention coefficient $\omega$ interfering with the reconstruction of the features of abnormal samples, a sparsification strategy is adopted to eliminate insignificant attention coefficients, which are finally obtained after activation. The specific calculation formulas are as follows:

${{\omega }^{\prime }}_i=\{\begin{array}{cc}{\omega }_i& {\omega }_i>\xi \\ 0,& \text{otherwise}\end{array}$ (8)

${{\omega }^{\prime }}_i=\frac{\text{Relu}\left({\omega }_i-\xi \right)\cdot {\omega }_i}{\left|{\omega }_i-\xi \right|+\delta }$ (9)

where $\xi =0.002$, and $\delta ={10}^{-12}$. Finally, multiply the sparsely processed attention coefficient $\omega$ by the memory matrix to obtain the new latent feature $z^{\prime }$. The formula is as follows:

$z^{\prime }={\omega }^{\prime }M={\displaystyle \underset{i}{\overset{N}{\sum }}{{\omega }^{\prime }}_i{m}_i}$ (10)

Subsequently, a One-Class Bottleneck Embedding (OCBE) module is employed to collect feature information, and its structure is shown in Figure 4 . Before single-stage embedding, OCBE enhances feature diversity with the aid of a Multi-Scale Feature Fusion (MFF) block. First, the module uses a convolutional layer to downsample the shallow layer features. Then, batch normalization and ReLU activation are performed on the downsampled features to align the connected feature representations. Subsequently, a convolutional layer is used for feature dimensionality reduction. Finally, after batch normalization and ReLU activation, a compact feature representation is generated and transmitted to the decoder.

During training, the OCBE module and the decoder are optimized collaboratively to improve the accuracy and robustness of feature representations.

During the training stage, the loss value is calculated based on the loss constraints of features from multiple intermediate layers to simulate the generalization representation of the teacher network for normal samples. The formula for the total loss is:

$L_{total}=Los{s}^1+Los{s}^2+Los{s}^3$ (11)

Here, $Los{s}^1$, $Los{s}^2$ and $Los{s}^3$ correspond to the output features $Q^1$, $Q^2$, $Q^3$ of the query set and the distillation output features $M^1$, $M^2$, $M^3$ respectively. The distillation output feature $M^i\left(i\in 1,2,3\right)$ consists of the output feature ${\tilde{S}}^i\left(i\in 1,2,3\right)$ of the support set after modulation and alignment and the feature $D^i\left(i\in 1,2,3\right)$ output by the student decoder. The loss of the $i$-th layer is:

$Los{s}^i=Loss\left(Q^i,D^i\right)+Loss\left(Q^i,{\tilde{S}}^i\right)$ (12)

Each loss term $Los{s}^i$ is composed of a value loss ( $L_{val}$) and a direction loss ( $L_{dir}$).

During the testing stage, the anomaly map ( $\Omega$) is obtained through calculation, with a size of $w\times h$. The pixel point $x_{i,j}$ in the input feature map represents the feature value at the position $\left(i,j\right)$. The calculation formula for the anomaly map is:

$\Omega \left(x\right)={\displaystyle {\prod }_{n=1}^3\text{Upsameple}\left({\Omega }_{i,j}^n\right)}$ (13)

where ${\Omega }_{i,j}^n$ represents the degree to which the pixel point at the position $\left(i,j\right)$ in the output feature map of the $n$-th convolutional layer deviates from the normal data flow. Finally, the anomaly maps from three different layers are upsampled to a unified size and then multiplied for fusion to generate a comprehensive anomaly detection result.

Our evaluation employs two widely-recognized anomaly detection (AD) benchmarks derived from real industrial scenarios, both designed for defect identification tasks.

In this study, the resolutions of all images in the MVTec AD [20] dataset were adjusted to ensure consistent input. The experiments followed the common procedures in the field of anomaly detection, performing anomaly detection and localization tasks on the data category by category. To optimize memory usage, the size of the memory cache item was set to $M=200$. In terms of the optimization strategy, the Adam optimizer was selected, with hyperparameters $\beta =\left(0.5,0.999\right)$ and an initial learning rate of 0.005. The model was trained for 200 epochs with a batch size of 16.

For industrial image anomaly detection, it’s necessary not only to identify anomalies but also to localize and segment abnormal regions. To comprehensively evaluate the performance of the algorithm, we adopt the following evaluation metrics:

$\text{ImageAUC}=\frac{\text{rank}\left(\text{Output}\cap \text{Label}\right)}{D_n}$ (14)

$\text{PixelAUC}=\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}}$ (15)

$\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}}$ (16)

$\text{F1-Score}=\frac{2\text{Precision}\times \text{Recall}}{\text{Precision}+\text{Recall}}$ (17)

To evaluate the effectiveness of the algorithm in this paper, the US [11] and MKD [12] method based on distillation, and the DiffNet [22] method based on few shot registration are selected as comparative experiments.

According to the AUC index in Table 1 , the BKD (Bidirectional Distillation) algorithm has significant advantages over the MKD and US methods.

Compared with other methods, our method improves the pixel-level and image-level AUROC indices at resolutions of 128 and 256. At high resolutions, the average indices perform well. Notably, at low resolutions, the image-level scores are more prominent. This is due to the fact that our method utilizes bidirectional knowledge distillation, giving full play to the advantages of anomaly detection. The experimental results are presented in Figure 5 .

Meanwhile, to verify the effectiveness of the method using few shot support set registration, the DiffNet [22] method based on few shot registration is selected for comparative experiments. The MVTec dataset is used for the F1-Score index test. One target category is selected, and support sets of different sizes are used for testing. The experimental results are shown in the following Table 2 . Compared with DiffNe [22] , the F1-Score index of BKD on the MVTec dataset has increased by 3.2%, 4.1%, and 3.8% respectively.

In cross-dataset benchmarking across MVTec AD and MPDD, we compared the proposed algorithm with state-of-the-art methods, using average F1-Score as the evaluation metric. Table 3 demonstrate that the synergistic optimization strategy combining few-shot registration and bidirectional distillation achieves remarkable robustness in complex industrial defect detection scenarios, particularly maintaining stable classification boundaries under high-noise and multi-scale deformation conditions.

In this section, an ablation study of anomaly detection and localization is carried out on the MVTEC dataset.

As shown in Table 4 , the “Augment” module refers to support-set augmentation, the “Memory” module is the memory module, and the “DCN” is the feature alignment network. For all categories within the dataset, the macro average AUC values are presented. The methods highlighted in bold demonstrate the best performance.

The results in the table clearly indicate that these modules significantly enhance the detection metrics.

A qualitative analysis from the perspective of the feature space was conducted for the memory module. As shown in Figure 6 , by visualizing the feature maps of the student network, it can be observed that the introduction of the memory module helps to increase the feature distance between anomalies, enabling better anomaly responses.

Additionally, an ablation study was performed on the data augmentation aspect for different transformation versions (when k = 2, T stands for shift, and R stands for rotation). Table 5 shows the macro average AUC values under various data augmentation methods. The best performing methods are shown in bold.

Additionally, under unified conditions (MVTec dataset, 256 × 256 resolution, RTX 4070 Ti GPU, WideResNet50 backbone), As is shown in Figure 7 , BKD achieves the highest F1-Score (87.5%) with 21.3 M parameters and 42.8 ms latency, DiffNet maintains efficiency with 15.4 M parameters and 32.1 ms, while US performs the worst (66.4%), and MKD sits in between with 13.5 M parameters and 40.5 ms. In industrial inspection scenarios, BKD significantly improves detection performance at an acceptable inference time cost, making it suitable for applications sensitive to missed detection rates.