In previous implicit representation frameworks, the conditioning of 3D query points typically relies on latent codes and sparse keypoint localization, while lacking direct guidance from 2D image features. This absence of image-level conditioning limits the recovery of high-frequency textures and geometric details, and may amplify projection ambiguities in single-image reconstruction.

To bridge this gap, we introduce a multi-scale contextual feature extraction module. Specifically, a pre-trained ResNet-50 [25] backbone is first employed to extract mid-level feature maps F ∈ ℝ H / 8 × W / 8 × C that capture hierarchical facial structures—such as landmark distributions and preliminary texture patterns H / 8 × H / 8 —while maintaining spatial resolution across multiple scales. Before feature extraction, a Dlib-based facial alignment module [26] is applied to normalize the input image into a 256 × 256 canonical frontal view, ensuring geometric consistency across samples.

To further aggregate contextual details and enrich semantic information, we design a multi-branch dilated convolutional module with dense connections. This structure effectively reduces channel redundancy and enhances receptive field diversity for facial geometry optimization. We set the number of branches to K = 5, with a dilation rate sequence of r = { r k } k = 1 K = { 1 , 6 , 12 , 18 , 24 } , thereby covering receptive fields ranging from local to global contexts. The outputs of all branches are concatenated along the channel dimension to form a unified multi-scale representation that encodes both fine-grained and large-scale contextual cues for subsequent implicit modeling.

The extracted image features are sampled through a weak-perspective projection, where the camera intrinsic K ∈ ℝ 3 × 3 and extrinsic parameters [ R | t ] are estimated from 2D facial landmarks via a least-squares optimization. Each 3D query point P is projected onto the corresponding UV coordinate ( u , v ) = π ( p ; K , [ R | t ] ) on the image plane. To mitigate depth ambiguity and sampling artifacts, we adopt a Gaussian-weighted 3 × 3 neighborhood sampling strategy, which aggregates local contextual information around each projected location, formulated as:

where, Ν ( u , v ) denotes the neighborhood centered at the projected point ( u , v ) , and ω ( u ′ , v ′ ) = exp ( − ‖ ( u ′ − u , v ′ − v ) ‖ 2 2 2 σ 2 ) / Z represents the Gaussian weighting coefficient for each sampled pixel. This sampling strategy primarily facilitates 2D - 3D alignment, providing image-guided supervision for the deformation field to enhance the consistency between image features and 3D geometry.

The multi-scale feature maps M are obtained through global average pooling across all dilated convolution branches, followed by a fusion convolution layer that aggregates contextual information from multiple receptive fields. The resulting feature maps are downsampled via bilinear interpolation to a fixed network resolution, effectively reducing computational complexity while preserving essential semantic cues.

Branches with low dilation rates focus on capturing local fine-grained details, whereas those with high dilation rates integrate global structural contours, thereby alleviating the detail loss typically caused by purely latent-code-based implicit representations. The proposed multi-scale feature extraction module thus provides rich multi-scale correlations between local and global facial regions, serving as a strong prior for implicit field learning.

Implicit Query Encoding

The core concept of implicit neural representations (INRs) is to train a neural network to approximate a continuous function f , which represents a 3D surface implicitly via its level set formulation. In this work, we employ a deep signed distance function (SDF) conditioned on latent embeddings of expression and identity to achieve a comprehensive and disentangled facial representation. For each spatial query point x ∈ ℝ 3 , the network outputs a signed distance value:

where, p ∈ R 3 denotes the spatial coordinate of the query point in 3D space, while z i d and z e x p represent the identity and expression deformation fields, respectively. The identity and expression deformation fields are constructed by first mapping each sampled point p into a canonical geometric space. The identity field receives only the canonical coordinates together with the identity latent code as input, and is responsible for modeling the stable and person-specific facial structure. The expression field is driven solely by the expression latent code and predicts localized displacement residuals, whose outputs are combined with the identity field’s canonical geometry to obtain the final dynamic facial surface.

Figure 2. Landmark-Net.

The latent codes are disentangled into identity and expression components through independent multilayer perceptron (MLP) projections. Each MLP consists of three fully connected layers activated by ReLU functions, with shared parameters to enhance model generalization. As shown in Figure 2, a Landmark-Net module is designed to determine the spatial position of query points p within the neural blend field. By conditioning on the input identity latent code, the network predicts a set of canonical facial landmarks, denoted as I = η ( z e x p , z i d ) ∈ ℝ 68 and I ′ = η ′ ( z i d ) ∈ ℝ 68 , which provide geometric priors for both deformation fields. Each of the three-layer MLPs within Landmark-Net guides the local semantic deformation of the identity and expression fields, ε and ζ , respectively.

In this work, we use S E ( 3 ) to represent the facial shape deformation, which exhibits superior capability in modeling jaw articulation and rotation. The corresponding rotation matrix e ω is formulated as:

All deformation fields are implemented using a shared mini-network architecture. A lightweight module conditioned on the spatial coordinates of query points is appended to the end of each mini-network, enabling adaptive local field blending to capture spatially varying deformations.

Implicit neural networks generally exhibit strong performance in representing low-frequency components but struggle to encode high-frequency geometric details. To enhance the network’s capability in modeling fine-grained geometry, we apply a multi-frequency Fourier positional encoding to each spatial point p , expanding the original low-dimensional 3D coordinates into a high-dimensional periodic feature representation:

Each coordinate component is independently encoded, ensuring that the network learns a continuous and smooth deformation field. The encoded positional features are then concatenated with the latent variables z i d and z e x p from Equation (1), forming a composite feature vector:

The introduction of implicit representations expands the latent space and enables precise optimization during single-image fitting, while maintaining identity consistency under facial expression editing.

Traditional approaches typically employ a single MLP decoder that performs a uniform mapping for all spatial points, which makes it difficult to capture region-specific high-frequency details. To address this limitation, we propose an adaptive local-expert decoding mechanism. Specifically, the proposed model preserves the local field structures of the Mini-Nets as backbone modules and introduces multiple independent local MLPs to model different facial regions separately. The human face exhibits significant variations across different regions in terms of geometric morphology, texture complexity, and sensitivity to expression changes. For example, areas such as the regions around the left and right eyes, the mouth, the nasal bridge, and the jaw demonstrate notably distinct geometric dynamics. Therefore, the face is partitioned into eight semantically consistent and structurally stable subregions based on these anatomical characteristics. We define K = 8 local subregions { e k } k = 1 K , each corresponding to a local field in the Mini-Nets Ψ n , which facilitates fine-grained modeling of structural details specific to each region.

By incorporating region-aware specialization, facial regions are partitioned during training using landmark-based guidance on the FaceScape dataset [27], dividing the face into semantically meaningful parts such as the eyes, mouth, and cheeks. Each expert network is trained to learn localized deformations within its assigned region. A two-layer gating network G ϕ receives the implicit feature vector h ( p ) has input and produces soft assignment weights across the K experts. This mechanism adaptively fuses local field predictions, enhancing both the representation capability and training stability of the model while reducing the risk of overfitting in geometrically complex regions.

Finally, the outputs of the local experts are aggregated to produce the local implicit predictions y k ( p ) , which are combined via weighted summation to yield the final signed distance field (SDF) value:

Several loss functions are employed to train the proposed method for learning a reliable facial shape representation.

Reconstruction Loss. A basic SDF structural loss is applied to learn the implicit field:

where s ¯ and n ¯ denote the ground-truth SDF value and field gradient, respectively, Ω i represents the sampled space of the i -th facial scan, and λ is the loss weight coefficient.

Embedding Loss. The embedding vectors are regularized using a zero-mean Gaussian prior:

Smooth Regularization Loss. Since the soft assignment of the gating network may lead to discontinuities or unstable boundaries among expert regions, we introduce a smoothness constraint to enforce local weight consistency. For each sampled neighboring point pair { ( p i , p j ) } :

where Ν denotes the neighborhood set of point p . This term encourages smooth transitions between adjacent regions, such as gradual blending from the forehead to the nose bridge.

Entropy Regularization Loss. To prevent degeneration caused by uniform expert activation in the gating network, an entropy-based regularization is introduced to promote sparse and discriminative expert selection:

Therefore, the total loss is:

FaceScape is a large-scale, high-quality 3D face dataset consisting of 938 individuals and 20 types of facial expressions. It is one of the most comprehensive and high-precision 3D facial datasets available. Among them, data from 365 subjects are publicly released and are used in this work. Specifically, 5323 facial scans from 355 subjects with 15 expressions are used for training, while 200 scans from the remaining 10 subjects with 20 expressions are used for testing.

1) Training

The proposed model is trained end-to-end using the Adam optimizer with an initial learning rate of 0.0001 for 1500 epochs. After 200 epochs, the learning rate decays by a factor of 0.95 every 10 epochs. The ResNet-50 backbone is frozen during the first 10 epochs, and the Mini-Nets component is fixed during pretraining to preserve deformation priors. Training is conducted on a single NVIDIA RTX 4090 GPU for approximately 2 days with a batch size of 72. During testing, optimizing 200 samples on a single GPU takes about 5 hours. Due to the lightweight architectures employed in all components of the proposed framework, the overall inference time is marginally reduced compared to IMFace. Nevertheless, the model is able to preserve and reconstruct finer geometric variations, demonstrating better detail representation without sacrificing computational efficiency.

2) Data Preprocessing

Since the implicit function requires strictly aligned inputs, we adopt the pseudo-watertight mesh generation method from [18]. Facial meshes are rigidly aligned to the frontal view using landmarks and normalized to a 10 cm unit scale. Sampling is performed within a sphere centered 4 cm behind the nose tip with a radius of 10 cm, and points outside the sphere are removed. To construct a directed pseudo-watertight mesh, holes around the nose and mouth are filled, enabling distance transformation to compute SDF values.

We use the proposed model to fit facial scans and compare reconstruction results against the geometric models of FLAME [6], i3DMM [22], and ImFace [18], demonstrating the superiority of our approach. The official FLAME code is used to fit full-face scans in the test set, which includes 300 identity parameters and 100 expression parameters. Since the FaceScape dataset is employed for both training and evaluation in this work, the publicly released models from the dataset are also used for comparison. For Imface, both the identity and expression embeddings are 128-dimensional, consistent with the dimensionality used in i3DMM and in the proposed method. Since the original i3DMM model is trained on only 58 subjects, we retrain it on the same dataset used in this paper for a fair comparison.

3.3.1. Quantitative Analysis

We adopt symmetric Chamfer Distance (CD) and F-score as evaluation metrics, where the F-score threshold is set to 0.001 as a strict criterion. The F-score is a comprehensive measure combining precision and recall, providing an overall assessment of model performance. A smaller Chamfer Distance indicates better reconstruction accuracy, while a higher F-score reflects improved consistency.

The results are summarized in Table 1, showing that our proposed method surpasses competing approaches across both metrics, clearly demonstrating its effectiveness.

Table 1. Quantitative analysis of facial reconstruction accuracy.

3.3.2. Qualitative Analysis

This section visualizes the reconstruction results obtained by different models, as shown in Figure 3, where each column corresponds to a test subject with a non-neutral facial expression.i3DMM is the first deep implicit model designed for human heads; however, when dealing with more complex scenes, it fails to capture intricate deformations and detailed features, resulting in noticeable artifacts on the reconstructed faces. FaceScape, benefiting from high-quality training scans and the inclusion of test subjects within its training set, performs well in preserving identity characteristics. Nevertheless, when handling nonlinear deformations, it tends to produce rigid facial expressions and struggles to capture fine texture details. ImFace achieves better overall performance compared to the above methods, as it incorporates both identity and expression attributes. However, it still lacks precision in reconstructing fine-grained facial structures. In contrast, our proposed method not only reconstructs faces with more accurate identity and expression representations but also maintains geometric robustness while effectively capturing subtle and rich nonlinear facial muscle deformations—such as frowning and pouting—that contribute to more realistic and expressive results.

Figure 3. Comparison of results with i3DMM, FaceScape, and Imface.

3.3.3. Ablation Study

The core components of the proposed algorithm include context feature extraction, local expert decoder, and regularization losses. In this section, we conduct ablation experiments to verify the effectiveness of each key component.

Since the regularization losses are designed to prevent instability in the soft assignments of the gating network, we specifically examine cases where the local expert decoder is used alone or combined with the regularization losses.

Figure 4. Ablation study.

Table 2. Presents the results of the ablation study.

From the ablation experiments in Table 2 above, it can be seen that the core components proposed in this paper are effective improvements. While adding either the feature extraction or the local expert decoder individually is effective, the combined effect of both yields better results. As shown in Figure 4, the algorithm proposed in this paper is more refined and accurate.