Consider a deep neural network parameterized by θ ∈ ℝ m and a mapping

For input points { x i } i = 1 n , the output is

As θ varies, the set of outputs forms a smooth manifold:

To introduce a rich geometric structure, we embed the output into a RKHS ( ℋ K , 〈 ⋅ , ⋅ 〉 K ) associated with a positive kernel K :

The output manifold becomes an immersed submanifold:

This embedding provides a natural metric induced by the RKHS inner product, a smooth Riemannian structure, access to geometric tools (connection, curvature), and a physically interpretable energy.

The induced metric on S K is

The tangent space is

This structure enables intrinsic gradient flows, geometric energies, and curvature-based regularization.

Embedding the output manifold S K into a RKHS naturally endows it with a Riemannian structure. This allows the definition of:

covariant derivatives, geodesics, intrinsic and extrinsic curvature, intrinsic gradient flows.

These tools are essential for formulating a coherent energetic theory in which learning is interpreted as motion on a manifold equipped with an induced metric.

Consider the simple network

For input points { x i } , the output is

As ( w , b ) vary, the points y ( θ ) trace a smooth curve in ℝ n . Embedding this curve into a RKHS yields Φ ( y ( θ ) ) ∈ ℋ K , and the induced metric becomes

Even this simple network exhibits nontrivial intrinsic geometry.

On any Riemannian manifold ( S K , h ) , there exists a unique connection ∇ satisfying:

Metric compatibility:

Zero torsion:

Theorem 4.1 (Levi-Civita Connection). There exists a unique connection satisfying the two properties above. It is called the Levi-Civita connection.

The Levi-Civita connection can be interpreted as a transport rule that is energy-preserving (metric compatibility) and twist-free (zero torsion). It corresponds to sliding along the manifold without artificial rotation.

The Euclidean gradient ∇ ℋ K E is not tangent to S K . The Riemannian gradient is its tangential projection:

The learning dynamics becomes a geometric flow:

Let f θ ( x ) ∈ ℋ K denote the RKHS representation of the network output. The RKHS gradient of the energy E can be written as

where ℓ is the loss function. The parameter gradient is obtained by the chain rule:

Thus, the usual backpropagation update

can be interpreted as the projection of the ambient RKHS gradient onto the parameter manifold. This provides a geometric bridge between the abstract RKHS gradient and the concrete parameter updates implemented in practice.

For any point p ∈ S K , we have the orthogonal decomposition:

For tangent vector fields X , Y :

Definition 5.1. The second fundamental form is defined by:

It measures how the submanifold bends within the ambient space.

If ℒ = 0 , the manifold is flat in the ambient space. If ℒ is large, the manifold is highly curved. In deep networks, ℒ measures extrinsic sensitivity.

The Gauss decomposition of ∇ X Y into tangential and normal components is illustrated in Figure 1, showing how the ambient direction splits into intrinsic and extrinsic parts.

Figure 1. Gauss decomposition in the RKHS: decomposition of ∇ X Y into tangential and normal components.

We define the extrinsic geometric energy:

It penalizes excessive extrinsic deformation and encourages smoother output manifolds.

Consider the surface:

At the point p = ( 0 , 0 , 0 ) , the surface looks like a shallow bowl. The tangent plane is horizontal, the normal vector is n = ( 0 , 0 , 1 ) , and the mean curvature vector points upward.

The paraboloid example and its upward normal at the origin are shown in Figure 2, highlighting how curvature encodes geometric deformation.

Figure 2. Illustration of the paraboloid z = x 2 + y 2 and its upward normal at the origin.

This example illustrates how curvature encodes geometric deformation in a familiar setting and why controlling extrinsic curvature stabilizes neural representations.

Let { T 1 , ⋯ , T m } be an orthonormal basis of T p S K . The mean curvature vector is defined as:

where ℒ is the second fundamental form. Thus,

The mean curvature measures the tendency of the manifold to bend within the ambient space:

H = 0 : the manifold is minimal, large H : the manifold is highly curved, H indicates the direction of maximal bending.

Figure 3 depicts the mean curvature as the sum of normal components ℒ ( T i , T i ) , providing an intuitive geometric interpretation.

Figure 3. Mean curvature as the sum of normal components ℒ ( T i , T i ) .

We define:

This acts as a membrane tension: minimizing ‖ H ‖ 2 smooths the manifold.

H = 0 : membrane in equilibrium, large H : membrane bent or stretched, minimizing ‖ H ‖ 2 : smoother, more stable representations.

In deep networks, H measures global geometric complexity and the tendency to overfit via highly curved decision boundaries.

Consider a diffusion on S K :

where W t is a Brownian motion and σ ( θ t ) is a diffusion coefficient.

The associated generator is:

The first term encodes energy dissipation along the Riemannian gradient, while the second term encodes intrinsic diffusion on the manifold. Mini-batch noise induces a diffusion on the curved manifold S K , and curvature modulates the amplitude of stochastic fluctuations. Regions of high extrinsic curvature amplify noise, while flatter regions stabilize the diffusion. Thus, geometric regularization shapes both the deterministic and stochastic components of learning.

Consider the PDE:

where ℒ is the geometric generator and V is a potential.

The solution is:

The Feynman-Kac representation provides a probabilistic interpretation of the geometric PDE governing the evolution of u ( θ , t ) . Learning becomes a diffusion process evolving on the curved manifold S K , where the potential V ( θ ) acts as an energetic penalty. The exponential term

plays the role of a Boltzmann weight: trajectories passing through regions of high curvature or high energy are exponentially suppressed, while smoother, low-energy trajectories contribute more significantly.

The geometric structure induced by the RKHS embedding provides a natural way to define a total energy functional that governs learning. This energy combines three components: a data fitting term, an extrinsic curvature term, and a mean curvature term.

Each term has a clear physical meaning:

E data pulls the model toward the data, E geom penalizes extrinsic curvature, E MC acts as a membrane tension,the Riemannian gradient ensures intrinsic consistency.

We consider three families of datasets:

Synthetic datasets: two-moons, concentric circles, intertwined spirals. Image datasets: MNIST and Fashion-MNIST (reduced resolution). Tabular datasets: standard UCI-type classification tasks.

We evaluate several architectures:

MLP-3: fully connected network with three hidden layers. CNN-Small: small convolutional network. ResNet-18 (reduced): lightweight residual network.

Training schemes:

Baseline (ERM + cross-entropy), L 2 regularization, Spectral norm regularization, G-RKHS (extrinsic curvature), G-RKHS+MC (extrinsic + mean curvature).

Hyperparameters ( λ 1 , λ 2 ) are selected by cross-validation. All experiments are repeated over 5 random seeds.

We approximate:

using finite differences and local tangent bases.

Baseline models produce highly curved decision boundaries. Geometric regularization smooths these boundaries and improves accuracy.

The impact of geometric regularization on synthetic datasets is summarized in Table 1, which reports test accuracy together with extrinsic and mean curvature values.

Table 1. Synthetic datasets: test accuracy and curvature (mean over 5 runs).

Gaussian perturbations x ↦ x + ε , ε ~ N ( 0 , σ 2 I ) .

Robustness to Gaussian noise is presented in Table 2, showing that curvature-based regularization significantly improves performance under increasing perturbation levels.

Table 2. Robustness under gaussian noise.

Generalization performance on MNIST, Fashion-MNIST, and tabular datasets is reported in Table 3, where G-RKHS and G-RKHS + MC consistently outperform baseline and classical regularization schemes.

Table 3. Generalization performance (test accuracy in %).

Geometric regularization yields smoother, more stable energy descent and correlates with improved robustness and generalization.

The evolution of the total energy during training for Baseline, G-RKHS, and G-RKHS + MC is shown in Figure 4, illustrating the stabilizing effect of geometric regularization.

Figure 4. Illustrative evolution of the total energy across training epochs for Baseline, G-RKHS, and G-RKHS + MC.

The RKHS-geometry-energy framework coherently connects:

extrinsic differential geometry, stochastic diffusions, energy-based modeling, deep learning.

Learning becomes a geometric flow on an immersed manifold, driven by a total energy balancing data fitting and geometric regularity.

Future directions include:

1) Extension to deep CNNs, transformers, and recurrent networks.

2) Intrinsic optimization based on geodesics.

3) Geometric analysis of gradient dynamics.

4) Adaptive curvature-based regularization.

5) Adversarial robustness via curvature control.

6) Geometric interpretability in sensitive domains.