This paper considers the following composite convex optimization problem:

where f : ℝ d → ℝ ∪ { + ∞ } is a proper closed convex function that is differentiable on the open effective domain X = dom   f , g : ℝ d → ℝ ∪ { + ∞ } is a proper closed convex function satisfying dom   g ⊆ X .

Problem (1) has wide applications in fields such as compressed sensing, sparse phase retrieval [1][2], network quantization [3], and machine learning. Currently, researchers have proposed various classical methods for solving problem (1), including the subgradient algorithm, splitting algorithms [4], and the proximal gradient method [5].

The proximal gradient method (PGD) combines gradient descent with the proximal operator to solve optimization problems with nonsmooth regularization terms by decomposing complex objective functions. Its iterative scheme is as follows:

where λ t > 0 is the step size parameter, and prox λ t , g denotes the proximal operator of the function g defined as

In the convergence theory analysis of traditional PGD, it is often required that the gradient satisfy Lipschitz continuity. Under this smoothness condition, if f is convex, the algorithm can achieve a sublinear convergence rate of O ( 1 / k ) .

However, the classical smoothness condition of gradient Lipschitz continuity is relatively strict. For functions with rapidly changing curvature or uneven local geometric structures, this assumption is often difficult to satisfy. Particularly in machine learning and large-scale optimization, many objective functions, although differentiable or even twice differentiable, may not have globally Lipschitz continuous gradients. Even in some standard models, the global smoothness constant may be unbounded [6], such as the ℓ 2 -regression function. Additionally, when the function f is an augmented Lagrangian function or the dual function of the original problem (which is not necessarily uniformly convex), the objective function may also exhibit nonsmooth or non-uniformly smooth characteristics [7][8]. Therefore, how to establish algorithm convergence theory under conditions weaker than classical smoothness remains an important question.

Significant progress has been made in research on weakening the classical smoothness assumption. In 2020, Zhang et al. [9], based on an in-depth study of the training process of deep neural networks such as LSTM [10] and ResNet [11], first proposed a non-uniform smoothness condition— ( L 0 , L 1 ) -smoothness—to characterize the dependence between the variation of the function gradient and the gradient norm. Compared with the classical smoothness condition of gradient Lipschitz continuity, this type of condition allows the local smoothness degree to vary with the position of the iteration point or the gradient norm, thus covering a broader class of functions, such as univariate polynomials and exponential functions. Under this generalized smoothness condition, Zhang et al. proved the convergence rate of gradient clipping [12] and demonstrated its advantages over gradient descent methods in neural network training. Since the introduction of this generalized smoothness condition, related research has been extensively developed in various fields, including minimax optimization [13], bilevel optimization [14], and variational inequalities [15].

On this basis, Chen et al. [16] further proposed a more general class of symmetric generalized smoothness conditions— α -symmetric ( L 0 , L 1 ) -smoothness. This concept generalizes existing smoothness definitions and covers many mainstream machine learning problems and important function classes, such as distributionally robust optimization [17][18], higher-order polynomials, and exponential functions. Under this framework, the researchers established corresponding descent lemmas for different values of α . On this basis, to solve non-convex optimization problems, Chen et al. designed several deterministic normalized gradient descent algorithms [19], which achieve an optimal complexity of O ( ϵ − 2 ) . Meanwhile, they found that under the stochastic setting, the classic variance reduction algorithm SPIDER [20] can also achieve the optimal sample complexity of O ( ϵ − 3 ) .

In the same year, Li et al. [21] also generalized the ( L 0 , L 1 ) -smoothness condition by replacing the original affine function L 0 + L 1 ‖   ⋅   ‖ with a more general non-decreasing continuous function ℓ ( ⋅ ) to characterize the dependence between the norm of the Hessian matrix and the gradient norm, thereby proposing a more generalized smoothness concept— ℓ ( ⋅ ) -smoothness. That is, there exists a non-decreasing continuous function ℓ ( ⋅ ) such that f satisfies

This generalization significantly expands the applicability of the generalized smoothness theory, enabling researchers to handle function classes with more complex curvature variations. Based on this new framework, Li et al. systematically analyzed the convergence rates of gradient descent methods in convex, strongly convex, and non-convex settings. Notably, under the condition that the objective function is convex, they further proved that Nesterov’s accelerated gradient method [22] achieves an optimal computational complexity of O ( ϵ − 0.5 ) , which exactly matches the optimal bound under the classical smoothness condition. In addition, they also conducted an in-depth study of stochastic non-convex optimization problems and, under the assumption of bounded variance, proved that stochastic gradient descent achieves an optimal complexity of O ( ϵ − 4 ) , which also matches its optimal bound under the classical smoothness condition.

This paper mainly conducts research in the finite-dimensional Euclidean space ℝ d . Let X ⊆ ℝ d be a non-empty convex set. For any x , y ∈ ℝ d , the inner product is denoted by 〈 x , y 〉 , and the induced norm is defined as ‖ x ‖ : = 〈 x , x 〉 . In this thesis, B ( x , R ) denotes the Euclidean ball centered at x with radius R .

Definition 1( ℓ -smoothness [21]). Let f : X → ℝ be a real-valued differentiable function. If there exists a non-decreasing continuous function ℓ : [ 0 , + ∞ ) → ( 0 , + ∞ ) such that the spectral norm of the Hessian matrix of f satisfies

almost everywhere on X (with respect to the Lebesgue measure), then f is called an ℓ -smooth function.

Remark. When the function ℓ is a constant function ℓ ( μ ) = L , the definition of ℓ -smoothness degenerates to the classical smoothness definition; when ℓ is an affine function ℓ ( μ ) = L 0 + L 1 μ , the ℓ -smoothness corresponds to the ( L 0 , L 1 ) -smoothness definition [9].

Definition 2(( r , ℓ )-smoothness). Let f : X → ℝ be a real-valued differentiable function. Suppose there exist a non-decreasing continuous function ℓ : [ 0 , + ∞ ) → ( 0 , + ∞ ) and a non-increasing continuous function r : [ 0 , + ∞ ) → ( 0 , + ∞ ) such that

1) ∀ x ∈ X , B ( x , r ( ‖ ∇ f ( x ) ‖ ) ) ⊆ X ;

2) ∀ x 1 , x 2 ∈ B ( x , r ( ‖ ∇ f ( x ) ‖ ) ) , ‖ ∇ f ( x 1 ) − ∇ f ( x 2 ) ‖ ≤ ℓ ( ‖ ∇ f ( x ) ‖ ) ‖ x 1 − x 2 ‖ ;

Then f is called an ( r , ℓ ) -smooth function.

Remark. Here, the function r : [ 0 , + ∞ ) → ( 0 , + ∞ ) is assumed to be nonincreasing and the function ℓ : [ 0 , + ∞ ) → ( 0 , + ∞ ) is assumed to be nondecreasing. This restriction is usually without loss of generality. When r and ℓ do not satisfy the required monotonicity, the desired monotonicity can be achieved by the following construction technique:

1) Replace the original function r with r ^ : = inf 0 ≤ v ≤ u r ( v ) ≤ r ( u ) ;

2) Replace the original function ℓ with the nonincreasing function ℓ ^ : = sup 0 ≤ v ≤ u ℓ ( v ) ≥ ℓ ( u ) .

Next, we elaborate on the intrinsic relationship between the two types of smooth functions. In fact, under appropriate assumptions, ℓ -smooth functions and ( r , ℓ ) -smooth functions are equivalent.

Proposition 1. If f is ( r , ℓ ) -smooth, then f is also ℓ -smooth; if f is ℓ -smooth and f is a differentiable closed function on the open effective domain X , then f is an ( r , m ) -smooth function, where m ( u ) : = ℓ ( u + a ) , r ( u ) : = a / m ( u ) , and the constant a > 0 is arbitrary.

The proof of the above proposition can be found in appendix A.2 of [21]. The objective function considered in this paper obviously satisfies the assumptions of Proposition 1. In view of the equivalence between ℓ -smoothness and ( r , ℓ ) -smoothness in the above framework, the subsequent theoretical analysis will mainly focus on ( r , ℓ ) -generalized smooth functions.

Lemma 2(Descent Lemma [23]). Let f : X → ℝ be a M -smooth function, where X is a convex set. Then for all x , y ∈ X , we have

Lemma 3. [21] Suppose the differentiable function f : X → ℝ satisfies ( r , ℓ ) -smoothness, and for all x ∈ X , ‖ ∇ f ( x ) ‖ ≤ C , Then

1) B ( x , r ( C ) ) ⊆ X ;

2) For any x 1 , x 2 ∈ B ( x , r ( C ) ) ,

where L f : = ℓ ( C ) denotes an effective smoothness constant.

Proof of the above lemma can be found in appendix A.3 of [21].

Remark. From Lemma 3, it is not difficult to see that when the gradient norm of an ( r , ℓ ) -smooth function is bounded, it also satisfies the property of a classical smooth function in a local neighborhood B ( x , r ( C ) ) , i.e., the descent lemma.

Remark. From Proposition 1, the two classes of smooth functions are equivalent. Therefore, the conclusion of Lemma 3 obviously also holds for ℓ -smooth functions. When a : = C , it suffices to take L f = ℓ ( 2 C ) and r ( C ) = C / L f to obtain the corresponding conclusion [21].

Lemma 4(Nonexpansiveness of the Proximal Operator [23]). Let g : ℝ d → ℝ ∪ { + ∞ } be a closed proper convex function. Then for any x , y ∈ ℝ d , we have

and

To address the strict limitation of the classic gradient Lipschitz smoothness assumption on the applicability of the PGD, this paper introduces a novel class of generalized smoothness conditions— ℓ ( ⋅ ) —smoothness, establishing a more general framework for analyzing composite convex optimization problems. Under this generalized smoothness framework, we systematically derive the convergence theory of the constant-stepsize PGD algorithm and rigorously prove that: even under the generalized smoothness assumption that relaxes the classic Lipschitz gradient continuity, the constant-stepsize PGD can still achieve the exact same sublinear convergence rate O ( 1 / k ) as in the classic smoothness scenario, provided that the function f satisfies ℓ ( ⋅ ) —smoothness and Lipschitz continuity.

However, the analysis framework in this paper relies on the core assumption that the objective function f satisfies gradient Lipschitz continuity. While this assumption guarantees the gradient Lipschitz continuity of f in local regions and provides theoretical support for the convergence analysis of the constant-step-size PGD algorithm, it also significantly limits the scope of application of the method: for composite convex optimization problems that do not satisfy gradient Lipschitz continuity (such as scenarios with unbounded gradients or rapidly changing gradients), the sublinear convergence rate theory of the constant-step-size PGD algorithm established in this paper will no longer hold.

Sincere thanks to the members of JAMP for their professional performance, and special thanks to managing editor Hellen XU for a rare attitude of high quality.