Proximal gradient descent and its accelerated version are resultful methods for solving the sum of smooth and non-smooth problems. When the smooth function can be represented as a sum of multiple functions, the stochastic proximal gradient method performs well. However, research on its accelerated version remains unclear. This paper proposes a proximal stochastic accelerated gradient (PSAG) method to address problems involving a combination of smooth and non-smooth components, where the smooth part corresponds to the average of multiple block sums. Simultaneously, most of convergence analyses hold in expectation. To this end, under some mind conditions, we present an almost sure convergence of unbiased gradient estimation in the non-smooth setting. Moreover, we establish that the minimum of the squared gradient mapping norm arbitrarily converges to zero with probability one.
We consider the following composite optimization problem:
min x ∈ ℝ d F ( x ) = def f ( x ) + g ( x ) , (1)
where g : ℝ d → ℝ is the average of the smooth functions g 1 , ⋯ , g n , i.e. g ( x ) = 1 n ∑ i = 1 n g i ( x ) and f : ℝ d → ℝ is a closed proper function that can be non-differentiable. One of the most well-studied instances of this type of problem is l 1 -regularized least squares [
min x ∈ ℝ d 1 n ∑ i = 1 n 1 2 ( a i T x − b i ) 2 + λ ‖ x ‖ 1
where ‖ ⋅ ‖ denotes the standard l 1 -norm.
We frequently encounter optimization problems of this nature in various fields such as machine learning, statistics, signal processing, and imaging [
x k + 1 = prox t k f ( x k − t k ∇ g ( x k ) ) , (2)
where t k is the step size at the kth iteration and the proximity operator of f prox f ( ⋅ ) is defined by:
prox t f ( y ) = arg min x ∈ ℝ d { 1 2 ‖ x − y ‖ 2 + t f ( x ) } .
We note that prox t f ( y ) maps to a singleton since f ( y ) is proper and closed, see for example (Beck, Theorem 6.3, 2017 [
Another effective method for solving problem (1) is the accelerated proximal gradient (APG) method, initially proposed by Nesterov [
y k + 1 : = x k + β k ( x k − x k − 1 )
x k + 1 : = prox t k f ( y k + 1 − t k ∇ g ( y k + 1 ) )
where β k ∈ [ 0,1 ) is an extrapolation parameter and t k is the usual step size.
These parameters must be chosen in specific ways to achieve the convergence accelerated. One simple choice takes β k = k k + 3 [
With the emergence of big data, the efficiency of deterministic optimization algorithm has gradually become a bottleneck. In the PGD (2), we need to calculate the full gradient of g ( ⋅ ) . When the size of the datasets n is very large, where the calculation costs will be high, first-order stochastic gradient methods have proven to be very effective thanks to their low iteration complexity. Therefore, one way to reduce calculation is to use stochastic algorithms that take advantage of the finite sum structure of the problem (1) to use cheaper iterations while preserving fast convergence. When f ( x ) = 0 , problem (1) reduces to general minimization optimization problem: min x ∈ ℝ d F ( x ) = def 1 n ∑ i = 1 n g i ( x ) , where F ( x ) arise as averages of a very large number of smooth functions. This problem often arises by approximation of the stochastic optimization loss function: F ( x ) = E ξ ∈ D [ g ξ ( x ) ] , where ξ is a random variable, g ξ : ℝ d → ℝ is smooth for all ξ . First-order stochastic methods for the case of a non-smooth regularizer f ( x ) are an active research area. Non-asymptotic convergence results were first achieved in [
x k + 1 = prox t k f ( x k − t k ∇ g i k ( x k ) ) . (3)
where t k > 0 is a sequence of step size (also known as learning rate). Sampling the index i k over all indices [ n ] = { 1,2, ⋯ , n } with i.i.d., the gradient ∇ g i k ( x k ) satisfies the unbiased estimation:
E [ ∇ g i k ( x k ) ] = ∑ i = 1 n 1 n ∇ g i k ( x k ) = ∇ g ( x k ) . (4)
The advantage of PSGD over PGD lies in the fact that, at each iteration, PSGD only necessitates the computation of a single gradient ∇ g i k ( x k ) . In contrast, each iteration of PGD evaluates n g gradients. As a result, the computational cost of PSGD per iteration is 1 n of that of PGD. Consequently, the computation of ∇ g i k ( x k ) is approximately n times less expensive than that of ∇ g ( x ) . Numerous algorithms have been devised to tackle the composite optimization problem (1). Gorbunov et al. [
The vast majority of the convergence rates analysis results for stochastic gradient methods in the literature are obtained in terms of the expectation (see, e.g. SGD, stochastic heavy ball (SHB) [
Remark 1.1. It is worth noting that our proof does not rely on the convexity of the function f, so convexity of f is not assumed in the model. However, to ensure the continuity of the generalized gradient operator, we assume that f is convex.
Based on the above, we now consider an accelerated version of proximal stochastic
| Citation | g ( x ) | f ( x ) | Constraint | Momentum | Algorithm | a.s. |
|---|---|---|---|---|---|---|
| Khaled et al. [ | L-smooth | -- | ℝ d | × | SGD | × |
| Gower et al. [ | -- | -- | -- | -- | -- | |
| Mertikopoulos et al. [ | L-smooth, | -- | X | × | SGD | √ |
| G-Lipschitz | -- | -- | -- | |||
| Sebbouh et al. [ | L-smooth | -- | ℝ d | × | SGD | √ |
| Liu et al. [ | -- | -- | -- | -- | -- | |
| Sebbouh et al. [ | L-smooth | -- | ℝ d | √ | SHB | √ |
| Liu et al. [ | -- | -- | -- | -- | -- | -- |
| Liu et al. [ | L-smooth | -- | ℝ d | √ | SNAG | √ |
| Ward et al. [ | L-smooth | -- | ℝ d | × | AdaGrad | × |
| Alacaoglu et al. [ | -- | ρ-weakly convex | X | × | AdaGrad | × |
| Davis et al. [ | -- | ρ-weakly convex | ℝ d | × | PSGD | × |
| Cevher et al. [ | L-smooth | Convex | ℝ d | × | PSGD | × |
| Gorbunov et al. [ | -- | -- | -- | -- | -- | -- |
| Ghadimi et al. [ | L-smooth | Convex | X | × | SCOMID | × |
| Ours | L-smooth | Convex | ℝ d | √ | PSAG | √ |
SCOMID = Stochastic Composite Objective Mirror Descent; PSGD = Proximal Stochastic Gradient Descent; SGD = Stochastic Gradient Descent; AdaGrad = Adaptive Stochastic Gradient Descent; SHB = Stochastic Heavy Ball; SNAG = Stochastic Nesterov’s Accelerated graDient; PSAG = Proximal Stochastic Accelerated Gradient (in this paper).
gradient method and use the unbiased stochastic gradient (see the last column of
y k + 1 : = x k + β k ( x k − x k − 1 )
x k + 1 : = prox t k f ( y k + 1 − t k G k ) (5)
where β k ∈ [ 0,1 ) is an extrapolation parameter and t k is the usual step size, G k = ∇ g i k ( y k + 1 ) is an unbiased estimator of the gradient, where i k is randomly picked from [ n ] = { 1,2, ⋯ , n } with probability 1 n via the independent identically distribution. Then, we have:
x k + 1 ∈ arg min x ∈ ℝ d { t 〈 G k , x 〉 + t f ( x ) + 1 2 ‖ x − x k ‖ 2 } . (6)
where in (5), t = t k is the stepsize, G k is the stochastic gradient and x k = y k + 1 is an extrapolation step. For non-smooth composite problem (1), we establish almost sure convergence rates of PSAG that the minimum of the squared gradient mapping norm is arbitrarily close to zero with probability one. It is noted that PSAG method reduces to PSGD method when β k = 0 in (5).
The rest of this paper is organized as follows. In Section 2, we recall some definitions and known results for further analysis. Then, we present our convergence analysis of PSAG and its convergence rate in Section 3. Finally, we summarize our findings and draw conclusions in Section 4.
The optimal solution of problem (1) is denoted by x * , and the optimal set of problem (1) is non-empty and denoted by X * . The optimal value of the problem (1) is denoted by F * = F ( x * ) . In the rest of this work, for notational brevity, we will omit the subscript of norm ‖ ⋅ ‖ 2 for ‖ ⋅ ‖ .
Definition 2.1. (Convexity). A function g : ℝ d → ℝ is said to be convex; i.e. for all x , y ∈ ℝ d ,
g ( x ) ≥ g ( y ) + 〈 ∇ g ( y ) , x − y 〉 . (7)
Definition 2.2. (L-smoothness). A function g : ℝ d → ℝ is said to be L-smooth if the gradient ∇ g ( x ) is Lipschitz continuous with constant L, i.e. there exists a constant L > 0 such that:
‖ ∇ g ( x ) − ∇ g ( y ) ‖ ≤ L ‖ x − y ‖ , for all x , y ∈ ℝ d . (8)
It is well-known that Definition 2.2 implies the following inequality (see, e.g. Nesterov [
g ( x ) ≤ g ( y ) + 〈 ∇ g ( y ) , x − y 〉 + L 2 ‖ x − y ‖ 2 , ∀ x , y ∈ ℝ d . (9)
Remark 2.1. Definition 2.2 essentially implies that gradient descent with sufficiently small step size is well behaved, and we also have g ( x ) = ∑ i = 1 n g i ( x ) is L-smooth where L = ∑ i n L i ( g i ( x ) is L i -smooth).
Definition 2.3. (Unbiasedness). Given a random iterate { x k } k ≥ 0 . We call the stochastic gradient ∇ g i k ( x k ) ( i k is sampled randomly from { 1,2, ⋯ , n } ) is unbiased if:
E [ ∇ g i k ( x k ) ] = ∇ g ( x k ) (10)
holds.
Definition 2.4. (Prox-grad operator). Consider the composite optimization problem (1), we have:
x k + = T t k f , g ( x k )
where T t f , g : i n t ( d o m ( g ) ) → E ( t > 0 ) is the prox-grad operator defined by:
T t f , g ( x ) ≡ prox t f ( x − t ∇ g ( x ) ) (11)
Definition 2.5. (Gradient mapping). Suppose that f , g satisfy the problem (1) settings. Then, the gradient mapping is the operator G t f , g : i n t ( d o m ( g ) ) → E defined by:
G t f , g ( x ) ≡ 1 t ( x − T t f , g ( x ) ) (12)
for any x ∈ i n t ( d o m ( g ) ) .
The proof about almost sure convergence relies on the classical Robbins-Siegmund supermartingale convergence result (Theorem 1 in [
Lemma 3.1. (Theorem 1, [
1) E [ Y k + 1 | F k ] ≤ ( 1 + γ k ) Y k − X k + Z k .
2) ∑ k = 1 ∞ γ k < ∞ , ∑ k = 1 ∞ Z k < ∞ almost surely.
Then, Y k converges almost surely and ∑ k = 1 ∞ X k < ∞ almost surely (a.s.)
Lemma 3.2. (Lemma 3, [
∑ k = 1 ∞ t k + 1 ∑ s = 1 k t s = ∞ , ∑ k = 1 ∞ t k + 1 X k < ∞ a . s . (13)
then we have:
min 1 ≤ i ≤ k X i = o ( 1 ∑ j = 1 k t s ) a . s . (14)
where o denotes the higher-order infinitesimal. i.e. for two sequences { a k } → 0 and { b k } → 0 , a k = o ( b k ) if and only if lim k → ∞ a k / b k = 0 .
Reviewing the iteration of PSAG (5), using the Definition 2.4, 2.5, we rewrite the PSAG as follows:
y k + 1 : = x k + β ( x k − x k − 1 ) ,
x k + 1 : = y k + 1 − t k G t k ( y k + 1 ) . (15)
where G t k ( y k + 1 ) = 1 t k ( y k + 1 − T t k f , g i k ( y k + 1 ) ) , T t k f , g i k ( y k + 1 ) = prox t k f ( y k + 1 − t k ∇ g i k ( y k + 1 ) ) , t k is the step size and β = β k ∈ [ 0 , 1 ) and i k is randomly picked from [ n ] = { 1,2, ⋯ , n } with probability 1 n via the independent identically distribution.
Lemma 3.3. Suppose that u ∈ arg min x ∈ ℝ d { t 〈 z , x 〉 + t f ( x ) + 1 2 ‖ x − y ‖ 2 } for some z , y ∈ ℝ d . If g ( x ) is L-smooth, the for any x ∈ ℝ d , we have:
F ( u ) ≤ F ( x ) + 〈 ∇ g ( y ) − z , u − x 〉 + 1 2 t ‖ x − y ‖ 2 − 1 2 t ‖ u − y ‖ 2 + L 2 ‖ u − y ‖ 2 . (16)
Proof By the optimality of u, we can show that for any x ∈ ℝ d ,
t 〈 z , u 〉 + t f ( u ) + 1 2 ‖ u − y ‖ 2 ≤ t 〈 z , x 〉 + t f ( x ) + 1 2 ‖ x − y ‖ 2 (17)
Upon rearranging the above equation, we have:
f ( u ) ≤ f ( x ) + 〈 z , x − u 〉 + 1 2 t ‖ x − y ‖ 2 − 1 2 t ‖ u − y ‖ 2 . (18)
Since g ( x ) is L-smooth, we can apply Definition 2.2 to have:
g ( u ) ≤ g ( y ) + 〈 ∇ g ( y ) , u − y 〉 + L 2 ‖ u − y ‖ 2 . (19)
g ( x ) ≥ g ( y ) + 〈 ∇ g ( y ) , x − y 〉 + 1 2 L ‖ ∇ g ( x ) − ∇ g ( y ) ‖ 2 . (20)
Next, upon combining (19) and (20), we arrive at:
g ( u ) ≤ g ( y ) + 〈 ∇ g ( y ) , u − y 〉 + L 2 ‖ u − y ‖ 2 ≤ g ( x ) + 〈 ∇ g ( y ) , y − x 〉 − 1 2 L ‖ ∇ g ( x ) − ∇ g ( y ) ‖ 2 + 〈 ∇ g ( y ) , u − y 〉 + L 2 ‖ u − y ‖ 2 ≤ g ( x ) + 〈 ∇ g ( y ) , u − x 〉 + L 2 ‖ u − y ‖ 2 . (21)
By the fact that F ( u ) = g ( u ) + f ( u ) , we finally obtain:
F ( u ) = g ( u ) + f ( u ) ≤ g ( u ) + f ( x ) + 〈 z , x − u 〉 + 1 2 t ‖ x − y ‖ 2 − 1 2 t ‖ u − y ‖ 2 ≤ g ( x ) + 〈 ∇ g ( y ) , u − x 〉 + L 2 ‖ u − y ‖ 2 + f ( x ) + 〈 z , x − u 〉 + 1 2 t ‖ x − y ‖ 2 − 1 2 t ‖ u − y ‖ 2 = F ( x ) + 〈 ∇ g ( y ) − z , u − x 〉 + 1 2 t ‖ x − y ‖ 2 − 1 2 t ‖ u − y ‖ 2 + L 2 ‖ u − y ‖ 2 . (22)
Theorem 3.1. Suppose that F ( x ) is lower bounded and there exists σ > 0 , A k ≥ 0 such that:
E [ ‖ ∇ g ( y k + 1 ) − G k ‖ 2 | F k ] ≤ A k ( F ( x k ) − F * ) + σ 2 , (23)
where G k = ∇ g i k ( y k + 1 ) , i k is randomly picked from [ n ] = { 1,2, ⋯ , n } with probability 1 n via the independent identically distribution, the iteration of PSAG (15) is employed with a non-increasing stepsize t k such that:
∑ k = 1 ∞ t k 2 < ∞ , ∑ k = 1 ∞ A k t k < ∞ , ∑ k = 1 ∞ t k + 1 ∑ s = 1 k t s = ∞ and t k ≤ 1 4 L ,
where A k ≥ 0 is defined in (23), then we have:
{ F ( x k ) − F * } a . s .
If f : ℝ d → ℝ is convex, we obtain:
min 1 ≤ i ≤ k ‖ G t k ( y k + 1 ) ‖ 2 = o ( 1 ∑ s = 1 k t s ) a . s . (24)
Proof We begin with the conclusion in (16) of Lemma 3.3. Firstly, by the definition of x k + in Definition 2.4, let u = x k + , then set t = t k 2 , z = ∇ g ( x k ) and y = x k to meet the condition u ∈ arg min x ∈ ℝ d { t 〈 z , x 〉 + t f ( x ) + 1 2 ‖ x − y ‖ 2 } , i.e.
x k + ∈ arg min x ∈ ℝ d { t k 2 〈 ∇ g ( x k ) , x 〉 + t k 2 f ( x ) + 1 2 ‖ x − x k ‖ 2 } (25)
From the update in (2), we can show that the iterate x k ∈ ℝ d for any k ≥ 1 . Upon applying Lemma 3.3 with u = x k + , t = t k 2 , z = ∇ g ( x k ) and y = x k for x = x k ∈ ℝ d , we have:
F ( x k + ) ≤ F ( x k ) + 〈 ∇ g ( x k ) − ∇ g ( x k ) , x k + − x k 〉 + 1 t k ‖ x k − x k ‖ 2 − 1 t k ‖ x k + − x k ‖ 2 + L 2 ‖ x k + − x k ‖ 2 = F ( x k ) + ( L 2 − 1 t k ) ‖ x k + − x k ‖ 2 . (26)
Next, by the update rule in (6) of PSAG algorithm, we choose u = x k + 1 , then set t = t k , z = G k = ∇ g i ( y k + 1 ) and y = y k + 1 to meet the condition u ∈ arg min x ∈ ℝ n { t 〈 z , x 〉 + t f ( x ) + 1 2 ‖ x − y ‖ 2 } , i.e.
x k + 1 ∈ arg min x ∈ ℝ d { t k 〈 G k , x 〉 + t k f ( x ) + 1 2 ‖ x − y k + 1 ‖ 2 } (27)
Similarly, from the definition of x k + in (25), we can show that for any k ≥ 1 , the sequence x k + ∈ ℝ d . Now, upon applying Lemma 3.3 with u = x k + 1 , t = t k , z = G k , and y = y k + 1 , for x = x k + ∈ ℝ d , we then obtain:
F ( x k + 1 ) ≤ F ( x k + ) + 〈 ∇ g ( y k + 1 ) − G k , x k + 1 − x k + 〉 + 1 2 t k ‖ x k + − y k + 1 ‖ 2 − 1 2 t k ‖ x k + 1 − y k + 1 ‖ 2 + L 2 ‖ x k + 1 − y k + 1 ‖ 2 = F ( x k + ) + ( L 2 − 1 2 t k ) ‖ x k + 1 − y k + 1 ‖ 2 + 1 2 t k ‖ y k + 1 − x k + ‖ 2 + 〈 ∇ g ( y k + 1 ) − G k , x k + 1 − x k + 〉 . (28)
Now, we consider 〈 ∇ g ( y k + 1 ) − G k , x k + 1 − x k + 〉 ,
〈 ∇ g ( y k + 1 ) − G k , x k + 1 − x k + 〉 ≤ ‖ ∇ g ( y k + 1 ) − G k ‖ ⋅ ‖ x k + 1 − x k + ‖ ≤ 2 t k ‖ ∇ g ( y k + 1 ) − G k ‖ 2 + 1 8 t k ‖ x k + 1 − x k + ‖ 2 = 2 t k ‖ ∇ g ( y k + 1 ) − G k ‖ 2 + 1 8 t k ‖ x k + 1 − y k + 1 + y k + 1 − x k + ‖ 2 ≤ 2 t k ‖ ∇ g ( y k + 1 ) − G k ‖ 2 + 1 4 t k ‖ x k + 1 − y k + 1 ‖ 2 + 1 4 t k ‖ y k + 1 − x k + ‖ 2 , (29)
where we use the Cauchy-Schwarz | 〈 a , b 〉 | ≤ ‖ a ‖ ‖ b ‖ in the first inequality, and the second inequality follows from a b ≤ ( 1 / 2 ) ( a 2 + b 2 ) with a = 2 t k ‖ ∇ g ( y k + 1 ) − G k ‖ and b = 1 2 t k ‖ x k + 1 − x k + ‖ . The last inequality holds by ‖ a + b ‖ 2 ≤ 2 ‖ a ‖ 2 + 2 ‖ b ‖ 2 with a = x k + 1 − y k + 1 and b = y k + 1 − x k + .
Upon substituting (29) back into (28), we further have:
F ( x k + 1 ) ≤ F ( x k + ) + ( L 2 − 1 2 t k ) ‖ x k + 1 − y k + 1 ‖ 2 + 1 2 t k ‖ y k + 1 − x k + ‖ 2 + 〈 ∇ g ( y k + 1 ) − G k , x k + 1 − x k + 〉 ≤ F ( x k + ) + ( L 2 − 1 2 t k ) ‖ x k + 1 − y k + 1 ‖ 2 + 1 2 t k ‖ y k + 1 − x k + ‖ 2 + 2 t k ‖ ∇ g ( y k + 1 ) − G k ‖ 2 + 1 4 t k ‖ x k + 1 − y k + 1 ‖ 2 + 1 4 t k ‖ y k + 1 − x k + ‖ 2
= F ( x k + ) + ( L 2 − 1 4 t k ) ‖ x k + 1 − y k + 1 ‖ 2 + 3 4 t k ‖ y k + 1 − x k + ‖ 2 + 2 t k ‖ ∇ g ( y k + 1 ) − G k ‖ 2 ≤ F ( x k ) + ( L 2 − 1 t k ) ‖ x k + − x k ‖ 2 + 2 t k ‖ ∇ g ( y k + 1 ) − G k ‖ 2 + ( L 2 − 1 4 t k ) ‖ x k + 1 − y k + 1 ‖ 2 + 3 4 t k ‖ y k + 1 − x k + ‖ 2 ≤ F ( x k ) + 2 t k ‖ ∇ g ( y k + 1 ) − G k ‖ 2 − 1 8 t k ‖ x k + 1 − y k + 1 ‖ 2 + 3 4 t k ‖ y k + 1 − x k + ‖ 2 . (30)
where the last inequality holds by the stepsize condition t k ≤ 1 / 4 L , i.e. L / 2 − 1 / 4 t k ≤ − 1 / 8 t k .
Next, we consider 3 4 t k ‖ y k + 1 − x k + ‖ 2 and combine Definition 2.4 with (5) and (15), we have:
x k + = T t k f , g i k ( y k + 1 ) = prox t k f ( y k + 1 − t k ∇ g i k ( y k + 1 ) ) ,
and
G t k ( y k + 1 ) = 1 t k ( y k + 1 − T t k f , g i k ( y k + 1 ) ) = 1 t k ( y k + 1 − x k + ) ,
Due to the convexity of f, we know that there exists M > 0 such that:
‖ G t k ( y k + 1 ) ‖ < M .
Therefore,
3 4 t k ‖ y k + 1 − x k + ‖ 2 = 3 t k 4 ‖ G t k ( y k + 1 ) ‖ 2 ≤ 3 M 2 4 t k . (31)
Then, by (15), we also have:
− 1 8 t k ‖ x k + 1 − y k + 1 ‖ 2 = − t k 8 ‖ G t k ( y k + 1 ) ‖ 2 . (32)
Upon combining (31) and (32) with (30), we have:
F ( x k + 1 ) ≤ F ( x k ) − t k 8 ‖ G t k ( y k + 1 ) ‖ 2 + 2 t k ‖ ∇ g ( y k + 1 ) − G k ‖ 2 + 3 M 2 4 t k . (33)
Now, the conditional expectation on (33) with respect to F k gives:
E [ F ( x k + 1 ) | F k ] ≤ E [ F ( x k ) | F k ] − t k 8 E [ ‖ G t k ( y k + 1 ) ‖ 2 | F k ] + 2 t k E [ ‖ ∇ g ( y k + 1 ) − G k ‖ 2 | F k ] + 3 M 2 4 t k = F ( x k ) − t k 8 ‖ G t k ( y k + 1 ) ‖ 2 + 2 t k E [ ‖ ∇ g ( y k + 1 ) − G k ‖ 2 | F k ] + 3 M 2 4 t k ≤ F ( x k ) − t k 8 ‖ G t k ( y k + 1 ) ‖ 2 + 2 t k ( A k ( F ( x k ) − F * ) + σ 2 ) + 3 M 2 4 t k = F ( x k ) − t k 8 ‖ G t k ( y k + 1 ) ‖ 2 + 2 A k t k ( F ( x k ) − F * ) + ( 2 σ 2 + 3 M 2 4 ) t k . (34)
where the first equality holds by the fact that x k is F k -measurable, i.e. E [ F ( x k ) | F k ] = F ( x k ) and E [ ‖ G t k ( y k + 1 ) ‖ 2 | F k ] = ‖ G t k ( y k + 1 ) ‖ 2 . The last inequality follows from E [ ‖ ∇ g ( y k + 1 ) − G k ‖ 2 | F k ] ≤ A k ( F ( x k ) − F * ) + σ 2 .
Upon subtracting F * from both sides of (34), we have:
E [ F ( x k + 1 ) − F * | F k ] ≤ F ( x k ) − F * − t k 8 ‖ G t k ( y k + 1 ) ‖ 2 + 2 A k t k ( F ( x k ) − F * ) + ( 2 σ 2 + 3 M 2 4 ) t k = ( 1 + 2 A k t k ) ( F ( x k ) − F * ) − t k 8 ‖ G t k ( y k + 1 ) ‖ 2 + ( 2 σ 2 + 3 M 2 4 ) t k . (35)
Upon multiplying (35) by t k + 1 , we can obtain:
E [ t k + 1 ( F ( x k + 1 ) − F * | F k ) ] ≤ ( 1 + 2 A k t k ) t k + 1 ( F ( x k ) − F * ) − t k t k + 1 8 ‖ G t k ( y k + 1 ) ‖ 2 + ( 2 σ 2 + 3 M 2 4 ) t k t k + 1 ≤ ( 1 + 2 A k t k ) t k ( F ( x k ) − F * ) − t k t k + 1 8 ‖ G t k ( y k + 1 ) ‖ 2 + ( 2 σ 2 + 3 M 2 4 ) t k 2 . (36)
where the last inequality follows from the non-increasing behaviour of the stepsize t k , i.e. t k + 1 < t k .
Finally, let Y k = t k ( F ( x k ) − F * ) , γ k = 2 A k t k , X k = t k 8 ‖ G t k ( y k + 1 ) ‖ 2 and Z k = ( 2 σ 2 + 3 M 2 4 ) t k 2 , then (36) becomes:
E [ Y k + 1 | F k ] ≤ ( 1 + γ k ) Y k − t k + 1 X k + Z k .
Recalling the stepsize conditions ∑ k = 1 ∞ t k 2 < ∞ and ∑ k = 1 ∞ A k t k < ∞ , we know that ∑ k = 1 ∞ Z k < ∞ , ∑ k = 1 ∞ γ k < ∞ . Thus, by Lemma 3.1, we have:
∑ k = 1 ∞ t k + 1 X k < ∞ a . s . and { F ( x k ) − F * } a . s .
Finally, with the condition ∑ k = 1 ∞ t k + 1 ∑ s = 1 k t s = ∞ of Lemma 3.2, we have:
min 1 ≤ r ≤ k t r ‖ G t r ( y k + 1 ) ‖ 2 = o ( 1 ∑ s = 1 k t s ) a . s . (37)
This completes the proof.
Remark 3.1. (23) is a new variance assumption on the stochastic gradient, where G k is weaker than the bounded variance assumption (i.e. A k = 0 in [
Remark 3.2. It is noting that we get the boundedness of the gradient mapping G t k ( y k + 1 ) due to the continuity of the prox-grad operator T t k f , g i k ( y k + 1 ) which is deduced by the convexity of f.
Corollary 3.1. Following the setting of Theorem 3.1 and choosing the stepsize t k = β 1 + γ β k 1 2 + ε for any γ , β ≥ 0 where ε ∈ ( 0, 1 2 ) gives:
min 1 ≤ r ≤ k ‖ G t r ( y k + 1 ) ‖ 2 = o ( 1 ) a . s . (38)
Proof Stepsize satisfying t k = β 1 + γ β k 1 2 + ε has been studied in [
1 + γ β k 1 2 + ε ≤ k 1 2 + ε + γ β k 1 2 + ε = ( 1 + γ β ) k 1 2 + ε , (39)
which implies that t k ≥ β ( 1 + γ β ) k 1 2 + ε . Upon by the integral test inequality, we have:
∑ s = 1 k t s ≥ ∑ s = 1 k β ( 1 + γ β ) s 1 2 + ε ≥ ∫ 1 k β ( 1 + γ β ) x 1 2 + ε d x = β ( k 1 2 − ε − 1 ) ( 1 + γ β ) ( 1 2 − ε ) ≥ β k − 1 2 − ε ( k − 1 ) 1 + γ β , (40)
where the last inequality follows from the concavity of h ( x ) = x 1 2 − ε , so that:
h ( y ) ≤ h ( x ) + h ′ ( x ) ( y − x ) .
In other words, by taking y = 1 and x = k , we can get k 1 2 − ε − 1 ≥ ( 1 2 − ε ) k − 1 2 − ε ( k − 1 ) .
Next, combining (39) and (40), we can obtain:
( ∑ s = 1 k t s ) t k ≥ β k − 1 2 − ε ( k − 1 ) 1 + γ β β ( 1 + γ β ) k 1 2 + ε = β 2 k − 1 − 2 ε ( k − 1 ) ( 1 + γ β ) 2 . (41)
Upon setting G r = ‖ G t r ( y k + 1 ) ‖ 2 , and applying the inequality min 1 ≤ r ≤ k t r G r ≥ t k min 1 ≤ r ≤ k G r , we have:
( ∑ s = 1 k t s ) min 1 ≤ r ≤ k t r G r ≥ ( ∑ s = 1 k t s ) t k min 1 ≤ r ≤ k G r ≥ β 2 k − 1 − 2 ε ( k − 1 ) ( 1 + γ β ) 2 min 1 ≤ r ≤ k G r ≥ 0. (42)
For the left hand side of (42), we apply Theorem 3.1 to get:
lim k → ∞ ( ∑ s = 1 k t s ) min 1 ≤ r ≤ k t k G r = 0 a . s .
The application of Squeeze theorem in conjunction with (42) gives:
lim k → ∞ k − 1 − 2 ε ( k − 1 ) min 1 ≤ r ≤ k G r = 0 a . s .
When ε → 0 , we have k − 1 k 1 + 2 ε → 1 , which implies that:
lim k → ∞ min 1 ≤ r ≤ k G r = 0 a . s .
Therefore, we finally obtain:
min 1 ≤ r ≤ k G r = min 1 ≤ r ≤ k ‖ G t r ( y k + 1 ) ‖ 2 = o ( 1 ) a . s .
This completes the proof.
Remark 3.3. Corollary 3.1 indicates that min 1 ≤ r ≤ k ‖ G t r ( y k + 1 ) ‖ 2 is arbitrarily close to 0 with probability one.
This paper presents PSAG with unbiased gradient estimation and analyzes its almost sure convergence for solving composite optimization problems, wherein the objective function comprises a smooth component and a non-smooth component. By leveraging certain key assumptions, we have established the almost sure convergence of PSAG with unbiased gradient estimation. Furthermore, we have demonstrated that the minimum of the squared gradient mapping norm approaches zero arbitrarily closely with probability one.
This paper was supported by the Natural Science Foundation of China (Grant No. 11801455), the Sichuan Science and Technology Program (Grant No. 2023NSFSC1922), the Innovation Team Funds of China West Normal University (Grant No. KCXTD2023-3), and the Fundamental Research Funds of China West Normal University (Grant No. 23kc010).
The authors declare no conflicts of interest regarding the publication of this paper.
Xiang, X. and Xia, H.M. (2024) Almost Sure Convergence of Proximal Stochastic Accelerated Gradient Methods. Journal of Applied Mathematics and Physics, 12, 1321-1336. https://doi.org/10.4236/jamp.2024.124081