Consider the following nonparametric multiplicative regression model

Y = exp ( g ( Z ) ) ε , (1)

where Y is the response variable, Z is a p-vector of covariates, ε is the positive random error and independent of Z , g ( ⋅ ) is an unknown link function. Without loss of generality and for simplicity, we will assume that p = 1 in the following, i.e., the covariate Z is univariate. But the method discussed later can be easily be extended to the general case. Assume that an i.i.d. sample ( Y i , Z i ) , i = 1, ⋯ , n from model (1) are observed. For model identifiability, it is required that E ( ε ) = E ( ε − 1 ) in [5] . On the other hand, after taking logarithmic transformation on bosh sides of Equation (1), it follows that

ln ( Y ) = g ( Z ) + ln ( ε ) = ( g ( Z ) + c ) + ( ln ( ε ) − c ) ≜ g ˜ ( Z ) + ln ( ε ˜ ) , (2)

holds for any real number c, which means that the former requirement is not enough to ensure the nonparametric component unique. This phenomenon has been found by [8] , where they imposed a strong condition on the model, namely, E ( ln ( ε ) ) = 0 . In our opinion, this is not necessary. The unknown constant c can be set to be E ( ln ( ε ) ) such that E ( ln ( ε ˜ ) ) = 0 holds. Thus the function g ( ⋅ ) is identifiable. Meanwhile, suppose that one obtains a initial estimator of g ( ⋅ ) , denoted by g ˜ n ( ⋅ ) . Combining the Equation (2), a more efficient estimator of g ( ⋅ ) , denoted by g ^ n ( ⋅ ) , can be obtained by subtracting the mean of E ( ln ( ε ) ) from the initial estimator g ˜ n ( ⋅ ) , where

g ^ n ( z ) = g ˜ n ( ⋅ ) − c ¯ , c ¯ = 1 n ∑ i = 1 n [ ln ( Y i ) − g ˜ n ( Z i ) ] . (3)

In this process, it only needs that E ( ln ( ε ) ) , the expectation of ln ( ε ) is finite, which is allowed to be nonzero. Therefore, our condition is weaker than that stated in [8] .

Following the LPRE technique used in [5] , the estimator of g ( ⋅ ) in model (1) can be obtained by minimizing the objective function

S n ( g ) = 1 n ∑ i = 1 n { | Y i − exp ( g ( Z i ) ) Y i | × | Y i − exp ( g ( Z i ) ) exp ( g ( Z i ) ) | } . (4)

When all Z i ’s are mutually different, the computation involved in Equation (4) can be accomplished individually for each g ( Z i ) . However, the resulting estimator is not continuous, which is unsatisfactory in some applications. To overcome these drawbacks, we approximate g ( ⋅ ) by B-spline functions. Exactly speaking, let − ∞ < a = t 0 < t 1 < ⋯ < t k n < t k n + 1 = b < ∞ be a sequence of knots, where [ a , b ] denotes the support of Z. As discussed in [19] , a set of B-spline basis function of degree d can be constructed and denoted by φ j ( ⋅ ) , j = 1 , ⋯ , K n = k n + 1 + d , where d = 2 or 3 corresponds to the quadratic spline or the cubic spline. Then g ( z ) can be approximated by a linear combination of basis functions, i.e., g ( z ) ≈ ∑ j = 1 K n     α j φ j ( z ) ≜ α Τ B ( z ) , where α = ( α 1 , ⋯ , α K n ) Τ is the spline coefficient vector and B ( z ) = ( φ 1 ( z ) , ⋯ , φ K n ( z ) ) Τ . Further, the model (1) can be rewritten as

Y = exp ( α Τ B ( Z ) ) ε . (5)

Meanwhile, α can be estimated by α ⌣ , the minimizer of the loss function

L n ( α ) = 1 n ∑ i = 1 n { | Y i − exp ( α Τ B i ) Y i | × | Y i − exp ( α Τ B i ) exp ( α Τ B i ) | } = 1 n ∑ i = 1 n [ Y i exp ( − α Τ B i ) + Y i − 1 exp ( α Τ B i ) − 2 ] ,

where B i = B ( Z i ) . Then the B-spline estimator of g ( ⋅ ) can be obtained through Equation (3), where g ˜ n ( z ) is replaced by α ⌣ Τ B ( z ) .

However, for reasons discussed in Section 1, here we estimate the nonparametric function in model (1) using penalized splines by adding a roughness penalty to the above minimization problem. In the literature, there are mainly two kinds of definition of roughness. One is based on the integrated squared q-th derivative of the spline function, the other is based on the q-th order difference of the spline coefficient vector, which is more popular for its simplicity and adopted in this paper. Exactly, let Δ denote the backward difference operator. Then we have Δ α j = α j − α j − 1 and Δ 2 α j = α j − 2 α j − 1 + α j − 2 . In General, let Δ q = Δ Δ q − 1 . With these notations, the P-spline estimator of g ( ⋅ ) is defined as g ^ , the corrected spline function in (3), where g ˜ n ( z ) is replaced by α ^ Τ B ( z ) , and α ^ is the minimizer of

Q n ( α ) = L n ( α ) + λ ∑ j = q + 1 K n | Δ q α j | γ ,

where γ > 0 and λ > 0 is the smoothing parameter controlling the smoothness of the fitting vurve. If λ = 0 , the resulting estimator becomes the B-spline estimator. In practice, common choices of q and γ are q = 2 and γ = 2 for computational convenience. At this time, the above minimization problem can be rewritten as

Q n ( α ) = L n ( α ) + λ α Τ D Τ D α , (6)

where D is the ( K n − 2 ) × K n matrix representation of the second order difference operator Δ 2 .

Note that for given smoothing parameter and knots, the loss function (6) is differential and convex, minimizing it is not difficult, and some common algorithms can be applied. In the later simulation studies and real data analysis, we employ the function nmk in the R package dfoptim.

However, other alternatives are still helpful. As in the least square mean regression and penalized quantile regression (e.g. [20] ), an iterative reweighed least square (IRLS) algorithm can be developed. Meanwhile, the measure for the effective model degree of freedom can be introduced based on the trace of the hat matrix, as used in [20] . In the literature, the degree of freedom usually is adopted as the measure of model complexity to facilitate further model comparison, which has not been discussed for multiplicative regression using local smoothing technique. Thus, the smoothing parameter selection can be achieved naturally using some existing information criteria or cross-validation.

When the smoothing parameter and knots are prepared, α ^ can also be seen as the root of the first derivative of objective function (6), namely,

− 1 n ∑ i = 1 n [ Y i exp ( − α ^ Τ B i ) − Y i − 1 exp ( α ^ Τ B i ) ] B i + 2 λ D Τ D α ^ = 0, (7)

which can be rewritten as

− 1 n ∑ i = 1 n     2 ( ln Y i − α ^ Τ B i ) W i ( α ^ ) B i + 2 λ D Τ D α ^ = 0, (8)

where

W i ( α ^ ) = Y i exp ( − α ^ Τ B i ) − Y i − 1 exp ( α ^ Τ B i ) 2 ( ln Y i − α ^ Τ B i ) . (9)

Write Y ˜ = ( ln Y 1 , ⋯ , ln Y n ) Τ , B = ( B 1 , ⋯ , B n ) and W ( α ^ ) = d i a g ( W 1 ( α ^ ) , ⋯ , W n ( α ^ ) ) . The IRLS algorithm is implemented as follows.

Step 0. For given smoothing parameter λ , provide an initial value of α , denoted as α ^ 0 , which can be taken as the unpenalized B-spline estimator, or the ordinary least square estimator based on the logarithmic transformed data. Assume that we have the k-th iterative value α ^ k .

Step 1. At the k + 1 -th step, update the parameter by

α ^ k + 1 = ( B W ( α ^ k ) B Τ + 2 λ D Τ D ) − 1 B W ( α ^ k ) Y ˜ .

Repeat Step 1 until convergence. The value at the terminated step is defined as the final estimator of α , also denoted by α ^ . At the same time, as in the linear regression, the hat matrix here can be defined through

H λ = B Τ ( B W ( α ^ ) B Τ + 2 λ D Τ D ) − 1 B W ( α ^ ) .

To implement the estimator, the smoothing parameter λ and the number and location of knots had to be determined in advance. Compared with the knots, the smoothing parameter plays a more significant role in the penalized spline estimation. As done in [20] , we set the trace of the hat matrix mentioned above to be the model degrees of freedom, denoted by d f = t r ( H λ ) , where t r ( A ) denotes the trace of a square matrix A. In spline context, some criterions have been proposed to select the smoothing parameter λ , such as the Bayesian Information Criterion (BIC), Generalized Approximation Cross-Validation criterion (GACV) and Generalized Cross-Validation criterion (GCV). Especially, in this paper, they are implemented by minimizing the functions:

BIC ( λ ) = ln ( L 1 n ( α ^ ) ) + d f ⋅ ln n 2 n ,

GACV ( λ ) = L 1 n ( α ^ ) 1 − d f / n ,

GCV ( λ ) = ∑ i = 1 n W i ( α ^ ) ( ln ( Y i ) − α ^ Τ B i ) / n ( 1 − d f / n ) 2 ,

respectively.

[18] suggested some rules for the number and location of knots. In the same spirits, we follow the strategy in [20] , where quadratic B-splines with 20 equally-spaced quantiles knots are used unless otherwise specified. With the BIC, GACV, GCV criterions above, we use 51 equally-spaced log-scaled grid points on log 10 ( λ ) ∈ [ − 5,5 ] for the choice of λ . All these settings work well in the numerical studies and a real example analysis.

For any probability measure P, define L 2 -norm ‖ f ‖ 2 = ( ∫ f 2 d P ) 1 / 2 . Denote p = d + 1 . To derive the asymptotic results, some regularity conditions are needed and listed as follows.

(A1) The true function in model (1) is g 0 . Suppose that g 0 ∈ G = { g ∈ C r [ a , b ] : ‖ g ( j ) ‖ ∞ ≤ M 0 , j = 1, ⋯ , r , | g ( r ) ( z 1 ) − g ( r ) ( z 2 ) | ≤ M 1 | z 1 − z 2 | } , where M 0 and M 1 are some positive constants, ‖   ⋅   ‖ ∞ is the superior norm, q ≤ r ≤ d + 1 .

(A2) Define h i = t i − t i − 1 . Assume that max i | h i + 1 − h i | = o ( 1 / k n ) . Moreover, the ratio of maximum and minimum spacings of knots is uniformly bounded.

(A3) k n = o ( n ) . (A4) The covariate Z has abounded support [ a , b ] with corresponding density f Z ( . ) , which has second derivative and is bounded away from zero and infinity.

(A5) ε is independent of Z, and E ( ε ) = E ( ε − 1 ) , E ( ln ( ε ) ) is finite.

(A6) E ( ε + ε − 1 ) < + ∞ .

Conditions (A1)-(A3) are common requirements in the penalized spline theory. (A4) is the regularization condition used in the study of MR. (A5) is an identification condition for the LPRE estimation, which is similar to the zero mean condition in the classical linear mean regression. (A6) is required for proof, which is also used in [15] .

Theorem 2.1 Under conditions (A1)-(A6) mentioned above, we have that

1) If r = p , k n = O ( n 1 / ( 2 p + 1 ) ) and λ = O ( n − β ) with β > ( 1 + p − q ) / ( 2 p + 1 ) , then it yields that ‖ g ^ − g 0 ‖ 2 2 = O P ( n − 2 p / ( 2 p + 1 ) ) .

2) If r = q , λ = O ( n − 1 / ( 2 q + 1 ) ) and k n = O ( n β ) with 1 > β > 1 / ( 2 q + 1 ) , then it yields that ‖ g ^ − g 0 ‖ 2 2 = O P ( n − 2 q / ( 2 q + 1 ) ) .

Proof. Note that

Q n ( α ) = 1 n ∑ i = 1 n [ Y i exp ( − α Τ B i ) + Y i − 1 exp ( α Τ B i ) − 2 ] + λ α Τ D Τ D α = 1 n ∑ i = 1 n [ ε i exp ( R i + ( g ∗ ( Z i ) − α Τ B i ) ) + ε i − 1 exp ( − R i − ( g ∗ ( Z i ) − α Τ B i ) ) − 2 ]         + λ α Τ D Τ D α ,

where ε i = Y i / exp ( g 0 ( Z i ) ) , R i = g ( Z i ) − g ∗ ( Z i ) , and g ∗ ( Z ) = α ∗ B ( Z ) denotes the best spline function approximation. Define X = n − 1 ∑ i = 1 n B i B i Τ , G λ = X + λ D Τ D and θ = G λ 1 / 2 ( α ∗ − α ) . Then we have that g ∗ ( Z i ) − α Τ B i = B i Τ ( α ∗ − α ) = B i Τ G λ − 1 / 2 θ . Let a n denote the convergence rate in the Theory. Since θ ^ = a n ( α ^ − α ∗ ) = arg min θ [ Q n ( α ∗ + a n − 1 θ ) − Q n ( α ∗ ) ] , we only need to prove that for every ν > 0 , there exists a large constant δ > such that

P ( inf ‖ θ ‖ 2 = δ Q n ( α ∗ + a n − 1 θ ) − Q n ( α ∗ ) ) ≥ 1 − ν .

In the line of the proof of Theorem 1 in the Appendix of [17] , some similar techniques can be applied. Finally, by the Corollary 1 of [16] , the desirable results are derived. The proof is completed.

Model A. We generated ( Y , Z ) from the following model

Y = exp ( 1 + 2 sin ( π Z 30 ) ) ε , (10)

where Z ~ U n i f ( 0,50 ) and is independent with the random error ε . Further, three error distributions are considered, namely,

Case (i): ε = exp ( U ) , U ~ N ( 0,1 ) ,

Case (ii): ε = exp ( U ) , U ~ U n i f ( − 2,2 ) ,

Case (iii): ε ~ U n i f ( 0.0001,4.635506 ) .

Note that under case (i)-(ii), it holds that E ( ε − ε − 1 ) = 0 and E ( log ( ε ) ) = 0 . But under case (iii), E ( ε − ε − 1 ) = 0 , E ( log ( ε ) ) = 0.5339771 . We use the averaged Integrated Absolute Bias (IABIAS) and Mean Integrated Square Error (MISE), where for one estimator g ^ j ( j = 1 , ⋯ , 500 ) obtained from the j-th sample,

IABIAS = 1 500 ∑ j = 1 500 [ 1 n g r i d ∑ k = 1 n g r i d | g ^ j ( u k ) − g 0 ( u k ) | ] ,

MISE = 1 500 ∑ j = 1 500 [ 1 n g r i d ∑ k = 1 n g r i d [ g ^ j ( u k ) − g 0 ( u k ) ] 2 ] ,

at the fixed grid points { u k } equally-spaced in [0, 50] and n g r i d = 501 . The values below them are the associated sample standard deviation. To study the different smoothing parameter selection methods at a finer scale, we also calculate the mean and standard deviation of effective model degree of freedom (DF).

Table 1 reported the results. It can be seen that all three penalized estimators

have smaller IABIAS and MISE than those of the unpenalized B-spline estimator. This indicates the benefits of the proposed method. As the sample size increases, the differences between them decrease and become comparable. The model degree using GACV are almost larger than those of using BIC and GCV in all settings, and those using BIC are the smallest, which is also observed in [20] . These imply that GACV tends to select more complex models, BIC tends to select simpler models, GCV makes a compromise and lies between them. Figure 1 displays the average estimated curves and their 95% point-wise confidence band under cases (i) with sample size n = 300 for model A. We can see that all estimated curves are close to the true one, but the B-spline estimation has a drastic deviation near the boundaries. Figure 2 shows the boxplots of the estimates of g ( z ) at z = 15,30,45 , respectively. We can see that most estimates are centered around the true values of ( 3,1, − 1 ) with small deviations, although the BS estimator has a little larger standard deviation. The corresponding QQ-plots of them are presented in Figure 3, which show that the resulting estimators seem to converge asymptotically to the normal distribution. Estimated curves under other situations are similar, and are not displayed, but are available in the supplementary materials.

We generated ( Y , Z ) from the following model

Y = exp ( | Z − 25 | 3 5000 ) ε , (11)

where Z ~ U n i f ( 0,50 ) and is independent with the random error ε . Others are the same as defined in Model A. We reported the results in Table 2 and displayed the average estimated curves and their 95% point-wise confidence band under cases (i) with sample size n = 300 for model B in Figure 4. Again, results are similar to those presented in Table 1 and Figures 1-3 are found. More related figures can be available in the supplementary materials.