The paper aims to discuss three interesting issues of statistical inferences for a common risk ratio
(
RR
) in sparse meta-analysis data. Firstly, the conventional log-risk ratio estimator encounters a number of problems when the
number of events in the experimental or control group is zero in sparse data of a 2 × 2 table. The adjusted log-risk ratio estimator with the continuity correction points
Frequently, biostatisticians would like to evaluate the effects of treatments or risk factors in terms of risk difference, relative risk (risk ratio), and/or odds ratio between two independent sample groups (e.g., treatment or control, presence or absence of a risk factor) and binary outcomes (e.g., disease or non-disease, success or failure, dead or alive) in a 2 × 2 table. Let p 1 be the probability of outcome in the treatment or exposed group and p 2 be the probability of outcome in the control or unexposed group. On comparing two independent groups, the popular effect parameters are defined by the risk difference R D = p 1 − p 2 , the relative risk R R = p 1 / p 2 , or the odds ratio O R = p 1 / ( 1 − p 1 ) p 2 / ( 1 − p 2 ) . Obviously, all three effect sizes are related to the estimation of proportion p on each arm. It has widely been known that the conventional proportion estimator p ^ = X / n is a good choice for estimating p in general, but may not be in sparse data. For example, in sparse data coping with the small number of events X and the small sample size n, the variance of p ^ , estimated by p ^ ( 1 − p ^ ) n , can cause a problem with a value of 0 when X = 0 or X = n . To solve the problem, a continuity correction term c is often added to each cell of each group in the 2 × 2 table, yielding to p ^ c = ( X + c ) / ( n + 2 c ) in each arm. Yate [
Our focus of interest is not on the simple adjusted estimate p ^ c = ( X + c ) / ( n + 2 c ) , but on the logarithm of p ^ c instead, leading to the final interest to the logarithm of the risk ratio estimate, θ ^ c = log R R ^ c = log p ^ c 1 − log p ^ c 2 = log ( X 1 + c 1 n 1 + 2 c 1 ) − log ( X 2 + c 2 n 2 + 2 c 2 ) . The reason is that we are interested in the logarithm function, because it is widely known that the risk ratio has a non-symmetric and rather right-skewed distribution about the null value of 1; consequently, the natural logarithm of the risk ratio is needed to transform to be more reliable for the normal distribution. For estimating log ( p + c / n 1 + 2 c / n ) , Pettigrew, Gart and Thomas [
Therefore, the first objective of the study is to find the optimal points ( c 1 , c 2 ) based upon continuity correction of this adjusted log-risk ratio estimator under various settings, such as the smallest bias, the smallest average bias, the smallest MSE, and the smallest average MSE.
Secondly, after obtaining the optimal value of ( c 1 , c 2 ) which is the best choice when considering the smallest bias and/or the smallest MSE of θ ^ c . The next focus of interest is concerned with sparse data in meta-analysis studies that combine the various risk ratios from k studies to produce a single summary risk ratio. Under a common risk ratio overall studies or homogeneity of risk ratios across k studies, the second aim of the study is to find the optimal weights f ^ j of the pooled estimate θ ^ w = ∑ j = 1 k θ ^ c j f ^ j that minimize the MSE of θ ^ w subject to the constraint on ∑ j = 1 k f ^ j = 1 where θ ^ c j = log ( R R ^ c j ) = log ( p ^ c 1 j ) − log ( p ^ c 2 j ) , p ^ c 1 j = ( X 1 j + c 1 ) / ( n 1 j + 2 c 1 ) , p ^ c 2 j = ( X 2 j + c 2 ) / ( n 2 j + 2 c 2 ) . Indeed, these optimal weights f ^ j actually come from the roughly similar formula to the work of Viwatwongkasem et al. [
Finally, the final objective of the study of interest is about comparing the performance of well-known summary effect estimators, such as the Mantel-Haenszel (MH) estimator stated with its variance estimate by Greenland and Robin [
The rest of the paper is organized as follows. Section 2 contains the derivative methods and the results of the adjusted log-risk ratio estimator. Section 3 discusses on the methodology and the outcomes of the minimum MSE weights of the adjusted summary relative risk estimator. Section 4 states on the other well-known estimators and tests as the comparative candidates to compare the performances of making inference for a common relative risk across meta-analysis studies. Section 5 consists of the simulation plans for studying the performances in senses of the estimation, the type I errors and the power of the tests. Section 6 contains the results of Section 5. Section 7 is about the discussion and the recommendation.
In a single study ( k = 1 ), since the conventionally popular estimator of the logarithm of risk ratio, obtained by θ ^ = log R R ^ = log p ^ 1 − log p ^ 2 = log ( X 1 n 1 ) − log ( X 2 n 2 ) , may have a problem when data are sparse. To solve the problem, a continuity correction term ( c 1 , c 2 ) is often added to each cell of the 2 × 2 table, leading to the adjusted estimate as θ ^ c = log R R ^ c = log p ^ c 1 − log p ^ c 2 = log ( X 1 + c 1 n 1 + 2 c 1 ) − log ( X 2 + c 2 n 2 + 2 c 2 ) . In addition, the adjusted risk ratio estimator can be further found as R R ^ c = p ^ c 1 p ^ c 2 = ( X 1 + c 1 ) / ( n 1 + 2 c 1 ) ( X 2 + c 2 ) / ( n 2 + 2 c 2 ) . Due to the work of Pettigrew, Gart and Thomas [
E ( θ ^ c ) = ( log p 1 − log p 2 ) + B ( θ ^ c ) (1)
B ( θ ^ c ) = 2 c 1 − 1 + p 1 { 1 − 2 ( 2 c 1 ) } 2 n 1 p 1 + − 6 c 1 2 + 6 ( 2 c 1 p 1 ) 2 + 12 c 1 q 1 + 4 q 1 ( q 1 − p 1 ) − 9 q 1 2 12 ( n 1 p 1 ) 2 − [ 2 c 2 − 1 + p 2 { 1 − 2 ( 2 c 2 ) } 2 n 2 p 2 + − 6 c 2 2 + 6 ( 2 c 2 p 2 ) 2 + 12 c 2 q 2 + 4 q 2 ( q 2 − p 2 ) − 9 q 2 2 12 ( n 2 p 2 ) 2 ] + O ( n − 3 ) (2)
V ( θ ^ c ) = q 1 n 1 p 1 + q 1 ( 1 − 2 c 1 ) + 1 2 q 1 2 ( n 1 p 1 ) 2 + q 2 n 2 p 2 + q 2 ( 1 − 2 c 2 ) + 1 2 q 2 2 ( n 2 p 2 ) 2 + O ( n − 3 ) (3)
M S E ( θ ^ c ) = q 1 n 1 p 1 + ( c 1 − 2 c 1 p 1 ) ( c 1 − 2 c 1 p 1 − q 1 ) + q 1 ( 1 − 2 c 1 ) + 3 4 q 1 2 ( n 1 p 1 ) 2 + q 2 n 2 p 2 + ( c 2 − 2 c 2 p 2 ) ( c 2 − 2 c 2 p 2 − q 2 ) + q 2 ( 1 − 2 c 2 ) + 3 4 q 2 2 ( n 2 p 2 ) 2 + O ( n − 3 ) (4)
when q 1 = 1 − p 1 and q 2 = 1 − p 2 .
The first setting to find the optimal point ( c 1 , c 2 ) that minimizes the smallest bias of θ ^ c is investigated. The first derivatives of the bias of θ ^ c with respect to c 1 and c 2 are given by
∂ B ( θ ^ c ) ∂ c 1 = 1 + ( − 1 + n 1 ) p 1 − 2 n 1 p 1 2 + c 1 ( − 1 + 4 p 1 2 ) n 1 2 p 1 2 (5)
∂ B ( θ ^ c ) ∂ c 2 = − [ 1 + ( − 1 + n 2 ) p 2 − 2 n 2 p 2 2 + c 2 ( − 1 + 4 p 2 2 ) n 2 2 p 2 2 ] (6)
Setting ∂ B ( θ ^ c ) ∂ c 1 = 0 and ∂ B ( θ ^ c ) ∂ c 2 = 0 , the critical roots ( c 1 , c 2 ) are obtained as follows:
c 1 = − 1 + p 1 − n 1 p 1 + 2 n 1 p 1 2 − 1 + 4 p 1 2 , p 1 ≠ 0.5 (7)
c 2 = − 1 + p 2 − n 2 p 2 + 2 n 2 p 2 2 − 1 + 4 p 2 2 , p 2 ≠ 0.5 (8)
The sufficient conditions of the second-order derivatives of the bias of θ ^ c to guarantee the solutions of (7) and (8) being minimum points are that the determinants D 1 and D 2 , evaluated at critical points, are all positive where D 1 = ∂ 2 B ( θ ^ c ) ∂ c 1 2 and D 2 = | ∂ 2 B ( θ ^ c ) ∂ c 1 2 ∂ 2 B ( θ ^ c ) ∂ c 1 ∂ c 2 ∂ 2 B ( θ ^ c ) ∂ c 1 ∂ c 2 ∂ 2 B ( θ ^ c ) ∂ c 2 2 | . Unfortunately, it is impossible to find the minimum point ( c 1 , c 2 ) such that θ ^ c = log ( X 1 + c 1 n 1 + 2 c 1 ) − log ( X 2 + c 2 n 2 + 2 c 2 ) has the smallest bias for all ( p 1 , p 2 ) . In addition, the solution to (7) and (8) is practically inflexible because it cannot be applied when p 1 and p 2 equal 0.5. Therefore, we further consider the other settings, such as the smallest average bias and the smallest MSE, to find the optimal points ( c 1 , c 2 ) . However, these settings still fail, they cannot provide the optimal points ( c 1 , c 2 ) .
Another successful setting for finding the optimal values ( c 1 , c 2 ) is coming from an average MSE or equivalently known as Bayes risk. Suppose that the squared error loss function is given by Loss = ( θ ^ c − θ ) 2 . The risk as the expectation of loss or the MSE of θ ^ c in such this case, is given by Risk = M S E ( θ ^ c ) = E ( θ ^ c − θ ) 2 . The prior distribution is usually constructed based on the previously scientific knowledge or the prior observed data; especially, in this paper we set the prior distribution based on
Bayesrisk = m ( c 1 , c 2 ) = ∫ 0 1 ∫ 0 1 M S E ( θ ^ c ) g ( p 1 , p 2 ) d p 1 d p 2
| Center j | Treatment | Control | log ( X 1 j n 1 j ) − log ( X 2 j n 2 j ) | ||
|---|---|---|---|---|---|
| X 1 j | n 1 j | X 2 j | n 2 j | ||
| 1 | 3 | 4 | 1 | 3 | 0.811 |
| 2 | 3 | 4 | 8 | 11 | 0.031 |
| 3 | 2 | 2 | 2 | 3 | 0.405 |
| 4 | 2 | 2 | 2 | 2 | 0.000 |
| 5 | 2 | 2 | 0 | 3 | * |
| 6 | 1 | 3 | 2 | 3 | −0.693 |
| 7 | 2 | 2 | 2 | 3 | 0.405 |
| 8 | 1 | 5 | 4 | 4 | −1.609 |
| 9 | 2 | 2 | 2 | 3 | 0.405 |
| 10 | 0 | 2 | 2 | 3 | * |
| 11 | 3 | 3 | 3 | 3 | 0.000 |
| 12 | 2 | 2 | 0 | 2 | * |
| 13 | 1 | 4 | 1 | 5 | 0.223 |
| 14 | 2 | 3 | 2 | 4 | 0.288 |
| 15 | 2 | 4 | 4 | 6 | −0.288 |
| 16 | 4 | 12 | 3 | 9 | 0.000 |
| 17 | 1 | 2 | 2 | 3 | −0.288 |
| 18 | 3 | 3 | 1 | 4 | 1.386 |
| 19 | 1 | 4 | 2 | 3 | −0.981 |
| 20 | 0 | 3 | 0 | 2 | * |
| 21 | 2 | 4 | 1 | 5 | 0.916 |
*Either X 1 j or X 2 j is 0, leading to the infinity value problem of the natural logarithm.
m ( c 1 , c 2 ) = ∫ 0 1 ∫ 0 1 q 1 n 1 p 1 + ( c 1 − 2 c 1 p 1 ) ( c 1 − 2 c 1 p 1 − q 1 ) + q 1 ( 1 − 2 c 1 ) + 3 4 q 1 2 ( n 1 p 1 ) 2 + q 2 n 2 p 2 + ( c 2 − 2 c 2 p 2 ) ( c 2 − 2 c 2 p 2 − q 2 ) + q 2 ( 1 − 2 c 2 ) + 3 4 q 2 2 ( n 2 p 2 ) 2 d p 1 d p 2
m ( c 1 , c 2 ) = ( − 1 + c 1 + 3 c 1 2 ) n 2 2 − n 1 n 2 2 + n 1 2 ( 3.25 − c 2 − 3 c 2 2 + n 2 ) n 1 2 n 2 2 (9)
The first and second order of partial derivatives of m ( c 1 , c 2 ) with respect to c 1 and c 2 are as follows: ∂ m ( c 1 , c 2 ) ∂ c 1 = 1 + 6 c 1 n 1 2 , ∂ m ( c 1 , c 2 ) ∂ c 2 = − 1 − 6 c 2 n 2 2 , ∂ 2 m ( c 1 , c 2 ) ∂ c 1 2 = 6 n 1 2 , ∂ 2 m ( c 1 , c 2 ) ∂ c 2 2 = − 6 n 2 2 , ∂ 2 m ( c 1 , c 2 ) ∂ c 1 ∂ c 2 = 0 . Unfortunately, the result of Bayes risk shows that the critical point ( c 1 , c 2 ) = ( − 1 6 , − 1 6 ) is not a minimum point. With the conditions of D 1 = ∂ 2 m ( c 1 , c 2 ) ∂ c 1 2 and D 2 = | ∂ 2 m ( c 1 , c 2 ) ∂ c 1 2 ∂ 2 m ( c 1 , c 2 ) ∂ c 1 ∂ c 2 ∂ 2 m ( c 1 , c 2 ) ∂ c 1 ∂ c 2 ∂ 2 m ( c 1 , c 2 ) ∂ c 2 2 | , the critical point ( c 1 , c 2 ) is a saddle point since D 2 = − 36 n 1 2 n 2 2 < 0 . However, in particular case, we let c = c 1 = c 2 and try again to find the minimum point. Fortunately, with the condition of n 2 > n 1 , the optimal point c with the smallest average MSE is obtained by c = − 1 6 ; equivalently, with the condition of n 1 > n 2 , the minimum point is c = 1 6 . The solution of c = − 1 6 or c = 1 6 looks well and can be considered as an appropriate one in practice.
Under a common risk ratio overall k studies or homogeneity of risk ratios across k studies, the optimal weights f j are investigated to minimize the MSE of θ ^ w of the form θ ^ w = ∑ j = 1 k θ ^ c j f j , subject to the constraint on ∑ j = 1 k f j = 1 , where θ ^ c j = log ( R R ^ c j ) = log ( p ^ c 1 j ) − log ( p ^ c 2 j ) , p ^ c 1 j = ( X 1 j + c 1 ) / ( n 1 j + 2 c 1 ) and p ^ c 2 j = ( X 2 j + c 2 ) / ( n 2 j + 2 c 2 ) . The MSE of θ ^ w = ∑ j = 1 k θ ^ c j f j , used under a true common risk ratio θ across all k studies, is given by
M S E ( θ ^ w ) = E ( θ ^ w − θ ) 2 = E ( ∑ j = 1 k θ ^ c j f j − θ ) 2
To find the optimal weights f j with the constraint on ∑ j = 1 k f j = 1 , we form the auxiliary function ϕ in extending MSE ( θ ^ w ) under the Lagrange’s method where λ is a Lagrange multiplier as follows:
ϕ = E ( ∑ j = 1 k θ ^ c j f j − θ ) 2 + λ ( ∑ j = 1 k f j − 1 )
ϕ = E ( ∑ j = 1 k θ ^ c j f j ) 2 − 2 θ E ( ∑ j = 1 k θ ^ c j f j ) + θ 2 + λ ( ∑ j = 1 k f j − 1 )
ϕ = V a r ( ∑ j = 1 k θ ^ c j f j ) + ( E ( ∑ j = 1 k θ ^ c j f j ) ) 2 − 2 θ E ( ∑ j = 1 k θ ^ c j f j ) + θ 2 + λ ( ∑ j = 1 k f j − 1 )
ϕ = ∑ j = 1 k f j 2 V a r ( θ ^ c j ) + ( ∑ j = 1 k f j E ( θ ^ c j ) ) 2 − 2 θ ( ∑ j = 1 k f j E ( θ ^ c j ) ) + θ 2 + λ ( ∑ j = 1 k f j − 1 )
ϕ = ∑ j = 1 k f j 2 V a r ( θ ^ c j ) + ( ∑ j = 1 k f j E ( θ ^ c j ) − θ ) 2 + λ ( ∑ j = 1 k f j − 1 )
From the previous section, we have the following results:
V j = V a r ( θ ^ c j ) = q 1 j n 1 j p 1 j + q 1 j ( 1 − 2 c 1 ) + 1 2 q 1 j 2 ( n 1 j p 1 j ) 2 + q 2 j n 2 j p 2 j + q 2 j ( 1 − 2 c 2 ) + 1 2 q 2 j 2 ( n 2 j p 2 j ) 2 + O ( n − 3 )
and
E j = E ( θ ^ c j ) = ( log p 1 j − log p 2 j ) + ( 2 c 1 − 1 + p 1 j { 1 − 2 ( 2 c 1 ) } 2 n 1 j p 1 j − 2 c 2 − 1 + p 2 j { 1 − 2 ( 2 c 2 ) } 2 n 2 j p 2 j ) + O ( n − 2 )
The partial derivatives with respect to λ and f j yield
∂ ϕ ∂ λ = ∑ j = 1 k f j − 1
∂ ϕ ∂ f j = 2 f j V j + 2 ( ∑ j = 1 k f j E j − θ ) E j + λ
Setting ∂ ϕ ∂ λ = 0 , ∂ ϕ ∂ f j = 0 and solving the equations, it yields
λ = − 2 a − 2 b ( ∑ j = 1 k f j E j − θ ) a
f j = ( V j − 1 ( 1 + τ j θ ) a ) − ( V j − 1 τ j a + ∑ j = 1 k τ j E j V j − 1 ) ( ∑ j = 1 k V j − 1 E j ( 1 + τ j θ ) a )
where a = ∑ j = 1 k 1 V j = ∑ j = 1 k V j − 1 , b = ∑ j = 1 k E j V j , τ j = a E j − b .
More details of derivation can be found from the work of Viwatwongkasem et al. [
θ ^ w = ∑ j = 1 k θ ^ c j f ^ j (10)
where
θ ^ c j = log ( R R ^ c j ) = log ( p ^ c 1 j ) − log ( p ^ c 2 j ) , p ^ c 1 j = ( X 1 j + c 1 ) / ( n 1 j + 2 c 1 ) ,
p ^ c 2 j = ( X 2 j + c 2 ) / ( n 2 j + 2 c 2 ) , q ^ c 1 j = 1 − p ^ c 1 j , q ^ c 2 j = 1 − p ^ c 2 j
f ^ j = ( V ^ j − 1 ( 1 − τ ^ j θ ^ p o o l ) a ^ ) − ( V ^ j − 1 τ ^ j a ^ + ∑ j = 1 k τ ^ j E ^ j V ^ j − 1 ) + ( ∑ j = 1 k E ^ j V ^ j − 1 ( 1 + τ ^ j θ ^ p o o l ) a ^ )
E ^ j = ( log p ^ c 1 j − log p ^ c 2 j ) + ( { c 1 − 0.5 + ( 0.5 − 2 c 1 ) p ^ c 1 j } n 1 j p ^ c 1 j − { c 2 − 0.5 + ( 0.5 − 2 c 2 ) p ^ c 2 j } n 2 j p ^ c 2 j ) + O ( n − 2 )
V ^ j = q ^ c 1 j n 1 j p ^ c 1 j + q ^ c 1 j ( 1 − 2 c 1 ) + 0.5 q ^ c 1 j 2 ( n 1 j p ^ c 1 j ) 2 + q ^ c 2 j n 2 j p ^ c 2 j + q ^ c 2 j ( 1 − 2 c 2 ) + 0.5 q ^ c 2 j 2 ( n 2 j p ^ c 2 j ) 2 + O ( n − 3 )
θ ^ p o o l = log ( p ^ 1 ) − log ( p ^ 2 )
where
p ^ 1 = ∑ j = 1 k n 1 j p ^ 1 j ∑ j = 1 k n 1 j = ∑ j = 1 k X 1 j ∑ j = 1 k n 1 j , p ^ 2 = ∑ j = 1 k n 2 j p ^ 2 j ∑ j = 1 k n 2 j = ∑ j = 1 k X 2 j ∑ j = 1 k n 2 j
a ^ = ∑ j = 1 k 1 V ^ j = ∑ j = 1 k V ^ j − 1 , b ^ = ∑ j = 1 k E ^ j V ^ j , τ ^ j = a ^ E ^ j − b ^
Assuming that a normal approximation is reliable, the minimum MSE weights Z-test for testing H 0 : θ = θ 0 , ( θ 0 = log R R 0 ) is
Z c w = ∑ j = 1 k f ^ j θ ^ c j − θ 0 ∑ j = 1 k f ^ j 2 V ^ a r ( θ ^ c j | H 0 ) = ∑ j = 1 k f ^ j θ ^ c j − θ 0 ∑ j = 1 k f ^ j 2 V ^ j
We will reject H 0 at α level for two-sided test if | Z c w | > Z α / 2 where Z α / 2 is the upper 100 ( α t h ) percentile of the standard normal distribution. Alternatively, reject H 0 when the p-value (p) is less than or equal to α where p = 2 ( 1 − Φ ( | Z c w | ) ) and Φ ( Z ) is the cumulative standard normal distribution.
Under a common risk ratio or homogeneity of risk ratios across k studies, we wish to compare the performance of the minimum MSE weighted estimator adjusted by various points c = 1 / 6 , c = 1 / 3 , c = 1 / 2 with the other well-known summary risk ratio estimators, such as the Mantel-Haenszel (MH) estimator and the weighted least square (WLS) estimator or equivalently known as the inverse-variance weighted estimator via a simulation study. According to these well-known estimators, we will present briefly both estimators.
Mantel-Haenszel Weights (MH)
For Mantel-Haenszel (MH) relative risk estimator overall centers/studies from binomial data, the estimator has been proposed by
R R ^ M H = ∑ j = 1 k n 2 j X 1 j / N j ∑ j = 1 k n 1 j X 2 j / N j (11)
where N j = n 1 j + n 2 j .
The variance estimator of the log-relative risk of Mantel-Haenszel was proposed by Green and Robin [
V ^ [ log R R ^ M H ] = ∑ j = 1 k D j ( ∑ j = 1 k R j ) ( ∑ j = 1 k S j ) (12)
where:
D j = ( n 2 j n 1 j t j − x 1 j x 2 j N j ) / N j 2 , R j = x 1 j n 2 j / N j , S j = x 2 j n 1 j / N j , t j = x 1 j + x 2 j and N j = n 1 j + n 2 j . Note that under a binomial sparse-data model, the Mantel-Haenszel relative risk is consistent in sparse stratification.
Assuming that a normal approximation is valid, the Mantel-Haenszel’s Z-test for testing H 0 : θ = θ 0 , ( θ 0 = log R R 0 ) is
Z M H = log R R ^ M H − θ 0 V ^ ( log R R ^ M H | H 0 )
We will reject H 0 at α level for two-sided test if | Z M H | > Z α / 2 where Z α / 2 is the upper 100 ( α t h ) percentile of the standard normal distribution. Alternatively, reject H 0 when the p-value (p) is less than or equal to α where p = 2 ( 1 − Φ ( | Z M H | ) ) and Φ ( Z ) is the cumulative standard normal distribution.
Weighted least square (WLS) estimator
The weighted least square (WLS) estimator or equivalently known as the inverse-variance weighted estimator of the log-relative risk overall centers/studies is
θ ^ W L S = log R R ^ W L S = ∑ j = 1 k w j log R R ^ j / ∑ j = 1 k w j (13)
where
log R R ^ j = log p ^ 1 j − log p ^ 2 j = log ( X 1 j n 1 j ) − log ( X 2 j n 2 j )
w j = 1 V [ log R R ^ j ] = ( 1 − p 1 j n 1 j p 1 j + 1 − p 2 j n 2 j p 2 j ) − 1 (14)
Practically, the weights w j are often replaced by their estimates.
w ^ j = 1 V ^ [ log R R ^ j ] = ( 1 − p ^ 1 j n 1 j p ^ 1 j + 1 − p ^ 2 j n 2 j p ^ 2 j ) − 1 = ( 1 X 1 j − 1 n 1 j + 1 X 2 j − 1 n 2 j )
The variance of the summary estimator θ ^ W L S is given by
V ( θ ^ W L S ) = ∑ j = 1 k w j 2 V [ log R R ^ j ] ( ∑ j = 1 k w j ) 2 = ∑ j = 1 k w j 2 ( 1 w j ) ( ∑ j = 1 k w j ) 2 = 1 ∑ j = 1 k w j (15)
Assuming that a normal approximation is valid, the weighted least square Z-test for testing H 0 : θ = θ 0 , ( θ 0 = log R R 0 ) is
Z W L S = ∑ j = 1 k w ^ j log R R ^ j / ( ∑ j = 1 k w ^ j ) − θ 0 1 / ( ∑ j = 1 k w ^ j )
We will reject H 0 at α level for two-sided test if | Z W L S | > Z α / 2 where Z α / 2 is the upper 100 ( α t h ) percentile of the standard normal distribution. Alternatively, reject H 0 when the p-value (p) is less than or equal to α where p = 2 ( 1 − Φ ( | Z W L S | ) ) and Φ ( Z ) is the cumulative standard normal distribution.
We present here a simulation study using the following designs:
Parameters: Let the common relative risk be some constants (RR = 1, 2 and 4) and generate the baseline proportion p 2 j in the control arm for the jth study from a uniform distribution in which the range corresponds to the values of RR. If R R = 1 , then p 2 j ~ U ( 0 , 0.9 ) ; if R R = 2 , then p 2 j ~ U ( 0 , 0.45 ) ; and if R R = 4 , then p 2 j ~ U ( 0 , 0.23 ) . The correspondent proportion risks in the exposure arm is p 1 j = p 2 j × R R . The sample size n 1 j and n 2 j in each study are fixed and varied as 4, 8, 16, 32 and 100. The number of studies is 1, 2, 4, 8, 16, 32 and 100.
Statistic 1 (estimation of RR): Generate x 1 j and x 2 j from the binomial distribution with parameter ( p 1 j , n 1 j ) and ( p 2 j , n 2 j ) for each study j ( j = 1 , 2 , ⋯ , k ). All summary estimates are calculated. The procedure is replicated 5000 times. From these replicates, we compute bias, variance, and mean square error (MSE) for the adjusted relative risk estimator with the proposed weight to compare the performance with Mantel-Haenszel (MH) estimator and the weighted least square estimator.
Statistic 2 (studying the type I error under H 0 : R R = R R 0 ): Generate x 1 j and x 2 j from the binomial distribution with parameter ( p 1 j , n 1 j ) and ( p 2 j , n 2 j ) and replicate these generations 5000 times for every procedure. From these replicates, the number of null hypothesis rejections when H 0 is true under three Z-tests is counted for the actual type I error.
TheactualtypeIerror = Numberofrejectionsof H 0 when H 0 istrue Numberofreplicates ( 5000 times )
Statistic 3 (studying the power of the test under H 1 ): Before comparing the power of tests, all test statistics should be calibrated to handle the same type I error rate under the null hypothesis. Under the alternative hypothesis with the random effects model, the powers of three candidate tests are compared. We need to revise the parameter setting for studying the power of the test. Let p 2 j be a uniform distribution over [0, 0.25] and we assume that θ j = log R R j follows as θ j = 0.1 + U m = 0.1 + m m ( 2 U − 1 ) where U m is a uniform [ − m m , m m ] random variable for a given m m = 0.2 , 0.4 , 0.6 , or equivalently, U is a uniform over (0, 1). Note that these parameter settings provide: E ( θ j ) = 0.1 and V a r ( θ j ) = ( 2 × m m ) 2 / 12 . Consequently, we still have p 1 j = p 2 j × R R . Binomial variates x 1 j and x 2 j are also generated with parameter ( p 1 j , n 1 j ) and ( p 2 j , n 2 j ) , respectively. All proposed test statistics under this alternative hypothesis are computed and replicated 5000 times. From these replicates, the number of null hypothesis rejection is counted for the power of the test.
Theactualpowerofthetest = Numberofrejectionsof H 0 when H 1 istrue Numberofreplicates ( 5000 times )
Under a constant of relative risk (RR = 1, 2 and 4), the performance in terms of bias, variance, and mean square error (MSE) of several summary relative risk estimators are compared. Results show that increasing k can decrease the variance and the MSE of all estimators and the increase of both n i j can also decrease the variance of all estimators while fixing k. The unbalance cases of n i j ( i = 1 , 2 and j = 1 , ⋯ , k ) have less affected on the order performance of MSE estimators. The summary adjusted relative risk estimator in meta-analysis study of size k has shrinkage estimator to be a simple adjusted relative risk estimator in one single study case. The optimal point ( c = c 1 = c 2 ) providing the bias, variance, and MSE of θ ^ w , adjusted by c = − 1 / 6 is identical to c = 1 / 6 . By these, the numerical evidence has confirmed the derivation process of finding the root c and it is very useful in practice.
For a single center study ( k = 1 ), regardless of a true value of RR, the proposed estimator adjusted by c = 1 / 3 performs the best with smallest MSE.
For a multi-center study of size k, when R R = 1 , the WLS estimator is the best in sense of the smallest MSE ignoring the sample size n i j . In case R R = 2 , k = 4 the proposed estimator adjusted by c performs the best with the smallest MSE. Another issue, when R R = 2 , k = 16 , the MH estimator achieves the smallest MSE when sample sizes are small. The MSE of the WLS estimator and the proposed estimator adjusted by c are close together when the sample sizes are moderate and large ( n i j ≥ 16 ). For the case R R = 4 and k = 4 or 16, the MH estimator well performs generally with the smallest MSE. Some comparisons of the performances of all summary estimators when n 1 j = 4 , 8 and n 2 j = 4 , 8 , 16 , 32 are depicted in Figures 1(a)-(f) for bias; Figures 2(a)-(f) for variance; and Figures 3(a)-(f) for MSE.
In summary for the performance of estimators, regardless of the true values of RR, the MH estimator achieves the best performance with the smallest MSE when the study size is rather large ( k ≥ 16 ) and the sample sizes within each study are small. The MSE of WLS estimator and the proposed-weight estimator adjusted by c = 1 / 6 , c = 1 / 3 , c = 1 / 2 are close together and they are the best when the sample sizes are moderate to large ( n i j ≥ 16 ) while the study size is rather small.
A type I error comparison between the tests is considered by comparing the actual (empirical) type I error ( α ^ ) with the nominal level of significance. In this study, the evaluation of the ability to control type I error probability for two-sided tests is based on Cochran limits as follows.
At α = 0.01 significant level, the actual α ^ value is between [0.005, 0.015].
At α = 0.05 significant level, the actual α ^ value is between [0.04, 0.06].
At α = 0.10 significant level, the actual α ^ value is between [0.08, 0.12].
If the actual type I error or the empirical alpha lies within those Cochran limits, then the statistical test can control type I error rate.
For a single study ( k = 1 ), regardless of the true values of RR, it is unfortunate that almost all tests shown in
For a meta-analysis study of size k, also displayed in
Usually, the empirical power of the tests will be compared under the same type I error value. In summary, the Mantel-Haenszel Z-test performs best when n i j is moderate to large with satisfying the type I error value within Cochran’s range limit, regardless of the study size k. The inverse variance weighted Z-test is good when n i j is large. For the proposed Z-test adjusted by c = 1 , c = 2 , the Z-tests perform well under the same type I error value when k is small ( k ≤ 4 ) and n i j is large ( n i j ≥ 32 ). The results in
| RR | k | n 1 j | n 2 j | Z M H | Z W L S | Z c w ( c = 0.05 ) | Z c w ( c = 1 ) | Z c w ( c = 2 ) | Z c w ( c = 1 / 6 ) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 1 | 16 | 16 | 2.34 | 1.90 | 0.74 | 1.36 | 13.72 | 0.74 |
| 16 | 32 | 3.88 | 3.22 | 2.64 | 2.46 | 10.06 | 2.96 | ||
| 32 | 32 | 3.12 | 3.06 | 2.00 | 2.54 | 4.44 | 2.14 | ||
| 32 | 100 | 5.08 | 4.96 | 3.82 | 4.46 | 9.44 | 3.78 | ||
| 100 | 100 | 4.28 | 4.28 | 3.78 | 3.96 | 4.52 | 3.80 | ||
| 2 | 1 | 16 | 16 | 2.94 | 2.96 | 1.68 | 3.52 | 18.62 | 1.68 |
| 16 | 32 | 2.46 | 0.46 | 1.06 | 2.62 | 14.00 | 1.06 | ||
| 32 | 32 | 2.90 | 2.90 | 1.96 | 3.64 | 14.28 | 1.96 | ||
| 32 | 100 | 3.90 | 3.90 | 2.70 | 4.02 | 6.58 | 2.78 | ||
| 100 | 100 | 4.14 | 4.14 | 3.44 | 4.38 | 8.16 | 3.40 | ||
| 1 | 4 | 16 | 16 | 3.56 | 2.06 | 2.20 | 4.28 | 20.20 | 2.14 |
| 16 | 32 | 4.64 | 4.36 | 4.32 | 5.86 | 15.32 | 4.34 | ||
| 32 | 32 | 5.00 | 4.46 | 4.34 | 5.84 | 10.06 | 4.32 | ||
| 32 | 100 | 4.60 | 6.36 | 5.34 | 6.40 | 8.96 | 5.30 | ||
| 100 | 100 | 4.80 | 4.44 | 5.16 | 5.80 | 6.56 | 5.12 | ||
| 2 | 4 | 16 | 16 | 3.78 | 4.82 | 2.66 | 5.78 | 21.24 | 2.52 |
| 16 | 32 | 4.52 | 3.48 | 2.14 | 5.04 | 18.28 | 2.06 | ||
| 32 | 32 | 4.12 | 5.04 | 2.90 | 5.04 | 10.86 | 2.86 | ||
| 32 | 100 | 5.18 | 4.56 | 3.88 | 5.36 | 8.50 | 3.80 | ||
| 100 | 100 | 5.12 | 5.24 | 4.38 | 5.24 | 6.40 | 4.30 | ||
| 1 | 16 | 16 | 16 | 4.50 | 2.54 | 6.20 | 9.32 | 29.64 | 5.52 |
| 16 | 32 | 4.76 | 6.20 | 7.02 | 10.88 | 24.38 | 6.88 | ||
| 32 | 32 | 5.06 | 3.90 | 7.08 | 9.38 | 15.74 | 7.04 | ||
| 32 | 100 | 5.22 | 10.66 | 7.90 | 10.64 | 14.26 | 7.74 | ||
| 100 | 100 | 4.88 | 4.94 | 7.74 | 8.60 | 9.64 | 7.70 | ||
| 1 | 32 | 16 | 16 | 4.56 | 1.82 | 7.24 | 10.70 | 30.96 | 6.20 |
| 16 | 32 | 4.94 | 9.16 | 9.06 | 14.18 | 27.12 | 8.64 | ||
| 32 | 32 | 4.54 | 3.86 | 7.18 | 9.50 | 15.80 | 7.10 | ||
| 32 | 100 | 4.64 | 15.92 | 9.06 | 13.16 | 17.02 | 8.76 | ||
| 100 | 100 | 5.48 | 5.36 | 8.05 | 8.87 | 9.24 | 7.89 |
Bold Values denote that the statistical tests can control the type I error.
| mm | k | n 1 j | n 2 j | Z M H | Z W L S | Z c w ( c = 0.05 ) | Z c w ( c = 1 ) | Z c w ( c = 2 ) | Z c w ( c = 1 / 6 ) |
|---|---|---|---|---|---|---|---|---|---|
| 0.2 | 1 | 32 | 32 | 2.08# | 2.08# | 0.44# | 2.42# | 9.36 | 0.40# |
| 32 | 100 | 4.80 | 4.80 | 3.24# | 6.38 | 24.02# | 3.24# | ||
| 4 | 32 | 32 | 5.10 | 3.34 | 1.02 | 7.16 | 26.94# | 0.86 | |
| 32 | 100 | 9.74 | 11.68# | 8.16 | 13.30# | 22.84# | 8.00 | ||
| 100 | 100 | 10.68 | 9.38 | 7.92 | 11.70 | 18.18# | 7.70 | ||
| 16 | 32 | 32 | 10.08# | 6.28# | 2.60# | 18.12# | 53.30# | 2.22# | |
| 32 | 100 | 17.80# | 30.98# | 19.70# | 17.28# | 21.16# | 19.72# | ||
| 100 | 100 | 23.44 | 21.46 | 18.70# | 27.02# | 38.56# | 18.24# | ||
| 32 | 32 | 32 | 15.06 | 9.60 | 5.70# | 25.10# | 62.38# | 4.92# | |
| 32 | 100 | 31.08# | 57.40# | 39.24# | 24.68# | 20.74# | 39.84# | ||
| 100 | 100 | 40.02# | 37.30# | 33.50# | 45.56# | 58.48# | 32.70# | ||
| 100 | 100 | 100 | 83.40 | 81.52 | 76.68 | 86.24 | 92.26# | 78.28 | |
| 0.4 | 1 | 32 | 32 | 2.86# | 2.86# | 0.80# | 3.34# | 10.80 | 0.72# |
| 32 | 100 | 7.32 | 7.32 | 5.50# | 8.90 | 26.70 | 5.50# | ||
| 4 | 32 | 32 | 6.78 | 4.60 | 1.80 | 9.64 | 32.10 | 1.68 | |
| 32 | 100 | 13.12 | 15.94# | 11.40 | 17.20# | 27.42 | 11.14 | ||
| 100 | 100 | 16.24 | 14..78 | 12.94 | 17.92 | 25.90 | 12.56 | ||
| 16 | 32 | 32 | 13.54# | 9.74# | 4.92# | 21.88# | 57.70 | 4.34# | |
| 32 | 100 | 24.56# | 42.44# | 27.56# | 23.20# | 26.10 | 27.66# | ||
| 100 | 100 | 32.74 | 31.76 | 27.98# | 36.58# | 47.58 | 27.72# | ||
| 32 | 32 | 32 | 20.70 | 15.72 | 8.84# | 31.88# | 67.96 | 8.22# | |
| 32 | 100 | 39.74# | 68.88# | 47.60# | 32.58# | 27.36 | 48.12# | ||
| 100 | 100 | 53.04# | 52.20# | 47.32# | 56.42# | 66.48 | 46.82# | ||
| 100 | 100 | 100 | 89.74 | 90.14 | 87.24 | 91.42 | 94.80 | 87.04 | |
| 0.6 | 1 | 32 | 32 | 4.74# | 4.74# | 1.70# | 5.22# | 12.94 | 1.64# |
| 32 | 100 | 10.48 | 10.48 | 7.88# | 11.98 | 30.46 | 7.88# | ||
| 4 | 32 | 32 | 10.58 | 10.54 | 5.02 | 16.94 | 58.58 | 4.72 | |
| 32 | 100 | 19.36 | 23.52# | 17.22 | 23.34# | 34.18 | 16.94 | ||
| 100 | 100 | 24.86 | 23.70 | 20.44 | 26.56 | 34.90 | 20.26 | ||
| 16 | 32 | 32 | 20.86# | 16.38# | 9.94# | 30.10# | 64.52 | 9.00# | |
| 32 | 100 | 33.58# | 55.00# | 36.32# | 32.20# | 34.86 | 36.50# | ||
| 100 | 100 | 45.72 | 45.96 | 40.72# | 48.60# | 57.60 | 40.22# |
| 32 | 32 | 32 | 32.26 | 27.66 | 17.90# | 42.94# | 75.32 | 16.84# | |
|---|---|---|---|---|---|---|---|---|---|
| 32 | 100 | 52.74# | 82.28# | 60.64# | 45.68# | 36.78 | 61.36# | ||
| 100 | 100 | 64.74# | 66.66# | 60.44# | 67.14# | 74.80 | 60.08# | ||
| 100 | 100 | 100 | 95.78 | 96.82 | 94.56 | 96.22 | 97.66 | 94.36 |
#denotes that the statistical tests cannot control type I error rate under H 0 : R R = 1 .
The main question rises which continuity correction values are the best choice for the adjusted relative risk estimator in a center study and multi-center study with sparse data. Due to the conventional continuity correction, most of investigators such as Yate [
For estimation of fixing θ ( θ = log R R ) in a center ( k = 1 ), regardless of a true value of RR, the proposed estimator adjusted by c = c 1 = c 2 = 1 / 3 performs the best with the smallest MSE. For a meta-analysis study of size k, in general the MH estimator achieves the smallest MSE when the sample size n i j is small while the study size is rather large ( k ≥ 16 ). The MSE of the WLS estimator and the proposed estimator adjusted by the various values of c are closed together and they are the best when the sample sizes are moderate to large ( n 1 j ≥ 16 and n 2 j ≥ 16 ) while the study size is rather small. This finding is consonant with the work of Viwatwongkasem et al. [
For the empirical power of the test under H 1 : θ j = 0.1 + U m ( − m m , m m ) , regardless of the study size k, the MH Z-test performs the best with the highest power when both n i j are moderate to large. The inverse-variance weighted Z-test is good when n i j is large. In accordance with Soulakova and Bright [
If we don’t know information about parameter RR, we recommend to use the adjusted estimator θ ^ by using continuity correction defined by c 1 = c 2 = 1 / 6 , or 1/3, or 1/2 in a center study. For a multi-center study of size k, we recommend to use adjusted θ ^ w defined by c 1 = c 2 = 1 / 6 , or 1/3, or 1/2, including optimal weights f ^ j .
Generally, the effect of exposures or the effect of treatments with binary outcomes covers the risk difference, the relative risk, and the odds ratio. Obviously, all three conventional effect estimators have the same problem of the zero values in sparse data. The conventional proportion estimate of p ^ = X / n is in need of replacement by p ^ c = ( X + c ) / ( n + 2 c ) to solve this problem. Therefore, the recommendation for a further study is to use these ideas, such as the smallest MSE, the smallest Bayes risk to fine the appropriate point ( c 1 , c 2 ) in estimating the odds ratio parameter.
The study was partially supported for publication by the Faculty of Public Health, Mahidol University, Bangkok, Thailand.
The authors declare no conflicts of interest regarding the publication of this paper.
Viwatwongkasem, C., Srisawad, S., Soontornpipit, P., Sillabutra, J., Satitvipawee, P., Kitidamrongsuk, P. and Chootrakool, H. (2022) Minimum MSE Weighted Estimator to Make Inferences for a Common Risk Ratio across Sparse Meta-Analysis Data. Open Journal of Statistics, 12, 49-69. https://doi.org/10.4236/ojs.2022.121004