One of the most commonly used statistical methods is bivariate correlation analysis. Sometimes it is of interest to compare the correlation of two variables X and Y that have been calculated using two independent samples. For example, in an unpublished Master’s thesis [1], Stuart evaluated several potential biomarkers for severity of symptoms of dry mouth (xerostomia). As part of her assessment of these biomarkers, she examined the association of one of the potential biomarkers, p21, with another potential biomarker, PCNA. Stuart wished to compare two independent groups in terms of the association between p21 and PCNA: 1) healthy subjects and 2) dental patients with xerostomia. An important part of the planning of this study is to ask the question “How large a sample is needed in the two groups to achieve adequate power?”

Tools are widely available for performing sample size and power calculations when the analysis involves the comparison of two Pearson correlation coefficients (PCCs). These include, for example, tables [2], software packages (PASS, nQuery, G*Power), and internet-based tools (e.g., https://www.unistat.com/guide/sample-size-and-power-two-correlations/). However, as best we can determine, there are no easily accessible tools for sample size calculation when the planned analysis will be a comparison of either two Spearman rank correlation coefficients (SCCs) or two Kendall coefficients of concordance (KCCs). The PCC is the most commonly used measure of bivariate association; however, the SCC and KCC are also widely used. For example, Brough et al. [3] used the SCC to measure the associations between peanut protein levels found in various household environments, including dust, surfaces, bedding, furnishings and air. Heist et al. [4] used the KCC as the measure of association in their study of the use of bevacizumab as a chemotherapeutic agent for the treatment of advanced non-small cell lung cancer. The KCC is also frequently used in the analysis of environmental data. For example, Helsel ( [5], pp. 227-228) used the KCC in his examination of concentrations of dissolved iron in water samples.

In May and Looney [6], we provided formulas and charts that can be used to determine the sample size needed for a hypothesis test of a single Spearman or Kendall coefficient. In this article, we extend those results to the situation in which it is desired to compare two independent Spearman or Kendall coefficients.

In Section 2, we briefly describe our previously published methods for sample size determination for a single Spearman or Kendall coefficient [6]. This will facilitate the description of our methods for determining the sample size when comparing two independent coefficients since the results in the two situations are similar in many ways. In Section 3, we present the formulas and charts for comparing two independent Spearman or Kendall coefficients; in Section 4, we provide some notes on the use of the sample size charts; and, in Section 5, we discuss our results.

Let ξ denote the population value of the measure of association to be used (either the PCC, SCC or KCC). Suppose that we wish to test a hypothesis involving ξ. In the two-sided case, for example, the hypotheses to be tested can be stated as:

H 0 : ξ = ξ 0     vs .     H a : ξ ≠ ξ 0 , (1)

where ξ₀ is the pre-specified null value of the desired measure of association and − 1 < ξ 0 < 1 . For the PCC, the most common approach for testing the hypotheses in (1) is to use a test statistic based on the Fisher z-transform of the sample value of the PCC, denoted by r. For any − 1 < r < 1 , the Fisher z-transform of r is given by [7]:

z ( r ) = tanh − 1 r = ln ( ( 1 + r ) / ( 1 − r ) ) / 2 , (2)

which is asymptotically distributed as N ( tanh − 1 ρ , σ z 2 ) , where ρ denotes the population value of the PCC and σ z 2 denotes the asymptotic variance of z(r). The transformation in (2) can be applied to the sample SCC or KCC; this also yields an approximately normally distributed transformed coefficient. For the PCC, σ z 2 = 1 / ( n − 3 ) [7] and, for the KCC, σ z 2 = 0.437 / ( n − 4 ) [8]. For the SCC, the preferred value of σ z 2 depends on ρ_s, the population value of the Spearman coefficient: for | ρ s | < 0.95 , σ z 2 = ( 1 + ρ s 2 / 2 ) / ( n − 3 ) [9]; for | ρ s | ≥ 0.95 , σ z 2 = 1.06 / ( n − 3 ) [8]. The value σ z 2 = 1.06 / ( n − 3 ) is more commonly used for the SCC than the improved value proposed by Bonett and Wright [9] when | ρ s | < 0.95 ; however, the results in this article are based on the Bonett and Wright values.

A test statistic for testing H 0 : ξ = ξ 0 based on the Fisher z-transform is given by:

z ξ 0 = z ( ξ ^ ) − z ( ξ 0 ) c 2 / ( n − b ) , (3)

where ξ ^ denotes the sample estimate of ξ, ξ₀ denotes the hypothesized value of ξ, n denotes the sample size, z ( · ) denotes the Fisher z-transform, and b and c² are obtained from Table 1.

An approximate p-value is obtained using the appropriate tail probability for z_ξ0 given in (3) using the standard normal distribution.

Suppose we wish to test H 0 : ξ = ξ 0 using the test statistic in (3). Let ξ₁ denote the alternative value of the measure of association that we wish to detect with our planned hypothesis test. The required sample size for detecting the value ξ₁ with power φ using a two-tailed test of H 0 : ξ = ξ 0 with significance level α is given by:

n = b + c 2 [ z α / 2 + z 1 − φ z ( ξ 1 ) − z ( ξ 0 ) ] 2 , (4)

^{a# ρ s 0} denotes the null value of the SCC. c 2 = 1 + ( ρ s 0 2 / 2 ) is used in the sample size calculation if | ρ s 0 | < 0.95 ; otherwise 1.06 is used.

where z_γ = upper γ-percentage point of the standard normal distribution, z ( · ) denotes the Fisher z-transform, and b and c² are obtained from Table 1.

For a one-tailed test of H 0 : ξ = ξ 0 , replace z_α/2 in (4) by z_α. If the formula in (4) does not yield an integer value, round up to the next largest integer.

In our previous article [6], we provided charts based on (4) for finding the sample size needed to achieve 80% power for tests of the SCC and KCC using significance level 0.05.

Suppose that we wish to test the equality of two coefficients ξ₁ and ξ₂, where ξ denotes either the SCC or the KCC, and that ξ₁ will be estimated using a sample that is independent of the sample used to estimate ξ₂. In the two-sided case, for example, the hypotheses to be tested are given by:

H 0 : ξ 1 = ξ 2     vs .     H a : ξ 1 ≠ ξ 2 . (5)

Let ξ₀ denote the common value of ξ₁ and ξ₂ in H₀.

We can use the approximate distributional results given in Section 2 for the Fisher z-transform of the estimators of the SCC and KCC to derive a test statistic for testing the hypotheses in (5). Let z ( ξ ^ 1 ) and z ( ξ ^ 2 ) denote the Fisher z-transformed estimators of ξ₁ and ξ₂, respectively. Assume that ξ₁ is estimated using a sample that is independent of the sample used to estimate ξ₂; hence, ξ ^ 1 and ξ ^ 2 are independent. Suppose that the same sample size n will be used to estimate ξ₁ and ξ₂. (The assumption of equal sample sizes will be relaxed in Section 4.5.) Under the null hypothesis H₀ in (5), the random variable z ( ξ ^ 1 ) − z ( ξ ^ 2 ) is asymptotically normally distributed with mean 0 and variance c 2 ( 2 n − b ) , where the appropriate values of b and c² are given in Table 1. A test statistic for testing the hypotheses in (5) is given by:

z ξ 0 = z ( ξ ^ 1 ) − z ( ξ ^ 2 ) 2 c 2 / ( n − b ) , (6)

where ξ ^ 1 and ξ ^ 2 denote the sample estimates of ξ₁ and ξ₂, respectively, n denotes the sample size in each group, z ( · ) denotes the Fisher z-transform, and b and c² are given in Table 1.

An approximate p-value is then obtained by calculating the appropriate tail probability for z_ξ0 in (6) using the standard normal distribution.

For the SCC, the hypotheses in (5) would be written as

H 0 : ρ s 1 = ρ s 2     vs .     H a : ρ s 1 ≠ ρ s 2 . (7)

Let ρ_s0 denote the common value of the two Spearman coefficients under the null hypothesis in (7); namely, ρ s 0 = ρ s 1 = ρ s 2 . The value of c² in the test statistic in (6) when testing two SCCs is therefore c 2 = 1 + ( ρ s 0 2 / 2 ) if | ρ s 0 | < 0.95 ; otherwise c² = 1.06 is used.

Assume that the two independent measures of association ξ₁ and ξ₂ (either two PCCs, two SCCs, or two KCCs) will be estimated using samples of equal size. Let ξ₀ denote the common value of ξ₁ and ξ₂ in the null hypothesis in (5) and let ξ₂₁ denote the alternative value of ξ₂ that one wishes to detect, assuming that ξ₁ = ξ₀. For a two-tailed test of the null hypothesis in (5), the required sample size for detecting the alternative value ξ₂₁ with power φ using a test based on (6) with significance level α is given by:

n = b + 2 c 2 ( z α / 2 + z 1 − φ ) 2 [ z ( ξ 0 ) − z ( ξ 21 ) ] 2 , (8)

where z_γ = upper γ-percentage point of the standard normal distribution, and z ( · ) denotes the Fisher z-transform.

The values of b and c² in (8) are given in Table 1. For a one-tailed test of H 0 : ξ 1 = ξ 2 , replace z_α/2 in (8) by z_α. If the formula in (8) does not yield an integer value, round up to the next largest integer.

A chart for finding the required per-group sample size that will yield 80% power for comparing two independent Spearman coefficients based on samples of equal size using significance level 0.05 is provided in Figure 1 for a two-tailed test and in Figure 2 for a one-tailed test. The corresponding charts for two independent Kendall coefficients are given in Figure 3 and Figure 4, respectively.

To use the chart in Figure 1, first locate the larger of the two SCCs associated with the alternative hypothesis along the horizontal axis. Next, draw a vertical line that intersects with the curve corresponding to the smaller value of the SCC associated with the alternative hypothesis. Finally, draw a horizontal line from the curve to the vertical axis. The point of intersection is the required sample size in each group.

For example, suppose one wishes to test

H 0 : ρ s 1 = ρ s 2     vs .     H a : ρ s 1 ≠ ρ s 2

and that the common null value under H₀ is ρ s 0 = ρ s 1 = ρ s 2 = 0.6 and the alternative value of the SCC that one wishes to detect is 0.4. In other words, the alternative difference that one wishes to detect is ρ s 1 − ρ s 2 = 0.6 − 0.4 = 0.2 . To use Figure 1 to determine the sample size needed in each group to detect this difference with 80% power using a significance level of 0.05, locate max ( | ρ s 1 | , | ρ s 2 | ) = 0.6 on the horizontal axis and locate the curve corresponding to min ( | ρ s 1 | , | ρ s 2 | ) = 0.4 . After drawing the horizontal and vertical lines as described above, we find n = 258 (Figure 5). For ρ_s1 = 0.4 and ρ_s2 = 0.2, locate max ( | ρ s 1 | , | ρ s 2 | ) = 0.4 on the horizontal axis and locate the curve corresponding to min ( | ρ s 1 | , | ρ s 2 | ) = 0.2 ; the resulting sample size is n = 351. For a one-tailed test, the sample sizes obtained using Figure 2 are 204 and 277, respectively.

If we use the KCC instead of the SCC as the measure of association, the required sample size for a two-tailed test, assuming τ₁ = 0.6 and τ₂ = 0.4 is n = 99, according to Figure 3. For τ₁ = 0.4 and τ₂ = 0.2, the sample size is n = 145. For a one-tailed test, the required sample sizes obtained from Figure 4 are 79 and 115, respectively. Note that, in the KCC charts, we use the maximum and minimum values associated with the alternative hypothesis differently than in the SCC charts. For the KCC charts, we first locate min ( | τ 1 | , | τ 2 | ) (rather than the max) along the horizontal axis, and then draw a vertical line that intersects with the curve corresponding to max ( | τ 1 | , | τ 2 | ) (rather than the min).

In this article, we have presented charts that can be used for sample size determination when planning a study in which hypothesis testing will be used to compare two independent Spearman or Kendall coefficients. In addition to the charts, we have provided simple sample size formulas that can be used for more accurate calculations. We have found Microsoft Excel^© to be particularly useful for performing these calculations, and a spreadsheet that accomplishes this is available from the second author.

Despite the widespread use of correlation analysis, it is usually the case that, when planning a study in which correlation will be the primary analysis, little or no attention is given to sample size determination. This general impression was confirmed by our review of studies that used correlation as the primary analysis; none of the 111 studies published in clinical research journals in 2014 provided a power analysis or sample size calculation.

While it is true that two of the key references in this article ( [7] [9] ) are rather old, they provide valid results that are directly relevant to the present article. As stated in the Introduction, we were unable to locate any modern tools (software, tables, graphs, etc.) that can be used to determine the sample size needed for comparing two independent Spearman or Kendall coefficients. Hence, we made use of the classical results from these two articles to develop our new tools for addressing this problem. The present article represents an application of the results in [7] [8], and [9] to extend our results of our recently published article [6], which considered only a single Spearman or Kendall coefficient.

We hope that, by making available the easy-to-use tools presented in this article, analysts will be encouraged to perform sample size calculations for correlation coefficient inference. Given the adverse consequences that can occur when studies are either under- or over-powered, it is extremely important that such calculations be made prior to beginning a research study. Our future research efforts will focus on extending our results in [6] to sample size estimation for the intra-class correlation.

The authors declare no conflicts of interest regarding the publication of this paper.

May, J.O. and Looney, S.W. (2022) On Sample Size Determination When Comparing Two Independent Spearman or Kendall Coefficients. Open Journal of Statistics, 12, 291-302. https://doi.org/10.4236/ojs.2022.122020