The R script consists of two sections, each included in a different appendix, and is available as a Word file (Appendices_with_R_Scripts.docx) in a GitHub repository: https://github.com/josemoraldelarubia579/R_script-for_MVN_K2_and_H_tests.git.

Appendix 1 presents the script for computing the multivariate skewness coefficient (B₁) and its asymptotic significance test (Q), the multivariate kurtosis coefficient (B₂) and its asymptotic significance test (Z), as well as the K² statistic and its significance tests using both asymptotic and bootstrap approaches. The asymptotic significance of the K² statistic represents the probability of obtaining a value greater than or equal to the observed one under a chi-square distribution with 1 + k(k + 1)(k + 2)/6 degrees of freedom.

For the bootstrap approach, both the normative bootstrap distribution and the empirical bootstrap distribution are generated. Let us begin with the normative bootstrap distribution. Given a sample of size n composed of k correlated variables, a normative sample of the same size and dimension is generated to follow a multivariate normal distribution with the same correlational structure. This process begins by creating a square matrix of k independent variables with standard normal distribution, each consisting of n observations. These values are obtained using the probit (quantile) function of the standard normal distribution. First, equally spaced quantile orders are generated, ranging from 0.5/n to 1 − 0.5/n. The quantile orders are then randomly permuted for each variable using a fixed seed (123) to ensure reproducibility [8]. In this way, each of the k variables shares the same set of quantile orders in a different sequence, forming mutually independent vectors.

The resulting matrix of independent Gaussian vectors is multiplied by the lower triangular matrix from the Cholesky decomposition of the original sample’s correlation matrix [15], yielding a normative sample with the same correlational structure as the original. From this sample, 1000 samples are randomly drawn with replacement by rows—not by columns—in order to preserve the correlational structure. The K² statistic is computed for each resample. These 1000 values constitute the normative bootstrap distribution. In this distribution, which replaces the theoretical chi-square distribution with 1 + k(k + 1)(k + 2)/6 degrees of freedom, the probability of observing the original sample’s K² statistic is calculated using the right tail. This yields the bootstrap p-value. The 95th percentile of the distribution serves as the normative bootstrap critical value, which is then used to estimate statistical power.

In contrast, the empirical bootstrap distribution is generated by drawing with replacement 1000 samples directly from the original sample, again using row-wise resampling to preserve the correlational structure. The K² statistic is calculated for each resample, and these 1000 values form the empirical bootstrap distribution. Using the normative bootstrap critical value (i.e., the 95th percentile), the proportion of resamples in which the null hypothesis of multivariate normality is rejected (i.e., when the K² estimate exceeds the critical value) is calculated. This proportion represents the bootstrap estimate of statistical power at the 5% significance level. If the null hypothesis of multivariate normality holds, power should be below 0.5, preferably under 0.2. Conversely, if the null hypothesis is false, power should exceed 0.5, preferably above 0.8. Otherwise, the result may be considered ambiguous or questionable.

Finally, the script enables visualization of the sampling distribution of the K² statistic through a figure that displays several components. The empirical bootstrap distribution is presented as a histogram with an overlaid smooth density curve, estimated using kernel density estimation via the density() function in R’s base Stats package. By default, this function employs a Gaussian kernel and determines the bandwidth with the nrd0 method, an adaptation of Silverman’s rule of thumb (Silverman, 1986). The normative bootstrap and chi-square distributions (with 1 + k(k + 1)(k + 2)/6 degrees of freedom) are represented by smooth density curves computed in the same way. In addition, the observed value of the K² statistic and the critical values from both the normative bootstrap and chi-square distributions are indicated with vertical lines.

Appendix 2 provides the R script for computing Royston’s MVN H-test. Using an asymptotic approach, the script calculates both the p-value and statistical power at a 5% significance level. This test is included as a comparative or complementary method for assessing multivariate normality. The aim is to evaluate the consistency of results across the asymptotic and bootstrap variants of the K² test and the H test in terms of retaining or rejecting the null hypothesis of multivariate normality.

The operation of the script is illustrated using two examples. The first example is executed with data generated from a multivariate normal distribution; the code used to generate these data is provided in Script 1 (Appendix 1), and the resulting sample is shown in Script 2 (Appendix 2). The second example is executed using the non-normal data provided in Appendix 3.

A total of 200 samples—each consisting of 50, 75, 100, 125, 150, 200, 250, or 500 observations and 2, 3, 4, 5, or 6 variables with homogeneous correlations of 0, 0.3, 0.5, 0.7, or 0.9—were drawn from a multivariate t-distribution with five degrees of freedom (MVTD). An additional set of 200 samples with identical characteristics was drawn from a multivariate normal distribution (MVND). For both sets, two variants of the K² test (K²_a and K²_b), along with Royston’s H test, were computed. The R package mvtnorm was used to generate the 400 samples [16]. A single random seed (123) was set at the beginning of each simulation condition to ensure reproducibility [8]. The random number generator state was allowed to advance naturally across successive data generations, so that the 200 samples correspond to independent realizations rather than identical repetitions. The correlation matrices were specified using a compound symmetry (equicorrelation) structure, with unit variances and a common off-diagonal correlation coefficient ρ. Further details can be found in Appendix 4, which is also accessible in the GitHub repository: https://github.com/josemoraldelarubia579/R_script-for_MVN_K2_and_H_tests.git.

Since the samples were generated using the same seed, the hit rate and statistical power data for the three multivariate normality tests represent repeated measurements. Therefore, statistical tests appropriate for repeated measures were applied.

Confidence intervals for the hit rates were calculated using the Wilson method with continuity correction [17]. Differences in hit rates among the five multivariate normality tests were assessed using Cochran’s Q test [18]. Effect size was measured with the eta-squared coefficient [19]. Pairwise comparisons of hit rates were conducted using the paired z-test with an estimated common standard error [20]. Bonferroni correction was applied to control the familywise error rate [21]. Effect size for pairwise comparisons was calculated using a z-based analogue of Cohen’s d: d = z/ n [20]. The resulting d values were interpreted according to Cohen’s thresholds: [0, 0.2) = trivial, [0.2, 0.5) = small, [0.5, 0.8) = medium, and ≥0.8 = large (Cohen, 1988) [22]. All omnibus and pairwise comparisons were performed using SPSS version 27 [23].

The comparison of mean statistical power (based on the 200 samples from the MVT distribution) and its complement (based on the 200 samples from the MVN distribution) was conducted using repeated measures ANOVA. Sample size, number of variables, and homogeneous correlation among variables were included as covariates in the general linear model (GLM), with distribution type (MVT vs. MVN) as a between-subjects factor. The assumption of sphericity—i.e., homogeneity of variances of the differences between all possible pairs of repeated measures—was tested using Mauchly’s test [24]. When this assumption was violated, a multivariate approach to within-subject effects was employed using Pillai’s trace [25], as recommended for repeated measures ANOVA [26]. Effect sizes for each model component were estimated using partial eta-squared coefficients.

Pairwise comparisons were performed using the paired-samples Student’s t-test with Bonferroni correction. Cohen’s d [22], with the Hedges–Olkin correction [27], was used to quantify the effect size. The 95% confidence intervals for the mean statistical power of the three MVN tests, as well as the correlations between statistical power and the three cofactors (in the GLM), were computed using the bias-corrected and accelerated (BCa) percentile method based on 1000 bootstrap samples [7]. The reported correlations correspond to Pearson product-moment coefficients. These analyses were executed using SPSS version 27 [23].

In addition to the violation of the sphericity assumption, the assumption of multivariate normality in the repeated measures component (MVN tests), assessed using Royston’s H test [9], and the assumption of univariate normality in the distribution of residuals for each repeated measure, assessed using Shapiro–Wilk’s W test [28][29], were not satisfied. Therefore, a nonparametric repeated measures ANOVA [30] was also applied. Pairwise comparisons were conducted using the rank-sum test [31]. Effect size in the omnibus test was estimated using Kendall’s W coefficient [32], and in pairwise comparisons using the rank-biserial correlation [33]. The JASP program version 0.19.2 [34] was used to perform these power comparisons.

Three psychometric aptitude scales—assessing mechanical, spatial, and numerical reasoning—were administered to 61 applicants for a Master of Architecture degree. It is assumed that the sample composed of 61 observations of 3-uples follows a multivariate normal distribution. This assumption is tested using both the asymptotic and bootstrap variants of the Mardia’s K² test, as well as Royston’s H test.

The sample was generated using an R script provided in Appendix 1, which also includes the code for running the asymptotic and bootstrap versions of K² test. Appendix 2 presents the three scoring vectors and the code for performing H test.

At a 0.05 significance level, the multivariate distribution—based on 61 observations of 3-tuples—shows symmetry and mesokurtosis, as confirmed by Mardia’s tests. Both versions of the K² test support the assumption of multivariate normality, although the bootstrap variant exhibits lower power (ϕ = 0.016 < 0.2) compared to the asymptotic one (ϕ = 0.2516 < 0.5). Royston’s H test also supports multivariate normality, with low power (ϕ = 0.1216 < 0.2). Notably, when the null hypothesis is not rejected, statistical power should fall below 0.5—ideally under 0.2. Otherwise, the result is contradictory or questionable.

Figure 1 shows that the three sampling distributions associated with the K² test—namely, the asymptotic (chi-square) distribution in green, the normative bootstrap distribution in red, and the empirical bootstrap distribution in yellow—are similar in overall shape. Among them, the normative bootstrap distribution is the least peaked and exhibits the greatest positive skewness. This divergence indicates that the correlation between multivariate skewness and kurtosis in the empirical data inflates the variability of the K² statistic, causing it to behave more like a generalized chi-square variable with substantial tail weight than like the nominal chi-square distribution.

Sample size: n = 61.

Number of variables in the original sample: k = 3.

Significance level: α = 0.05.

Average of the correlation among variables: m(R_3×3) = 0.6846.

Mardia’s multivariate skewness and kurtosis tests

Mardia’s multivariate skewness statistic: B₁ = 4.5598.

Two-tailed p-value for the null hypothesis of multivariate symmetry: p = 0.9186.

Mardia’s multivariate kurtosis statistic: B₂ = −0.5925.

Two-tailed p-value for the null hypothesis of multivariate mesokurtosis:

p = 0.5535.

Asymptotic variant of the MVN K-square test

Mardia’s multivariate normality statistic: MVN K-square = 4.9108.

Degrees of freedom: df = 11

Critical value: 0.95χ² [11] = 19.6751

Figure 1. Histogram (yellow) with density curves for the empirical bootstrap (yellow), normative bootstrap (red), and chi-square (green) distributions. The value of the k-square statistic is indicated by a blue vertical line (K² = 4.9108), the critical value of the normative bootstrap distribution by a red vertical line (0.95 quantile: 0.95qboot = 24.0928), and the critical value of the chi-square distribution with 11 degrees of freedom by a green vertical line (0.95χ² [11] = 19.6751).

Right-tailed p-value under a chi-square distribution with 11 degrees of freedom:

p = 0.9354.

H₀ (multivariate normality) is maintained at the 0.05 level based on the asymptotic p-value (chi-2 distribution).

Asymptotic statistical power for the K-square statistic: ϕ = 0.2516 < 0.5.

If the null hypothesis is rejected, statistical power should exceed 0.5—preferably 0.8 or higher. If the null hypothesis is not rejected, statistical power should be below 0.5—preferably under 0.2. Otherwise, the result is contradictory.

Bootstrap variant of the MVN K-square test

Bootstrap critical value (0.95 quantile): 0.95qboot = 24.0928.

Bootstrap p-value: pboot = 0.9790.

H₀ (multivariate normality) is maintained at the 0.05 level based on the bootstrap p-value.

Bootstrap power: ϕ = 0.016 < 0.2.

Royston’s multivariate normality H test

H statistic: H = 1.7955.

Degrees of freedom: df = 2.6596.

p-value for the null hypothesis of multivariate normality: p = 0.5498.

Statistical power of the H test at a significance level of 0.05: ϕ = 0.1844 < 0.2.

Next, an illustrative example is presented using real-world data with characteristics typical of psychology studies. A non-probability sampling method was employed, resulting in an incidental sample of 509 participants, including 300 women (59%) and 209 men (41%). The questionnaire was administered digitally and began with a request for informed consent. The Subjective Happiness Scale [35], adapted for Mexico and validated in the general Mexican population by Quezada et al. [36], was used as the measurement instrument. Appendix 3 provides the list of variables used in this example, which is available in the GitHub repository at https://github.com/josemoraldelarubia579/R_script-for_MVN_K2_and_H_tests.git. It consists of four Likert-type items, that is, ordinal variables with seven ordered categories. The data frame only needs to be modified to include these four variables —original_data <- data.frame(x1, x2, x3, x4)— and no further changes are required in the scripts presented in Appendices 1 and 2.

The results clearly reject the null hypothesis of multivariate normality, both for the K² test under its asymptotic and bootstrap approaches, and for Royston’s H test under its asymptotic or standard approach, which is the only one considered. Figure 2 displays the histogram with density curves corresponding to the empirical bootstrap, the normative bootstrap, and the chi-square distributions for the K² test.

Figure 2. Histogram (yellow) with density curves for the empirical bootstrap (yellow), normative bootstrap (red), and chi-square (green) distributions. The value of the k-square statistic is indicated by a blue vertical line (k-square = 507.174), the critical value of the normative bootstrap distribution by a red vertical line (0.95qboot = 62.1757), and the critical value of the chi-square distribution with 21 degrees of freedom by a green vertical line (0.95χ²[21] = 32.6706).

Compared to Figure 1, which shows the sampling distribution of the K² statistic with three variables following a multivariate normal distribution, the empirical distribution is much more leptokurtic and positively skewed. This arises because the four items were negatively asymmetric, and two of them were leptokurtic.

A further aspect worth noting is the marked difference in scale between Figure 1 and Figure 2. In Figure 1, the sampling distribution of the K² statistic is concentrated within a narrow range, and the empirical, normative-bootstrap, and chi-square curves overlap closely. By contrast, in Figure 2 the empirical K² values extend over a much wider range, with extreme values far exceeding those expected under multivariate normality. This stretching of the horizontal axis reflects the strong departure from normality, while the clear separation between the three density curves shows that neither the chi-square approximation nor the normative bootstrap can capture the heavy-tailed behaviour of the empirical bootstrap.

Sample size: n = 509.

Number of variables in the original sample: k = 4.

Significance level: α = 0.05.

Average of the correlation among variables: m(R) = 0.4281.

Mardia’s multivariate skewness and kurtosis tests

Mardia’s multivariate skewness statistic: B1 = 261.5066.

Two-tailed p-value for the null hypothesis of multivariate symmetry: p = 0.

Mardia’s multivariate kurtosis statistic: B2 = 15.6738.

Two-tailed p-value for the null hypothesis of multivariate mesokurtosis: p = 0.

Asymptotic variant of the MVN K-square test

Mardia’s multivariate normality statistic: MVN K-square = 507.174.

Degrees of freedom: df = 21.

Critical value: 0.95 _chi-sq[21] = 32.6706.

Right-tailed p-value under a chi-square distribution with 21 degrees of freedom:

p_value = 0.

H_0 (multivariate normality) is rejected at the level 0.05 based on the asymptotic p-value (chi-2 distribution).

Asymptotic statistical power for the K-square statistic: phi = 1.

If the null hypothesis is rejected, the statistical power should exceed 0.5, preferably 0.8 or higher.

If the null hypothesis is not rejected, the statistical power should be below 0.5, preferably under 0.2.

Otherwise, the result is contradictory or questionable.

Bootstrap variant of the MVN K-square test

Bootstrap critical value (0.95 quantile): q_boot(0.95) = 62.1757.

Bootstrap p-value: p_boot = 0.

H_0 (multivariate normality) is rejected at the level 0.05 based on the bootstrap p-value.

Bootstrap power: phi_boot = 1.

Royston’s multivariate normality H test:

H statistic: H = 281.542.

Degrees of freedom: df = 3.9544.

p-value for the null hypothesis of multivariate normality: p < 0.0001.

Statistical power of the H test at a significance level of 0.05: phi = 1.

In the 200 samples drawn from the multivariate t distribution, the hit rate (HR) of the H test was the lowest, and its confidence interval lay entirely below those of the two variants of the Q test, whose intervals overlapped (Table 1 and Figure 3).

Figure 3. Diagram of the hit rates of the three MVN tests based on 200 samples drawn from multivariate t-distributions, 200 samples drawn from multivariate normal distributions, and a combined sample of 400 observations.

Table 1. Hit rates and 95% continuity-corrected Wilson confidence intervals based on 200 samples from multivariate t distributions, 200 samples from multivariate normal distributions, and a combined sample of 400 observations.

Note. Distribution: MTD = 200 samples drawn from multivariate t distribution, MND = 200 samples drawn from multivariate normal distribution, and combined = 400 samples drawn MTV or MND. MVN test: H = Royston’s H test, K²_a = asymptotic variant of the Q test, and K²_b = bootstrap variant of Q test. Statistic: HR = hit rate, LL = lower limit and UL = upper limit of a 95% continuity-corrected Wilson confidence interval.

When comparing HRs among the three MVN tests, a significant difference was found using Cochran’s Q test (Χ²[2, N = 200] = 34, p < 0.001), with a medium effect size (0.06 < η² = 0.085 < 0.14). According to Bonferroni-adjusted paired z tests, the HRs of both Q test variants were significantly higher than that of the H test, although the effect sizes were small (Table 2).

In the combined sample, the HRs of the three MVN tests overlapped (Table 1 and Figure 3). However, when comparing HRs among the three MVN tests, a significant difference was found (Χ²[2, N = 400] = 9.176, p < 0.010), with a small effect size (0.01 < η² = 0.011 < 0.06). After applying Bonferroni’s correction, only one significant difference remained: the HR of the bootstrap variant of the Q test was significantly higher than that of the H test, although the effect size was trivial (Table 2).

The HR of the H test was significantly higher for samples drawn from multivariate normal distributions than from multivariate t-distributions (d = −0.075, se = 0.021, z = −3.526, p < 0.001). In contrast, the HRs of both the asymptotic variant (d = 0.075, se = 0.019, z = 3.948, p < 0.001) and the bootstrap variant (d = 0.025, se = 0.011, z = 2.250, p = 0.024) of the K² test were significantly higher for samples drawn from MVT than from MVN distributions.

Table 2.Pairwise comparisons of hit rates using Bonferroni-adjusted paired z-tests on 200 samples from multivariate t-distributions, 200 samples from multivariate normal distributions, and a combined sample of 400 observations.

Note. md = mean difference in hit rate between MVN test 1 and MVN test 2, se = standard error, z = paired z statistic, p = two-tailed p-value, p_Bonf = Bonferroni-adjusted two-tailed p-value, d = Cohen’s d (effect size measure).

3.4.1. In Samples Drawn from MVT Distributions

In the 200 samples drawn from the MVT distributions, the asymptotic variant of the K² test exhibited the highest mean statistical power, while the H test showed the lowest. However, the 95% BCa confidence intervals for the mean power overlapped across the three MVN tests (Table 3 and Figure 4).

Table 3. Mean statistical powers and 95% BCa confidence intervals based on 200 samples from multivariate t distributions, 200 samples from multivariate normal distributions, and a combined sample of 400 observations.

Note. Point estimates and 95% BCa confidence intervals for the average statistical power (its complement for 200 samples drawn from multivariate normal distributions) based on 1000 simulations.

The sphericity assumption was strongly violated (Mauchly W = 0.075, Χ²[2, N = 200] = 504.182, p < 0.001); therefore, a Huynh-Feldt epsilon correction (ε = 0.528 < 0.7) was applied to adjust the degrees of freedom in the analysis of within-subject effects. Additionally, a multivariate test was included, as it is more appropriate when the sphericity assumption is not met.

Figure 4. Diagram of the mean statistical power (its complement of the statistical power for 200 samples drawn from multivariate normal distributions) of the three MVN tests.

The effect of the MVN test type was not significant (F[1.056, 206.928] = 1.591, p = 0.209, η² = 0.008, ϕ = 0.247), nor were its interactions with sample size (F[1.056, 206.928] = 1.4, p = 0.240, η² = 0.007, ϕ = 0.223), the number of variables (F[1.056, 206.928] = 1.424, p = 0.236, η² = 0.007, ϕ = 0.226), or the homogeneous correlation among variables (F[1.056, 206.928] = 1.881, p = 0.171, η² < 0.010, ϕ = 0.283).

In contrast to these results, the multivariate test of within-subject effects revealed a significant main effect of the MVN test type (Pillai’s Λ = 0.158; F[2, 195] = 18.316, p < 0.001; η² = 0.158; ϕ = 1), as were its significant interactions with two of the three cofactors (Pillai’s Λ = 0.072; F[2, 195] = 7.610, p = 0.001; η² = 0.072; ϕ = 0.944 with sample size and Pillai’s Λ = 0.062; F[2, 195] = 6.465, p = 0.002; η² = 0.062; ϕ = 0.902 with number of variables). The effect size of the MVN test type on power was large, and the effect sizes of the interactions were medium. However, the interaction between the MVN test type and the homogeneous correlation among variables was not significant (Pillai’s Λ = 0.010; F[2, 195] = 0.964, p = 0.383; η² = 0.010; ϕ = 0.216).

Based on Pearson’s product-moment correlation, with confidence intervals obtained using the BCa method, the statistical power of the three MVN tests was independent of the homogeneous correlation among variables. However, power was positively correlated with sample size. Additionally, the power of both K² test variants was positively correlated with the number of variables. The strength of association was low for the H test and the asymptotic variant of the K² test, and moderate for the bootstrap variant (see Table 4).

In the pairwise comparisons, only one of the three differences was significant both before and after applying the Bonferroni correction. The asymptotic variant of the K² test showed significantly higher mean power than its bootstrap variant, with a small effect size (Hedges-Olkin g = 0.455, 95% asymptotic CI: 0.310, 0.600). See Table 5.

Apart from the failure to comply with the assumption of sphericity, the assumption of multivariate normality across the three repeated measures was not met (Royston’s test: H = 342.06, df = 3.013, p < 0.001, ϕ = 1). Additionally, univariate

Table 4. Correlations between the mean statistical powers of the three MVN tests and the three cofactors based on 200 samples drawn from multivariate t distributions.

Note. ϕ = average statistical power (its complement for 200 samples drawn from multivariate normal distributions); LL = lower limit, and UL = upper limit of the 95% BCa confidence interval based on 1000 simulations.

Table 5. Post hoc pairwise comparisons of mean statistical power between MVN tests (Bonferroni-adjusted) based on 200 samples from multivariate t distributions.

Note. md = mean difference in statistical power between Test 1 and Test 2 (LL = lower limit, and UL = upper limit, of the 95% asymptotic confidence interval adjusted using Bonferroni correction to control the Type I error rate for an alpha level of 0.05); se = asymptotic standard error; t[df] = paired t-test statistic [degrees of freedom = 196]; p = two-tailed asymptotic p-value; p_Bonf = two-tailed Bonferroni-adjusted asymptotic p-value.

normality was violated in the residuals for each dependent variable (Shapiro-Wilk W = 0.312, p < 0.001 for H; W = 0.472, p < 0.001 for K²_a; W = 0.724, p < 0.001 for K²_b). Consequently, the result of the analysis of variance should be interpreted with caution.

The Friedman test revealed a statistically significant difference in the central tendency of statistical power among the three MVN tests (Χ²[2, N = 200] = 174.162, p < 0.001), with a medium effect size for the MVN test type on statistical power (Kendall W = 0.435).

Table 6. Pairwise comparisons of statistical power using the Conover rank-sum test (Bonferroni-adjusted) based on 200 samples from multivariate t distributions.

Note. wᵢ = sum of ranks for Test 1, wⱼ = sum of ranks for Test 2, |t| = absolute value of the t-test statistic (degrees of freedom: df = 398), p = asymptotic two-tailed p-value, p_Bonf = Bonferroni-adjusted p-value for an alpha level of 0.05, |r_rb| = absolute value of the rank-biserial correlation, used as a measure of effect size.

Pairwise comparisons using the Conover rank sum test indicated that two of the three differences were statistically significant, even after applying the Bonferroni correction, with a large effect size based on biserial rank correlation. The central tendency of statistical power for the asymptotic variant of the K² test was significantly higher than that of both its bootstrap variant and the H test (Table 6).

3.4.2. In Samples Drawn from MVN Distributions

In the 200 samples drawn from the MVN distribution, the two variants of the K² test exhibited the lowest mean statistical power (HR = 0.235, 95% BCa CI: 0.202, 0.266 for asymptotic variant and HR = 0.234, 95% BCa CI: 0.203, 0.265), while H test showed the highest (HR = 0.268, 95% BCa CI: 0.244, 0.290). Since the null hypothesis should be retained in these 200 tests, the closer the statistical power is to 0, the better the test performs. However, the 95% BCa confidence intervals for the mean statistical power of the three MVT tests overlapped.

The sphericity assumption was violated (Mauchly W = 0.869, Χ² [2, N = 200] = 27.456, p < 0.001); therefore, a Huynh-Feldt epsilon correction (ε = 0.884 > 0.7) was applied to adjust the degrees of freedom in the analysis of within-subject effects. Additionally, the multivariate test was included.

The effect of the MVN type of test was significant (F[1.768, 346.492] = 34.591, p < 0.001, η² = 0.150, ϕ = 1), as were its interactions with sample size (F[1.768, 346.492] = 11.54, p < 0.001, η² = 0.056, ϕ = 0.988) and the number of variables (F[1.768, 346.492] = 94.702, p < 0.001, η² = 0.326, ϕ = 1). The effect size was large for the main effect and the interaction with the number of variables, while it was small for the interaction with sample size. However, the interaction between the MVN type of test and the homogeneous correlation among variables was not significant (F[1.768, 346.492] = 1.843, p = 0.165, η² = 0.009, ϕ = 0.360).

The multivariate test for the analysis of within-subject effects yielded very similar results. The effect of the MVN test type was significant (Pillai’s Λ = 0.350; F[2, 195] = 52.423, p < 0.001; η² = 0.350; ϕ = 1) as were its interactions with sample size (Pillai’s Λ = 0.091; F[2, 195] = 9.706, p < 0.001; η² = 0.091; ϕ = 0.981) and number of variables (Pillai’s Λ = 0.588; F[2, 195] = 139.404, p < 0.001; η² = 0.588; ϕ = 1). The effect size was large for both the main effect and the interaction with number of variables, while it was medium for the interaction with sample size. However, the interaction between the MVN test type and the homogeneous correlation among variables was not significant at the 5% level of significance, though it was significant at the 10% level (Λ of Pillai = 0.029; F[2, 394] = 2.871, p = 0.059; η² = 0.029; ϕ = 0.557).

Based on Pearson’s product–moment correlations, with confidence intervals obtained using the BCa method, the statistical power of the two variants of the Q test was independent of the homogeneous correlation among variables. This cofactor was only negatively correlated with the power of the H test, and the strength of this association was small. Sample size was negatively correlated with the power of both the H test and the bootstrap variant of the K² test, and positively correlated with the power of the asymptotic variant. In all three cases, the strength of association was small. The number of variables was negatively correlated with the power of both K² test variants, showing a strong association with the asymptotic variant and a moderate association with the bootstrap variant; in contrast, it was positively correlated with the power of the H test, with a moderate strength of association (Table 7).

Table 7. Correlations between the mean statistical powers of the three MVN tests and the three cofactors based on 200 samples drawn from multivariate normal distributions.

Note. n = sample size, k = number of variables, and r = homogeneous correlation among variables. *An asterisk superscript indicates statistical significance at the 0.05 confidence level, as the confidence interval obtained via the BCa method (based on 1000 simulations) includes zero.

In the pairwise comparisons, two significant differences were observed before applying the Bonferroni correction. Both variants of the K² test showed lower mean statistical power than the H test. However, the effect sizes were trivial and not statistically significant (Hedges-Olkin g = 0.098, 95% asymptotic CI: −0.041, 0.237 for H - K²_a; and g = 0.120, 95% asymptotic CI: −0.019, 0.259 for H - K²_b). After applying the Bonferroni correction, neither of these two differences remained statistically significant (Table 8).

Table 8. Post hoc pairwise comparisons of mean statistical power between MVN tests (Bonferroni-adjusted) based on 200 samples from multivariate normal distributions.

Note. md = mean difference in statistical power between Test 1 and Test 2 (LL = lower limit, and UL = upper limit, of the 95% asymptotic confidence interval adjusted using Bonferroni correction to control the Type I error rate for an alpha level of 0.05); se = asymptotic standard error; t[df] = paired t-test statistic [degrees of freedom = 196]; p = two-tailed asymptotic p-value; p_Bonf = two-tailed Bonferroni-adjusted asymptotic p-value.

Apart from the violation of the sphericity assumption, the assumption of multivariate normality across the three repeated measures was not met (Royston’s test: H = 132.84, df = 2.939, p < 0.001, ϕ = 1). Additionally, univariate normality was violated in the residuals for each dependent variable (Shapiro-Wilk: W = 0.966, p < 0.001 for H; W = 0.939, p < 0.001 for K²_a; W = 0.960, p < 0.001 for K²_b). Although these deviations are smaller than those observed in the general linear model based on 200 samples drawn from multivariate t-distributions, the results of the analysis of variance should still be interpreted with caution.

The Friedman test revealed a statistically significant difference in the central tendency of statistical power among the three MVN tests (Χ²[2, N = 200] = 275.16, p < 0.001), with a large effect size of test type on power (Kendall W = 0.688).

In pairwise comparisons using the Conover rank sum test, all three differences in central tendency were statistically significant, even after applying the Bonferroni correction. Based on the rank-biserial correlation, the effect size was large in both comparisons involving the H test, and small in the comparison between the two K² test variants. The H test had the highest sum of ranks (Table 9).

Table 9.Pairwise comparisons of statistical power using the Conover rank-sum test (Bonferroni-adjusted) based on 200 samples from multivariate normal distributions.

Note. wᵢ = sum of ranks for Test 1, wⱼ = sum of ranks for Test 2, |t| = absolute value of the t-test statistic (degrees of freedom: df = 398), p = asymptotic two-tailed p-value, pBonf = Bonferroni-adjusted p-value for an alpha level of 0.05, |r_rb| = absolute value of the rank-biserial correlation, used as a measure of effect size.

Table 10. Critical values for the K² test at the 10% and 5% significance levels.

Note. k = number of variables; n = sample size. Urzua = critical values for the K² test at the 10% and 5% significance levels, based on Urzúa (1997, p. 352); Boot = bootstrap critical value (i.e., the 0.90 or 0.95 quantile of the normative bootstrap distribution); Chi-2 = 0.90 or 0.95 quantile of the chi-squared distribution with 1 + k(k + 1))(k + 2)/6 degrees of freedom.

The 10% and 5% critical values reported by Urzua (1997, p. 352) [3], based on Monte Carlo simulations, were lower than the corresponding critical values from both the chi-square distribution with 1 + k(k + 1)(k + 2)⁄6 degrees of freedom and those generated by the script used in this study (from the normative bootstrap distribution). The latter were the highest.

The average absolute difference between Urzua’s critical values and those from the chi-square distribution was 12.31, while the difference with the critical values obtained by the script was 10.01 (Table 10).

3.4.3. In the Combined Sample of 400 Observations

In the combined sample, the asymptotic variant of the K² test exhibited the highest mean power, while the H test showed the lowest. However, the 95% BCa confidence intervals for the mean statistical power overlapped among the three tests (Table 3 and Figure 4).

The sphericity assumption was violated (Mauchly W = 0.758, Χ²[2, N = 400] = 109.11, p < 0.001); therefore, a Greenhouse-Geisser epsilon factor (ε = 0.805 > 0.7) was applied to adjust the degrees of freedom in the analysis of within-subject effects and the multivariate test was included.

The effect of the MVN test type was significant (F[1.61, 636.127] = 6.652, p = 0.003, η² = 0.017, ϕ = 0.864), as were its interactions with sample size (F[1.61, 636.127] = 7.062, p = 0.002, η² = 0.018, ϕ = 0.884) and number of variables (F[1.61, 636.127] = 22.406, p < 0.001, η² = 0.054, ϕ = 1). In all three cases, the effect size was small. However, the interaction between MVN test type and homogeneous correlation among variables was not significant (F[1.61, 636.127] = 0.055, p = 0.914, η² < 0.001, ϕ = 0.058), nor was the interaction with distribution type (F[1.61, 636.127] = 0.526, p = 0.552, η² = 0.001, ϕ = 0.128).

The multivariate test for the analysis of within-subject effects yielded the same result. The effect of the MVN test type was significant (Pillai’s Λ = 0.022; F[2, 394] = 4.479, p = 0.012; η² = 0.022; ϕ = 0.765) as were its interactions with sample size (Pillai’s Λ = 0.052; F[2, 394] = 10.700, p < 0.001; η² = 0.052; ϕ = 0.99) and number of variables (Pillai’s Λ = 0.082; F[2, 394] = 17.639, p < 0.001; η² = 0.082; ϕ = 1). All three significant effects were small. The other two interactions involving the MVN test type were not significant: neither with homogeneous correlation among variables (Pillai’s Λ = 0; F[2, 394] = 0.037, p = 0.963; η² = 0; ϕ = 0.056), nor with distribution type (Pillai’s Λ = 0.005; F[2, 394] = 1.026, p = 0.359; η² = 0.005; ϕ = 0.229).

Based on Pearson’s product-moment correlation, with confidence intervals obtained using the BCa method, the statistical power of the three tests was independent of the homogeneous correlation among variables. The statistical power of the H test correlated positively with sample size, with a small strength of association (r = 0.114, 95% BCa CI: 0.034, 0.270). The statistical power of the asymptotic variant of the K² test correlated positively with the number of variables, with a moderate strength of association. The statistical power of the bootstrap variant of the K² test correlated positively with both sample size and number of variables, with a small strength of association in each case (Table 11).

Table 11.Correlations between the statistical powers of the three MVN tests and the three cofactors based on a combined sample of 400 observations.

Note. n = sample size, k = number of variables, and r = homogeneous correlation among variables. * An asterisk superscript indicates statistical significance at the 0.05 confidence level, as the confidence interval obtained via the BCa method (based on 1000 simulations) includes zero.

In the pairwise comparisons, only one of the three differences was statistically significant before applying the Bonferroni correction. The asymptotic approximation of the K² test showed significantly higher mean power than the H test, with a trivial effect size (Hedges-Olkin g = −0.104, 95% asymptotic CI: −0.202, −0.006). However, after applying the Bonferroni correction, none of the differences remained statistically significant (see Table 12).

Table 12. Post hoc pairwise comparisons of mean statistical power between MVN tests (Bonferroni-adjusted) based on a combined sample of 400 observations.

Note. md = mean difference in statistical power (its complement for 200 samples drawn from multivariate normal distributions) between Test 1 and Test 2 (LL = lower limit, and UL = upper limit, of the 95% asymptotic confidence interval adjusted using Bonferroni correction to control the Type I error rate for an alpha level of 0.05); se = asymptotic standard error; t[df] = paired t-test statistic [degrees of freedom = 395]; p = two-tailed asymptotic p-value; p_Bonf = two-tailed Bonferroni-adjusted asymptotic p-value.

It should be noted that the assumption of homogeneity across the matrices of observed covariances of the dependent variables was not met (Box M = 987.872, F[6, 1147681.811] = 163.3, p < 0.001), nor was the assumption of homogeneity of variance between the residuals of two dependent variables (F[1, 398] = 30.373, p < 0.001 for K²_a and F[1, 398] = 109.609, p < 0.001 for K²_b). However, this assumption was met for the H test (F[1, 398] = 0.044, p = 0.833). Additionally, the assumption of multivariate normality was violated across the three repeated measures (Royston’s test: H = 368.72, df = 3.003, p < 0.001, ϕ = 1), as well as the univariate normality in the residuals for each dependent variable (Shapiro-Wilk W = 0.377, p < 0.001 for H; W = 0.905, p < 0.001 for K²_a; W = 0.947, p < 0.001 for K²_b). Consequently, the results of the analysis of variance should be interpreted with caution.

The Friedman test revealed a statistically significant difference in the central tendency of statistical power among the three MVN tests (Χ²[2, N = 400] = 84.241, p < 0.001), with a small effect size of test type on statistical power (Kendall W = 0.105).

In the pairwise comparisons using the Conover rank sum test, all three differences in central tendency were significant, even after applying the Bonferroni correction. The effect size was medium for the comparison between the asymptotic variant of the K² test and the H test, and small for the other two comparisons, based on rank-biserial correlation (Table 13).

Table 13. Pairwise comparisons of statistical power using the Conover rank-sum test (Bonferroni-adjusted) based on a combined sample of 400 observations.

Note. wᵢ = sum of ranks for Test 1, wⱼ = sum of ranks for Test 2, |t| = absolute value of the t-test statistic (degrees of freedom: df = 798), p = asymptotic two-tailed p-value, p_Bonf = Bonferroni-adjusted p-value for an alpha level of 0.05, |r_rb| = absolute value of the rank-biserial correlation, used as a measure of effect size.