Mehta and Patel [7] clarify that the SPSS program calculates the two-tailed probability using the definition in Equation (7). In this equation, R represents the variable or distribution of the number of runs with parameters n₀ and n₁, r is the number of runs in the sample, and E(R) is the mathematical expectation of the distribution.

p 2   c o l a s = P ( | R − E ( R | n 0 , n 1 ) | ≥ | r − E ( R | n 0 , n 1 ) | ) (7)

Following Equation (7), the sides of the distribution are defined by the change in sequence of |r – E(R)|, with the values of r ordered from 2 (minimum) to maximum. The downward, or negative sequence with no absolute value, occurs when r ≤ E(R) o r − E(R) ≥ 0, forming the left side. The ascending, or positive sequence with no absolute value, occurs when r > E(R) or r − E(R) < 0, forming the right-hand side. Therefore, the two sides of the distribution are defined based on the mathematical expectation of R (Equation (8)).

E ( R | n 0 , n 1 ) = 2 × n 0 × n 1 n 0 + n 1 + 1 (8)

Next, the one-sided probability corresponding to the side on which r is located is calculated. Let r be the number of runs in a sample of size n from a variable X with n₀ sample data less than the criterion (median, arithmetic mean, mode, or other statistic of the n sample data of X) and n₁ sample data greater than or equal to the criterion, where n₀ + n₁ = n. When r is located on the left side, the point probabilities from 2 to r are summed (Equation (9)). When r is located on the right side, the point probabilities from r to the maximum (max(R) = n when n₀ = n₁ or 2 × min(n₀, n₁) + 1 when n₀ ≠ n₁) are summed (Equation (10)).

r ≤ E ( R | n 0 , n 1 ) p l e f t t a i l = P ( R ≤ r ) = ∑ i = 2 r P ( R = i ) (9)

r > E ( R | n 0 , n 1 ) p r i g h t t a i l = P ( R ≥ r ) = ∑ i = r max ( R ) P ( R = i ) (10)

To each value of r corresponds a single value |r − E(R)|. If n₀ = n₁, each value of |r − E(R)| corresponds to two values of r, one on each side of the distribution. However, if n₀ ≠ n₁, it corresponds to a single value. To obtain the two-tailed probability, the one-tailed probability of r is added to the probability toward the other tail of the value of R that is equal (when n₀ = n₁) or immediately greater (when n₀ ≠ n₁) than |r − E(R)| in that tail. If there is none, the probability to the other tail is 0. In Equation (11), the algorithm for calculating the exact two-tailed probability for a value of r on the left side is shown; such value is denoted by r_LS, and the value of R corresponding to the right side is denoted by r_RS. In Equation (12), it is shown for a value of r on the right side, which is denoted by r_RS, and the value of R corresponding to the left side by r_LS.

r L S ≤ E ( R | n 0 , n 1 ) p l e f t t a i l = P ( R ≤ r L S → | r L S − E ( R ) | ) = ∑ i = 2 r L S P ( R = i ) p o t h e r t a i l = P ( R ≥ r R S →   ≥ | r L S − E ( R ) | ) = ∑ i = r R S max ( R ) P ( R = i ) p 2   t a i l s = P ( | R − E ( R ) | ≥ | r L S − E ( R ) | ) = p l e f t t a i l + p o t h e r t a i l (11)

r R S > E ( R | n 0 , n 1 ) p r i g h t t a i l = P ( R ≥ r R S ) = ∑ i = r R S max ( R ) P ( R = i ) p o t h e r t a i l = P ( R ≤ r L S →   ≥ | r R S − E ( R ) | ) = ∑ i = 2 r L S P ( R = i ) p 2   t a i l s = P ( | R − E ( R ) | ≥ | r R S − E ( R ) | ) = p r i g h t t a i l + p o t h e r t a i l (12)

See the distribution of the number of runs (R) with parameters: n₀ = 7 (values less than the criterion: median) and n₁ = 2 (values greater than or equal to the criterion), where n = n₀ + n₁ = 7 + 2 = 9 (sample size). In this distribution taken as an example, the minimum value is: min(R|n₀ = 7, n₁ = 2) = 2, the maximum value is: max(R|n₀ = 7, n₁ = 2) = 2 × min(n₀, n₁) +1 = 5, and the expected value is: E(R|n₀ = 7, n₁ = 2) = 1 + (2 × n₀ × n₁)/n = 4.111. If a significance level of 0.1 is adopted, taking into account the small sample size (n = 9), the null hypothesis of randomness is rejected with a number of runs of 2. However, with three other numbers of runs (r = 3, 4, or 5), the null hypothesis holds. Table 1 shows the exact point probabilities, calculated using Equations (5) and (6), as well as the one-tailed probability, obtained using Equations (9) and (10), and the other-tailed and two-tailed probabilities, computed using Equations (11) and (12).

The R program calculates the two-tailed exact probability by simply doubling the

Note. r = number of runs, P(R = r) = exact point probability, |r − E(R)| = absolute difference between the number of runs and the mathematical expectation of the R|n₀, n₁ distribution, Side = Left when r ≤ E(R) = 4.111 and Right when r > E(R), One-tailed p-value = sum of the exact point probabilities from 2 to r on the left side and from r to the maximum on the right side, r_{other side} = value of r on the other side of the distribution that corresponds to a value immediately greater than |r − E(R)|, other-tail p-value = one-tailed probability value of r_{other side} y Two-tailed p-value = sum of the one-tailed and the other-tailed probability values.

one-tailed exact probability [3] . It’s important to note that a value greater than 1 may appear (Equation (13)). The distribution is divided into two parts based on the mathematical expectation or arithmetic mean of the number of runs (Equation (8)). Values of R less than or equal to its mean constitute the left-hand side of the distribution, while values of R greater than the mean form the right-hand side. If r is less than or equal to E(R), the exact probability for a tail (left tail) is obtained by adding the exact point probabilities from 2 (minimum value of R) to r (Equation (13)). If r is greater than E(R), the one-tailed exact probability (right tail) is calculated by summing the pointwise exact probabilities from r to the maximum of R: n₀ + n₁ when n₀ = n₁ ≤ n or 2 × min(n₀, n₁) +1 when n₀ ≠ n₁ (Equation (14)).

p 2   t a i l s = 2 × p 1   t a i l (13)

p 1   t a i l = { ∑ i = 2 r P ( R = i ) r ≤ E ( R ) ∑ i = r max ( R ) P ( R = i ) r > E ( R ) (14)

The script for executing the runs test using the R program is displayed. Hypotheses can be formulated as two-tailed (alternative = “two.sided”), left-tailed (alternative = “left.sided”), or right-tailed (alternative = “right.sided”). The dichotomization criterion for the sample is specified in ‘threshold,’ with the default being the median. If the data are dichotomous with coding 0 and 1, the value of 0.5 can serve as the dichotomization criterion. The probability value calculation can be exact (p value = “exact”) or asymptotic (p value = “default”). By default, it provides the asymptotic probability, unless n₀ + n₁ is less than 10. A two-dimensional plot can be included (plot = TRUE) or excluded (plot = FALSE). This plot allows visualization of the sequence of the n sample data. On the abscissa axis, the random order of the sample data is displayed, and on the ordinate axis, the value of the data on the X scale.

In Table 2, the previous example is revisited. Using the R program, the null hypothesis of randomness holds with the four values of the distribution of the number of runs with parameters: n₀ = 7 (values less than the criterion: median) and n₁ = 2 (values greater than or equal to the criterion) in a two-tailed test, as corresponds to this type of hypothesis, and a level of significance of 0.01 considering the small sample size (n = n₀ + n₁ = 9).

Like the R program, the RSUE package also has a simple procedure for calculating exact two-tailed probabilities [4] . The two parts of the distribution are defined by the median of the number of runs or the value of R with a cumulative probability of at least 50%. If the number of runs is less than the median of R, the one-sided probability is the sum of the point probabilities from 2 to r, and the two-sided probability is twice this sum (Equation (15)).

r < M d n ( R | n 0 , n 1 ) ⇒ P ( R ≤ M d n ( R | n 0 , n 1 ) ) ≥ 0.5 p 1   t a i l = ∑ i = 2 r P ( R = i ) p 2   t a i l s = 2 × ∑ i = 2 r P ( R = i ) (15)

If the number of runs is greater than or equal to the median of R, the one-tailed probability is the complement of the sum from 2 to r, and the two-tailed probability is twice that complement (Equation (16)).

Note. r = number of runs, P(R = r) = exact point probability, Side = Left when r ≤ E(R) = 4.111 and Right when r > E(R), One-tailed p-value = sum of the exact point probabilities from 2 to r on the left side and from r to the maximum on the right side, and Two-tailed p-value = doubling of one-tailed probability value.

r ≥ M d n ( R | n 0 , n 1 ) p 1   t a i l = 1 − ∑ i = 2 r P ( R = i ) p 2   t a i l s = 2 × ( 1 − ∑ i = r n P ( R = i ) ) (16)

In Table 3, the example seen previously with the SPSS and R programs is revisited. The null hypothesis of randomness is rejected with a value of r equal to 5, even though the point probability is not zero, P(R = 5) = 0.417, and this hypothesis holds when tested using SPSS and R program. This result is because the value of r equal to 5 accumulates 100% of the distribution, P(R ≤ 5) = 1, so the complement of its cumulative probability is 0 (one-tailed probability), and the duplicate complement is 0 (two-tailed probability).

The exact two-tailed probability is obtained by adding the one-sided probability of the side where r is located and the probability corresponding to the other tail. Taking the median as the axis of symmetry, a simple procedure to find the probability corresponding to the other tail is through the quantile order when r is located on the left side and the complementary quantile order when r is situated on the right side.

We begin by calculating the median of R to define the left and right sides of the distribution. The median of R is the value that accumulates at least 50% of the probability mass of the Wald-Wolfowitz distribution of the number of runs [2] . Both the probabilities on the left side of the distribution, from 2 to r, and the probabilities on the right side, from r to the maximum: n (n₀ = n₁) or 2 × min(n₀, n₁) + 1 (n₀ ≠ n₁), are calculated. Additionally, the mode of R (the r value with the highest probability) is computed, and Bickel’s robust skewness measure based on the mode is calculated. This measure is obtained by subtracting from one twice

Note. r = number of runs, P(R = r) = exact point probability, P(R ≤ r) = exact cumulative exact probability, Side = Left when r < Mdn(R) = 4 and Right when r ≥ Mdn(R), One-tailed p-value = sum of the exact point probabilities from 2 to r on the left side and from r to the maximum on the right side, and Two-tailed p-value = doubling of one-tailed probability value.

the cumulative distribution function evaluated at the mode [12] . Denoted by A_B, it varies from −1 to 1, where 0 indicates symmetry, negative values show left-tailed skewness, and positive values exhibit right-tailed skewness (Equation (17)).

A B = 1 − 2 × F X ( M o ) (17)

The distribution of the number of runs varies from symmetry (A_B = 0) when n₀ = n₁, where n₀ + n₁ = n, to extreme negative skewness (A_B = −1) when n₀ = 1 and n₁ = n − 1 or n₀ = n − 1 and n₁ = 1 (Figure 1 and Figure 2). With non-extreme skewness values (0 ≥ A_B > −1), if r is less than the median of R, we calculate the probability that accumulates from 2 (minimum value) to r or the left-tailed probability, representing the order of the quantile. The value with a right-tailed probability equal to or immediately greater is sought, and this probability is the one corresponding to the other tail. The sum of both probabilities provides the two-sided probability. If there is none equal or greater, the one-sided probability is doubled, resulting in the two-tailed probability.

Similarly, if the number of runs is greater than the median of R, we sum the probabilities from r to max(R) or the right-tailed probability, representing the order of the complementary quantile. The probability to the left tail equal or immediately greater is sought, and this probability is the one corresponds to the other tail. The two-sided probability is the sum of both probabilities. If there is neither, the one-sided probability is doubled, resulting in the two-tailed probability.

If r is the median, the other-tailed (right) probability is its complement, and the two-tailed probability is the sum of both probabilities, yielding a value of 1.

With an extreme skewness value of A_B = −1, where the right tail does not exist or one value dominates all the probability, we proceed in the same manner as with non-extreme skewness to calculate the two-tailed probability of the median, which is 1. We double the one-sided probability of the r value that defines the shortened tail to obtain its two-tailed probability. If this doubling exceeds unity,

the two-tailed probability is given a value of 1. The one-tailed probabilities of the values of the elongated part of the distribution are taken as their two-tailed probabilities, considering that the probabilities of the other tail are zero.

Table 4 shows the exact two-tailed probabilities yielded by the newly proposed algorithm following the previously applied example. Results closely align with SPSS. At a significance level of 10%, the null hypothesis is rejected when r = 2. When r = 5, the null hypothesis is maintained, and the two-tailed exact probability is slightly higher than that provided by SPSS (0.833 versus 0.833), which is the only difference with SPSS.

In determining the randomness of the sample data sequence, the decision relies on the bilateral significance. If the two-tailed probability value equals or exceeds the significance level α, the null hypothesis of randomness is upheld. If it falls below α, the hypothesis is rejected. Typically, the significance level is set at 0.05. However, for very small samples (n ≤ 10), it might be raised to 0.1 to counterbalance the conservatism of the test associated with the null hypothesis.

Note. r = number of runs, P(R = r) = exact point probability, P(R ≤ r) = exact cumulative probability, Side = Left when r < Mdn(R) = 4 and Right when r ≥ Mdn(R), One-tailed p-value = sum of the exact point probabilities from 2 to r on the left side and from r to the maximum on the right side, Other-tailed p-value = examine the procedure with extreme skewness, since A_B = −1 in this distribution, and Two-tailed p-value = sum of the one-tailed and the other-tailed probability values, as when SPSS is used; it is the doubling of one-tailed probability when utilizing R program and RSUE.

The decision can alternatively be based on the critical values or quantiles of the distribution for the Wald-Wolfowitz number of runs with parameters n₀ and n₁. As the test is two-tailed test, there are two critical values.

R α / 2 n 0 , n 1 = the quantile of order α/2 in the Wald-Wolfowitz distribution for the number of runs with parameters n₀ and n₁, or the critical value in the left tail for a two-tailed test with a significance level α (Equation (18)).

P ( R ≤ R α 2 n 0 n 1 ) ≥ α 2 (18)

R 1 − α / 2 n 0 , n 1 = the quantile of order 1 − α/2 in the Wald-Wolfowitz distribution for the number of runs with parameters n₀ and n₁, or the critical value in the right tail for a two-tailed test with a significance level α (Equation (18)).

P ( R ≥ R 1 − α 2 n 0 n 1 ) = 1 − P ( R < R 1 − α 2 n 0 n 1 ) = 1 − P ( R ≤ R 1 − α 2 n 0 n 1 − 1 ) ≥ α 2 (19)

The null hypothesis of randomness is upheld when the number of runs (r) falls within the range of the two critical values, inclusive: R α / 2 n 0 , n 1 ≤ r ≤ R 1 − α / 2 n 0 , n 1 . Conversely, the null hypothesis is rejected if r is below the lower critical value (r < R α / 2 n 0 , n 1 ) or exceeds the upper critical value (r > R 1 − α / 2 n 0 , n 1 ). RSUE facilitates the computation of these quantiles. To calculate them using the R program, the following script can be employed [3] , where the first number represents the order of the quantile, the second number is the parameter n₀, the third number is the parameter n₁, and “lower.tail = TRUE” is used for left-tailed calculations, while “lower.tail = FALSE” is used for right-tailed calculations.

Fifty-nine samples with dichotomous data were generated to compare the four algorithms, comprising 29 samples with an even number of data (n = 10) and 30 samples with an odd number of data (n = 11). See Table 5.

On the one hand, nine samples (r from 2 to 10) were derived from a distribution of the number of runs with parameters n₀ = n₁ = 5 (R5|5), eight samples (r from 2 to 9) from a distribution with parameters n₀ = 4 and n₁ = 6 (R4|6), six samples (r from 2 to 7) from a distribution with parameters n₀ = 3 and n₁ = 7 (R3|7), four samples (r from 2 to 5) from a distribution with parameters n₀ = 2 and n₁ = 8 (R2|9), and two samples (r from 2 to 3) from a distribution of parameters n₀ = 1 and n₁ = 9 (R1|9).

On the other hand, ten samples (r from 2 to 11) are drawn from a distribution with parameters n₀ = 5 and n₁ = 6 (R5|6), eight samples (r from 2 to 9) from a distribution with parameters n₀ = 4 and n₁ = 7 (R4|7), six samples (r from 2 to 7) from a distribution with parameters n₀ = 3 and n₁ = 8 (R3|8), four samples (r from 2 to 5) from a distribution with parameters n₀ = 2 and n₁ = 9 (R2|9), and two samples (r from 2 to 3) from a distribution with parameters n₀ = 1 and n₁ = 10 (R1|10).

The central tendency comparison of the two-tailed exact probabilities among the four algorithms, whose distributions deviated from normality, was conducted using Friedman’s test with a significance level of 5%. Post-hoc comparisons were performed using Conover’s test [13] with Benjamini-Yekutielli correction for the familywise error rate [18] . The effect size of the algorithm’s impact on the probabilities was estimated by Kendall’s W coefficient of concordance and the average Spearman’s rank correlation [13] .

Table 6 summarizes the computational processes of the four algorithms. It shows the criteria for defining the symmetry axis, how the two sides of the distribution are determined, as well as how the one-tailed and two-tailed probabilities are obtained.

Table 7 shows the definition of the (left and right) sides of the 10 distributions of the number of runs. It includes one-tailed and two-tailed probabilities calculated using RSUE and R programs, along with mathematical expectations, exact point, and cumulative probabilities for these distributions. The R program uses mathematical expectations for defining distribution sides, while the RSUE program relies on the median.

Table 8 presents the calculations for one-tailed and two-tailed probabilities using the SPSS programs and the new proposal.

The analysis revealed a significant difference among the four algorithms (Friedman Q-statistic = 58.890, df = 3, p-value < 0.001; Iman-Davenport

Note. Each sample is identified by the parameters n₀ and n₁ of the R distribution from which it is drawn and its number of runs (r). Runs test: n = n₀ + n₁ = sample size, n₀ = number of zeros, n₁ = number of ones, r = number of runs (threshold: 0.5).

Note. RSUE = Real Statistics using Excel, r = number of runs, varying from 2 to Max(R) = n₀ + n₁ when n₀ = n₁ or 2 × min(n₀, n₁) + 1 when n₀ ≠ n₁, E(R) = mathematical expectation and Mdn(R) = median of the number of runs distribution with parameters n₀ and n₁. * If there is no equal or greater probability, the one-tailed probability is doubled to obtain two-tailed probability, unless there is extreme negative skewness (A_B = −1). In this case, the probabilities corresponding to the other tail of the values of R in the lengthened tail are 0 and, with the value of R in the shortened tail, the one-tailed probability is doubled with the restriction that the maximum value of the two-tailed probability is 1.

Note. Each sample is identified by the parameters n₀ and n₁ of the R distribution from which it is drawn and its number of runs (r), n = n₀ + n₁ = sample size, n₀ = number of zeros, n₁ = number of ones, r = number of runs (threshold: 0.5).

Note. Sample: n₀|n₁_r = parameters: n₀ (numbers of 0 s) and n₁ (numbers of 1 s), and r = number of runs, p = probability value. |r − E(R|n₀, n₁)| = absolute difference of the number of runs r and the mathematical expectation of the distribution of the number of runs; a horizontal line separates the two sides of the distribution. Mdn (R R|n₀, n₁) = median of the distribution of the number of runs. Sk_B = Bickel’s skewness coefficient based on the mode.

Note. E = Real Statistics using Excel, R = R program, SPSS = Statistical Package for the Social Sciences, N = new proposal. Frequency: <0 (number of negative differences), = 0 (number of null differences or equivalences), and >0 (number of positive differences). Pairwise comparisons using Friedman-Conover test with Benjamini-Yekutielli correction for familywise error: MD = mean signed differences of two-tailed exact probabilities, DRM = difference in mean ranks, SE = standard error of difference in mean ranks, t-statistic = DRM/SE = DRM/0.188, p-value = two-tailed probability value following a t-distribution with 174 degree of freedom, α_c = corrected significance level, significance: “yes” when p-value is less than α_c and “no” when p-value is greater o equal than α_c.

F-statistic = 28.919, df₁ = 3, df₂ = 174, p-value < 0.001). The effect size was medium (Kendall’s w = 0.333, r ¯ S = 0.321). Utilizing Conover’s test with the Benjamini-Yekutieli correction, significant differences of central tendency in probability were observed between RSUE and the R program (RSUE < R), as well as the new proposal (RSUE < NP), and similarly between SPSS and the R program (SPSS < R) and the new proposal (SPSS < NP). The central tendencies in probability between RSUE and SPSS, as well as between R program and the new proposal, were found to be statistically equivalent (Table 9).