Odds ratio is commonly used in the analysis of association of two factors that both have two categories. In epidemiological studies and clinical trials, these two factors usually refer to the exposure (treatment/intervention/risk) factor X and the outcome factor Y respectively. The association between X and Y, however, could be modified or confounded by a third factor Z. For example, in a multi-center clinical trial, factor Z could be the center. Each center is corresponding to a stratum of Z. Because the presence of the heterogeneity of odds ratios may lead to different methods of analysis, researchers usually want to test whether the odds ratios are homogeneous across the strata of the factor Z or not. This type of tests is called the tests for homogeneity of the odds ratios, or the tests for null interaction.

A few procedures have been developed for testing the homogeneity of odds ratios. They are usually categorized into two classes: exact tests and asymptotic tests. Most of the asymptotic statistics were derived for “large-stratum” settings, where the sample size is large, and the number of strata is small. Liang and Self [1] developed two asymptotic statistics—score statistics for the “sparse-data” setting, where there are many cells with small counts and/or zeros. The asymptotic tests could be poor if there were some cells with small cell counts in the contingency tables even when the sample size was quite large. The empirical sizes of the asymptotic tests were shown to be conservative when the data were “small-stratum” setting [2] . Jones et al. [3] showed in their simulation study that, in the “sparse-data” setting, five asymptotic tests suffered in both empirical size and power. Zelen [4] constructed an exact test for homogeneity that employs the ordering principle for a single 2 by 2 table, which was reexamined by Halperin et al. [5] . Algorithms were developed for the Zelen exact test later [6] [7] [8] [9] . Hirji et al. [10] showed that their algorithm for the Zelen statistic was more versatile and efficient in terms of the accuracy and the usage of memory in the computer computation. Hirji et al. [10] also constructed five other exact statistics from their asymptotic counterparts, which are score, likelihood ratio, Pearson χ² and mixture model χ² tests. The exact tests are computationally intensive, particularly when the cell counts are large. Reis, Hirji and Afifi [11] examined the performance of the empirical power of six asymptotic statistics and six exact tests mentioned above; the results of the study showed that the power of both exact and asymptotic tests was low when sample sizes were small even with relatively large heterogeneity among odds ratios. Baghri et al. [12] compared three tests of homogeneity of odds ratios. A recent study investigated profile likelihood tests for common odds ratios in meta-analysis [13] .

Our study presented in this paper compared a class of U-statistics with the established tests such as the Zelen test and the Breslow-Day test.

The statistics of the form

U = [ l ! ( n − l ) ! / n ! ] ∑ ( n , l ) h ( X i 1 , ⋯ , X i l ) (1)

are known as U-statistics, where { X i 1 , ⋯ , X i l } is a set of l-subset of { 1 , ⋯ , n } ;

the sum ∑ ( n , l ) is taken over all subsets 1 ≤ i 1 < ⋯ < i l ≤ n of { 1 , 2 , ⋯ , n } ; h is the

kernel function and symmetric in its arguments. In our study, we investigated the applicability of this class of statistics with l = 2 in testing homogeneity of odds ratios among 2 by 2 tables. U-statistics were first identified as a minimum-variance unbiased estimator by Halmos [14] and were named by Hoeffding [15] , who demonstrated the asymptotic normality of this class of statistics in his seminal paper [15] , Many statistics in common use are members of this class, including the sample mean, the sample variance and the sample covariance [16] [17] .

Table 1 shows a 2 by 2 table of a stratum k.

Assume that a_k and b_k are counts of independent binomial outcomes from number n_k and m_k of trials with or without exposure at the stratum k respectively; N_k is the total sample size of the stratum k; and the tables are independent among strata. The commonly used estimate of the odds ratio of the kth stratum is expressed as: ψ ^ k = ( a k / ( n k − a k ) ) / ( b k / ( m k − b k ) ) . We want to test whether the odds ratios are homogeneous among all K strata (or all levels of the third variable Z), that is to test H₀: ψ 1 = ψ 2 = ⋯ = ψ K = ψ against Ha: ψ i ≠ ψ j for at least one pair of (i, j), where i , j = 1 , ⋯ , K , i ≠ j .

Our study evaluated a class of weighted U-statistics ∑ i < j w i j h ( ψ ^ i , ψ ^ j ) as well as

a class of unweighted U-statistics ( 2 ! ( K − 2 ) ! / K ! ) ∑ i < j h ( ψ ^ i , ψ ^ j ) , where ψ ^ i and ψ ^ j are the estimates of the odds ratios in the ith and jth stratum of Z, h ( ψ ^ i , ψ ^ j ) is a function of ψ ^ i and ψ ^ j , and w i j is the weight associated with h ( ψ ^ i , ψ ^ j ) . Based on the simulation results, we only focus our attention on the following two statistics in this paper:

U3: ( 2 ! ( K − 2 ) ! / K ! ) ∑ i < j | log ψ ^ i − log ψ ^ j | , (2)

WU3: ∑ i < j K w i j | log ψ ^ i − log ψ ^ j | . (3)

The base of log in all formula is e. The sample distribution of the estimated odds ratio is highly skewed when the sample size is small or moderate. Because of this, we used the natural logarithm of ψ ^ in U3 and WU3 to reduce the skewness. Consider that a large stratum offers more accurate estimate for the odds ratio, a weight was selected for each h ( ψ ^ i , ψ ^ j ) in expression (3), which is proportional to the stratum’s size. It has the following form:

w i j = ( w i w j ) / ∑ w k (4)

where w i = 1 / var ( log ( ψ ^ i ) ) . The log transform of the sample odds ratio has an asymptotic variance in a simple form, which is, var ( log ψ ^ k ) ≈ 1 / a k + 1 / ( n k − a k ) + 1 / b k + 1 / ( m k − b k ) . If there was any cell count that equals to zero, the odds ratio estimate ψ ^ k and var ( log ψ ^ k ) from the above formula would be undefined. We added 0.5 to each cell count of that table in the calculations to get the amended estimators [18] . In our simulation, we studied other forms of U-statistics. Due to the relatively poor performance of the others, we only report the results of the U3 and the WU3 and their comparisons to the Zelen’s test and the Breslow-Day test in this paper. Detailed results are available upon request.

A total of 10,000 data sets were simulated using the SAS subroutine RANBIN. Each data set contains pre-specified K sets of 2 by 2 tables. The cell counts a_k and b_k were independently generated from binomial distributions (n_k, p_1k) and (m_k, p_0k), where p x k = P ( Y = 1 | X = x , Z = k ) is the probability of Y = 1 when X = x (x = 1, 0) in the kth stratum. Each set of the tables was simulated with a given n_k, m_k, the number of the strata K and the odds ratios. For a given odd ratio ψ_k and a binomial proportion p_0k, p_1k was calculated by solving:

ψ k = [ p 1 k / ( 1 − p 1 k ) ] / [ p 0 k / ( 1 − p 0 k ) ] .

Following the previous simulation study by Reis, Hirji and Afifi [11] , five factors that might influence the sizes and power of the test statistics were studied. These five factors are: 1) the sample size for X = 1 (n_k); 2) the ratio of the sample sizes of X = 0 and X = 1 (m_k:n_k); 3) the probability of Y = 1 among X = 0 (p_0k); 4) the odds ratio (OR) ψ k ; 5) the number of 2 × 2 tables, K (strata). In order to study the performance of the statistics, different values of the above five factors (parameters) were chosen to simulate different data sets. In choosing the values of n_k, m_k:n_k, p_0k, ψ k , and K, we took the following factors into account: the situations in practice, the characteristics of U-statistics and the design of the simulation study by Reis, Hirji and Afifi [11] . When the effect of one factor was studied, the other four factors’ values were fixed.

We compared the performance of the U-statistics with the Breslow-Day statistic and Zelen’s exact test in our simulation study. A C++ program was written to calculate these statistics’ exact P-values, empirical sizes and power. The empirical size was calculated as the percentage of times that the test rejected the null hypothesis of a common odds ratio at a pre-specified α level among 10,000 tests that were simulated with same odds ratios among K tables. The empirical power was calculated as the percentage of times that a test rejected the null hypothesis of a common odds ratio at a prescribed α level when data were simulated under alternative hypotheses. Because the U-statistics studied here are functions of the sums across all the absolute distances between all possible pairs of the estimated odds ratios in log scale, a large value of U statistics indicates the heterogeneity of the odds ratio.

Theoretically, under suitable conditions, [ T − E ( T ) ] / var ( T ) will be asymptotically following N ( 0 , 1 ) as K → ∞ , where T represents a U-statistic. In our study, the sample mean and the sample variance of 10,000 statistics under the null hypothesis were used to estimate the E(T) and the var(T). In an actual application, the sample mean and variance may be estimated as suggested in our application section. In the simulation, one would use the result from expression [9] for the variance. In so doing, one would get a different numerical value of the variance for each realization, which was inconsistent with our assumption of common variance. The effect of unequal variance is still under investigation.

Generally, U3 and WU3 performed well in terms of both size and power. Their empirical sizes were stable under various situations and had relatively high power.

With the assumption that the odds ratios of K 2 × 2 tables are the same; and log ( ψ ^ i ) is normally distributed with variance equal to σ 2 , we can derive the estimated expected value of U3 and WU3. They are:

E ( U 3 ) = ( 2 ! ( K − 2 ) ! / K ! ) E ( ∑ i < j | log ψ ^ i − log ψ ^ j | ) = 2 ( 1 / ( 2 σ π ) ) ∫ 0 ∞ x exp ( − x 2 / ( 4 σ 2 ) ) d x = 2 σ π (5)

E ( W U 3 ) = E ( ∑ i < j K w i j | log ψ ^ i − log ψ ^ j | ) = E ( | log ψ ^ i − log ψ ^ j | ) ∑ i < j K w i j = ∑ i < j K w i j 2 σ π (6)

The variance of U3 can be also expressed as:

var ( U 3 ) = [ 2 ! ( K − 2 ) ! / K ! ] [ 2 ( K − 2 ) 1.436 σ 2 + ( K − 2 ) 2 σ 2 ] . (7)

The variance of WU3 can be also expressed as:

var ( W U 3 ) = [ 1.436 σ 2 ∑ j ≠ k w i j w i k + 2 σ 2 ∑ i < j w i j 2 ] . (8)

To estimate the σ 2 , consider the Mantel-Haenzel estimator of common odds ratio ψ ^ M H as a weighted average of odds ratios. Given the weight, we can solve the σ 2 as a function of var ( log ( ψ ^ M H ) ) , which is:

σ 2 = var ( log ( ψ ^ M H ) ) [ ∑ ( b k c k / N k ) ] 2 / ∑ ( b k c k / N k ) 2 (9)

And the value of var ( log ( ψ ^ M H ) ) can be calculated by the following formula:

var ( log ( ψ ^ M H ) ) = ( ∑ G i P i ) / [ 2 [ ∑ G i ] 2 ] + ∑ ( G i Q i + H i P i ) / [ 2 ∑ G i ∑ H i ]     + ∑ H i Q i / 2 [ ∑ H i ] 2

where G i = a i d i / N i , H i = b i c i / N i , P i = ( a i + d i ) / N i , Q i = ( b i + c i ) / N i .

To illustrate the application of these two U-statistics, we applied and compared them to the Breslow-Day statistic and the Zelen statistic in two published data sets: 1) Alcohol assumption data (Table 2) are from Statistical Methods in Cancer Research [19] , volume 1 page 137, which is an example for introducing the use of Breslow-Day statistic; 2) new drug data (Table 4) are from a study of comparing a new drug with a controlled drug among 22 hospital sites [20] . Because of the sparseness of the data, asymptotic tests (Breslow-Day statistic) might not be able to yield an accurate p-value. The test results for the homogeneity of odds ratios are presented in Table 2 and Table 4. For data set 1), according to the p-values of the test statistics, U3 rejected the null hypothesis; the WU3, Breslow-Day, and Zelen statistics accepted the null hypothesis at 0.05 level. For data set 2), Zelen and WU3 rejected the null hypothesis, yet Breslow-Day and U3 accepted the null hypothesis of no difference among odds ratios at 0.05 level (Tables 2-5).

To summarize the simulation study that we conducted, the following are some remarks: When the number of strata is not very small, (K ≥ 6), the empirical size of U3 and WU3 were very stable under various situations and stay very close to the nominal of 0.05. In terms of size and power, U3 and WU3 performed better than the Breslow-Day statistic and the Zelen’s exact test. Therefore, U3 and WU3 are considered as better statistics for testing the homogeneity of odds ratios in this situation. The test statistic U3 is recommended when the sample size is the same in each stratum, the number of strata is large and the sample size in each stratum is not large. Otherwise, WU3 is recommended.

Breslow-Day test is conservative in most situations; its empirical size is close

Source: Statistical Methods in Cancer Research, volume 1, page 137.

to 0.05 when the sample size is large; when sample size is small, Breslow-Day test is not recommended. Breslow-Day test is never recommended for sparse data;

When the sample size is small and the number of strata is small, say less than 5, Zelen’s exact test is recommended;

In our application, the sample mean and the variance were estimated based on certain assumptions. The empirical power and size of U3 and WU3 would be highly dependent on how well the estimator of σ 2 would be.

This work was partially supported by Cancer Prevention Research Institute of Texas (RP170668).

The authors declare no conflicts of interest regarding the publication of this paper.

Wei, Q. and Lai, D.J. (2019) Test for Homogeneity of Odds Ratios Using U-Statistics. Open Journal of Statistics, 9, 347-360. https://doi.org/10.4236/ojs.2019.93024