To make our task easier, we assume that the sum all probabilities of the Multinomial distribution defined in Formula (2) equals to $r<1$, and thus allowing for the possibility of an extra outcome whose probability is $1-r$, denoting its observed total by $W$. This substantially simplifies subsequent development by removing the singularity of the variance-covariance (V-C) matrix of the $X_{i,j}$ variables, while letting us reach correct conclusions in the $r\to 1$ limit. Correspondingly modifying Formula (2) is easy: extend the bottom row of the multinomial coefficient by $w=n-{\displaystyle {\sum }_{i=1}^K{\displaystyle {\sum }_{j=1}^M{x}_{i,j}}}$ and further multiply by ${\left(1-r\right)}^w$. Since all variables must add up to $n$, the resulting $KM+1$ by $KM+1$ V-C matrix is singular, but the singularity disappears when replacing $W$ by $n-{\displaystyle {\sum }_{i=1}^K{\displaystyle {\sum }_{j=1}^M{X}_{i,j}}}$ and considering the distribution of the remaining $X_{i,j}$’s only. The corresponding moments are

$\mathbb{E}\left(X_{i.j}\right)=n{p}_i{q}_j$

$\text{Var}\left(X_{i,j}\right)=n{p}_i{q}_j\left(1-p_i{q}_j\right)$

$\text{Cov}\left(X_{i,j},X_{k,m}\right)=-n{p}_i{q}_j{p}_k{q}_m\text{when}i\ne k\text{or}j\ne m$

which remains correct even with the extra $W$. We then define

$Y_{i,j}:=\frac{X_{i,j}-n{p}_i{q}_j}{\sqrt{n}}$ (4)

and substitute

$x_{i,j}=n{p}_i{q}_j+y_{i,j}\sqrt{n}$

$w=-\underset{i=1}{\overset{K}{{\displaystyle \sum }}}\underset{j=1}{\overset{M}{{\displaystyle \sum }}}{y}_{i,j}$

into the natural logarithm of Formula (2), after is has been modified by introducing the extra $W$. Adding ln of the corresponding Jacobian (namely $KM\cdot \ln \sqrt{n}$, since the distribution of the $Y_{i,j}$’s is $KM$ dimensional) and taking the $n\to \infty$ limit of the resulting expression (all limits of this article are routinely done by a computer equipped with Mathematica or a similar software) yields

(5)

This identifies the asymptotic distribution of the $Y_{i,j}$’s to be multivariate Normal, with a probability density function (PDF) whose natural logarithm is given by the last expression. Its linear and quadratic moments agree with those of the original discrete distribution, being equal to 0 for the expected value of each $Y_{i,j}$ and to

$p_i{\delta }_{i,k}{q}_j{\delta }_{j,m}-p_i{p}_k{q}_j{q}_m$ (6)

for the covariance between $Y_{i,j}$ and $Y_{k.m}$ (for any combination of indices); the corresponding V-C matrix is thus

$\mathbb{V}=\mathbb{P}\otimes \mathbb{Q}-p\cdot p^{\text{T}}\otimes q\cdot q^{\text{T}}$ (7)

where $p$ and $q$ are column vectors whose elements are the $p_i$ and $q_j$ probabilities (respectively), and $\mathbb{P}$ and $\mathbb{Q}$ are diagonal matrices with the $p_i$ and $q_j$ probabilities (respectively) on the main diagonal. The PDF can then be written, in correspondence with Formula (5), as

$\frac{\text{exp}\left(-\frac{Y^{\text{T}}\circ \mathbb{A}\circ Y}{2}\right)}{{\left(2\pi \right)}^{KM/2}}\sqrt{\det \mathbb{A}}$ (8)

where $Y$ is a column vector whose $KM$ elements are $Y_{i,j}$ (organized in accordance with our Kronecker’s product, i.e. $Y_{1,1},Y_{1,2},\cdots ,Y_{1,M},Y_{2,1},\cdots ,Y_{K,M}$),

$\mathbb{A}={\mathbb{P}}^{-1}\otimes {\mathbb{Q}}^{-1}+\frac{1_k\cdot 1_k^{\text{T}}\otimes 1_m\cdot 1_m^{\text{T}}}{1-r}={\mathbb{P}}^{-1}\otimes {\mathbb{Q}}^{-1}+\frac{1_k\otimes 1_m\circ 1_k^{\text{T}}\otimes 1_m^{\text{T}}}{1-r}$ (9)

and

$\begin{array}{c}\text{det}\left(\mathbb{A}\right)=\text{det}\left({\mathbb{P}}^{-1}\otimes {\mathbb{Q}}^{-1}\right)\text{det}\left(1+\frac{1_k^{\text{T}}\otimes 1_m^{\text{T}}\circ \mathbb{P}\otimes \mathbb{Q}\circ 1_k\otimes 1_m}{1-r}\right)\\ =\frac{1}{{\left({{\displaystyle \prod }}_{i=1}^K{p}_i\right)}^M{\left({{\displaystyle \prod }}_{j=1}^M{q}_j\right)}^K\left(1-r\right)}\end{array}$ (10)

the last two implied by Formula (5).

Alternately, they can be derived from Formula (7) using the Woodbury and Silvester identities. Bypassing the details, we only verify that $\mathbb{A}$ is the inverse of $\mathbb{V}$, by multiplying

$\begin{array}{l}\left({\mathbb{P}}^{-1}\otimes {\mathbb{Q}}^{-1}+\frac{1_k\cdot 1_k^{\text{T}}\otimes 1_m\cdot 1_m^{\text{T}}}{1-r}\right)\circ \left(\mathbb{P}\otimes \mathbb{Q}-p\cdot p^{\text{T}}\otimes q\cdot q^{\text{T}}\right)\\ ={\mathbb{I}}_k\otimes {\mathbb{I}}_m-1_k\cdot p^{\text{T}}\otimes 1_m\cdot q^{\text{T}}+\frac{1_k\cdot p^{\text{T}}\otimes 1_m\cdot q^{\text{T}}}{1-r}-\frac{r\left(1_k\cdot p^{\text{T}}\otimes 1_m\cdot q^{\text{T}}\right)}{1-r}\\ ={\mathbb{I}}_k\otimes {\mathbb{I}}_m\end{array}$

where ${\mathbb{I}}_k$ and ${\mathbb{I}}_m$ are $k$ by $k$ and $m$ by $m$ identity matrices respectively.

We now take the $n\to \infty$ limit of Formula (1) and of Formula (3), first replacing $X_{i,j}$ with $Y_{i,j}$ by using Formula (4); this results in the same answer in both cases, namely in

$\underset{i=1}{\overset{K}{{\displaystyle \sum }}}\underset{j=1}{\overset{M}{{\displaystyle \sum }}}\frac{Y_{i,j}^2}{p_i{q}_j}-\underset{i=1}{\overset{K}{{\displaystyle \sum }}}\frac{Y_{i,•}^2}{p_i}-\underset{j=1}{\overset{M}{{\displaystyle \sum }}}\frac{Y_{•,j}^2}{q_i}$ (11)

where $Y_{•,j}={\displaystyle {\sum }_{i=1}^K{Y}_{i,j}}$ and $Y_{i,•}:={\displaystyle {\sum }_{j=1}^M{Y}_{i.j}}$. To get the corresponding moment generating function (MGF) of $U$, we need to compute the expected value of $\text{exp}\left(tU\right)$, i.e.

$\frac{{\displaystyle \int \cdots }{\displaystyle {\int }_{-\infty }^{\infty }\text{exp}\left(-\frac{Y^{\text{T}}\circ \left(\mathbb{A}-2t\mathbb{U}\right)\circ Y}{2}\right)\text{d}Y}}{{\left(2\pi \right)}^{KM/2}}\sqrt{\det \mathbb{A}}$ (12)

where

$\mathbb{U}={\mathbb{P}}^{-1}\otimes {\mathbb{Q}}^{-1}-{\mathbb{P}}^{-1}\otimes 1_m\cdot 1_m^{\text{T}}-1_k\cdot 1_k^{\text{T}}\otimes {\mathbb{Q}}^{-1}$ (13)

which is the matrix version of Formula (11); to see this, it helps to re-write Formula (11) as follows

$\underset{i=1}{\overset{K}{{\displaystyle \sum }}}\underset{k=1}{\overset{K}{{\displaystyle \sum }}}\underset{j=1}{\overset{M}{{\displaystyle \sum }}}\underset{m=1}{\overset{M}{{\displaystyle \sum }}}{Y}_{i.j}\left(\frac{{\delta }_{i,k}{\delta }_{j,m}}{p_i{q}_j}-\frac{{\delta }_{i,k}}{p_i}-\frac{{\delta }_{j,m}}{q_j}\right)Y_{k,m}$

To find the result of Formula (12) necessitates computing the determinant of $\mathbb{A}-2t\mathbb{U}$; this matrix can be expressed in the following form

$\left(1-2t\right){\mathbb{P}}^{-1}\otimes {\mathbb{Q}}^{-1}+\frac{1_k\cdot 1_k^{\text{T}}\otimes 1_m\cdot 1_m^{\text{T}}}{1-r}+2t\left[\begin{array}{ll}{\mathbb{P}}^{-1}\otimes 1_m\hfill & 1_k\otimes {\mathbb{Q}}^{-1}\hfill \end{array}\right]\circ \left[\begin{array}{l}{\mathbb{I}}_k\otimes 1_m^{\text{T}}\hfill \\ 1_k^{\text{T}}\otimes {\mathbb{I}}_m\hfill \end{array}\right]$ (14)

which follows from Formula (9) and Formula (13); the square brackets indicate that the matrix consists of two (and subsequently also of four) blocks.

Pre-multiplying Formula (14) by $\frac{\mathbb{P}\otimes \mathbb{Q}}{1-2t}$, whose determinant is

$D_1:=\frac{{\left({{\displaystyle \prod }}_{i=1}^K{p}_i\right)}^M{\left({{\displaystyle \prod }}_{j=1}^M{q}_j\right)}^K}{{\left(1-2t\right)}^{KM}}$

leaves us with the task of finding the determinant of

$\begin{array}{c}ℂ:={\mathbb{I}}_k\otimes {\mathbb{I}}_m+\frac{p\cdot 1_k^{\text{T}}\otimes q\cdot 1_m^{\text{T}}}{\left(1-r\right)\left(1-2t\right)}+\frac{2t}{1-2t}\left[\begin{array}{ll}{\mathbb{I}}_k\otimes q\hfill & p\otimes {\mathbb{I}}_m\hfill \end{array}\right]\circ \left[\begin{array}{l}{\mathbb{I}}_k\otimes 1_m^{\text{T}}\hfill \\ 1_k^{\text{T}}\otimes {\mathbb{I}}_m\hfill \end{array}\right]\\ :=\mathbb{F}+\frac{2t}{1-2t}\mathbb{G}\circ ℍ\end{array}$ (15)

This needs to be done in two steps: for the determinant of $\mathbb{F}$, we get

$D_2:=\det \mathbb{F}=1+\frac{1_k^{\text{T}}\otimes 1_m^{\text{T}}\circ p\otimes q}{\left(1-r\right)\left(1-2t\right)}=1+\frac{r}{\left(1-r\right)\left(1-2t\right)}=\frac{1-2t\left(1-r\right)}{\left(1-r\right)\left(1-2t\right)}$

while its inverse equals to

${\mathbb{F}}^{-1}={\mathbb{I}}_k\otimes {\mathbb{I}}_m-\frac{p\cdot 1_k^{\text{T}}\otimes q\cdot 1_m^{\text{T}}}{1-2t\left(1-r\right)}$ (16)

easily verifiable by simple multiplication, utilizing $p\cdot 1_k^{\text{T}}\otimes q\cdot 1_m^{\text{T}}\circ p\cdot 1_k^{\text{T}}\otimes q\cdot 1_m^{\text{T}}=r\left(p\cdot 1_k^{\text{T}}\otimes q\cdot 1_m^{\text{T}}\right)$.

Sylvester’s identity implies that the determinant of $ℂ$ is a product of $\det \mathbb{F}$ and the determinant of (at this point, we can start replacing $r$ by 1)

$\begin{array}{l}{\mathbb{I}}_{k+m}+\frac{2t}{1-2t}\left[\begin{array}{l}{\mathbb{I}}_k\otimes 1_m^{\text{T}}\hfill \\ 1_k^{\text{T}}\otimes {\mathbb{I}}_m\hfill \end{array}\right]\circ \left({\mathbb{I}}_k\otimes {\mathbb{I}}_m-p\cdot 1_k^{\text{T}}\otimes q\cdot 1_m^{\text{T}}\right)\circ \left[\begin{array}{ll}{\mathbb{I}}_k\otimes q\hfill & p\otimes {\mathbb{I}}_m\hfill \end{array}\right]\\ ={\mathbb{I}}_{k+m}+\frac{2t}{1-2t}\left[\begin{array}{cc}{\mathbb{I}}_k\otimes 1& p\otimes 1_m^{\text{T}}\\ 1_k^{\text{T}}\otimes q& 1\otimes {\mathbb{I}}_m\end{array}\right]-\frac{2t}{1-2t}\left[\begin{array}{cc}p\cdot 1_k^{\text{T}}\otimes 1& p\otimes 1_m^{\text{T}}\\ 1_k^{\text{T}}\otimes q& 1\otimes q\cdot 1_m^{\text{T}}\end{array}\right]\\ =\frac{1}{1-2t}\left[\begin{array}{cc}\left({\mathbb{I}}_k-2tp\cdot 1_k^{\text{T}}\right)\otimes 1& 0_k\otimes 0_m^{\text{T}}\\ 0_m\otimes 0_k^{\text{T}}& 1\otimes \left({\mathbb{I}}_m-2tq\cdot 1_m^{\text{T}}\right)\end{array}\right]\end{array}$ (17)

where $0$ is a zero column-vector of indicated length and 1 is a scalar. The determinant of the last matrix is

$D_3:=\frac{{\left(1-2t\right)}^2}{{\left(1-2t\right)}^{K+M}}$

since the determinant of each main-diagonal block is equal to $1-2t,$ due to Sylvester’s identity.

In the $r\to 1$ limit Formula (12), whose value is given by $\sqrt{\frac{\det \mathbb{A}}{\det \left(\mathbb{A}-2t\mathbb{U}\right)}}=\sqrt{\frac{\det \mathbb{A}}{D_1{D}_2{D}_3}}$, then equals to

${\left(\frac{1}{1-2t}\right)}^{\left(K-1\right)\left(M-1\right)/2}$ (18)

proving the well-known result that, to this level of approximation, the distribution of $U$ is ${\chi }^2$ with $\left(K-1\right)\left(M-1\right)$ degrees of freedom.

Making the integrand of Formula (12) into a PDF of a Normal distribution results in

$\frac{\text{exp}\left(-\frac{Y^{\text{T}}\circ \left(\mathbb{A}-2t\mathbb{U}\right)\circ Y}{2}\right)}{{\left(2\pi \right)}^{KM/2}}\sqrt{\det \mathbb{A}-2t\mathbb{U}}$ (19)

whose higher (quartic and hexic in particular) moments are key to finding $\frac{1}{n}$-proportional corrections to the MGF of each Formula (1) and Formula (3). Furthermore, these moments are polynomial functions of its second moments (a special property of a multivariate Normal distribution), which in turn are simply the elements of ${\left(\mathbb{A}-2t\mathbb{U}\right)}^{-1}$.

To compute this inverse, we first write Formula (14) as a product of $\left(1-2t\right){\mathbb{P}}^{-1}\otimes {\mathbb{Q}}^{-1}$ (whose inverse is easy) and of $ℂ=\mathbb{F}+\frac{2t}{1-2t}\mathbb{G}\circ ℍ$, introduced in Formula (15). From Formula (16) we know that

${\mathbb{F}}^{-1}={\mathbb{I}}_k\otimes {\mathbb{I}}_m-p\cdot 1_k^{\text{T}}\otimes q\cdot 1_m^{\text{T}}$

as $r$ is no longer needed and can be set equal to 1.

To complete the exercise, we need to simplify

${\mathbb{F}}^{-1}-\frac{2t}{1-2t}{\mathbb{F}}^{-1}\circ \mathbb{G}\circ {\left({\mathbb{I}}_{k+m}+\frac{2t}{1-2t}ℍ\circ {\mathbb{F}}^{-1}\circ \mathbb{G}\right)}^{-1}\circ ℍ\circ {\mathbb{F}}^{-1}$

obtained from Woodbury’s identity as a part of inverting $ℂ$. The last line of Formula (17) has already done exactly that for the matrix in parentheses; the corresponding inverse is

$\left(1-2t\right)\left[\begin{array}{cc}{\mathbb{I}}_k\otimes 1& 0_k\otimes 0_m^{\text{T}}\\ 0_m\otimes 0_k^{\text{T}}& 1\otimes {\mathbb{I}}_m\end{array}\right]+2t\left[\begin{array}{cc}p\cdot 1_k^{\text{T}}\otimes 1& 0_k\otimes 0_m^{\text{T}}\\ 0_m\otimes 0_k^{\text{T}}& 1\otimes q\cdot 1_m^{\text{T}}\end{array}\right]$

easily verifiable by simple matrix multiplication (note that both matrices in the last expression are idempotent).

This inverse still needs to be pre-multiplied by $\left[\begin{array}{ll}{\mathbb{I}}_k\otimes q\hfill & p\otimes {\mathbb{I}}_m\hfill \end{array}\right]$ and post-multiplied by $\left[\begin{array}{l}{\mathbb{I}}_k\otimes 1_m^{\text{T}}\hfill \\ 1_k^{\text{T}}\otimes {\mathbb{I}}_m\hfill \end{array}\right]$, thus getting

$\left(1-2t\right)\left({\mathbb{I}}_k\otimes q\cdot 1_m^{\text{T}}+p\cdot 1_k^{\text{T}}\otimes {\mathbb{I}}_m\right)+4tp\cdot 1_k^{\text{T}}\otimes q\cdot 1_m^{\text{T}}$

and further pre and post-multiplied by ${\mathbb{F}}^{-1}$, resulting in

$\left(1-2t\right)\left({\mathbb{I}}_k\otimes q\cdot 1_m^{\text{T}}+p\cdot 1_k^{\text{T}}\otimes {\mathbb{I}}_m-2p\cdot 1_k^{\text{T}}\otimes q\cdot 1_m^{\text{T}}\right)+4t{0}_k\cdot 0_k^{\text{T}}\otimes 0_m\cdot 0_m^{\text{T}}$

Multiplying the last expression by $\frac{2t}{1-2t}$ and subtracting the result from ${\mathbb{F}}^{-1}$ yields ${ℂ}^{-1}$, namely

${\mathbb{I}}_k\otimes {\mathbb{I}}_m+\left(4t-1\right)p\cdot 1_k^{\text{T}}\otimes q\cdot 1_m^{\text{T}}-2t\left({\mathbb{I}}_k\otimes q\cdot 1_m^{\text{T}}+p\cdot 1_k^{\text{T}}\otimes {\mathbb{I}}_m\right)$

Finally, post-multiplying by $\frac{\mathbb{P}\otimes \mathbb{Q}}{1-2t}$ results in the following expression for ${\left(\mathbb{A}-2t\mathbb{U}\right)}^{-1}$

$\frac{\mathbb{P}\otimes \mathbb{Q}}{1-2t}+\left(\frac{1}{1-2t}-2\right)p\cdot p^{\text{T}}\otimes q\cdot q^{\text{T}}+\left(1-\frac{1}{1-2t}\right)\left(\mathbb{P}\otimes q\cdot q^{\text{T}}+p\cdot p^{\text{T}}\otimes \mathbb{Q}\right)$

and the corresponding formula for computing the desired second moments, namely

$\begin{array}{c}\mathbb{E}\left(Y_{i,j}{Y}_{k,m}\right)=-2p_i{p}_k{q}_j{q}_m+{\delta }_{i.k}{p}_i{q}_j{q}_m+{\delta }_{j,m}{p}_i{p}_k{q}_j\\ +\frac{p_i\left({\delta }_{i.k}-p_k\right)q_j\left({\delta }_{j,m}-q_m\right)}{1-2t}\end{array}$ (20)

(note the expected agreement with Formula (6) when $t=0$).

Consequently,

$\begin{array}{l}\mathbb{E}\left(Y_{i,j}{Y}_{k,•}\right)=p_i{q}_j\left({\delta }_{i,k}-p_k\right)\\ \mathbb{E}\left(Y_{i,j}{Y}_{•,m}\right)=p_i{q}_j\left({\delta }_{j,m}-q_m\right)\\ \mathbb{E}\left(Y_{i,•}{Y}_{k,•}\right)=p_i\left({\delta }_{i.k}-p_k\right)\\ \mathbb{E}\left(Y_{•,j}{Y}_{•,m}\right)=q_j\left({\delta }_{j,m}-q_m\right)\\ \mathbb{E}\left(Y_{i,•}{Y}_{•,j}\right)=0\end{array}$ (21)

provide the remaining moments needed to proceed with computing the desired $\frac{1}{n}$-proportional corrections. Note that fourth-order moments are then found from

$\mathbb{E}\left(abcd\right)=\mathbb{E}\left(ab\right)\mathbb{E}\left(cd\right)+\mathbb{E}\left(ac\right)\mathbb{E}\left(bd\right)+\mathbb{E}\left(ad\right)\mathbb{E}\left(bc\right)$

(with subsequent special cases, such as $\mathbb{E}\left(a^2{b}^2\right)=2\mathbb{E}{\left(ab\right)}^2+\mathbb{E}\left(a^2\right)\mathbb{E}\left(b^2\right)$ etc.), and the sixth-order moments from

$\mathbb{E}\left(abcdef\right)=\mathbb{E}\left(ab\right)\mathbb{E}\left(cd\right)\mathbb{E}\left(ef\right)+\cdots +\mathbb{E}\left(af\right)\mathbb{E}\left(cd\right)\mathbb{E}\left(be\right)$

(where the RHS consists of 15 terms corresponding to all possible ways of pairing the six arguments), usually needed in its special form such as

$\mathbb{E}\left(a^3{b}^3\right)=9\mathbb{E}\left(a^2\right)\mathbb{E}\left(b^2\right)\mathbb{E}\left(ab\right)+6\mathbb{E}{\left(ab\right)}^3$

etc.

This time we get the following extra terms in the expansion of Formula (3)

$\begin{array}{c}\mathfrak{U}=\cdots +\frac{2{{\displaystyle \sum }}_{i=1}^K\frac{Y_{i,•}^3}{p_i^2}+2{{\displaystyle \sum }}_{i=1}^M\frac{Y_{•,j}^3}{q_j^2}}{3\sqrt{n}}+\frac{-2{{\displaystyle \sum }}_{i=1}^K{{\displaystyle \sum }}_{j=1}^M\frac{Y_{i,j}^3}{p_i^2{q}_j^2}}{3\sqrt{n}}\\ +\frac{{{\displaystyle \sum }}_{i=1}^K\frac{Y_{i,•}^4}{p_i^3}+{{\displaystyle \sum }}_{i=1}^M\frac{Y_{•,j}^4}{q_j^3}}{3n}+\frac{-{{\displaystyle \sum }}_{i=1}^K{{\displaystyle \sum }}_{j=1}^M\frac{Y_{i,j}^4}{p_i^3{q}_j^3}}{3n}\\ :=\frac{u_1}{\sqrt{n}}+\frac{u_2}{\sqrt{n}}+\frac{u_3}{n}+\frac{u_4}{n}\end{array}$

We then repeat the steps of the previous section, getting the following (surprisingly simple) correction to the corresponding PDF

$\frac{\tilde{P}\tilde{Q}+\left(M^2-1\right)\tilde{P}+\left(K^2-1\right)\tilde{Q}+\left(K^2-1\right)\left(M^2-1\right)}{12n}\left(\frac{x}{d}-1\right){\chi }_d^2\left(x\right)$