Tests to compare covariance matrices

Source: vignettes/Tests_Sigmas.Rmd

Introduction

Let 𝑿1i,𝑿2i,…,𝑿Nii\mathbf{X}_{1i}, \mathbf{X}_{2i}, \ldots, \mathbf{X}_{N_i i} be a random sample from the population Np(𝝁g,𝚺g)N_p(\boldsymbol{\mu}_g, \boldsymbol{\Sigma}_g) indexed by i=1,2,…,gi = 1,2,\ldots,g. In other words, we have gg samples as follows

𝑿11,𝑿21,…,𝑿N11𝑿12,𝑿22,…,𝑿N22⋮𝑿1g,𝑿2g,…,𝑿Ngg \begin{matrix} \mathbf{X}_{11}, \mathbf{X}_{21}, \ldots, \mathbf{X}_{N_1 1} \\ \mathbf{X}_{12}, \mathbf{X}_{22}, \ldots, \mathbf{X}_{N_2 2} \\ \vdots \\ \mathbf{X}_{1g}, \mathbf{X}_{2g}, \ldots, \mathbf{X}_{N_g g} \end{matrix}

It is not necessary to have equal NiN_i. The main objective in this vignette is to use test to study the following hypothesis.

H0:Σ1=Σ2=⋯=Σg=Σ H_{0}: \Sigma_{1} = \Sigma_{2} = \cdots = \Sigma_{g} = \Sigma

HA:at least one Σ is different H_{A}: \text{at least one } \Sigma \text{ is different}

Box-M test

Box (1949) proposed this test and the statistic test φ\varphi is given by:

φ=−2ρlog⁡(λ) \varphi = -2 \rho \log(\lambda)

Under true H0H_{0}, the statistic

φ∼χp(p+1)(g−1)/22 \varphi \sim \chi^{2}_{p(p+1)(g-1)/2}

Where ρ\rho, log⁡(λ)\log(\lambda) and SS are obtained by as

ρ=1−2p2+3p−16(p+1)(g−1)(∑i=1g1ni−1n) \rho = 1 - \frac{2p^{2} + 3p - 1}{6(p + 1)(g - 1)} \left( \sum_{i=1}^{g} \frac{1}{n_i} - \frac{1}{n} \right)

log⁡(λ)=nlog⁡(|S|)−∑i=1gnilog⁡(|Si|)−2 \log(\lambda) = \frac{n \, \log(|S|) - \sum_{i=1}^{g} n_i \, \log(|S_i|)}{-2}

S=1n∑i=1gniSi S = \frac{1}{n} \sum_{i=1}^{g} n_i S_i

with

ni=Ni−1 n_i = N_i - 1

n=n1+n2+…ng n = n_1 + n_2 + \ldots n_g

This test seems to be good if each NiN_i exceeds 20, and if gg and pp do not exceed 5 (Mardia et al. (1992), page 140).

Bartlett’s test or modified LRT

xxx proposed this test and the statistic is given by:

M=nlog⁡(|S|)−∑i=1gnilog⁡(|Si|) M = n \log(|S|) - \sum_{i=1}^{g} n_i \log(|S_i|)

Under true H0H_{0}, the statistic

M∼χp(p+1)(g−1)/22 M \sim \chi^{2}_{p(p+1)(g-1)/2}

The matrix SS and nn are the same as in the Box-M test.

Note: Schott (2007) claims that since the sample covariance matrix SiS_i is singular if ni<pn_i < p, this likelihood ratio test is valid only if ni≥pn_i \ge p for i=1,2,…,gi = 1,2, \ldots, g.

Wald Schott test

Schott (2001) (page 27) proposed this test and the statistic is given by:

W=n2{∑i=1gnintr(SiS−1SiS−1)−∑i=1g∑j=1gninjn2tr(SiS−1SjS−1)}. W = \frac{n}{2} \left\{ \sum_{i=1}^{g} \frac{n_i}{n} \, \mathrm{tr}(S_i S^{-1} S_i S^{-1}) - \sum_{i=1}^{g} \sum_{j=1}^{g} \frac{n_i n_j}{n^{2}} \, \mathrm{tr}(S_i S^{-1} S_j S^{-1}) \right\}.

Under true H0H_{0}, the statistic

W∼χp(p+1)(g−1)/22 W \sim \chi^{2}_{p(p+1)(g-1)/2}

The matrix SS and nn are the same as in the Box-M test.

References

Box, George EP. 1949. “A General Distribution Theory for a Class of Likelihood Criteria.” Biometrika 36 (3/4): 317–46.
Mardia, Kanti V., John M. Bibby, and J. T. Kent. 1992. Multivariate Analysis. Acad. Pr.
Schott, James R. 2001. “Some Tests for the Equality of Covariance Matrices.” Journal of Statistical Planning and Inference 94 (1): 25–36.
Schott, James R. 2007. “A Test for the Equality of Covariance Matrices When the Dimension Is Large Relative to the Sample Sizes.” Computational Statistics & Data Analysis 51 (12): 6535–42.