Tests to compare covariance matrices

library(stests)

Introduction

Let $\mathbf{X}_{1i}, \mathbf{X}_{2i}, \ldots, \mathbf{X}_{N_i i}$ be a random sample from the population $N_p(\boldsymbol{\mu}_g, \boldsymbol{\Sigma}_g)$ indexed by $i = 1,2,\ldots,g$. In other words, we have $g$ samples as follows

$$\begin{matrix} \mathbf{X}_{11}, \mathbf{X}_{21}, \ldots, \mathbf{X}_{N_1 1} \\ \mathbf{X}_{12}, \mathbf{X}_{22}, \ldots, \mathbf{X}_{N_2 2} \\ \vdots \\ \mathbf{X}_{1g}, \mathbf{X}_{2g}, \ldots, \mathbf{X}_{N_g g} \end{matrix}$$

It is not necessary to have equal $N_i$. The main objective in this vignette is to use test to study the following hypothesis.

$$H_{0}: \Sigma_{1} = \Sigma_{2} = \cdots = \Sigma_{g} = \Sigma$$

$$H_{A}: \text{at least one } \Sigma \text{ is different}$$

Box-M test

Box (1949) proposed this test and the statistic test $\varphi$ is given by:

$$\varphi = -2 \rho \log(\lambda)$$

Under true $H_{0}$, the statistic

$$\varphi \sim \chi^{2}_{p(p+1)(g-1)/2}$$

Where $\rho$, $\log(\lambda)$ and $S$ are obtained by as

$$\rho = 1 - \frac{2p^{2} + 3p - 1}{6(p + 1)(g - 1)} \left( \sum_{i=1}^{g} \frac{1}{n_i} - \frac{1}{n} \right)$$

$$\log(\lambda) = \frac{n \, \log(|S|) - \sum_{i=1}^{g} n_i \, \log(|S_i|)}{-2}$$

$$S = \frac{1}{n} \sum_{i=1}^{g} n_i S_i$$

with

$$n_i = N_i - 1$$

$$n = n_1 + n_2 + \ldots n_g$$

This test seems to be good if each $N_i$ exceeds 20, and if $g$ and $p$ do not exceed 5 (Mardia, Bibby, and Kent (1992), page 140).

Bartlett’s test or modified LRT

xxx proposed this test and the statistic is given by:

$$M = n \log(|S|) - \sum_{i=1}^{g} n_i \log(|S_i|)$$

Under true $H_{0}$, the statistic

$$M \sim \chi^{2}_{p(p+1)(g-1)/2}$$

The matrix $S$ and $n$ are the same as in the Box-M test.

Note: Schott (2007) claims that since the sample covariance matrix $S_i$ is singular if $n_i < p$, this likelihood ratio test is valid only if $n_i \ge p$ for $i = 1,2, \ldots, g$.

Wald Schott test

Schott (2001) (page 27) proposed this test and the statistic is given by:

$$W = \frac{n}{2} \left\{ \sum_{i=1}^{g} \frac{n_i}{n} \, \mathrm{tr}(S_i S^{-1} S_i S^{-1}) - \sum_{i=1}^{g} \sum_{j=1}^{g} \frac{n_i n_j}{n^{2}} \, \mathrm{tr}(S_i S^{-1} S_j S^{-1}) \right\}.$$

Under true $H_{0}$, the statistic

$$W \sim \chi^{2}_{p(p+1)(g-1)/2}$$

The matrix $S$ and $n$ are the same as in the Box-M test.

References

Box, George EP. 1949. “A General Distribution Theory for a Class of Likelihood Criteria.” Biometrika 36 (3/4): 317–46.

Mardia, Kanti V., John M. Bibby, and J. T. Kent. 1992. Multivariate Analysis. Acad. Pr.

Schott, James R. 2001. “Some Tests for the Equality of Covariance Matrices.” Journal of Statistical Planning and Inference 94 (1): 25–36.

———. 2007. “A Test for the Equality of Covariance Matrices When the Dimension Is Large Relative to the Sample Sizes.” Computational Statistics & Data Analysis 51 (12): 6535–42.