Spotfire Statistica® Variance Components & Mixed Model ANOVA ANOCOVA

Variance Components and Mixed Model ANOVA/ANCOVA is a specialized module for designs with random effects and/or factors with many levels; options for handling random effects and for estimating variance components. The module provides the ability to: 

• compute the standard Type I, II, and III analysis of variance sums of squares and mean squares for the effects in the model
• compute the table of expected mean squares for the effects in the design, the variance components for the random effects in the model, the coefficients for the denominator synthesis, and the complete ANOVA table with tests based on synthesized error sums of squares and degrees of freedom using Satterthwaite's method 
• other methods for estimating variance components are also supported (e.g., MIVQUE0, Maximum Likelihood [ML], Restricted Maximum Likelihood [REML])
• for maximum likelihood estimation, both the Newton-Raphson and Fisher scoring algorithms are used, and the model will not be arbitrarily changed (reduced) during estimation to handle situations where most components are at or near zero

Users can analyze a variety of models containing random effects, obtain estimates of the contribution of the random factors to the variation on the dependent variables, and test the significance of the components of variance.

Mixed Model ANOVA

Mixed Model ANOVA techniques are used when random effects are included in the model. The method for estimating the variance of random factors begins by constructing the Sums of squares and cross products (SSCP) matrix for the independent variables. The sums of squares and cross products for the random effects are then residualized on the fixed effects, leaving the random effects independent of the fixed effects, as required in the mixed model. Using the method of synthesis (Hartley, 1967), the residualized Sums of squares and cross products for each random factor are then divided by their degrees of freedom to produce the coefficients in the Expected mean squares matrix. The Expected mean squares is then used to estimate the variation of the random effects in the model. The Expected mean squares matrix can be produced using Type I, Type II, or Type III Sums of squares (Milliken & Johnson, 1992).

To test the significance of effects in mixed or random models, error terms must be constructed that contain all the same sources of random variation except for the variation of the respective effect of interest. This is done using Satterthwaite's method of denominator synthesis (Satterthwaite, 1946), which finds the linear combinations of sources of random variation that serve as appropriate error terms for testing the significance of the respective effect of interest. To perform the tests of significance of effects, ratios of appropriate Mean squares are then formed to compute F statistics and p-values for each effect. Denominator degrees of freedom for corresponding synthesized error terms are computed using Satterthwaite's method. The resulting F tests generally are approximate, rather than exact, and can be based on fractional degrees of freedom, reflecting fractional sources of random variation from which the error terms are synthesized. These approximate F tests become undefined, and thus are not displayed, when the denominator degrees of freedom approach or become zero.

Overparameterized and sigma-restricted models

 In one line of literature, the analysis of multi-factor ANOVA designs is generally discussed as the Sigma-restricted model. The ANOVA parameters are constrained to sum to zero. In this manner, given k levels of a factor, the k-1 parameters (corresponding to the k-1 degrees of freedom) can readily be estimated (e.g., Lindeman, 1974, Snedecor and Cochran, 1989, p. 322). Another tradition discusses ANOVA in the context of the unconstrained and thus over-parameterized general linear model (e.g., Kirk, 1968). The results for mixed random and fixed effect models can be different applying the two approaches.

For example, the user has a two-way mixed model design: Subject (random) by Treatment (fixed). If the user started with the sigma-restricted model, the expected mean square for the random effect (i.e., Subject) does not contain the two-way interaction. However without the sigma restriction, it does (compare tables 4.6 and 4.7 in Searle, Casella, & McCulloch, 1992; the derivations of the expected mean squares in the two cases are also discussed in that reference).

The next question, of course, is which expected mean square is right. The answer to this question is "it depends." Searle, Casella, and McCulloch (1992) give a detailed discussion of the advantages and disadvantages of the two approaches. They conclude that the question "has no definitive, universally acceptable answer," p. 126). However, Searle, et al. also point out that when the parameters of the fixed effects "are being taken as realized values of random variables, it is not realistic to have them summing to zero" (Searle, et al. p. 123). Moreover, for the case of unbalanced data, this restriction is usually never even considered. Therefore, most general linear model routines, including this module, that estimate expected mean squares for mixed models will usually use the solution for the over-parameterized model.

Note: Spotfire Statistica® ANOVA MANOVA uses, by default, the means model approach. It will construct F-tests for mixed models that are consistent with the sigma restricted model. This is an ANOVA "tradition" most commonly discussed in statistics textbooks in the biological and social sciences.