Section 1: The Ideal Conditions of the General Linear Model The analysis of variance (ANOVA) and multiple linear regression analysis are perhaps the primary statistical techniques for modeling data. SAS, SPSS, and many other programs contain modules that will compute General Linear Models (often called GLM) and also called linear statistical models (as described in the book of that name by Neter, Wasserman, and Kutner among many others). Programs to compute linear models were originally designed for fixed effects analysis of variance models (i.e., all levels of interest for each classification factor are included in the design and only one observation from each subject or experimental unit). The following three fundamental conditions are implied: 1. Observations come from independent random samples (that is, only one observation for each subject who have been randomly assigned to two or more classification groups) 2. The residuals of the model are normally distributed 3. The residuals have equal variances within each classification group (i.e., homoskedasticity, the population sigma^2 is the same "unknown" value in each group) Very important portions of the printed output from these ANOVA programs are t-tests and F-tests along with their all important pvalues. In order for them to actually mean something these three ideal conditions need to be checked.If two or more data values are collected from a subject, then you have repeated measures or random effects design (which is a topic of great interest in these pages). Checking these Conditions The first condition is generally met from the statistical design and following through on the data collection process, namely, by randomly assigning subjects to the independent groups and taking one observation per subject. The statistical design associated of study planning is an important component of any analysis but will not be covered in these pages. The second condition focuses on normality of the residuals and not the observations themselves. A procedure for checking residuals for normality from the general linear model is found in Section 6, Analysis of Fixed Effects with PROC MIXED. The third condition implies equal "spread" of the residuals across the groups as measured by the pooled variance (assuming an equal variance model). Side-by-side boxplots or hilo (range) plots of the data for each group are particularly helpful methods for visually detecting violations in this condition and should always be among the initial steps of an ANOVA. How to examine the equal variance assumption statistically is presented in Section 10: Tests for Homogeneity of Variances. The MIXED procedure has the capability to make computations with unequal variances for the classification factors defined by a grouping variable (see Section 10 for details and a worked out example). Fortunately, ANOVA works reasonably well with minor violations of the conditions just described for normality and unequal variances; however, the ANOVA assumes independent observations which implies it is not a good choice for data analysis if the first condition, independent observations, is not met. A Variety of Designs In addition to fixed effects, GLM procedures can analyze data from unbalanced designs and and also designs with a few random effects to a somewhat limited extent (the levels included in a factor are an "exchangeable" selection from all possible values, that is other levels of the factors would also be suitable). Once a researcher understands the concepts of ANOVA and regression with the basic ANOVA and regression routines, the next logical step has been to compute statistical analyses with a GLM under designs such as a randomized block, split plot, or the analysis of covariance (a process that combines features from the analysis of variance and regression techniques). In addition, repeated measures data have been analyzed through entering the REPEATED statement in PROC GLM (with SAS) or the Repeated Measures ANOVA routine with SPSS. Despite its popularity and long history, are these GLM procedures destined to become obsolete? If your data clearly satisfy the three conditions listed above, a correct analysis will result, even from an "older" procedure. Data that legitimately require a two-sample t-test or fixed effects ANOVA will produce the "correct" results if you apply a t-test or GLM respectively, or if you decide to compute the sums of squares and corresponding p-values yourself with a calculator (and perhaps a large dose of patience). However, in most real-life data analysis situations, at least one of the three ideal conditions will be violated to the extent it will affect interpretation of the model. If you have clustered data, autocorrelated errors, or repeated measurements, or multiple observations from each subject for any reason, then a statistical procedure with the most recent developments in computing technology is worth investigating. Motivation for Further Study of Mixed Models To clarify the implications of the three conditions for ANOVA: in any situation where you collect two or more data points from the same subject or experimental unit, whether they are multiple measurements on one subject at each testing session, data collected over time, multiple test scores of students within a classroom, job satisfaction ratings from several workers collected within the same company, or opinions of registered voters who live in the same district, you have clustered data, i.e., you should not treat the data as if they are independent. One way to describe this clustering is in terms of heirarchical linear models (HLM) as defined by Raudenbush and Bryk (2002). For example, when you have data collected across multiple trials at each testing session, the individual observations are denoted as level 1 (micro units) and the aggregated data from each cluster as level 2 (macro units). Researchers have typically analyzed the level 2 data (such as means or medians computed from the level 1 data) as if they were the "independent" observations and applied independent groups or repeated measures ANOVA with PROC GLM (or its equivalent). What is most mis-leading in this situation is if the interpretation of the data is made at level 1 (the actual data) when in fact the summary measure applied to the data should always be kept in the foreground. Even though it is commonly done, even in some text book examples, for many research situations this approach is almost always not necessary and may not be desireable. One common example is computing correlations or regression lines. If it clearly makes sense to only talk in terms of averages, that is, in terms of the macro or level 2 units, perhaps a correlation of two average responses would make sense; however, there is a HUGE disclaimer which MUST be made here. Continually go out of the way to remind readers at all times that you are talking about _average_ responses (i.e., the means of 2, 3, 5, or more data values) are correlated at say r=.75 and make a special point to state this correlation has nothing to do with how the individual response values (the micro or level 1 units) may be correlated. They should NOT be interpreted to be correlated as the averages indicate, and they will likely be considerably less correlated depending on how variable the micro units are around their means. Of course, this also implies it makes sense to talk about an average response and rather than an individual response; most subjects don't produce average responses on each trial, so you must always go out of your way to state: "the average of var 1 is found to be correlated with the average of var 2" If these averages are based on several repeats, e.g., 5-10 trials, it probably is a reasonably accurate number to state the mean is a representative value. And finally, if you have different levels of a fixed factor, that factor should first be considered before you unwittingly take averages across levels of it. More details on how to work with data effectively at the micro level will appear throughout the following pages.