Section 8: The Various Types of Within-Subject Variance/Covariance Matrices With PROC MIXED you can avoid potentially confusing or ambiguous discussions of sphericity, Mauchly's test, and the Greenhouse-Geisser and Huynh-Feldt corrections to univariate repeated measures ANOVA. Time spent learning them may be better utilized by evaluating the many patterns of the variance/covariance matrix that PROC MIXED provides and comparing the various fit statistics available (e.g., AIC, BIC) to choose among them. The Residual Variance/Covariance Matrix Whenever you compute a repeated measures analysis of variance (that is, you have collected more than one observation from a subject) a within subject covariance matrix is a crucial component of the analysis. The size of this matrix is determined by the number of observations collected. For example, one measurement collected at pre-test and one measurement collected at post-test generates a 2x2 covariance matrix. If you take measurements from 4 treatments applied to each subject, you will have a 4x4 matrix. The structure of this matrix is also crucial (a few of the most common ones are presented below). The reason is the variance of a difference in two within-subject means should be memorized by anyone who runs repeated measures analyses and is stated here: VAR(y1_mn - y2_mn) = Var(y1_mn) + Var(y2_mn) - 2*COV(y1_mn,y2_mn) The covariance term on the far right makes within subject comparisons more powerful than between subject comparisons, since this term is usually positive. And the value of the covariance is determined by the choice of matrix structure. Choices With mixed linear models the REPEATED option allows you to specify the structure of the R (variance/covariance) matrix which produces the within-subject correlation matrix. The latter presents the correlations observed among observations defined by levels of the within-subject factors. A description of the most commonly applied variance/covariance matrices supported by the MIXED procedure follows. Knowing about the first three types is fundamental understanding how GLM repeated measures works. The following descriptions are only a few of the many possibilities. Independence: R = sig^2 * I Given that a dataset consists of two or more repeated measurements within individuals, the simplest possible correlation structure is to assume independence of the observations (most likely incorrectly). This is equivalent to the assumption that any observation collected from one individual is uncorrelated with all other observations measured from that individual. That is, correlations are assumed to be 0 for all pairwise combinations of the within-subject variables. If rho_jk is the correlation between observations j and k, rho_jj=1 and rho_jk=0, for j not equal to k. _ _ | sig^2 0 0 0 0 | | sig^2 0 0 0 | R= | sig^2 0 0 | | symm. Sig^2 0 | | sig^2 | - - If you specify TYPE=simple on the REPEATED statement of PROC MIXED, then the correlation matrix will be the identity matrix. This choice produces the same results as invoking the independence model with PROC GLM. Compound Symmetry: Corr(Y_ij,Y_ik) = a, j NE k Compound Symmetry assumes non-zero covariance matrix, yet equal covariances exist for all combinations of observations defined by the within-subject factors. That is, every observation collected from a subject is equally correlated with every other observation from that subject. Formally, the diagonal elements equal 1, (rho_jj=1) and and the off-diagonal elements equal a, (rho_jk =a, j not equal to k) where a is also called the intraclass correlation coefficient. Assuming the covariance structure is compound symmetry (TYPE=cs) the following structure defines the variance/covariance matrix for the within-subject factor: - - | sig1 + sig2 | | sig1 sig1 + sig2 | R= | sig1 sig1 sig1+sig2 | | sig1 sig1 sig1 sig1+sig2 | _ _ For any of the six possible comparisons of two means chosen from the 4 time points, the variance of a difference is: VAR(lsm_j - lsm_k) = (sig1 + sig2) + (sig1 + sig2) - (2 * sig1) = 2*sig2 The resulting within-subjects correlation matrix has this structure where a = sig1 / (sig1 + sig2) _ _ | 1 a a a | | 1 a a | Rcorr = | 1 a | | 1 | - - Huynh-Feldt This choice is best described with an example since it is directly connected with the sphericity assumption. Consider the following 4x4 within-subject variance/covariance matrix: c1 c2 c3 c4 - - | 10 5 10 15 | R = | 5 20 15 20 | | 10 15 30 25 | | 15 20 25 40 | - - The variance of the difference in any two means is computed with the following formula: S(c_i - c_j)^2 = s_ci^2 + s_cj^2 - 2*(s_ci_cj) i=1..4, j=1..4 S(c1-c2)^2 = 10 + 20 - 2*(5) = 20 S(c1-c3)^2 = 10 + 30 - 2*(10) = 20 S(c1-c4)^2 = 10 + 40 - 2*(15) = 20 S(c2-c3)^2 = 20 + 30 - 2*(15) = 20 S(c2-c4)^2 = 20 + 40 - 2*(20) = 20 s(c3-c4)^2 = 30 + 40 - 2*(25) = 20 The variance of any selected difference of the six possible pairs of means is equal to 20. Although this exact relation is not likely to be found with real data, it demonstrates that lack of compound symmetry (equal covariances) does not necessarily imply that sphericity is violated. (TYPE=cs for Compound symmetry is a sufficient but not a necessary requirement for sphericity to be met.) Compound symmetry in this example is clearly not met (e.g., the largest variance is 4 times bigger than the smallest), but sphericity still holds. Note that if the variances along the diagonal are equal, then compound symmetry must be the covariance structure. One of the features of the matrix demonstrated here is the variances get larger as the within subject factor increases. For the assumption of sphericity to be true, as the two variances increase, the covariances must also get larger so the variance of a difference remains a constant. Huynh has stated that it would be difficult to conceptualize a situation that produces a covariance matrix satisfying the sphericity condition but not having compound symmetry (1978). The choices of correlation structures defined above may not be reasonable with measurements collected over equally spaced points in time. Correlations most likely will diminish as the time lag between observations increases. Thus other types of matrices are available. This is point where MIXED models now greatly extend themselves beyond what GLM can do. Mixed models offer several additional choices (in addition to those above) of the working correlation matrix, from the simple to complex, for the observations collected within subjects. Autoregressive(AR^-1) Autoregressive is a term associated with times series analysis that assumes observations are related to their own past values through the number of lags between them. That is, an autoregressive correlation structure implies that two observations close to each other over time or space are more highly correlated than two observations spread further apart. Formally, the correlation coefficient between observation j and k, rho_jk, (where j is not equal to k) decreases as the absolute difference between j and k gets larger. A first-order autoregressive correlation structure specifies that rho_jk = rho**|j-k| = where rho is the computed correlation when |j-k|=1. Corr(Y_i,j,Y_i,j+t) = a_t = %alpha^t, t=1,2,...,n_i-j %alpha is estimated by %alpha^hat = _ _ | 1 | | %a_1 1 | V= | %a_2 %a_1 1 | | %a_3 %a_2 %a_1 1 | | %a_4 %a_3 %a_2 %a_1 1 | - - It may help to think of AR(1) as stating the within-subject correlations drop off exponentially with the number of time periods between measurements. This is often assumed to be connected to equally spaced gaps between measurements. If gaps are uneven and the correlations still show this exponential dropoff, then AR(1) may not handle this as well as one of the spatial covariance forms. Unstructured: Corr(Y_ij,Y_ik) = a_ij = %alpha_jk The most complex model is based on the unstructured matrix (type=un) estimates t*(t-1)/2 correlations from the data. This is the correlation found with GLM MANOVA. Unstructured assumes pair-wise correlations are not constrained by the data. No assumptions, as existed in the previous examples, are made about the relative magnitude of the correlation between any two pairs of observations. Formally, rho_jj=1 and rho_ij is free to take any value between -1 and +1. _ _ | 1 | | a_21 1 | V= | a_31 a_32 1 | | a_41 a_42 a_43 1 | | a_51 a_52 a_53 a_54 1 | - - A key feature of the REPEATED statement is the ability to specify one of many possible covariance structures for the residual matrix with the TYPE= option. In the example presented earlier, the value TYPE=cs tells SAS to compute a within-subject covariance matrix based on compound symmetry (the pattern assumed by PROC GLM with its REPEATED statement or when treating subjects as a random variable with a univariate ANOVA). However, other choices may be more appropriate for your data including TYPE=ar(1), an autoregressive structure (where measurements taken at disjunct time intervals become less correlated as the time span increases) or TYPE=un, the unstructured covariance matrix (no distinct pattern exists, the type assumed in MANOVA from GLM). The tradeoff of various correlation matrices is less complex structures require the estimation of fewer parameters, whereas the number of parameters to estimate in unstructured matrices may be extremely large, especially with many values collected over the repeated measures factors. If you don't know the structure of the covariance matrix of the residuals, you can check it by entering TYPE=un (as long as the estimation procedure converges) along with RCORR on the REPEATED statement. Then observe the within-subject correlation matrix to see if some pattern emerges which could perhaps be approximated by a simpler structure. Goodness of fit summaries such as AIC or BIC will help you choose which structure works better (smaller numbers are preferred). If TYPE=un (unstructured residual variance/covariance matrix) gives you the best fit according to the AIC, review the data to see if some smaller set of data points are causing the covariance structure to differ from some more constrained structure. One of these correlation structures may fit the theoretical pattern you expect (i.e., autoregressive for data collected over time). In the end, you may decide to choose a statistical model which is not optimal by these measures (i.e., the smallest); rather, one which gives you a more interpretable model. If AIC and BIC are not considerably smaller when compared to TYPE=un, favor the simpler model. Double Multivariate The double multivariate covariance structure allows you to fit a covariance matrix for two within subject factors (actually more by recoding multiple levels of 2 or more factors into 1). First, assume you have two levels of a within subject factor, wfct coded as A and B, collected from each subject. Also for each condition A and B, three measurements are collected at equally spaced time intervals (tm=1,2,3 which implies an autoregressive correlation) for repeated measurements from both conditions A and B. With the PROC MIXED the REPEATED statement looks like: REPEATED wfct tr / subject=subject type=un@ar(1) rcorr r; This statement fits the UN@AR(1) covariance structure where the covariance parameter estimates UN(1,1), UN(2,1), and UN(2,2) along with AR(1) values are transformed into the R matrix as follows: First, notice the table of Covariance Parameter Estimates Cov Parm Subject Estimate Visit UN(1,1) Subject 2.1248 UN(2,1) Subject 0.1887 UN(2,2) Subject 1.3870 tr AR(1) Subject 0.7222 The estimated R Matrix is defined in 4 blocks by levels of A and B and the three repeated measurements: Col1 Col2 Col3 Col4 Col5 Col6 wfct A A A B B B tm 1 2 3 1 2 3 Row 1 A 1 2.1248 1.5345 1.1082 0.1887 0.1363 0.0984 2 A 2 1.5345 2.1248 1.5345 0.1363 0.1887 0.1363 3 A 3 1.1082 1.5345 2.1248 0.0984 0.1363 0.1887 4 B 1 0.1887 0.1363 0.0984 1.3870 1.0017 0.7234 5 B 2 0.1363 0.1887 0.1363 1.0017 1.3870 1.0017 6 B 3 0.0984 0.1363 0.1887 0.7234 1.0017 1.3870 The UN(1,1) value is the diagonal element for the upper left portion of the R matrix for factor level A and UN(2,2) is the diagonal element for the lower right portion corresponding to factor B for all three time points. The UN(2,1) value, the covariance term, is the diagonal entry for the lower left and upper right blocks of the R matrix. The other elements (off diagonals in each block) are obtained by multiplying the respective UN(_,_) values by the autocorrelation estimate 0.7222 for lag 1 and (.7222)*(.7222) for the lag 2 elements of the matrix. This R matrix then produces this within subject correlation matrix printed with the rcorr option on the REPEATED statement: Row Col1 Col2 Col3 Col4 Col5 Col6 1 1.0000 0.7222 0.5216 0.1099 0.0794 0.0573 2 0.7222 1.0000 0.7222 0.0794 0.1099 0.0794 3 0.5216 0.7222 1.0000 0.0573 0.0794 0.1099 4 0.1099 0.0794 0.0573 1.0000 0.7222 0.5216 5 0.0794 0.1099 0.0794 0.7222 1.0000 0.7222 6 0.0573 0.0794 0.1099 0.5216 0.7222 1.0000