Example #2: Factorial Independent Groups Design. A data set was created using summary data presented by Wicherts, Dolan and

Transcription

1 Example #2: Factorial Independent Groups Design A data set was created using summary data presented by Wicherts, Dolan and Hessen (2005). These authors examined the effects of stereotype threat on women s mathematics ability. Study participants were assigned to one of six groups defined by crossing the independent factors of test condition (J: control, nullified, stereotype threat) and sex (K: male, female). There were four different tests administered to study participants (arithmetic, number series, word problems, and sums tests); we focus here on scores for the arithmetic test (out of 40) because these scores exhibited a greater magnitude of variance heterogeneity than scores for the other tests. This is an unbalanced design with cell sizes ranging from 45 to 50 participants, and a total sample size of 283. The following sample program code illustrates how the WJGLM program is used to conduct omnibus tests of the interaction and main effects: y={10, 10, 16, 20, 35, 13, 12, 0, 17, 9, 10, 14, 5, 12, 16, 10, 23, 20, 22, 7, 1, 17, 7, 9, 10, 7, 19, 1, 40, 9, 7, 2, 9, 14, 36, 12, 15, 23, 18, 25, 40, 13, 24, 13, 9, 17, 18, 40, 9, 9, 28, 10, 5, 13, 13, 10, 10, 12, 11, 16, 10, 16, 19, 19, 17, 15, 21, 4, 17, 23, 11, 11, 13, 20, 13, 9, 16, 14, 18, 9, 6, 6, 25, 8, 12, 12, 3, 4, 32, 18, 16, 12, 13, 14, 22, 19, 13, 11, 17, 40, 20, 17, 15, 20, 16, 6, 11, 15, 13, 26, 14, 29, 13, 32, 12, 6, 39, 9, 17, 11, 11, 27, 18, 17, 10, 7, 13, 14, 18, 20, 13, 17, 13, 12, 11, 18, 18, 32, 18, 19, 20, 10, 18, 15, 11, 6, 18, 9, 14, 6, 10, 6, 8, 14, 7, 10, 11, 11, 9, 12, 13, 13, 14, 15, 13, 9, 12, 10, 37, 12, 10, 12, 23, 11, 14, 11, 11, 17, 12, 5, 19, 10, 10, 1, 13, 28, 15, 13, 21, 13, 11, 17, 9, 8, 8, 17, 15, 20, 8, 9, 9, 17, 8, 25, 20, 14, 9, 14, 8, 20, 11, 8, 16, 17, 10, 12, 12, 5, 6, 11, 30, 6, 11, 14, 30, 5, 12, 14, 4, 10, 13, 14, 8, 0, 13, 10, 10, 15, 13, 7, 20, 15, 12, 2, 8, 16, 6, 16, 0, 8, 40, 8, 9, 15, 10, 40, 0, 0, 7, 0, 36, 15, 12, 17, 20, 12, 12, 5, 15, 2, 1, 35, 4, 8, 40, 9, 0, 0, 14, 4, 40, 5, 1}; nx = { }; cj={1 0-1, 0 1-1}; c=cj@ck; print 'Test Of Interaction Effect - ADF Solution'; opt1=0; 1

2 cj={1 0-1, 0 1-1}; ik={1 1}; c=cj@ik; print 'Test Of Condition Main Effect - ADF Solution'; opt1=0; ij={1 1 1}; c=ij@ck; print 'Test Of Sex Main Effect - ADF Solution'; opt1=0; cj={1 0-1, 0 1-1}; c=cj@ck; print 'Test Of Interaction Effect - ADF Solution With Trimming'; cj={1 0-1, 0 1-1}; ik={1 1}; c=cj@ik; print 'Test Of Condition Main Effect - ADF Solution With Trimming'; ij={1 1 1}; c=ij@ck; print 'Test Of Sex Main Effect - ADF Solution With Trimming'; 2

3 cj={1 0-1, 0 1-1}; c=cj@ck; print 'Test Of Interaction Effect - ADF Solution With Trimming & Bootstrap'; opt2=1; seed=651332; cj={1 0-1, 0 1-1}; ik={1 1}; c=cj@ik; print 'Test Of Condition Main Effect - ADF Solution With Trimming & Bootstrap'; opt2=1; seed=651332; ij={1 1 1}; c=ij@ck; print 'Test Of Sex Main Effect - ADF Solution With Trimming & Bootstrap'; opt2=1; seed=651332; The first two SAS/IML statements define the dataset and the number of observations in each group. The order of entry for Y and NX must correspond, so that the first n 11 rows of Y correspond to the sample size for the first cell of the design, the next n 12 rows of Y correspond to the sample size for the second cell of the design, and so on. The data for males in the control condition are entered first, followed by the data for females in the control condition, then males in the nullified condition, females in the nullified condition, and so on. As in the one-way example, the commas used to separate the 3

4 individual data points serve to delineate the rows of Y, so that Y is a column vector with 283 elements. p={10, 10, 16, 20, 35, 13, 12, 0, 17, 9, 10, 14, 5, 12, 16, 10, 23, 20, 22, 7, 1, 17, 7, 9, 10, 7, 19, 1, 40, 9, 7, 2, 9, 14, 36, 12, 15, 23, 18, 25, 40, 13, 24, 13, 9, 17, 18, 40, 9, 9, 28, 10, 5, 13, 13, 10, 10, 12, 11, 16, 10, 16, 19, 19, 17, 15, 21, 4, 17, 23, 11, 11, 13, 20, 13, 9, 16, 14, 18, 9, 6, 6, 25, 8, 12, 12, 3, 4, 32, 18, 16, 12, 13, 14, 22, 19, 13, 11, 17, 40, 20, 17, 15, 20, 16, 6, 11, 15, 13, 26, 14, 29, 13, 32, 12, 6, 39, 9, 17, 11, 11, 27, 18, 17, 10, 7, 13, 14, 18, 20, 13, 17, 13, 12, 11, 18, 18, 32, 18, 19, 20, 10, 18, 15, 11, 6, 18, 9, 14, 6, 10, 6, 8, 14, 7, 10, 11, 11, 9, 12, 13, 13, 14, 15, 13, 9, 12, 10, 37, 12, 10, 12, 23, 11, 14, 11, 11, 17, 12, 5, 19, 10, 10, 1, 13, 28, 15, 13, 21, 13, 11, 17, 9, 8, 8, 17, 15, 20, 8, 9, 9, 17, 8, 25, 20, 14, 9, 14, 8, 20, 11, 8, 16, 17, 10, 12, 12, 5, 6, 11, 30, 6, 11, 14, 30, 5, 12, 14, 4, 10, 13, 14, 8, 0, 13, 10, 10, 15, 13, 7, 20, 15, 12, 2, 8, 16, 6, 16, 0, 8, 40, 8, 9, 15, 10, 40, 0, 0, 7, 0, 36, 15, 12, 17, 20, 12, 12, 5, 15, 2, 1, 35, 4, 8, 40, 9, 0, 0, 14, 4, 40, 5, 1}; nx = { }; C is defined as the Kronecker (i.e., direct) product of two matrices; one for the condition effect, and one for the sex effect. To test the interaction effect, C is a Kronecker product of CJ and CK, each of which defines linearly independent contrasts among the between-subjects factor levels. To test the condition main effect, C is a Kronecker product of CJ, which defines linear contrasts among the conditions and an indicator matrix that sums the levels of sex. A similar syntax is used to define the contrast matrices to test the sex main effect. If C J is a matrix of dimension (J 1) x J and C K is a matrix of dimension (K 1) x K, then the Kronecker product is defined as C J C K = c C c 11 K 1J K c C c C C (J-1)1 K (J-1)J K. The program code to test the interaction and main effects adopts the default options for trimming (20% symmetric trimming) and bootstrapping (999 bootstrap samples). A 4

5 starting seed for the bootstrap has been specified to ensure reproducibility of results. Once a seed has been specified, it is not necessary to specify another seed for subsequent invocations of RUN WJGLM or RUN BOOTCOM, unless a QUIT statement is specified. The means and SEs for both the least squares and trimmed solutions are reported in Table 2. The following results are obtained for the test of the interaction effect when least-squares estimators were adopted: T WJ /c = 2.52 with 2 and df (p =.0837). When robust estimators were adopted the corresponding values are T /c = 3.13, with WJt 2 and df (p =.0483). When the bootstrap is used, p = Table 2: Means and SEs for least-squares and trimmed estimators (20% symmetric trimming) for a factorial independent groups design Leastsquares mean Leastsquares SE Trimmed mean Winsorized SE Control men Control women Nullified men Nullified women Stereotype threat men Stereotype threat women /c T WJ For the marginal main effect of condition when least-squares estimators are used, = 2.15, with 2 and df (p =.1199). When trimming is used, T /c = 5.21, with 2 and df (p =.0072). When the bootstrap is applied to the trimmed data the p-value is For the condition effect, trimming produces significant results, while results based on least-squares estimators do not. For the marginal main effect of sex WJt 5

6 when least-squares estimators are adopted, T WJ /c = 2.93 with 1 and df (p =.0882). For the trimmed data, T /c = 5.75, with 1 and df (p =.0179). When the WJt bootstrap is adopted, p = Hence for the sex effect, the test based on trimming produces significant results, while the test based on least-squares estimators does not. Given that the interaction between condition and sex is statistically significant when robust estimators are adopted, one might wish to probe this interaction using a series of tetrad (i.e., interaction) contrasts. SAS code to produce all possible tetrad contrasts when robust estimators are adopted is given below: cjj={1-1 0}; print 'Interaction Contrast for J1 Vs J2 and K1 Vs K2 - ADF Solution With Trimming'; cjj={1 0-1}; print 'Interaction Contrast for J1 Vs J3 and K1 Vs K2 - ADF Solution With Trimming'; cjj={0 1-1}; print 'Interaction Contrast for J2 Vs J3 and K1 Vs K2- ADF Solution With Trimming'; 6

7 The SAS code to produce and test all three tetrad contrasts when robust estimators and the bootstrap are adopted and the FWER is controlled at α =.05 is: cjj={1-1 0, 1 0-1, 0 1-1}; print 'Interaction Contrasts - ADF Solution With Trimming & Bootstrap'; opt2=1; run bootcom; The following results are obtained when each of the tetrad contrasts is conducted (with robust estimators): K1 vs K2 and J1 vs J2: T /c = 5.77 with 1 and df (p =.0184); K1 vs K2 and J1 vs J3: = 0.47 with 1 and df (p =.4964); K1 vs K2 and J2 vs J3: T /c =1.98 with 1 and df (p =.1628). Applying Hochberg's step-up WJt T WJt/c Bonferroni procedure (See Keselman, 1998; Keselman et al., 2004), only the first contrast is statistically significant. For the ADF solution with trimming and bootstrapping, the critical value is 5.46; the first tetrad contrast is also statistically significant. WJt SAS code to product ES estimates for tetrad contrasts when the default standardizer is used (i.e., the cell corresponding to the first level of condition and the first level of sex) is as follows (a scaling factor is always adopted): cjj={1-1 0}; print 'Effect Size For J1 Vs J2 And K1 Vs K2 - ADF Solution With Trimming'; opt3=1; cjj={1 0-1}; 7

8 print Effect Size For J1 Vs J3 And K1 Vs K2 - ADF Solution With Trimming'; opt3=1; cjj={0 1-1}; print Effect Size For J2 Vs J3 And K1 Vs K2 - ADF Solution With Trimming'; opt3=1; The ES estimates and their 95% CIs are as follows: (-1.23, -0.06), (-0.85, 0.40), and 0.37 (-0.12, 0.96). If the square root of the average of the variances were used to compute the ES estimates and their confidence intervals instead of a single cell of the design, the following results would be obtained: (-1.41, -0.09), (-0.83, 0.44), and 0.46 (-0.19, 1.07). Suppose we are interested in testing a contrast of J1 and J2 (control/nullified) versus J3 (stereotype threat), collapsing over sex. Either of the following code fragments could be used: or cjj={1 1-2}; ckk={1 1}; print 'Effect Size For J1 and J2 Vs J3 - ADF Solution With Trimming'; cjj={.5.5-1}; ckk={.5.5}; 8

9 print 'Effect Size For J1 and J2 Vs J3 - ADF Solution With Trimming'; The contrast vectors would be c cjj k = [ ] c k = , andc = cjj ck k = [ ], for the two programs, respectively. Because the second contrast vector is a.25 times the first, both code fragments would yield the same test statistic. Now suppose we want to construct a CI relevant to the comparison of J1 and J2 (control/nullified) versus J3 (stereotype threat). Changing opt3=0 to opt3=1 in either code fragment will produce an ES and a CI, but both will vary across the two programs. One must carefully choose the contrast coefficient values for computing the ES estimate for marginal effects. The correct approach is cjj={.5.5-1}; ckk={.5.5}; print 'Effect Size For J1 and J2 Vs J3 - ADF Solution With Trimming'; opt3=1 Here c = [ ], producing an ES that is the comparison of the average of the marginal mean for J1 (the average of the means for males and females under J1) and the marginal mean for J2 versus the marginal men for J3. By contrast if we used cjj={1 1-2}; ckk={1 1}; 9

10 print 'Effect Size For J1 and J2 Vs J3 - ADF Solution With Trimming'; opt3=1; c = [ ] and would produce an effect size for a contrast of the average of the marginal sum for J1 (the sum of the means for males and females under J1) and the marginal sum for J2 versus the marginal sum for J3. Finally, a QUIT statement would be specified at the end of all programming statements, to finish the program. 10

11 Reference Wicherts, J.M., Dolan, C.V., & Hessen, D.J. (2005). Stereotype threat and group differences in test performance: a question of measurement invariance. Journal of Personality and Social Psychology, 89, 5,