Variance? which variance? R squared effect size measures in simple mediation models

Transcription

1 Vaiance? which vaiance? squaed effect size measues in simple mediation models M This is it? med di X de Heus, P. (01). squaed effect size measues and ovelap between diect and indiect effect in mediation analysis. Behavio eseach Methods, 44, Y Pete de Heus Methodology & Statistics + Social & Oganisational Psychology Univesity of Leiden 1 Oveview 1. Effect size in mediation analysis. squaed effect size (Faichild et al.) 3. Stange effects, and thei explanation 4. Solution 1. Squaing betas (with nice pictues) 5. Solution. Back to Faichild (and to MacKinnon) 6. Pactical advice, unfotunately with limitations

2 1. Effect size in mediation analysis The total, the diect, and the indiect effect Simplifying assumption: All vaiables ae standadized. tot YX YX di YX. M YX MX YM 1 MX X ß YX Total effect Y ( MX YM MX YX ind MX YM. X 1 MX ) X ß MX M ß YM.X Y ß YX.M tot di ind Diect (ed) and indiect (blue) effect 3 Effect size befoe Faichild Most commonly used: Popotion mediated (PM). PM effect b effect b tot effect tot Intepetation Effect size as popotion of total egession weight. Allows us to say things like: One exta standad deviation of X leads to a pedicted incease of.5 standad deviations in Y, of which 70 % (PM di ) is due to the diect effect and 30 % (PM ind ) to the indiect effect. Advantages Easy to apply Easy to undestand (well, in a way...) 4

3 PM effect b effect b tot effect tot Disadvantages Only elative measue of effect size (lage PM can efe to vey small effect if total effect is small). Unstable and biased in small to medium (N < 500) samples (Faichild et al., 009). Can be negative, o lage than 1 (if diect and indiect effect have opposite signs). Eveyone seems to want vaiance explained (which PM is not). Even when we know total vaiance explained, popotion mediated does not give popotion of total vaiance explained. 5. squaed effect size (Faichild et al.) Faichild et al. (009). Standad decomposition of vaiance explained in egession analysis (e.g. Cohen et al., 003). tot YX di med Y ( X. M ) tot So... Total vaiance is just squaed coelation of Y and X. di YX YM MX 1 MX YX Y ( X. M ) Diect effect vaiance is popotion of Y vaiance shaed with X, but not with M (= squaed semipatial coelation). Indiect effect vaiance is popotion of Y vaiance shaed with both X and M. Faichild, A. J., MacKinnon, D. P., Taboga, M. P., & Taylo, A. B. (009). effectsize measues fo mediation analysis. Behavio eseach Methods, 41, X med di M Y 6

4 Why it looks nice Social scientist love vaiance expained (we ae used to it). Bette statistical popeties (stability, low bias) than popotion mediated (Faichild et al., 009). Intuitive claity. (a) Diect effect vaiance (squaed semipatial coelation) coesponds to unique vaiance explained by pedicto in egession analyis. (b) Indiect effect. Idea that indiect effect vaiance is vaiance shaed by all thee vaiables also sounds natual (although pehaps a little vague). 7 The vedict by Peache & Kelley (011) Po Staightfowad intepetation. Inceases as indiect effect appoaches total effect. Does not depend on sample size. Allows making confidence intevals. Conta Can take negative values (in case of suppession), so it is not eally a popotion of vaiance. Vedict Limited, but may be heuistically useful in some situations Peache, K. J., & Kelley, K. (011). Effect size measues fo mediation models: Quantitative stategies fo communicating indiect effects. Psychological Methods, 16,

5 3. Stange effects, and thei explanation Counteintuitive couse example Effects. Diect effect much stonge than indiect effect: -.8 vs Popotion mediated. Idem:.688 (PM di ) vs..31 (PM ind ). Vaiance explained. Diect effect appeas much weake (!!) than indiect effect. Why? I had no idea, but I pomised my students to ty to find out. esult. This pesentation..345 Social suppot (X) Active coping (M) -.8 Coelations (N = 85) X M M.345 Y Depession (Y) Effects (betas) beta PM Total(ß yx ): Diect (ß yx.m ): Indiect (ß mx ß ym.x ) Effect sizes (Faichild et al.) Total.113 Diect.046 Indiect diect much lage indiect much lage 9 Asymmety of effect sizes Intepetation of effect: One exta SD in X leads to ß exta SDsin Y. Applies to both diect and indiect effect, so both have completely compaable effects on Y. Theefoe, one would expect completely symmetic effects on Y vaiance explained. Moe specifically: - Diect and indiect effect should have same ange of possible values. - Intechange of values of diect and indiect effect should lead to simple intechange of vaiances explained. But both ae untue fo the Faichild et al. method. - Indiect, but not diect effect vaiance explained can be negative. - Exchanging diect and indiect effect leads to completely diffeent vaiances explained. Whee does this asymmety come fom? 10

6 Diect and indiect effect ae heavily intedependent Assumption in Faichild et al. (009). Total vaiance can be uniquely decomposed into diect effect vaiance and indiect effect vaiance, as if these effects ae independent. But they ae not. Once ß di and ß ind have been estimated, diect and indiect effect ae pefectly coelated, because they ae both linea functions of X only (M is cucial fo decomposing ß tot into ß di and ß ind, but aftewad disappeas fom the stoy). Diect: Y = ß di X ^ Indiect: Y = ß ind X ^ Total: Y = ß tot X = (ß di + ß ind )X Fom this follows fo vaiance explained: tot ^ tot ( di ind ) di ind di ind Not two, but thee pats. What to do with the joint pat ß di ß ind? Solution 1. Squaing betas (with nice pictues) Two appoaches 1. Othe-is-zeo appoach. Just take ß di and ß ind as vaiance explained fo the diect and indiect effect espectively. di( OZ ) di ind ( OZ ) ind Intepetation. Vaiance explained by effect if othe effect wee zeo, if the effect wee completely on its own. Advantages. (i) Clea causal intepetation. (ii) No negative vaiances. Disadvantage. Does not add up to total vaiance explained, because joint pat ß di ß ind is ignoed. 1

7 . Hieachical appoach. Take one effect as pimay and othe as seconday. pimay( H ) pimay seconday(h) seconday pimay seconday Intepetation. Pimay as in othe-is-zeo; seconday gives additional vaiance explained ove and above what was explained by pimay. Advantages. (i) Clea causal intepetation. (ii) Vaiances add up to total vaiance explained. Disadvantage (o challenge). How to decide which effect is pimay? Theoetical o pactical consideations. Expected o intended effect vs. possible side-effect(s). If effects have opposite sign. Take lagest effect as pimay. 13 New dawings fo vaiance explained (without negative aea) Use squaes instead of cicles (distinction between diect and indiect effect does not matte hee, so I use b1 and b). Take the lagest of total, diect, and indiect effects as the big squae, containing the othe two effects as smalle squaes. Now all aeas ae positive. A E D 1 ( ß 1 ) 1 1 B H C ( 1 ) 1 ( ß 1 ) 1 1 F G 1 Case 1. ß 1 and ß same sign (hee both positive) Case. ß 1 and ß diffeent signs (hee ß 1 positive with lagest absolute value) 1 14

8 Disadvantage of squaed betas appoach Not wong, but vey unnatual. Fist, in computing effects we coect fo thei ovelap. Then, when estimating thei vaiance explained, we act as if othe effect does not exist at all. Mainly useful fo illustative puposes Solution. Back to Faichild (and MacKinnon) Why was the indiect effect favoued? The Faichild et al. is usually smalle than squaed beta. di Y (X.M ) (1 XM ) di Basically, without saying so, they always take the diect effect as pimay. As a consequence, the indiect effect ( YX Y(X.M) ) always gets the joint pat, which: is positive when diect and indiect effect have the same sign indiect effect is favoued (as in my counteintuitive example); is negative when diect and indiect effect have opposite signs indiect effect is disfavoued (may become negative). 16

9 Anothe theeway decomposition If we want symmety in vaiance explained by diect and indiect effect, we need a theeway decomposition of vaiance explained with: di ind (1 XM ) di (1 XM ) ind ovelap di ind MX ( di ind ) This theeway decomposition: shows the desied symmety between diect and indiect effect; neve gives negative vaiance explained fo the indiect effect (it does fo the ovelap pat, but that is no poblem). 17 Undestanding ind We can ewite ind as follows: ind ( 1 XM ) ind (1 XM ) YM. X MX YM. X MX Unique popotion of Y vaiance explained by M (but not by X) Popotion of M vaiance explained by X So apat fom satisfying symmety demands, it can also be undestood as the popotion of Y vaiance uniquely explained by M, but only as fa as this was pedicted by X. Was mentioned in MacKinnon (008, p. 84, equation 4.6) as one of thee measues equiing moe development (but he neve did). 18

10 6. Conclusions effect size measues do not wok as well as they should, because, - eithe we leave the ovelap vaiance out (and then a lot of vaiance emains unexplained), - o we assign the ovelap to one of the effects (which equies a athe abitay choice). My solutions ae moe helpful in explaining what goes wong than in giving applied eseaches helpful effects size measues (soy!). My pesonal advice: in many situations di and ind themselves give a easonable indication fo the stength of both effects. But see also Peache & Kelley (011), fo othe possibilities. 19