Performance of Some Ridge Parameters for Probit Regression:

Transcription

1 Performance of Some Rdge Parameters for Probt Regresson: wth Applcaton on Swedsh Job Search Data Håkan Lockng 1, Krstofer Månsson and Ghaz Shukur 1, 1 Department of Economcs and Statstcs, Lnnaeus Unversty, Sweden. Department of Economcs, Fnance and Statstcs, Jönköpng Unversty, Sweden. Abstract In rdge regresson the estmaton of the rdge parameter s an mportant ssue. Ths paper generalzes some methods for estmatng the rdge parameter for probt rdge regresson (PRR) model based on the work of Kbra et al. (011). The performance of these new estmators are udged by calculatng the mean square error (MSE) usng Monte Carlo smulatons. In the desgn of the experment we chose to vary the sample sze and the number of regressors. Furthermore, we generate explanatory varables that are lnear combnatons of other regressors, whch s a common stuaton n economcs. In an emprcal applcaton regardng Swedsh ob search data we also llustrate the benefts of the new method. Keywords: probt regresson; maxmum lkelhood; multcollnearty; rdge regresson; MSE; ob search. Mathematcs Subect Classfcaton: Prmary 6J07; Secondary 6J0 1

2 1. Introducton In ths paper we nvestgate the effect of havng explanatory varables, that are a lnear combnaton of other regressors, on the probt regresson model. Ths problem s very common n the area of mcroeconometrcs and t leads to hgh varance and nstablty when estmatng the unknown vector of coeffcents by applyng the tradtonal maxmum lkelhood (ML) method. A popular soluton to ths type of problem s rdge regresson ntroduced for the lnear regresson model by Hoerl and Kennard (1970a,b). The authors showed n that paper that the rdge regresson estmator has better mean squared error (MSE) propertes than ordnary least squares (OLS) when the explanatory varables are collnear. Rdge regresson estmator for other models such as the logt and probt has then, based on the result from Hoerl and Kennard (1970a,b), been derved for the non-lnear logt and Posson models by Schaeffer et al. (1984), Månsson and Shukur (011a,b), among others. The purpose of ths paper s to develop probt rdge regresson (PRR) by generalzng some methods of estmatng the rdge parameter evaluated n Kbra et al. (011) so they can be used for ths estmaton method. In order to be able to udge the performance of the dfferent methods of estmatng k we calculate the mean squared error (MSE) usng Monte Carlo smulatons. In the desgn of the experment we chose to vary the sample sze, the number of explanatory varables and the degree of correlaton. Furthermore, we chose to generate explanatory varables that are lnear combnatons of other regressors and we evaluate the effect of both contnuous regressors and dummy varable. Hence, n the smulaton study we replcate an emprcally relevant stuaton whch s usually not consdered when dfferent rdge parameters are evaluated. The result from the smulaton study shows that the PRR always outperforms the ML n the presence of hghly correlated lnear combnatons of the regressors. Then, n an emprcal applcaton the beneft of usng PRR nstead of ML s llustrated to practtoners. We show that usng ths new estmaton method we obtan estmators of the unknown vector of coeffcents wth much lower varances than the ML method.

3 The paper s organzed as follows: n Secton, we descrbe the statstcal methodology. The desgn of the experment and smulated results are provded n Secton 3. In Secton 4 we provde an emprcal example whle n secton 5 we gve a bref summary and conclusons.. Methodology Ths secton defnes the probt regresson model and descrbes the PRR and the tradtonal ML estmaton methods..1 The Probt Rdge Regresson Estmator Consder the followng regresson model: where y * y * x ' u (.1) s an latent varable, x s the th row of X whch s an 1 p explanatory varables, s a p 1 1 vector of coeffcents and u n p data matrx wth s an error term assumed to be normally dstrbuted. The latent varable s not observable n realty; nstead we may analyze the followng dummy varable: y 1 f y* 0 = (.) 0 otherwse whch s dstrbuted as Be where x' and s the dstrbuton functon of the standard normal dstrbuton. In ths stuaton the probt regresson model should be used whch s estmated by ML by applyng the subsequent teratve weghted least square (IWLS) algorthm dscussed n Cameron and Trved (1998): where zˆ -1 ' ˆ ' ˆˆ ML X WX X Wz (.3) Wˆ dag x ' x ' 1 x ' and ẑ s a vector where the th element equals y ˆ log ˆ. The MSE of the ML estmator corresponds to: ˆ 1 ˆ 3

4 J 1, (.4) ' ˆ -1 ML tr X WX E L 1 where s the th egenvalue of the ˆ X ' WX matrx. When the explanatory varables are collnear some egenvalues wll be small whch nflate the MSE. In ths stuaton the followng PRR estmator mght be a better alternatve: RR ' ˆ -1 The MSE of ths estmator equals: X WX ki X ' WX ˆ. (.5) ML E L J J ML k 1 1 k k, (.6) where the frst term corresponds to the varance and the second term equals the squared bas. The PRR estmator wll have a lower MSE than the ML estmate f we fnd a value of k such that the reducton n the varance term s greater than the ncrease of the squared bas.. Suggested estmators of the rdge parameter There s not a defnte rule of how to estmate the rdge parameter k. However, many suggestons have been gven for the lnear regresson model and some of them wll be generalzed n ths paper so they are applcable for PRR. The frst one that we suggest s based on the classcal rdge parameter proposed by Hoerl and Kennard (1970a,b): ˆ K1, ˆ max where we defne ˆ max to be the maxmum element of ML and ˆ corresponds to the sum of square devance resduals dvded by the degrees of freedom ( n p 1). In Schaeffer et al. (1984) a modfed verson of ths estmator was proposed: 1 K ˆ. max Furthermore, two rdge regresson estmators wll be proposed based on Kbra (003): 4

5 ˆ K3 l ˆ 1 1 l, and K4 medan m, where m ˆ. ˆ We then propose the followng rdge parameter evaluated by Kbra et al. (011): 1 K5 max m, K6 max m, p 1 K7 1 m 1 p p, K8 m 1 1 p 1 K9 medan m 1, K11 max q, K10 medan m, K1 max q, 1 p p 1 K13 1 q, K14 1 p p q, 1 1 K15 medan q, K16 medan q, where q n p ˆ max ˆ max s defned as the maxmum egenvalue of X ' WX ˆ. and max.3 Judgng the performance of the estmators To nvestgate the performance of the PRR and ML method we calculate the MSE usng the followng equaton: MSE R SE R 1 1 R ˆ ' ˆ R, (.7) where ˆ s the estmator of obtaned from ML or PRR and R equals 000 whch corresponds to the number of replcates used n the Monte Carlo smulaton. 3. The Monte Carlo smulaton In ths secton we descrbe the desgn of the experment and dscuss the result of the smulaton study. 5

6 3.1 The Desgn of the Experment Followng Kbra (003) we generate p explanatory varables usng the followng equaton, 1 1/ 1 1 p p x z z z z, 1,,... n, (3.1) where p 1 represents to whch degree the explanatory varable s determned by the other regressors, and z are pseudo-random numbers from the standard normal dstrbuton. When dummy varables are used nstead we consder the x to be latent varables and we make the explanatory varables bnary by applyng equaton (.). The dependent latent varable s then generated usng the followng formula: y * = x x u 1 1 l p (3.) where u are pseudo-random numbers from the standard normal dstrbuton. Ths latent varable s also gong to be made bnary by usng equaton (.). The factors we chose to vary n the Monte Carlo experment are the degree of correlaton, the number of observatons and the number of explanatory varables. Three dfferent values of correspondng to 0.85, 0.95 and 0.99 are consdered. We study sample szes wth, 50, and 0 observatons and equatons wth 5 and 10 regressors. We wll generate models consstng of only 5 or 10 contnuous regressors. Furthermore, models consstng of a mxture between contnuous and dscrete random varables wll be consdered. In the mxture models 40 % of the regressors wll be dummy varables and 60 % contnuous varables. 3. Result Dscusson The estmated MSEs of the dfferent estmaton methods can be found n Tables 1 and. The factors that have an mpact on the estmated MSE are to what degree the explanatory varables are determned by the other regressors, the number of observatons and the number of explanatory varables. Increasng whle holdng n and p fxed leads, n general, to a hgher estmated MSE for ML and PRR when applyng most of the dfferent rdge parameters. The 6

7 least robust opton of estmatng k s to use ether the K1 and K that are proposed by Hoerl and Kennard (1970a,b) and Schaeffer et al. (1984), respectvely. Other rdge parameters that are better than these two but stll do not work well n the presence of multcollnearty are the those based on q (.e, K1, K14 and K16). However, for PRR when the rdge parameter s estmated usng ether the nverse of m (rdge parameters K5, K7 and K9) or the nverse of q (.e, K11, K13 and K15) the estmated MSE occasonally decreases. The rdge parameters that are calculated based on the nverse of q are the ones wth the lowest estmated MSE for all dfferent values of when the sample sze s low. However, n contrast to most of the other rdge parameters, the estmated MSE of K11, K13 and K15 ncreases wth the sample sze. Hence, when the number of observatons s large the rdge parameters K3 and K6 should be preferred. These results hold for both 5 and 10 lnear combnatons and both when we have only contnuous varables and a mxture between dscrete and contnuous varables. 7

8 Table 1: Estmated MSE when all regressors are contnuous =0.85 Estmated MSE when p=5 ML K1 K K3 K4 K5 K6 K7 K8 K9 K10 K11 K1 K13 K14 K15 K = = Estmated MSE when p=10 = = =

9 Table : Estmated MSE when there s a mx between contnuous and dscrete regressors =0.85 Estmated MSE when p=5 ML K1 K K3 K4 K5 K6 K7 K8 K9 K10 K11 K1 K13 K14 K15 K = = Estmated MSE when p=10 = = =

10 4. Emprcal Applcaton A standard ob search model predcts that optmal search behavors generate a reservaton wage and the worker wll accept any offer above hs reservaton wage. Frms, on the other hand, create vacances to maxmze proft whch generate an exogenous flow of offers to the workers. However, the probablty of recevng a ob offer wll be nfluenced by the effort an unemployed person exerts. The hrng stuaton s also characterzed by mperfect nformaton, so the employer have to relay on attrbutes n there attempt to value the ob-seekers. Such attrbutes could be age, chosen search channel, educaton and all ndvdual attrbutes the frm could observe. Böhem and Taylor (00) found that drect contact s the most effectve search method n Brtan and that other search methods were not sgnfcant. In another UK study, Frters et al. (005) usng a panel of unemployed men durng found that search channels such as drect contact, socal networks and agences are more effectve than usng ob centers and newspaper. Usng data from 1981, 1983 and 1986 for Canada, Osberg (1993) found sgnfcant effects for dfferent search methods dependng on sample and year, and n most of the estmates there were only 1- search methods for each year that were sgnfcant. The results usng US data show the same thng, only the search trough newspapers was sgnfcant out of 5 studed methods n Holzer (1988). The above studes report that ob seekers use about 3 methods on average, out of 5-6 studed alternatves. As Böhem and Taylor (00) pont out, ob search does not appear to be a sngle, unform actvty for the unemployed seekng work. Thus researchers have numerous nomnal varables that are not mutually exclusve so we can expect a large degree of multcollnearty between the ncludng varables, especally as other varables are ncluded n the studes as well or put n another way, the data could contan to lttle varaton to be able to answer detaled questons about the search effectveness of ndvdual channels. Ths study uses a dataset earler used by Bolnder (1999) contanng a random selecton of 1806 regstered unemployed n the begnnng of 1996 n Sweden. The data was kndly suppled by Mattas Strandh, Umeå Unversty. The outcome s f the respondent has got a ob or not durng a year perod after the frst contact. Our search channels nclude usng newspaper advertsng, usng frends or own contact. Each varable s graded from never used t, sometmes used t to usng t often. We do not use search through publc employment 10

11 servce as a varable as t s mandatory for ganng access to unemployment benefts. Thus our reference pont ncludes search through publc employment servces. The other exogenous varables, observed n the ntal perod, used n ths study are Tme spent n search, Number of contacts wth employers, Work experence n desred ob dvded up nto categores, Gender, Age, Age Squared, Educaton dvde up nto 3 categores, Ctzenshp, Cvl status, Handcap, Length of unemployment spell, Earler work classfcaton dvded up nto 6 categores and f the ndvdual has worked before or not. We also nclude a varable that measures atttude or motvaton towards work. The varable s defned as a summaton of categorcal values on Importance of workng, Lke to work even f you have money, Dslkes beng unemployed, Become borng f you don t have a ob. To have a ob s among the most mportant thngs n lfe. Thus, we expect Atttude toward work to have a postve mpact on the lkelhood to get a ob. To allow for nonlnear effect from the categorcal varable we nclude t squared as well. The results can be found n Tables 3 and 4 are for values of the estmated coeffcents together wth the vector bootstrapped standard errors (n parenthess). Some estmators of k parameters have very hgh values and push all coeffcents to zero whle others gve a very low value of k so they do not adust the coeffcents. The results are broadly consstent wth the smulatons study, the suggested parameterzaton of the method K1, K13 and K15 reduce the standards errors, although method K5, K7, K9 and K11 are smlar n ths sample. The average reducton n standard errors are between 0 and 57% for ndvdual coeffcents for the suggested method K1, K13, K15 and the unweghted average reducton s about 40%. However, the number of sgnfcant coeffcents at the 5% level does not change that much. The K13 s a clear excepton from ths concluson n ths sample. The K13 produces estmated parameters that are very close to those from the ML (whch are consstent n large samples) and at the same tme t heavly reduces the standard errors of the coeffcent so that these estmated parameters become statstcally sgnfcant. The overall results are broadly n lne wth earler cted studes. The results show a large mprovement n the precson of the estmated effects on the dfferent search channels, and 11

12 searchng through frends are the outstandng channel, f the obectve s to fnd a ob. More astonshng s that an extensve search on your own and through newspaper seems to be counter productve, when we control for tme spend n search. Thus ndcate that regulatons that requre oblged search and employer contact attempts for workers on unemployment benefts are not an effectve method to mprove ther ob chances. The results nstead suggest that socal networkng s effectve as a means to get a ob durng the hgh unemployment perod of ths study. Moreover, the results emphasze that hghly educated and persons wth sklled blue collar or hgh whte collar work experence have a large advantage n the ob search market. 1

13 Table 3: Impact of search strateges and human captal varables on the probablty of obtanng a ob durng condtoned on beng unemployed 006. Results for Maxmum Lkelhood and Probt Rdge Regresson methods. ML K1 K K3 K4 K5 K6 K7 K8 Medum Educaton ( ) (0.0458) ( ) ( ) ( ) ( ) (0.0464) ( ) ( ) Hgh Educaton (0.1613) ( ) ( ) ( ) (0.0043) ( ) (0.018) (0.1611) ( ) Age (0.0710) (0.0154) (0.0655) ( ) ( ) (0.0516) ( ) (0.0697) ( ) Newspaper (sometmes) ( ) ( ) (0.0976) ( ) (0.0068) ( ) (0.0491) ( ) ( ) Newspaper (often) (0.166) ( ) (0.1544) ( ) (0.0053) (0.1560) ( ) (0.1619) ( ) Own contact (sometmes) (0.0940) ( ) ( ) ( ) ( ) (0.0936) ( ) ( ) ( ) Own contact (often) (0.137) ( ) (0.146) (0.004) ( ) (0.16) ( ) (0.130) (0.0643) Frends (sometmes) (0.0989) ( ) ( ) ( ) ( ) ( ) (0.0167) ( ) ( ) Frends (often) (0.1377) ( ) (0.1305) (0.0046) ( ) (0.131) ( ) (0.1370) ( ) Tme spent n search (Hours) (0.0055) ( ) ( ) ( ) (0.0045) (0.0055) ( ) (0.0055) ( ) Number of Contacts wth employer ( ) ( ) ( ) ( ) (0.0104) ( ) (0.0111) ( ) ( ) Female ( ) ( ) ( ) (0.0055) ( ) ( ) (0.0170) ( ) ( ) Sngle (0.0930) (0.0503) ( ) (0.0070) ( ) (0.0917) (0.0380) (0.096) ( ) Foregn Ctzenshp (0.1686) ( ) ( ) (0.0060) ( ) ( ) ( ) ( ) ( ) Handcap ( ) (0.0450) (0.1658) (0.0070) ( ) (0.1655) ( ) ( ) ( ) Some earler work experence n the desred ob ( ) ( ) (0.1163) ( ) ( ) (0.1166) ( ) (0.1137) ( ) Good earler work experence n the desred ob (0.1089) ( ) (0.1018) ( ) ( ) (0.109) ( ) (0.1083) ( ) 13

14 Length of unemployment spell (0.0000) (0.0000) (0.0000) (0.0000) (0.0000) (0.0000) (0.0000) (0.0000) (0.0000) Worked before ( ) (0.0485) (0.1531) ( ) ( ) (0.1516) ( ) ( ) ( ) Sklled Blue collar worker ( ) ( ) ( ) ( ) ( ) ( ) ( ) (0.1147) ( ) Low whte collar worker (0.157) ( ) (0.1445) (0.0039) ( ) (0.1445) ( ) (0.1518) ( ) Md whte collar worker (0.1689) (0.0460) ( ) (0.0065) ( ) ( ) (0.0146) ( ) ( ) Hgh whte collar worker (0.6063) ( ) (0.569) (0.0010) (0.0048) (0.566) (0.0149) (0.600) ( ) Executve or had own busness (0.378) (0.0346) (0.981) (0.0078) (0.009) (0.881) (0.0165) (0.38) ( ) Atttude to work ( ) (0.0319) ( ) ( ) ( ) ( ) ( ) (0.0608) ( ) Age Squared ( ) (0.0000) ( ) ( ) ( ) ( ) ( ) ( ) (0.000) Length of unemployment spell Squared ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Atttude to work Squared ( ) ( ) ( ) ( ) ( ) (0.0015) ( ) ( ) (0.0010) Note: The standard errors are n parenthess. 14

15 Table 4: K9 K10 K11 K1 K13 K14 K15 K16 Medum Educaton ( ) (0.0616) ( ) (0.0001) (0.0464) ( ) ( ) ( ) Hgh Educaton ( ) ( ) ( ) ( ) (0.018) (0.1611) ( ) ( ) Age (0.0695) ( ) (0.0637) ( ) ( ) (0.0697) ( ) (0.0695) Newspaper (sometmes) ( ) (0.0641) ( ) (0.0009) (0.0491) ( ) ( ) ( ) Newspaper (often) (0.1617) ( ) (0.1610) (0.0001) ( ) (0.1619) ( ) (0.1617) Own contact (sometmes) ( ) (0.0593) ( ) ( ) ( ) ( ) ( ) ( ) Own contact (often) (0.1319) ( ) (0.131) (0.000) ( ) (0.130) (0.0643) (0.1319) Frends (sometmes) ( ) ( ) (0.0988) ( ) (0.0167) ( ) ( ) ( ) Frends (often) (0.1369) ( ) (0.1361) (0.0000) ( ) (0.1370) ( ) (0.1369) Tme spent n search (Hours) (0.0055) ( ) (0.0055) (0.0010) ( ) (0.0055) ( ) (0.0055) Number of Contacts wth employer ( ) (0.0151) ( ) ( ) (0.0111) ( ) ( ) ( ) Gender ( ) ( ) ( ) (0.0007) (0.0170) ( ) ( ) ( ) Sngle (0.096) ( ) (0.0911) (0.0004) (0.0380) (0.096) ( ) (0.096) Foregn Ctzenshp ( ) (0.0718) ( ) ( ) ( ) ( ) ( ) ( ) Handcap ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Some earler work experence n the desred ob (0.1136) ( ) ( ) ( ) ( ) (0.1137) ( ) (0.1136) Good earler work experence n the desred ob (0.108) ( ) (0.1074) (0.0004) ( ) (0.1083) ( ) (0.108) Length of unemployment spell

16 (0.0000) (0.0000) (0.0000) ( ) (0.0000) (0.0000) (0.0000) (0.0000) Worked before (0.1541) (0.0753) ( ) ( ) ( ) ( ) ( ) (0.1541) Sklled Blue collar (0.1146) ( ) ( ) ( ) ( ) (0.1147) ( ) (0.1146) Low whte collar (0.1517) ( ) (0.1507) ( ) ( ) (0.1518) ( ) (0.1517) Md whte collar ( ) ( ) (0.1679) ( ) (0.0146) ( ) ( ) ( ) Hgh whte collar (0.601) (0.0687) (0.5970) ( ) (0.0149) (0.600) ( ) (0.601) Executve or own busness (0.31) ( ) (0.318) ( ) (0.0165) (0.38) ( ) (0.31) Sumatt (0.0604) ( ) ( ) ( ) ( ) (0.0608) ( ) (0.0604) Age Squared ( ) (0.000) ( ) ( ) ( ) ( ) (0.000) ( ) Length of unemployment spell Squared ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Sumatt Squared ( ) ( ) ( ) (0.0001) ( ) ( ) (0.0010) ( ) Note: The standard errors are n parenthess. 16

17 5. Conclusons In ths paper we generalze some new methods of estmatng the rdge parameter k, evaluated for lnear regresson by Kbra et al. (011), to be applcable for probt rdge regresson (PPR). These new methods of estmatng k for PRR are evaluated by means of Monte Carlo smulatons along wth the tradtonal ML method. In the smulaton study we focus on the problem that several explanatory varables n the regresson model are determned by lnear combnatons of other regressors. To udge the performance of the dfferent estmaton methods n ths crcumstance we estmate the MSE. We show that the degree of whch an explanatory varable s determned by other regressors s mportant and ncreasng ths factor yelds an mmense ncrease of the estmated MSE of ML. Instead of applyng the ML we may recommend usng PRR and estmate k usng K11, K13 or K15 when the number of observatons s low and the K3 and K6 estmators for large sample szes, although they showed to heavly shrnk the estmated parameter toward zero n the emprcal study. In the emprcal applcaton we show that the problem of explanatory varables beng a functon of other regressors s an emprcal relevant ssue n mcroeconometrcs and we also llustrate the PRR method. In the applcaton we fnd that the average reducton n standard errors are between 0 and 57% for ndvdual coeffcents for the suggested method K1, K13, K15 and the unweghted average reducton s about 40%. More specfcally, the K13 has shown to outperform the others n the emprcal study n the sense that t produces parameter estmates that are very close to those of the ML method and at the same tme have the smallest varances. The results show that marred, hghly educated and persons wth sklled blue collar or hgh whte collar work experence have a large advantage n the ob search market and the most effectve search method s to use frends. 17

18 Referenser: Alkhams, M. A., Khalaf, G. and Shukur, G. (006). Some modfcatons for Choosng Rdge Parameter. Communcatons n Statstcs- Theory and Methods, 35: Bolnder M (1999) Sökbeteendets betydelse för chanson att htta ett obb. Arbetsmarknad & Arbetslv, år g 5, nr 1. Böhem R. and Taylor M. P. (00). Job search methods, ntensty and success n Brtan n the 1990s. Workng paper No 006, Department of economcs, Johannes Kepler Unversty of Lnz. Cameron, A. C. and Trved P. K. (1998). Regresson analyss of count data. Cambrdge Unversty Press, New York. Frters, P., Shelds, M. A. and Prce, S. W. (005), Job Search Methods and Ther Success: A Comparson of Immgrants and Natves n the UK. The Economc Journal, 115: F359 F376. Hoerl, A.E. and Kennard, R.W. (1970a). Rdge regresson: based estmaton for nonorthogonal Problems. Technometrcs, 1, Holzer, H. (1988). Search method use by unemployed youth, Journal of Labor Economcs, 6, 1-0. Hoerl, A. E. and Kennard, R. W. (1970b). Rdge Regresson: Applcaton to Non- Orthogonal Problems. Technometrcs, 1, Kbra, B.M.G. (003). Performance of some new rdge regresson estmators. Communcatons n Statstcs- Theory and Methods 3: Kbra, B. M. G., Månsson, K. and Shukur, G. (011). Performance of some logstc rdge regresson parameters. To appear n Computatonal Economcs. Månsson, K. and Shukur, G. (011a). On Rdge Parameters n Logstc Regresson, Communcatons n Statstcs, Theory and Methods, 40, Issue 18, Månsson, K. and Shukur, G (011b). A Posson Rdge Regresson estmator. Economc Modellng, 8, Issue 4, Osberg, L. (1993). Fshng n dfferent pools: ob search strateges and ob-fndng success n Canada n the early 1980s, Journal of Labor Economcs, 11, Schaefer, R.L., Ro, L. D. and Wolfe, R. A. (1984). A rdge logstc estmator. Communcatons n Statstcs- Theory and Methods, 13,