Forecasting Stock Returns using Evolutionary Artificial Neural Networks 1

Transcription

1 Forecastng Stoc Returns usng Evolutonary Artfcal eural etwors 1 Prsadarng Solpadunget, Keshav Dahal, apat Harnporncha MOSAIC Research Group, Unversty of Bradford, Great Horton Road, Bradford, BD7 1DP, Great Brtan. CAMT, Chang Ma Unversty, Chang Ma, Thaland. {p.solpadunget,.p.dahal}@bradford.ac.u, tomnapat@camt.nfo Abstract. Several models and technques have been used to forecast stoc returns. Some prevous researches have appled Evolutonary Artfcal eural etwors to predct stocs prces. The most of prevous researches have been concentrated on ether predct stoc ndexes or trends rather than ndvdual stoc returns. In ths paper, we use the adaptve EAs to predct ndvdual stoc returns based on multvarate tme seres (AR wth state varables) models. Comparng wth the tradtonal Lnear Regressons, the As show promsng results and our proposed EAs can mprove the performances of the As as we expected even the mprovements are slghts 1 Introducton For stoc maret nvestors and portfolo managers, t s crucal to have most accurate forecast of stoc returns. However, stoc returns are dffcult to forecast accurately. Several models and technques have been used to forecast stoc returns. The relevant models nclude autoregressve models (AR), autoregressve-movng-average models (ARMA) and AR wth state varables (explanng varable) models. It s reported that the AR models wth state varables are superor to the rest both n short-run and long-run [1]. In developng a model wth many state varables as model nputs to forecast stoc returns, a crucal part s to dentfy nput varables. There are numerous theores and models that determne the nput varables rangng from techncal analyss based on tradng data complcated multvarate tme seres models. The multvarate tme seres models, whch are based on fundamental factors, are consdered more theoretcally sound than those based on techncal factors (e.g. tradng volume, prce trend, etc.)[1]. They are mpled that stoc prces and stoc returns can be explaned and thus predcted by a number of fundamental economc factors as proposed by Captal Maret Theory (CAPM-sngle factor.e. stoc maret ndex) [2,3] and Asset Prcng Theory (APT mult-factors)[4]. The nputs suggested by an emprcal research were changng n economc and fnancal varables such as changng n nflaton, changng n yeld spreads, etc [5]. The technques that have been deploy to forecast stoc returns are lnear regresson (tme-seres), artfcal neural networs (As), decson trees, rule nductons, Bayesan belef networs, evolutonary algorthms (EAs), classfer systems and assocaton rules [6]. Researches found that As shows better performances than most of technques especally lnear regressons [7]. However, As wth sub-optmal ntal weghts can be trapped n local mnma. In dynamc envronments as the nature of learnng objects are always changng, the topologes of the As also should be adapted accordngly. Evolutonary Algorthms can be appled to evolve As at many levels e.g. connecton weghts, topologes both the number of hdden nodes as well as the number of hdden layers, and learnng rules [8]. Some prevous researches have appled Evolutonary Artfcal eural etwors (EAs) to predct stocs prces [6,7]. In ther research, Kwon et al. [6] amed to predct stoc prce trends (only up or down) by usng EAs that could evolve ther ntal connecton weghts. Comparng buy-and-hold strategy, Recurrent As (RAs)and EAs, they found that ther proposed EAs outperformed Recusrrent As, whle EAs were sgnfcantly outperformed the buy-andhold strategy. The same author n another paper [7], attempted to predct stoc returns

2 based on stoc correlatons (wth other stoc n the same maret). They proposed EAs (they called Feature Selecton Genetc Algorthm FSGA) that could evolve set of nputs (selectons of nputs). By comparng predcton performance (only stoc prce up or down) of buy-and-hold, Recurrent As and EAs (FSGAs), they found that the order of performance was the same.e. EAs then RAs and then buy and hold. A related research by Armano et al. [9] proposed an EA algorthm called XCS, essentally a set of genetc classfers desgned to control feed forward As actvaton for performng forecastng at dfferent partcular local scopes, to predct stoc ndexes (rather than ndvdual stoc prces or returns also up or down only.) The predcton then results from experts nteractons n the populaton. The research found that the proposed methodology repeatedly outperformed buy-and-hold strategy. The most of prevous researches have been concentrated on ether predct stoc ndexes or trends rather than ndvdual stoc returns [1,8]. For stoc tradng, merely predcton on trends of stoc prces or ndexes would be adequate. But for portfolo optmsaton especally mean-varances analyss (Marowtz) portfolo optmsaton model, to construct an effcent portfolo of stocs, a portfolo manager needs to most accurately predct ndvdual stoc returns as well as ther varances[11]. Our research n ths paper proposes an evolutonary scheme of neural networs wth evolvng connecton weghts and step-up addng more hdden nodes and layers n order to search for optmal structure. We use the adaptve EAs to predct ndvdual stoc returns based on multvarate tme seres (AR wth state varables) models. Snce the nput tme seres are qute lmted we also apply Mult-fold Cross Valdaton methods for the sectons of the optmal A structures. The predcton results have been compared wth those of smple regressons (Least Square Estmaton) and of smple (non-evolutonary) As (Bacpropagaton and Elman Recurrent As.) Ths paper s structured as follows: Secton 2 gves a bref revew of the tme seres models for explanaton and predcton of stoc returns. Secton 3 descrbes the As, ts encodng representaton for the evolutonary algorthm and the nonlnear cross-valdaton calculaton as the objectve functon for evolutonary selecton as well as the complete loop of evoluton algorthm. Secton 4 detals for the dataset and experment settng. Where as secton 5 shows the results wth dscusson. Secton 6 provdes for conclusons and suggeston for future wors 2 Tme Seres Models of Stoc Returns Generally, asset returns are the dfference n prces from the begnnng perod (nvestng tme) to (dsnvestng tme) plus dvdends f any. For convenence and by assumng that ether the tme s qute short or dvdend payouts are always reflected n asset prces, we wll dsregard dvdends n the models. We can generalze models of asset return nto 3 categores of models. In the smplest AR Model for tme-varaton expected returns, the expected returns follow auto-regressve (AR) processes. The second category s called the ARMA model. The logarthmc prces of assets have two components, a permanent part and a transtory part. The permanent part follows an AR process. On the other hand, the transtory part follows a movng average (MA) process. The last category s the state varable model. In ths nd of model, the transtory component of prce not only depends on ts own past value but also on state varable (x) whch are relevant fnancal and economc varables. The AR Model of expected returns has consderable capacty to capture the movement of stoc returns over short-horzons but has a medocre capacty to predct the expected stoc return over longer-horzons. On the other hand, the ARMA model does a good job of forecastng long-horzon returns, but has no adequate flexblty to capture the pattern of expected return at short horzons. Whle, the State Varable model s the best n at least four aspects namely, usng only the recent past returns, parsmonous, good predcton n the short-run and good predcton n the longrun[1]. A State Varable model wth K state varables and tme lags can be stated as

3 t K = φ rt + = 1 r γ x + Where, r t,, r t- s stoc returns at the tme perod t and t- respectvely. φ s stoc return s autoregressve coeffcent for tme lag. x t- s the th state varable at prevous th tme perod, γ ι s the regresson coeffcent for the prevous th perod of the th state varable, e τ s the error term. Factors that have evdences of nfluencng stoc returns and nclude n our model are prevous stoc returns (R), unemployment (U), money supply (M), stoc ndex (SP500), nflaton (CPI), default spread (DS), term spread (TS), reference nterest rate (FED), ndustral producton (IP), and January effect (JA-crcumstantal varable)[11]. ote that we exclude some factors that have no monthly data e.g. trade defcts, GDP etc. The model deploy n ths paper can be stated as follow: R(t=0) = R(t=-1 to -12) + U(t=-1 to -12) + M(t=-1 to -12) + SP500(t=-1 to -12) + CPI(t=-1 to -12) + DS(t=-1 to -12) + TS(t=-1 to -12) + FED(t=-1 to -12) + IP(t=-1 to -12) +JA(t=-1 to -12) (2) The model consttutes 1 predctng output and 120 nputs (10 nds wth 12 month lags each.) For January effect (JA), the nput s 1 f the month s January and 0 otherwse. t e t (1) 3 Evolutonary A Desgn 3.1 The A and EA Encodng In ths paper, BPs and Elman RAs are used to predct stoc return for next T perod ahead. The proposed encodngs for the BPs and RAs for evolvng proposes uses drect encodng for connecton weghts and ndrect encodng for the number of hdden nodes and the number of layers. There are two set of evolvng encodng gene namely weght matrces and layer specfcaton. A weght matrx descrbes weghts of connectons for each node (hdden and output nodes) to other hdden nodes n the mmedate prevous layer (may be the nput layer for hdden nodes n the frst hdden layer.) The weght values are n between -1 and 1. In the frst generaton the weghts are randomly set up. And also when the structure of an A s changng.e. addng a hdden node or addng a hdden layer, the weghts are also randomly re-assgned. In evolutonary process, each weght mutates to ts current value plus random number between -0.5 and 0.5. If the mutated values exceed 1 or below -1, the weght wll be set to 1 or -1 respectvely. A layer specfcaton s a vector of nteger descrbng the number of hdden nodes n each layer excludng nput layers thus the length of layer specfcaton s equal to the number of hdden node plus one of output layer. In evolutonary process, after a mutaton on connecton weghts cannot mproves the performance of an A, a hdden node s added nto the frst hdden layer wth all connecton weghts are randomly reassgned. If the ncluson of a hdden node n the frst hdden layer also cannot mprove the performance of the A, a new hdden layer wth 2 hdden nodes s put before the frst hdden layer (then become the fst hdden layer.)

4 Error! Random ntalzaton Connecton Wgt. mutatons As tranng Intal partal tranng Improved? Y Selecton Hdden node mutatons As tranng Step-up mutaton Improved? Y Connecton mutatons Obtan new generaton Stop? RAs tranng Y Further Tranng Fg 1: The Stepwse Mutaton EA Algorthm 3.2 Evolutonary Algorthm The proposed Evolutonary Algorthm used n ths paper s a modfed EP et Algorthm from [2]. The proposed EA s named Step-up Mutaton Evolutonary A. As the name suggests, the algorthm begns wth a mutaton that has least effect on the structure of As then step-up to whch have more effects f the prevous mutaton fals to mprove the performance of the As. On the other hand, f the mutaton can mprove the performance, the mutant wll be selected whle ts parent wll be dscarded (dual tournament wth ts parent) and the loop wll contnue to the next teraton. The man loop s shown n Fgure 1. The Algorthm begns wth random ntalzaton a set of A encodng genetc boxes then creates the correspondng As. All of the As are ntally traned to collect ther prelmnary ftness values. The selected As then goes under the step-up mutaton wth 3 condtonal sub-steps, namely, connecton weght mutatons, hdden node mutatons and connecton mutatons. The mutatons are condtonally step-up n such a way that t wll step at a sub-step f the traned correspondng A s ftness value (n ths case, MCV value as descrbed n the next secton) s mproved. The process repeats untl the pre-specfed round count s met. The algorthm does not have matng operator but based the evoluton solely on the mutaton operators. 3.3 The EA Objectve Multfold Cross-Valdaton (MCV) s a method that maes effcent use of the avalable data. It s a sample re-used method to estmate predcton rs. Our EA objectve s to mnmze predcton rs of the RA. MCV s essentally a perturbaton refnement of Cross-Valdaton (CV) methods. The method can be descrbed as follows: Let the data set D be dvded nto m randomly chosen dsjont subsets D j of roughly equal sze.

5 U m D D, D D for j 1 j = j = φ = For each dsjont set j, CV s defned as CV D j ( λ ) 1 = ( t j ( x, t ) D j ˆ µ ( D λ j, x Where, µ λ (D j, x ) s an estmator traned on all data except (x, t) ε D j, t s the realzed (actual) output, x s the vector of all nputs, j s the number of observaton n subset D j. )) 2 j (3) (4) Cross-Valdaton (CV) for all avalable data set of an A s a non parametrc estmaton of the predcton rs. 1 CV ( λ ) = CV D j ( λ ) (5) m j A refnement s requred for CV to become MCV. An A s traned on the entre set of data D to obtan estmates µ λ (D, x ) wth set of weghts W 0. The weghts W 0 are used as startng pont m-fold cross valdaton producton procedure. Each subset D j s removed from the tranng data n turn. The A s then retraned usng the remanng data (startng at W 0, not random ntal weghts) assumng that deletng a subset from tranng data set does not lead to a sgnfcant dfferent n the locally-optma weghts. These perturbed retranng from W 0 yeld W ( = 1 to m.) The MCV error s calculated for each perturbed model by the sum (t - µ λ (D j, x )) as an estmaton of predcton rs of the model wth W 0 [3.] Tranng Set Valdatng Set Fg 2: Mult-fold Cross Valdaton for Selecton of Optmal A Structure 4 The Experment The forecast As and the experments were conducted wth the Step-up Mutaton EA Algorthm proposed above wth BPs and Elman RAs wthout evolutons as well as forecastng from Lnear Least Square Regresson (LS) n order to compare ther performance. All forecastng have been traned and tested wth monthly dvdend and splt adjusted return seres from 1971 to 2007 on 10 selected stocs n US Stoc marets, namely Alcoa (AA), Boeng (BA), Caterpllar (CAT), Dupont (DD), Dsney (DIS), General Electrc (GE), General Motor (GM), Honeywell (HO), HP (HPQ) and IBM (IBM). The ndependent varables are 12 month tme lag (from lag = -1 to -12) of changng on nflaton (CPI), default yeld spread, term yeld spread, fed fund rate, ndus-

6 tral product, money quantty (M), S&P 500, unemployment rate, January effect (dummy varable) also stoc returns own lags. The sets of data are pared between a dependent varable and a set of tme lags ndependent varables (10 nomnal varables wth 12 lags, n totalty 120 ncludng lags) to form pattern sets (10 stocs n consderaton thus 10 pattern sets.) For tranng and testng the regresson model (LS), BPs and Elman Rs, each pattern sets are brea nto 16 subsets: 8 subsets for tranng and 8 subsets for testng ( for tranng and 2000 for testng, for tranng and 2001 for testng correspondngly to the eghth set for tranng and 2007 for testng). But for tranng and testng the Evolutonary As, each pattern set from s brea nto 5 subsets (12 months for 5 years thus 60 patterns each.) The subsets then form 5 tranng sets correspondng wth a valdatng set for tranng and testng to obtan MCVs value (see Fgure 2). Then the best group of genes, n whch they have mnmum MVCs of the last generaton for each stoc, s deployed to structure As. The same pattern sets are used to tran and test LS, BPs and Elman Rs. The parameters for all BPs, Elman Rs and fnal tranng testng for EAs are as follows: Epoch lmt s 100. Error lmt s 0.0. The learnng constant s 1. The (ntal for EA) archtecture s 1 hdden layer wth 2 nodes. The step-up evoluton algorthm was run 50 generatons on populaton of 10 As (5 BPs and 5 Elman RAs) wth condtonal mutaton probablty s 1 (a mutaton wll affect only there s an mprovement). 5 The Results LS BP Elman EA 0 AA BA CAT DIS DD GE GMHO HP IBM Fg 3: Comaprng Average CV values of Lnear Regresson (LS), Bacpropagaton A (BP) and Elman Recurrent A (Elman) In comparson between the four methods of forecastng, namely Least Square Regresson (LS), Bacpropagaton A, Elman RA and Evolutonary A (EA) for ten stoc returns, we found that BPs have a better performances n all stoc return forecasts than those of LS and Elman RAs. Ths results show that to ncrease complexty of A by ntroducng recurrent networs are not always mprove forecastng performances. For the EAs, they are all but one (namely GM s) at least slghtly better than those of BPs. However, the mprovements are not substantal. Most of EAs have evolved only n ntal weghts of BPs (AA, BA, CAT, DD and HO) only a few have evolved both ntal weghts and structures of BPs (GE, HO and HPQ). Only for GM and IBM, evolved Elman RAs are selected.

7 6 Conclusons and Future Wor As have potentals to mae a better forecastng of fnancal and economcs tme seres. In ths paper, we go a step further to automatcally evolve both ntal and structures (number of hdden nodes and number of hdden layers). Comparng wth the tradtonal Lnear Regressons, the As show promsng results and most of our proposed EAs can mprove the performances of the As as we expected even the mprovements are slghts. There s an ample room for further researches. Frstly, the runnng tme s qute long about 36 hours for each stoc return forecast. Ths causes us, due to lmted computer power, unable to experment wth many populatons and many generatons as we ntally wsh. To run the experment n parallel hgh performance computer may show some more mprovements. The EA algorthm also can be modfed for further mprovements such as ntroducng more varatons of As or selecton of nputs. To apply the EA to other related forecastng problems such as to predct stoc volatltes, exchange rates, etc. s qute a natural step to do. References 1. C. Zhou, Forecastng long- and short- horzon stoc returns n a unfed framewor, Board of Governors of the Federal Reserve System Fnance and Economcs Dscusson Seres FEDS, Paper no. 96-4, Jan W.E. Shape, Captal Assets Prces: A Theory of Maret Equlbrum and Condtons of Rs, The Journal of Fnance, 19(3), 1964, pp J. Lntner., The Valuaton of Rs Assets and the Selecton of Rsy Investments n Stoc Portfolo and Captal Budgets, Revew of Economcs and Statstcs, 47(1), 1965, pp S.A. Ross, The Arbtrage Theory of Captal asset Prcng, The Journal of Economc Theory 13(3), 1967, pp R.R. Roll and S.A. Ross, An Emprcal Investgaton of the Arbtrage Prcng Theory, The Journal of Fnance, 39(5), 1980, pp Y. Kwon, S. Cho and B. Moon, Stoc Predcton Based on Fnancal Correlaton, In Preceedng of GECCO Y. Kwon and B. Moon, A hybrd neurogenetc approach for stoc forecastng, IEEE Transactons on eural etwors, vol. 18, o. 3, pp May X. Yao, Evolutonary artfcal neural networs, Proceedng of the IEEE, vol. 87, o. 9, Sep J. Moody, Forecastng the economy wth neural nets : A survey of challenges and solutons, n eural etwors : Trcs of the Trade, G. B. Orr and K. Muller, Eds., Berln, Germany : Sprnger-Verlag, 1998, pp G. Armano, M. Marches and A. Murru, A hybrd genetc-neural archtecture for stoc ndexes forecastng, Informaton Scences, vol. 170, pp R. E. Oberuc, Dynamc Portfolo Theory and Management, ew Yor, Y : McGraw-Hll, 2004.