To Generate Rule for Software Defect Predication on Quantitative and Qualitative Factors using Artificial Neural Networks

Transcription

1 Proeedings of Interntionl onferene on Intelligent Computtionl Systems To Generte Rule for Softwre Defet Predition on Quntittive nd Qulittive Ftors using Artifiil Neurl Networks Neh Gutm, Prvinder S. Sndhu, Sunil Khullr Abstrt Fult-proneness of softwre module is the probbility tht the module ontins fults. A orreltion exists between the fult-proneness of the softwre nd the mesurble ttributes of the ode (i.e. the stti metris) nd of the testing (i.e. the dynmi metris). Stti ode metris suh s Hlsted omplexity, Cylomti omplexity, MCbe s omplexity mesure re ineffiient to mesure qulity. The use of single fetures of softwre to predit fults is uninformtive. Therefore Artifiil Neurl Network is used for softwre defet predition. An Artifiil Neurl Network (ANN) is n informtion-proessing prdigm tht is inspired by the wy biologil nervous system in humn brin works. Lrge number of neurons present in the humn brin forms the key element of the neurl network prdigm nd t s elementry proessing elements. These neurons re highly interonneted nd work in unison to solve omplex problems. Likewise, n Artifiil Neurl Network n be onfigured to solve number of diffiult nd omplex problems. Neurl Network Approhes suh s Multilyer Pereptron & RBF re used for softwre Defet predition on Quntittive nd Qulittive ftors. Keywords Rdil Bsis Funtion ( RBF), Artifiil neurl Network, Multilyer Pereptron, Cylomti omplexity, M Cbe s omplexity, Hlsted omplexity. F I. INTRODUCTION AULTt-proneness of softwre module is the probbility tht the module ontins fults. A orreltion exists between the fult-proneness of the softwre nd the mesurble ttributes of the ode (i.e. the stti metris) nd of the testing (i.e. the dynmi metris). Erly detetion of fultprone softwre omponents enbles verifition experts to onentrte their time nd resoures on the problem res of the softwre system under development. Softwre qulity models ensure the relibility of the delivered produts. It hs beome importnt to develop nd pply good softwre qulity models erly in the softwre development life yle, espeilly for lrge-sle development efforts. Softwre qulity predition models seek to predit qulity ftors suh s Neh Gutm is Student in Deprtment of Computer Siene, Ryt Institute of Engineering & Informtion Tehnology, Punjb, Indi. Dr. Prvinder S. Singh is Diretor-Prinipl in Ryt & Bhr Institute of Engineering & Biotehnology, Mohli, Indi; (e-mil: prvinder.sndhu@gmil.om). Sunil Khullr is Senior Leturer, Deprtment of Computer Siene, Ryt Institute of Engineering & Informtion Tehnology, Punjb, Indi. whether omponent is fult prone or not. Fults in softwre systems ontinue to be mjor problem. Mny systems re delivered to users with exessive fults. This is despite huge mount of development effort going into fult redution in terms of qulity ontrol nd testing. Despite this it is diffiult to identify relible pproh to identifying fult-prone softwre omponents. So, it is mde ttempt to mke use of Multilyer Pereptron nd RBF bsed Neurl Network pprohes to identify the reltion between the vrious qulittive s well s quntittive ftor of the modules with the number of fults present in the module tht will be helpful for the predition of fults. II. METHODOLOGY The methodology onsists of the following steps: A. Find the Qulittive nd Quntittive ttributes of softwre systems: First of ll, find the struturl ode nd design ttributes of softwre systems. Therefter, selet the suitble metri vlues s representtion of sttement The following re the quntittive metris used [1] Softwre size: the size, in KLoC of the developed ode nd the development lnguge Effort: development effort mesured in person hours for the softwre development, from speifition review to unit test Hlsted omplexity, Cylomti omplexity, MCbe s omplexity The quntittive ftors re grouped under five topis []: Speifition nd Doumenttion proess New Funtionlity Design nd Development proess Testing nd Rework Projet Mngement Eh ftor is nmed nd desribed by question to be nswered. The desriptive questions were speifilly tilored for the orgniztion providing the projet dt. The following re the Speifition nd doumenttion proess ttributes []: 1) Relevnt Experiene of Spe & Do Stff: How would

2 Proeedings of Interntionl onferene on Intelligent Computtionl Systems you rte the experiene nd skill set of your tem members for exeuting this projet during the requirements nd speifitions phse? ) Qulity of Doumenttion inspeted: How would you rte the qulity of the requirements given by the lient or other groups? 3) Regulrity of Spe & Do Reviews: Hve ll the Requirements, Design Douments nd Test Speifitions been reviewed in the projet? 4) Stndrd Proedures Followed: In your opinion, how effetive ws the review proedure? 5) Qulity of Doumenttion inspeted: Wht ws the review effetiveness in the projet for the requirements phse? 6) Spe Defets Disovered in Review: In your opinion, is the defet density of spe reviews on the high side? 7) Requirements Stbility: How stble were the requirements in your projet? The following re the detils of the New funtionlity ttributes []: 1) Complexity of new funtionlity: Wht ws the omplexity of the new development or new fetures tht hppened in your projet? ) Sle of New funtionlity implemented: How lrge ws the extent of working on new funtionlity rther thn just enhning the older funtionlities in your projet? 3) Totl no. of Inputs nd Outputs: For your produt domin, would you rte the totl no of outputs/inputs (newly developed / enhned) s high? The following re the Design nd development proess ttributes []: 1) Relevnt Development Stff Experiene: How would you rte the experiene nd skill set of your tem members for exeuting this projet during the design nd development phse? ) Progrmmer pbility: On n verge, how would you ssess the Qulity of ode produed by the tem members? 3) Defined proesses followed: Wht ws the review effetiveness in the projet for the Design nd Development phse? 4) Development Stff motivtion: Wht is your opinion bout the motivtion levels of your tem members? The following re the Testing nd Rework ttributes []: 1) Testing Proess Well Defined: How effetive ws the testing proess dopted by your projet? ) Stff Experiene Unit Test: Wht ws the level of softwre test ompetene of those performing the unit test? 3) Stff Experiene Independent Test: How would you rte the experiene nd skill set of the independent test engineers (Integrtion, funtionl or subsystem testing, Alph, Bet)? 4) Qulity of Doumented Test Cses: Wht ws the extent of the defets tht were found using forml testing ginst the intuitive/rndom testing? The following re the Projet Mngement ttributes [] : 1) Dev. Stff Trining Qulity: Wht is the overge of the identified projet / proess relted trinings s well s trinings identified s per the roles, by the tem members? ) Configurtion Mngement: How effetive is the projet s doument mngement nd onfigurtion mngement? 3) Projet Plnning: Hs the projet plnning been done dequtely? 4) Sle of Distributed Communition: How mny sites/groups were involved in the projet? 5) Stkeholder involvement: To wht extent were the key projet stkeholders involved? 6) Customer involvement: How good ws ustomer intertion in the projet? 7) Vendor Mngement: How would you rte the Vendor /Sub-ontrtor Mngement (if pplible)? 8) Internl ommunition/ intertion: How would you the rte the qulity of internl intertions / ommunition within the tem? 9) Proess Mturity: Wht s your opinion bout proess mturity in the projet? Qulittive dt re expressed on 5-point ordinl sle. The ordinl vlues used re: Very High, High, Medium, Low, Very Low. The dt vlues were gthered using questionnire, whih ws ompleted by the projet mnger, projet qulity mnger or other senior projet stff. The number of fults present in the modules is lso expressed on 5-point ordinl sle: Very High, High, Medium, Low nd Very Low. B. Selet the suitble metri vlues s representtion of sttement: The suitble metris like produt requirement metris nd produt module metris out of these dt sets re onsidered. [3]The term produt is used referring to module level dt. The term metris dt pplies to ny finite numeri vlues, whih desribe mesured qulities nd hrteristis of produt. The term produt refers to nything to whih defet dt nd metris dt n be ssoited. C. Anlyze, refine metris nd normlize the metri vlues nd Explore different Neurl Network Tehniques It is very importnt to find the suitble lgorithm for modeling of softwre omponents into different levels of fult severity in softwre systems.[10] The following two Neurl Network lgorithms re experimented: Multilyer Pereptron RBF bsed Neurl Network Approhes. III. COMPARISON OF ALGORITHMS The omprisons re mde on the bsis of the more ury nd lest vlue of MAE nd RMSE error vlues.

3 Proeedings of Interntionl onferene on Intelligent Computtionl Systems Aury vlue of the predition model is the mjor riteri used for omprison. The men bsolute error is hosen s the stndrd error. The tehnique hving lower vlue of men bsolute error is hosen s the best fult predition tehnique. A. Men bsolute error Men bsolute error, MAE is the verge of the differene between predited nd tul vlue in ll test ses; it is the verge predition error [13]. The formul for lulting MAE is given in eqution n n (1) n Assuming tht the tul output is, expeted output is. B. Root men-squred error RMSE is frequently used mesure of differenes between vlues predited by model or estimtor nd the vlues tully observed from the thing being modeled or estimted []. It is just the squre root of the men squre error s shown in eqution 8. ( ) + ( ) ( ) 1 1 () n n n The men-squred error is one of the most ommonly used mesures of suess for numeri predition. This vlue is omputed by tking the verge of the squred differenes between eh omputed vlue nd its orresponding orret vlue. [17]The root men-squred error is simply the squre root of the men-squred-error. The root men-squred error gives the error vlue the sme dimensionlity s the tul nd predited vlues [3] The men bsolute error nd root men squred error is lulted for eh mhine lerning lgorithm i.e. vrious lgorithms for Neurl Networks. IV. RESULT The proposed Neurl bsed methodology is implemented in WEKA environment is one suh fility whih lends high performne lnguge for tehnil omputing. The Clssifier tht uses bkpropgtion to lssify instnes. This network is reted by the lgorithm. The network n lso be monitored nd modified during trining time. The nodes in this network re ll sigmoid (exept for when the lss is numeri in whih se the output nodes beome unthresholded liner units). The following prmeters re used for running the multi pereptron bsed progrm: hiddenlyers -- This defines the hidden lyers of the neurl network. This is list of positive whole numbers. 1 for eh hidden lyer. Comm seprted. To hve no hidden lyers put single 0 here. This will only be used if utobuild is set. There re lso wildrd vlues '' = (ttribs + lsses) /, 'i' = ttribs, 'o' = lsses, 't' = ttribs + lsses. We hve set vlue equl to lerningrte -- The mount the weights re updted. The vlue is set to 0.3. momentum -- Momentum pplied to the weights during updting. The vlue is set to 0.. nominltobinryfilter -- This will preproess the instnes with the filter. This ould help improve performne if there re nominl ttributes in the dt. The vlue is set to true. normlizeattributes -- This will normlize the ttributes. This ould help improve performne of the network. This is not relint on the lss being numeri. This will lso normlize nominl ttributes s well (fter they hve been run through the nominl to binry filter if tht is in use) so tht the nominl vlues re between -1 nd 1. The vlue is set to true. normlizenumericlss -- This will normlize the lss if it's numeri. This ould help improve performne of the network, It normlizes the lss to be between -1 nd 1. Note tht this is only internlly, the output will be sled bk to the originl rnge. The vlue is set to true. seed -- Seed used to initilize the rndom number genertor. Rndom numbers re used for setting the initil weights of the onnetions between nodes, nd lso for shuffling the trining dt. The vlue is set to 0 triningtime -- The number of epohs to trin through. If the vlidtion set is non-zero then it n terminte the network erly. The vlue is set to 500. vlidtionsetsize -- The perentge size of the vlidtion set.(the trining will ontinue until it is observed tht the error on the vlidtion set hs been onsistently getting worse, or if the trining time is rehed). The vlue is set to 0. If This is set to zero no vlidtion set will be used nd insted the network will trin for the speified number of epohs. vlidtion Threshold -- Used to terminte vlidtion testing. The vlue here dittes how mny times in row the vlidtion set error n get worse before trining is terminted. The vlue is set to 0. When Multi pereptron bsed neurl network is pplied, the results obtined fter 10-fold ross-vlidtion re: Men bsolute error Root men squred error In se of Rdil bsis funtion network, RBF Network, (Liner regression pplied to k-mens lusters s bsis funtions), the results obtined fter 10-fold ross-vlidtion re: Men bsolute Root men squred error Clss tht implements normlized Gussin rdil bsis bsis funtion network uses the k-mens lustering lgorithm to provide the bsis funtions nd lerns either logisti regression (disrete lss problems) or liner regression (numeri lss problems) on top of tht. Symmetri multivrite Gussins re fit to the dt from eh luster. If

4 Proeedings of Interntionl onferene on Intelligent Computtionl Systems the lss is nominl it uses the given number of lusters per lss. It stndrdizes ll numeri ttributes to zero men nd unit vrine. The following prmeters re used (s shown in figure 1): lustering Seed -- The rndom seed to pss on to k- mens. It is set to vlue 1. mx Its -- Mximum number of itertions for the logisti regression to perform. Only pplied to disrete lss problems. It is set to vlue -1. min Std Dev -- Sets the minimum stndrd devition for the lusters. It is set to vlue 0.1. num Clusters -- The number of lusters for K-Mens to generte. It is set to vlue s we hve only two lsses required. ridge -- Set the Ridge vlue for the logisti or liner regression. It is set to vlue 1.0 E -8. Fig. 1Snpshot of the prmeters used in the RBF Network Progrm In the Liner Regression Model the totl defets TD n be lulted with help of following eqution: TD = * pcluster_0_ * pcluster_0_ V. CONCLUSION Predition of Level of fults in modules supports softwre qulity engineering through improved sheduling nd projet ontrol. It is key step towrds steering the softwre testing nd improving the effetiveness of the whole proess. Fult predition is used to improve softwre proess ontrol nd hieve high softwre relibility. In this study, we investigte whether qulittive nd quntittive ftors n be used to identify level of number of fulty softwre modules. We ompre the performne of Rdil bsis funtion network, where Liner regression pplied to k-mens lusters s bsis funtions nd Multi Pereptron bsed neurl network for the fult dtset. Multi Pereptron bsed neurl network shows best results thn RBF Network with lower vlues of MAE nd RMSE lulted s error nd respetively. It is therefore, onluded the Multi Pereptron bsed neurl network model is implemented nd the best lgorithm for lssifition of the fult prone modules from the fultless modules of the softwre systems. REFERENCES [1] Seliy N., Khoshgoftr T.M. nd Zhong S. (005), Anlyzing softwre qulity with limited fult-proneness defet dt, in proeedings of the Ninth IEEE interntionl Symposium on High Assurne System Engineering, Germny, pp [] Normn Fenton, Mrtin Neil, Willim Mrsh, Peter Herty, Luksz Rdlinski, Pul Kruse, Projet Dt Inorporting Qulittive Ftors for Improved Softwre Defet, Proeedings of the PROMISE workshop, Yer: 007 [3] Jing Y., Cuki B. nd Menzies T. (007), Fult Predition Using Erly Lifeyle Dt. ISSRE 007, the 18th IEEE Symposium on Softwre Relibility Engineering, IEEE Computer Soiety, Sweden, pp [4] Bezdek J.C., Ehrlih R., nd Full W. (1984) FCM: Fuzzy -mens lgorithm. Computers nd Geosiene, Volume: 10, pp [5] Khoshgoftr T.M. nd Munson J.C. (1990), Prediting Softwre Development Errors using Complexity Metris, IEEE Journl on Seleted Ares in Communitions, Volume: 8 Issue:, pp [6] Pigoski M. nd Nelson E. (1994), Softwre Mintenne Metris: A Cse Study, Proeedings of IEEE Conferene on Softwre Mintenne, Cnd, pp [7] Khoshgoftr, T.M., Allen E.B., Ross F.D., Munikoti R., Goel N. nd Nndi A. (1997), Prediting fult-prone modules with se-bsed resoning. ISSRE 1997, the Eighth Interntionl Symposium on Softwre Engineering, Mexio, pp [8] Menzies T., Ammr K., Nikor A., nd Stefno S. (003), How Simple is Softwre Defet Predition? Journl of Empiril Softwre Engineering, Volume: 3, Issue:, pp [9] Emn K., Benlrbi S., Goel N., nd Ri S. (001), Compring sebsed resoning lssifiers for prediting high risk softwre omponents, Journl of Systems Softwre, Volume: 55 Issue: 3, pp [10] Munson J.C. nd Khoshgoftr T.M. (199), The detetion of fultprone progrms, IEEE Trnstions on Softwre Engineering, Volume: 18, Issue: 5, pp [11] Yun X., Khoshgoftr M. nd Allen B. (000), An Applition of Fuzzy Clustering to Softwre Qulity Predition. Informtion Sienes: An Interntionl Journl, Volume: 179, Issue: 8, pp [1] Seliy N. nd Khoshgoftr M. (007) Softwre Qulity Anlysis of Unlbeled Progrm Modules with Semi supervised Clustering. Softwre Qulity Journl, Volume: 37, Issue:, pp [13] Chllgul, Bstni B. nd Yen (006). A Unified frmework for Defet Dt Anlysis using the MBR Tehnique. Proeedings of the 18th IEEE Interntionl Conferene on Tools with Artifiil Intelligene (ICTAI'06), Wshington, pp [14] Dv e N. nd Krishnpurm R. (1997). Robust lustering methods: A unified view. IEEE Trnstions on Fuzzy Systems, Volume: 5, Issue:, pp [15] Sux B. nd Boujem N (00). Unsupervised robust lustering for imge Dtbse tegoriztion. In Proeedings of the IEEE-IAPR Interntionl Conferene on Pttern Reognition (ICPR 00), Turkey, pp [16] Strk E. (1996), Mesurements for Mnging Softwre Mintenne, Interntionl Conferene on Softwre Mintenne, USA, pp [17] Bellini P. (005), Compring Fult-Proneness Estimtion Models, 10th IEEE Interntionl Conferene on Engineering of Complex Computer Systems (ICECCS'05), Chin, pp [18] Lnubile F., Lonigro A., nd Visggio G. (1995) Compring Models for Identifying Fult-Prone Softwre Components, Proeedings of

5 Proeedings of Interntionl onferene on Intelligent Computtionl Systems Seventh Interntionl Conferene on Softwre Engineering nd Knowledge Engineering, USA, pp [19] Fenton N.E. nd Neil M. (1999), A Critique of Softwre Defet Predition Models, IEEE Trnstions on Softwre Engineering, Volume: 5, Issue: 5, pp [0] Runeson, Wohlin C. nd Ohlsson M.C. (001), A Proposl for Comprison of Models for Identifition of Fult-Proneness, Journl of System nd Softwre, Volume: 56, Issue: 3, pp [1] Deodhr M. (00), Predition Model nd the Size Ftor for Fultproneness of Objet Oriented Systems, Journl of System nd Softwre, Volume: 56, Issue: 3, pp [] M Y. nd Guo L. (006), A Sttistil Frmework for the Predition of Fult-Proneness, Produt Foused Proess Improvement, Edition: First, Publisher: Springer Berlin/Heidelberg, pp [3] Chllgull V.U.B., Bstni F.B., Yen I. L. nd Pul (005) Empiril ssessment of mhine lerning bsed softwre defet predition tehniques, 10th IEEE Interntionl Workshop on Objet-Oriented Rel-Time Dependble Systems, USA, pp