Learning Ensembles of Convolutional Neural Networks
|
|
- Horatio Hampton
- 5 years ago
- Views:
Transcription
1 Learnng Ensembles of Convolutonal Neural Networks Lran Chen The Unversty of Chcago Faculty Mentor: Greg Shakhnarovch Toyota Technologcal Insttute at Chcago 1 Introducton Convolutonal Neural Networks (CNN) have demonstrated mpressve performance n mage classfcaton. When fttng complex models wth nonconvex objectves to tran the network, the resultng model depends on stochastc learnng procedure,.e., the fnal network traned wth gradent descent depends on factors such as the order of data n each epoch, ntalzaton, learnng rates, etc. Ensemble learnng s a method for generatng multple versons of a predctor network and usng them to get an aggregated predcton. Gven a learnng set Ω conssts of data {(y n, x n ), n = 1,..., N} where y s the class label and x s the nputng feature, we tran a predctor ϕ( x, Ω). Wth dfferent ntalzaton, we obtan a seres of predctors {ϕ k }. Our object s to use the {ϕ k } to get a better predctor, ϕ A. In the last few years, several papers have shown that ensemble method can delver outstandng performance n reducng the testng error. Most notably, (Krzhevsky et al., 2012) showed that on the ImageNet 2012 classfcaton benchmark, ther ensemble model wth 5 convnets acheved a top-1 error rate of 38.1%, compared to the top -1 error rate of 40.7% gven by the sngle model. In addton, (Zeler & Fergus, 2013) showed that by the ensemble of 6 convnets, they reduced the top -1 error from 40.5% to 36.0%. In 1994, Breman ntroduced the concept of baggng, whch helped us gan some understandng of why the ensemble of classfcaton tree and regresson tree work when they were traned by random samples from the whole dataset (Breman, 1996). However there s stll no clear understandng of 1
2 why the ensemble of CNNs performs so well, what s the relaton between the number of models n ensemble and the amount of error reduced, or other methods of ensemble nstead of averagng predcton. 2 Experments 2.1 The Data Set The MNIST database (Mxed Natonal Insttute of Standards and Technology database) s a large database of handwrtten zp code dgts provded by the U.S. Postal Servce. Ths dataset s commonly used for tranng varous mage processng systems. The database contans 60,000 tranng mages and 10,000 testng mages. The dgts have been sze-normalzed and centered n a fxed-sze mage. 2.2 The Archtecture The archtecture of our network contans three learned layers - two convolutonal layers, H1, H2 and one fully-connected layer, H3. The output of the fully-connected layer s fed to a Softmax layer whch produces a vector of length 10. Each element of the vector represents the probablty that the nput belongs to a certan class (from 0 to 9). We construct our CNN n a smlar way as (LeCun et al., 1989) dd. The frst convolutonal layer, H1, s fed by a normalzed nput mage. Ths layer conssts feature maps, desgnated as H1.1, H1.2,..., H1.12. Each pxel n each feature map n H1 takes nput on a 5 5 receptve feld on the nput plane. In H1, pxels that are next to each other have ther receptve felds two pxels apart. For pxels n a certan feature map, all receptve felds share the same set of 5 5 weghts. However, pxels n another feature map share a dfferent sets of 5 5 weghts. Thus, n total, there are 12 sets of 5 5 weghts to create 12 features maps for H1. Before beng fed to H2, we operates a nonlnear ReLU transformaton to all pxels n all maps. From H1 to H2, a smlar process occurs - convoluton and nonlnear transformaton. H2 conssts feature maps, each contans 12 unts arranged n a 7 7 plane, desgnated as H2.1, H2.2,..., H2.12. For H2.1, pxels n the frst unt take a 5 5 receptve feld from H1 and share a set of 5 5 weghts, pxels n another unt share a dfferent set of 5 5 weghts, and so on. Thus, to obtan H2.1, we need a set of weghts szed
3 In total, H2 s created from 12 sets of weghts. The output of H2 are feature maps, whch are then fed to H3. H3 s a fully-connected layer, whch conssts of 30 unts, each s produced from the dot product of H2 and 30 sets of weghts each szed and nonlnear ReLU transformaton. Before beng fed to the Softmax layer, a bas term s added to H3. Thus, the output of H3 s 30 numercal numbers plus a bas. Wth back propagaton through the whole archtecture, a test error rate of 3% s acheved wth sngle model. Snce we want to nvestgate nto the effect of ensemble, we dumb the CNN on purpose by fxng H1 and H2 and let learnng occurs only n the fully-connected layer and the Softmax layer. 2.3 Tranng on Dfferent Epochs In our experment, we tran 30 CNNs ndependently and each CNN s traned from 1 to 20 epochs, desgnated as ϕ 1.1, ϕ 1.2,..., ϕ 1.20, ϕ 2.1, ϕ 2.2,..., ϕ 2.20,..., ϕ 30.1, ϕ 30.2,..., ϕ The testng error, the vertcal axs of Fgure 1, s plotted aganst the tranng epochs, the horzontal axs. Lower testng error ndcates better performance. The red lne n the mddle s the averaged testng error of all CNN traned under a certan number of epochs. Fgure 1 shows there s no obvous observaton that ncreasng the tranng number of epochs would lead to better performance. Note that wth dfferent structure and dataset, ths observaton may vary. 2.4 Averagng Predctons From the prevous settng, we have traned 30 groups of CNNs desgnated as ϕ 1.1, ϕ 1.2,..., ϕ 1.20, ϕ 2.1, ϕ 2.2,..., ϕ 2.20,..., ϕ 30.1, ϕ 30.2,..., ϕ Snce the output s numercal, an obvous procedure of ensemble learnng s averagng the predcton over the predctors traned under the same number of epochs. Desgnatng ϕ N.e as the ensemble of predctors each traned wth e epochs. In Fgure 2, the testng error s plotted aganst, the number of CNNs averaged. Fgure 2 shows that as ncreases, the testng error reduces, ndcatng that the ensemble va averagng the predcton contrbutes to achevng better performance. However, we notce that as more and more predctors beng averaged, the rate of reducng testng rate goes down and eventually the lnes go flat, thus t s unlkely that by averagng nfnte number of CNNs, testng error can be reduced to zero. 3
4 Fgure 1: Testng Error of CNNs Traned wth Dfferent Epochs Fgure 2: Testng Error of CNNs wth Averaged Predcton 2.5 Fxng the Total Number of Tranng Epochs Now that we are aware that for our archtecture and dataset, ncreasng, the number of models averaged, helps obtan better performance, whle 4
5 ncreasng e, the learnng duraton, does not. Snce both ncreasng and e lead to hgher costs, t s natural to thnk about the tradeoff between and e. If we fx e, then we fx the total cost of tranng. In Fgure 3, testng error s plotted aganst dfferent combnatons of and e. The blue lne shows that when e = 30, the testng error s reduced as ncreases. The red lne s the test error of CNNs wth = 1 and e equals that of the blue lne. Use the red lne as a control group, Fgure 3 shows that the predctors gan more and more accuracy as the number of models nvolve n the ensemble ncreases. Snce the models are traned ndependently, f we spread the tranng onto dfferent machnes, we can effectvely reduce the tranng tme. Fgure 3: Testng Error of the ensemble of CNNs wth Fxed Amount of Tranng Cost 2.6 Creatng New Softmax Layer Here we experment a new way of ensemble nstead of smply averagng the predcton numercally. As dscussed n the prevous secton, the output of a CNN s a vector of length 10. Wth a fxed tranng duraton,.e., e s a constant, we collect the output of 30 ndependently-traned CNNs and stack them as a new feature map. Ths map s then fed to a new Softmax layer whch outputs a vector of length 10. The elements of the new output stll 5
6 represent the probablty of the orgnal nput belongs to a certan class. In Fgure 4, testng error s plotted aganst e ranged from 1 to 20. The blue lne s the plot of testng error aganst epochs under the new way of ensemble, whle the red lne s testng error aganst epochs wthout ensemble. Use the red lne as a control group, Fgure 4 shows that wth ensemble by stackng the outputs from ndependently-traned models and creatng a new Softmax layer, the predctor performs better. Fgure 4: Testng Error of Ensemble by Creatng New Softmax Layer 3 Why Ensemble Works For a sngle network from the dstrbuton P (Net), the expected test error s e = E ϕ E x,y (y ϕ(x)) 2 (1) The aggregate predctor s ϕ A (x) = E ϕ P (Net) (ϕ(x)) (2) 6
7 The expected aggregate error s The emprcal mean of the predctors s The expected error of the mean of the predctor s To prove e e A, we have e A = E x,y (y ϕ A (x)) 2 (3) ϕ = 1 ϕ (x) (4) e = E ϕ E x,y (y 1 ϕ (x)) 2 (5) e = E ϕ E x,y (y ϕ(x)) 2 = E ϕ E x,y (y 2 2yϕ(x) + ϕ 2 (x)) 2 = E x,y E ϕ (y 2 2yϕ(x) + ϕ 2 (x)) 2 = E x,y (y 2 E ϕ (2yϕ(x)) + E ϕ (ϕ 2 (x))) E x,y (y 2 2yϕ A (x) + E 2 ϕ(ϕ(x))) = E x,y (y 2 2yϕ A (x) + ϕ 2 A(x)) = E x,y (y ϕ A (x)) 2 = e A (6) Thus we prove that the expected error from ensemble s always smaller than the expected error from a sngle predctor. So theoretcally, we always gan from ensemble n expectaton. We can decompose (1) by bas and varance as followng: e = E ϕ E x,y (y ϕ(x)) 2 = E ϕ E x,y (y E x,y ϕ A (x) + E x,y ϕ A (x) ϕ(x)) 2 = E ϕ E x,y (y E x,y ϕ A (x)) 2 + 2E ϕ E x,y (y E x,y ϕ A (x))(e x,y ϕ A (x) ϕ(x)) + E ϕ E x,y (ϕ(x) E x,y ϕ A (x)) 2 = E ϕ E x,y (y E x,y ϕ A (x)) 2 + E ϕ E x,y (ϕ(x) E x,y ϕ A (x)) 2 = E x,y (y E x,y ϕ A (x)) 2 + E ϕ E x,y (ϕ(x) E x,y ϕ A (x)) 2 (7) Smlarly, (3) can be decomposed as followng: e A = E x,y (y ϕ A (x)) 2 = E x,y (y E x,y ϕ A (x) + E x,y ϕ A (x) ϕ A (x)) 2 = E x,y (y E x,y ϕ A (x)) 2 + E x,y (ϕ A (x) E x,y ϕ A (x)) 2 (8) 7
8 Thus, e e A = E ϕ E x,y (ϕ(x) E x,y ϕ A (x)) 2 E x,y (ϕ A (x) E x,y ϕ A (x)) 2 = E x,y (E ϕ (ϕ 2 (x)) E 2 ϕ(ϕ(x))) = E x,y (V ar ϕ (ϕ(x))) 0 (9) However, n our experment, we can only observe the emprcal mean,(4), nstead of the expectaton,(2). To obtan the ensemble learnng effect, we decompose e = E ϕ E x,y (y 1 f (x)) 2 = E ϕ E x,y (y E x,y ϕ A (x) + E x,y ϕ A (x) 1 f (x)) 2 = E ϕ E x,y (y E x,y ϕ A (x)) 2 + 2E ϕ E x,y (y E x,y ϕ A (x))(e x,y ϕ A (x) 1 f (x)) + E ϕ E x,y ( 1 f (x) E x,y ϕ A (x)) 2 = E x,y (y E x,y ϕ A (x)) 2 + E ϕ E x,y ( 1 f (x) E x,y ϕ A (x)) 2 (10) In (10), the second term can be decomposed as E ϕ E x,y ( 1 f (x) E x,y ϕ A (x)) 2 = E x,y E ϕ (Ex,yϕ 2 A (x) 2E x,y ϕ A (x) 1 f (x) + ( 1 f (x)) 2 ) = E x,y E ϕ (Ex,yϕ 2 A (x) 2E x,y ϕ A (x)ϕ A (x) + ( 1 f (x)) 2 ) (11) E x,y E ϕ ( 1 f (x)) 2 = E x,y E ϕ ( 1 2 ( f 2 (x) +,j ϕ (x)ϕ j (j))) = 1 2 (E ϕ(ϕ 2 (x)) +,j E ϕ (ϕ (x)ϕ j (j))) (12) 8
9 Thus, e e = E x,y (E ϕ (ϕ 2 (x)) E ϕ (ϕ 2 (x)) +,j E ϕ (ϕ (x)ϕ j (j)) = E x,y ( 1 (E ϕ(ϕ 2 (x)) Eϕ(ϕ(x)))) 2 = 1 E x,y(v ar ϕ (ϕ(x))) 0 (13) From (13) we know that the more predctors nvolve n ensemble, the less of the error. However, as goes to nfnty, the margnal amount of error we reduce decrease to 0. Ths agrees wth the behavor of the error we observed from Fgure 2. 4 Concluson Ensemble s a powerful procedure whch mproves sngle network performance. It reduces the varance porton n the bas-varance decomposton of the predcton error. Our project has expermented wth dfferent ensemble methods that all tend to contrbute to dramatc error reducton. In addton, the tradeoff between number of models and ther complexty has been nvestgated and we show that ensemble learnng may lead to accuracy gans along wth reducton n tranng tme. 5 References Breman, L., Baggng Predctors, Machne Learnng, 24(2): , 1996 Krzhevsky, A., Sutskever, I., and Hnton, G.E. Imagenet classfcaton wth deep convolutonal neural networks. In NIPS, LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., and Jackel, L. D. Backpropagaton appled to handwrtten zp code recognton. Neural Comput., 1(4): , Zeler, M., Fergus, R., Vsualzng and Understandng Convolutonal Networks, ECCV 2014, Arxv , ) 9
Fast Code Detection Using High Speed Time Delay Neural Networks
Fast Code Detecton Usng Hgh Speed Tme Delay Neural Networks Hazem M. El-Bakry 1 and Nkos Mastoraks 1 Faculty of Computer Scence & Informaton Systems, Mansoura Unversty, Egypt helbakry0@yahoo.com Department
More informationarxiv: v1 [cs.lg] 8 Jul 2016
Overcomng Challenges n Fxed Pont Tranng of Deep Convolutonal Networks arxv:1607.02241v1 [cs.lg] 8 Jul 2016 Darryl D. Ln Qualcomm Research, San Dego, CA 92121 USA Sachn S. Talath Qualcomm Research, San
More informationTo: Professor Avitabile Date: February 4, 2003 From: Mechanical Student Subject: Experiment #1 Numerical Methods Using Excel
To: Professor Avtable Date: February 4, 3 From: Mechancal Student Subject:.3 Experment # Numercal Methods Usng Excel Introducton Mcrosoft Excel s a spreadsheet program that can be used for data analyss,
More informationPRACTICAL, COMPUTATION EFFICIENT HIGH-ORDER NEURAL NETWORK FOR ROTATION AND SHIFT INVARIANT PATTERN RECOGNITION. Evgeny Artyomov and Orly Yadid-Pecht
68 Internatonal Journal "Informaton Theores & Applcatons" Vol.11 PRACTICAL, COMPUTATION EFFICIENT HIGH-ORDER NEURAL NETWORK FOR ROTATION AND SHIFT INVARIANT PATTERN RECOGNITION Evgeny Artyomov and Orly
More informationSide-Match Vector Quantizers Using Neural Network Based Variance Predictor for Image Coding
Sde-Match Vector Quantzers Usng Neural Network Based Varance Predctor for Image Codng Shuangteng Zhang Department of Computer Scence Eastern Kentucky Unversty Rchmond, KY 40475, U.S.A. shuangteng.zhang@eku.edu
More informationResearch of Dispatching Method in Elevator Group Control System Based on Fuzzy Neural Network. Yufeng Dai a, Yun Du b
2nd Internatonal Conference on Computer Engneerng, Informaton Scence & Applcaton Technology (ICCIA 207) Research of Dspatchng Method n Elevator Group Control System Based on Fuzzy Neural Network Yufeng
More informationANNUAL OF NAVIGATION 11/2006
ANNUAL OF NAVIGATION 11/2006 TOMASZ PRACZYK Naval Unversty of Gdyna A FEEDFORWARD LINEAR NEURAL NETWORK WITH HEBBA SELFORGANIZATION IN RADAR IMAGE COMPRESSION ABSTRACT The artcle presents the applcaton
More informationA Preliminary Study on Targets Association Algorithm of Radar and AIS Using BP Neural Network
Avalable onlne at www.scencedrect.com Proceda Engneerng 5 (2 44 445 A Prelmnary Study on Targets Assocaton Algorthm of Radar and AIS Usng BP Neural Networ Hu Xaoru a, Ln Changchuan a a Navgaton Insttute
More informationA Comparison of Two Equivalent Real Formulations for Complex-Valued Linear Systems Part 2: Results
AMERICAN JOURNAL OF UNDERGRADUATE RESEARCH VOL. 1 NO. () A Comparson of Two Equvalent Real Formulatons for Complex-Valued Lnear Systems Part : Results Abnta Munankarmy and Mchael A. Heroux Department of
More informationHigh Speed, Low Power And Area Efficient Carry-Select Adder
Internatonal Journal of Scence, Engneerng and Technology Research (IJSETR), Volume 5, Issue 3, March 2016 Hgh Speed, Low Power And Area Effcent Carry-Select Adder Nelant Harsh M.tech.VLSI Desgn Electroncs
More informationDynamic Optimization. Assignment 1. Sasanka Nagavalli January 29, 2013 Robotics Institute Carnegie Mellon University
Dynamc Optmzaton Assgnment 1 Sasanka Nagavall snagaval@andrew.cmu.edu 16-745 January 29, 213 Robotcs Insttute Carnege Mellon Unversty Table of Contents 1. Problem and Approach... 1 2. Optmzaton wthout
More informationControl Chart. Control Chart - history. Process in control. Developed in 1920 s. By Dr. Walter A. Shewhart
Control Chart - hstory Control Chart Developed n 920 s By Dr. Walter A. Shewhart 2 Process n control A phenomenon s sad to be controlled when, through the use of past experence, we can predct, at least
More informationarxiv: v1 [cs.lg] 22 Jan 2016 Abstract
Mne Km MINJE@ILLINOIS.EDU Department of Computer Scence, Unversty of Illnos at Urbana-Champagn, Urbana, IL 61801 USA Pars Smaragds Unversty of Illnos at Urbana-Champagn, Urbana, IL 61801 USA Adobe Research,
More informationWavelet Multi-Layer Perceptron Neural Network for Time-Series Prediction
Wavelet Mult-Layer Perceptron Neural Network for Tme-Seres Predcton Kok Keong Teo, Lpo Wang* and Zhpng Ln School of Electrcal and Electronc Engneerng Nanyang Technologcal Unversty Block S2, Nanyang Avenue
More informationDevelopment of Neural Networks for Noise Reduction
The Internatonal Arab Journal of Informaton Technology, Vol. 7, No. 3, July 00 89 Development of Neural Networks for Nose Reducton Lubna Badr Faculty of Engneerng, Phladelpha Unversty, Jordan Abstract:
More informationA High-Sensitivity Oversampling Digital Signal Detection Technique for CMOS Image Sensors Using Non-destructive Intermediate High-Speed Readout Mode
A Hgh-Senstvty Oversamplng Dgtal Sgnal Detecton Technque for CMOS Image Sensors Usng Non-destructve Intermedate Hgh-Speed Readout Mode Shoj Kawahto*, Nobuhro Kawa** and Yoshak Tadokoro** *Research Insttute
More informationResearch Article Indoor Localisation Based on GSM Signals: Multistorey Building Study
Moble Informaton Systems Volume 26, Artcle ID 279576, 7 pages http://dx.do.org/.55/26/279576 Research Artcle Indoor Localsaton Based on GSM Sgnals: Multstorey Buldng Study RafaB Górak, Marcn Luckner, MchaB
More informationDigital Transmission
Dgtal Transmsson Most modern communcaton systems are dgtal, meanng that the transmtted normaton sgnal carres bts and symbols rather than an analog sgnal. The eect o C/N rato ncrease or decrease on dgtal
More informationUncertainty in measurements of power and energy on power networks
Uncertanty n measurements of power and energy on power networks E. Manov, N. Kolev Department of Measurement and Instrumentaton, Techncal Unversty Sofa, bul. Klment Ohrdsk No8, bl., 000 Sofa, Bulgara Tel./fax:
More informationRejection of PSK Interference in DS-SS/PSK System Using Adaptive Transversal Filter with Conditional Response Recalculation
SERBIAN JOURNAL OF ELECTRICAL ENGINEERING Vol., No., November 23, 3-9 Rejecton of PSK Interference n DS-SS/PSK System Usng Adaptve Transversal Flter wth Condtonal Response Recalculaton Zorca Nkolć, Bojan
More informationRC Filters TEP Related Topics Principle Equipment
RC Flters TEP Related Topcs Hgh-pass, low-pass, Wen-Robnson brdge, parallel-t flters, dfferentatng network, ntegratng network, step response, square wave, transfer functon. Prncple Resstor-Capactor (RC)
More informationPOLYTECHNIC UNIVERSITY Electrical Engineering Department. EE SOPHOMORE LABORATORY Experiment 1 Laboratory Energy Sources
POLYTECHNIC UNIERSITY Electrcal Engneerng Department EE SOPHOMORE LABORATORY Experment 1 Laboratory Energy Sources Modfed for Physcs 18, Brooklyn College I. Oerew of the Experment Ths experment has three
More informationNATIONAL RADIO ASTRONOMY OBSERVATORY Green Bank, West Virginia SPECTRAL PROCESSOR MEMO NO. 25. MEMORANDUM February 13, 1985
NATONAL RADO ASTRONOMY OBSERVATORY Green Bank, West Vrgna SPECTRAL PROCESSOR MEMO NO. 25 MEMORANDUM February 13, 1985 To: Spectral Processor Group From: R. Fsher Subj: Some Experments wth an nteger FFT
More informationCalculation of the received voltage due to the radiation from multiple co-frequency sources
Rec. ITU-R SM.1271-0 1 RECOMMENDATION ITU-R SM.1271-0 * EFFICIENT SPECTRUM UTILIZATION USING PROBABILISTIC METHODS Rec. ITU-R SM.1271 (1997) The ITU Radocommuncaton Assembly, consderng a) that communcatons
More informationEvaluate the Effective of Annular Aperture on the OTF for Fractal Optical Modulator
Global Advanced Research Journal of Management and Busness Studes (ISSN: 2315-5086) Vol. 4(3) pp. 082-086, March, 2015 Avalable onlne http://garj.org/garjmbs/ndex.htm Copyrght 2015 Global Advanced Research
More informationEnsemble Evolution of Checkers Players with Knowledge of Opening, Middle and Endgame
Ensemble Evoluton of Checkers Players wth Knowledge of Openng, Mddle and Endgame Kyung-Joong Km and Sung-Bae Cho Department of Computer Scence, Yonse Unversty 134 Shnchon-dong, Sudaemoon-ku, Seoul 120-749
More informationPhoneme Probability Estimation with Dynamic Sparsely Connected Artificial Neural Networks
The Free Speech Journal, Issue # 5(1997) Publshed 10/22/97 1997 All rghts reserved. Phoneme Probablty Estmaton wth Dynamc Sparsely Connected Artfcal Neural Networks Nkko Ström, (nkko@speech.kth.se) Department
More informationHigh Speed ADC Sampling Transients
Hgh Speed ADC Samplng Transents Doug Stuetzle Hgh speed analog to dgtal converters (ADCs) are, at the analog sgnal nterface, track and hold devces. As such, they nclude samplng capactors and samplng swtches.
More informationROBUST IDENTIFICATION AND PREDICTION USING WILCOXON NORM AND PARTICLE SWARM OPTIMIZATION
7th European Sgnal Processng Conference (EUSIPCO 9 Glasgow, Scotland, August 4-8, 9 ROBUST IDENTIFICATION AND PREDICTION USING WILCOXON NORM AND PARTICLE SWARM OPTIMIZATION Babta Majh, G. Panda and B.
More informationantenna antenna (4.139)
.6.6 The Lmts of Usable Input Levels for LNAs The sgnal voltage level delvered to the nput of an LNA from the antenna may vary n a very wde nterval, from very weak sgnals comparable to the nose level,
More informationTh P5 13 Elastic Envelope Inversion SUMMARY. J.R. Luo* (Xi'an Jiaotong University), R.S. Wu (UC Santa Cruz) & J.H. Gao (Xi'an Jiaotong University)
-4 June 5 IFEMA Madrd h P5 3 Elastc Envelope Inverson J.R. Luo* (X'an Jaotong Unversty), R.S. Wu (UC Santa Cruz) & J.H. Gao (X'an Jaotong Unversty) SUMMARY We developed the elastc envelope nverson method.
More informationComparative Analysis of Reuse 1 and 3 in Cellular Network Based On SIR Distribution and Rate
Comparatve Analyss of Reuse and 3 n ular Network Based On IR Dstrbuton and Rate Chandra Thapa M.Tech. II, DEC V College of Engneerng & Technology R.V.. Nagar, Chttoor-5727, A.P. Inda Emal: chandra2thapa@gmal.com
More informationNetworks. Backpropagation. Backpropagation. Introduction to. Backpropagation Network training. Backpropagation Learning Details 1.04.
Networs Introducton to - In 1986 a method for learnng n mult-layer wor,, was nvented by Rumelhart Paper Why are what and where processed by separate cortcal vsual systems? - The algorthm s a sensble approach
More informationaperture David Makovoz, 30/01/2006 Version 1.0 Table of Contents
aperture 1 aperture Davd Makovoz, 30/01/2006 Verson 1.0 Table of Contents aperture... 1 1 Overvew... 2 1.1 Input Image Requrements... 2 2 aperture... 2 2.1 Input... 2 2.2 Processng... 4 2.3 Output Table...
More informationElectricity Price Forecasting using Asymmetric Fuzzy Neural Network Systems Alshejari, A. and Kodogiannis, Vassilis
WestmnsterResearch http://www.westmnster.ac.uk/westmnsterresearch Electrcty Prce Forecastng usng Asymmetrc Fuzzy Neural Network Systems Alshejar, A. and Kodoganns, Vassls Ths s a copy of the author s accepted
More informationParameter Free Iterative Decoding Metrics for Non-Coherent Orthogonal Modulation
1 Parameter Free Iteratve Decodng Metrcs for Non-Coherent Orthogonal Modulaton Albert Gullén Fàbregas and Alex Grant Abstract We study decoder metrcs suted for teratve decodng of non-coherently detected
More informationEquity trend prediction with neural networks
Res. Lett. Inf. Math. Sc., 2004, Vol. 6, pp 15-29 15 Avalable onlne at http://ms.massey.ac.nz/research/letters/ Equty trend predcton wth neural networks R.HALLIDAY Insttute of Informaton & Mathematcal
More informationSTRUCTURE ANALYSIS OF NEURAL NETWORKS
STRUCTURE ANALYSIS OF NEURAL NETWORKS DING SHENQIANG NATIONAL UNIVERSITY OF SINGAPORE 004 STRUCTURE ANALYSIS OF NEURAL NETWORKS DING SHENQIANG 004 STRUCTURE ANANLYSIS OF NEURAL NETWORKS DING SHENQIANG
More informationOptimization Frequency Design of Eddy Current Testing
Optmzaton Frequency Desgn of Eddy Current Testng NAONG MUNGKUNG 1, KOMKIT CHOMSUWAN 1, NAONG PIMPU 2 AND TOSHIFUMI YUJI 3 1 Department of Electrcal Technology Educaton Kng Mongkut s Unversty of Technology
More informationarxiv: v1 [cs.ne] 2 Nov 2016
Deep counter networks for asynchronous event-based processng arxv:1611.71v1 [cs.ne] 2 Nov 216 Jonathan Bnas, Gacomo Indver, and Mchael Pfeffer Insttute of Neuronformatcs Unversty of Zurch and ETH Zurch,
More informationPerformance Analysis of Multi User MIMO System with Block-Diagonalization Precoding Scheme
Performance Analyss of Mult User MIMO System wth Block-Dagonalzaton Precodng Scheme Yoon Hyun m and Jn Young m, wanwoon Unversty, Department of Electroncs Convergence Engneerng, Wolgye-Dong, Nowon-Gu,
More informationGeneralized Incomplete Trojan-Type Designs with Unequal Cell Sizes
Internatonal Journal of Theoretcal & Appled Scences 6(1): 50-54(2014) ISSN No. (Prnt): 0975-1718 ISSN No. (Onlne): 2249-3247 Generalzed Incomplete Trojan-Type Desgns wth Unequal Cell Szes Cn Varghese,
More informationNEURAL PROCESSIN G.SYSTEMS 2 INF ORM.ATIO N (Q90. ( Iq~O) DAVID S. TOURETZKY ADVANCES CARNEGIE MELLON UNIVERSITY. ..F~ k \ """ Ct... V\.
....F~ k \ """ Ct... V\. ~.Le.- b;e ve-. ( Iq~O) ADVANCES IN NEURAL INF ORM.ATIO N PROCESSIN G.SYSTEMS 2 EDITED BY DAVID S. TOURETZKY CARNEGIE MELLON UNIVERSITY (Q90.MORGAN KAUFMANN PUBLISHERS 2929 CAMPUS
More informationArtificial Intelligence Techniques Applications for Power Disturbances Classification
Internatonal Journal of Electrcal and Computer Engneerng 3:5 28 Artfcal Intellgence Technques Applcatons for Power Dsturbances Classfcaton K.Manmala, Dr.K.Selv and R.Ahla Abstract Artfcal Intellgence (AI)
More informationChess players fame versus their merit
Ths artcle s ublshed n Aled Economcs Letters htt://www.tandfonlne.com/do/full/0.080/350485.05.0435 Chess layers fame versus ther mert M.V. Smkn and V.P. Roychowdhury Deartment of Electrcal Engneerng, Unversty
More informationproblems palette of David Rock and Mary K. Porter 6. A local musician comes to your school to give a performance
palette of problems Davd Rock and Mary K. Porter 1. If n represents an nteger, whch of the followng expressons yelds the greatest value? n,, n, n, n n. A 60-watt lghtbulb s used for 95 hours before t burns
More informationAdaptive System Control with PID Neural Networks
Adaptve System Control wth PID Neural Networs F. Shahra a, M.A. Fanae b, A.R. Aromandzadeh a a Department of Chemcal Engneerng, Unversty of Sstan and Baluchestan, Zahedan, Iran. b Department of Chemcal
More informationTHE INCREDIBLE SHRINKING NEURAL NETWORK: NEW PERSPECTIVES ON LEARNING REPRESENTA-
Under revew as a conference paper at ICLR 217 THE INCREDIBLE SHRINKING NEURAL NETWORK: NEW PERSPECTIVES ON LEARNING REPRESENTA- TIONS THROUGH THE LENS OF PRUNING Nkolas Wolfe, Adtya Sharma & Bhksha Ra
More informationA study of turbo codes for multilevel modulations in Gaussian and mobile channels
A study of turbo codes for multlevel modulatons n Gaussan and moble channels Lamne Sylla and Paul Forter (sylla, forter)@gel.ulaval.ca Department of Electrcal and Computer Engneerng Laval Unversty, Ste-Foy,
More informationAdaptive Modulation for Multiple Antenna Channels
Adaptve Modulaton for Multple Antenna Channels June Chul Roh and Bhaskar D. Rao Department of Electrcal and Computer Engneerng Unversty of Calforna, San Dego La Jolla, CA 993-7 E-mal: jroh@ece.ucsd.edu,
More informationA NSGA-II algorithm to solve a bi-objective optimization of the redundancy allocation problem for series-parallel systems
0 nd Internatonal Conference on Industral Technology and Management (ICITM 0) IPCSIT vol. 49 (0) (0) IACSIT Press, Sngapore DOI: 0.776/IPCSIT.0.V49.8 A NSGA-II algorthm to solve a b-obectve optmzaton of
More informationA MODIFIED DIFFERENTIAL EVOLUTION ALGORITHM IN SPARSE LINEAR ANTENNA ARRAY SYNTHESIS
A MODIFIED DIFFERENTIAL EVOLUTION ALORITHM IN SPARSE LINEAR ANTENNA ARRAY SYNTHESIS Kaml Dmller Department of Electrcal-Electroncs Engneerng rne Amercan Unversty North Cyprus, Mersn TURKEY kdmller@gau.edu.tr
More informationEstimation of Solar Radiations Incident on a Photovoltaic Solar Module using Neural Networks
XXVI. ASR '2001 Semnar, Instruments and Control, Ostrava, Aprl 26-27, 2001 Paper 14 Estmaton of Solar Radatons Incdent on a Photovoltac Solar Module usng Neural Networks ELMINIR, K. Hamdy 1, ALAM JAN,
More informationCS345a: Data Mining Jure Leskovec and Anand Rajaraman Stanford University
CS345a: Data Mnng Jure Leskovec and Anand Rajaraman Stanford Unversty HW3 s out Poster sesson s on last day of classes: Thu March 11 at 4:15 Reports are due March 14 Fnal s March 18 at 12:15 Open book,
More informationDevelopment of an UWB Rescue Radar System - Detection of Survivors Using Fuzzy Reasoning -
Development of an UWB Rescue Radar System - Detecton of Survvors Usng Fuzzy Reasonng - Iwak Akyama Shonan Insttute of Technology Fujsawa 251-8511 Japan akyama@wak.org Masatosh Enokto Shonan Insttute of
More informationMulti-objective Genetic Algorithm Based Selective Neural Networks Ensemble for Concentration Estimation of Indoor Air Pollutants Using Electronic Nose
IJCSI Internatonal Journal of Computer Scence Issues, Vol. 10, Issue 3, No, May 013 ISSN (Prnt): 1694-0814 ISSN (Onlne): 1694-0784 www.ijcsi.org 105 Mult-objectve Genetc Algorthm Based Selectve Neural
More informationFEATURE SELECTION FOR SMALL-SIGNAL STABILITY ASSESSMENT
FEAURE SELECION FOR SMALL-SIGNAL SABILIY ASSESSMEN S.P. eeuwsen Unversty of Dusburg teeuwsen@un-dusburg.de Abstract INRODUCION hs paper ntroduces dfferent feature selecton technques for neural network
More informationTHE GENERATION OF 400 MW RF PULSES AT X-BAND USING RESONANT DELAY LINES *
SLAC PUB 874 3/1999 THE GENERATION OF 4 MW RF PULSES AT X-BAND USING RESONANT DELAY LINES * Sam G. Tantaw, Arnold E. Vleks, and Rod J. Loewen Stanford Lnear Accelerator Center, Stanford Unversty P.O. Box
More informationTarget Response Adaptation for Correlation Filter Tracking
Target Response Adaptaton for Correlaton Flter Tracng Adel Bb, Matthas Mueller, and Bernard Ghanem Image and Vdeo Understandng Laboratory IVUL, Kng Abdullah Unversty of Scence and Technology KAUST, Saud
More informationEstimating Mean Time to Failure in Digital Systems Using Manufacturing Defective Part Level
Estmatng Mean Tme to Falure n Dgtal Systems Usng Manufacturng Defectve Part Level Jennfer Dworak, Davd Dorsey, Amy Wang, and M. Ray Mercer Texas A&M Unversty IBM Techncal Contact: Matthew W. Mehalc, PowerPC
More informationGraph Method for Solving Switched Capacitors Circuits
Recent Advances n rcuts, ystems, gnal and Telecommuncatons Graph Method for olvng wtched apactors rcuts BHUMIL BRTNÍ Department of lectroncs and Informatcs ollege of Polytechncs Jhlava Tolstého 6, 586
More informationApplication of Linear Discriminant Analysis to Doppler Classification
Applcaton of Lnear Dscrmnant Analyss to Doppler Classfcaton M. Jahangr QnetQ St Andrews Road, Malvern WORCS, UK, WR14 3PS Unted Kngdom mjahangr@qnetq.com ABSTRACT In ths wor the author demonstrated a robust
More informationJoint Power Control and Scheduling for Two-Cell Energy Efficient Broadcasting with Network Coding
Communcatons and Network, 2013, 5, 312-318 http://dx.do.org/10.4236/cn.2013.53b2058 Publshed Onlne September 2013 (http://www.scrp.org/journal/cn) Jont Power Control and Schedulng for Two-Cell Energy Effcent
More informationAn Improved Method in Transient Stability Assessment of a Power System Using Committee Neural Networks
IJCSNS Internatonal Journal of Computer Scence and Network Securty, VOL.9 No., January 9 9 An Improved Method n Transent Stablty Assessment of a Power System Usng Commttee Neural Networks Reza Ebrahmpour
More informationNOVEL ITERATIVE TECHNIQUES FOR RADAR TARGET DISCRIMINATION
NOVEL ITERATIVE TECHNIQUES FOR RADAR TARGET DISCRIMINATION Phaneendra R.Venkata, Nathan A. Goodman Department of Electrcal and Computer Engneerng, Unversty of Arzona, 30 E. Speedway Blvd, Tucson, Arzona
More informationMulti-Level Halftoning by IGS Quantization
Journal of Sgnal and Informaton Processng, 2013, 4, 351-358 Publshed Onlne ovember 2013 (http://www.scrp.org/journal/jsp) http://dx.do.org/10.4236/jsp.2013.44044 351 Mult-Level Halftonng by IGS Quantzaton
More informationDIMENSIONAL SYNTHESIS FOR WIDE-BAND BAND- PASS FILTERS WITH QUARTER-WAVELENGTH RES- ONATORS
Progress In Electromagnetcs Research B, Vol. 17, 213 231, 29 DIMENSIONAL SYNTHESIS FOR WIDE-BAND BAND- PASS FILTERS WITH QUARTER-WAVELENGTH RES- ONATORS Q. Zhang and Y. Lu School of Electrcal and Electroncs
More informationEfficient Large Integers Arithmetic by Adopting Squaring and Complement Recoding Techniques
The th Worshop on Combnatoral Mathematcs and Computaton Theory Effcent Large Integers Arthmetc by Adoptng Squarng and Complement Recodng Technques Cha-Long Wu*, Der-Chyuan Lou, and Te-Jen Chang *Department
More informationPassive Filters. References: Barbow (pp ), Hayes & Horowitz (pp 32-60), Rizzoni (Chap. 6)
Passve Flters eferences: Barbow (pp 6575), Hayes & Horowtz (pp 360), zzon (Chap. 6) Frequencyselectve or flter crcuts pass to the output only those nput sgnals that are n a desred range of frequences (called
More informationImage Compression Using Cascaded Neural Networks
Unversty of New Orleans ScholarWorks@UNO Unversty of New Orleans Theses and Dssertatons Dssertatons and Theses 8-7-2003 Image Compresson Usng Cascaded Neural Networks Chgoze Obegbu Unversty of New Orleans
More informationModelling Service Time Distribution in Cellular Networks Using Phase-Type Service Distributions
Modellng Servce Tme Dstrbuton n Cellular Networks Usng Phase-Type Servce Dstrbutons runa Jayasurya, Davd Green, John senstorfer Insttute for Telecommuncaton Research, Cooperatve Research Centre for Satellte
More informationTECHNICAL NOTE TERMINATION FOR POINT- TO-POINT SYSTEMS TN TERMINATON FOR POINT-TO-POINT SYSTEMS. Zo = L C. ω - angular frequency = 2πf
TECHNICAL NOTE TERMINATION FOR POINT- TO-POINT SYSTEMS INTRODUCTION Because dgtal sgnal rates n computng systems are ncreasng at an astonshng rate, sgnal ntegrty ssues have become far more mportant to
More informationEnhancement of Degraded Image Based on Neural Network
Engneerng scence Enhancement of Degraded Image Based on Neural Network Defa Hu School of Computer and Informaton Engneerng, Hunan Unversty of Commerce, Changsha 410205, Hunan, Chna Zhuang Wu* Informaton
More informationReview: Our Approach 2. CSC310 Information Theory
CSC30 Informaton Theory Sam Rowes Lecture 3: Provng the Kraft-McMllan Inequaltes September 8, 6 Revew: Our Approach The study of both compresson and transmsson requres that we abstract data and messages
More informationN( E) ( ) That is, if the outcomes in sample space S are equally likely, then ( )
Stat 400, secton 2.2 Axoms, Interpretatons and Propertes of Probablty notes by Tm Plachowsk In secton 2., we constructed sample spaces by askng, What could happen? Now, n secton 2.2, we begn askng and
More informationRobot Docking Based on Omnidirectional Vision and Reinforcement Learning
Robot Dockng Based on Omndrectonal Vson and Renforcement Learnng Davd Muse, Cornelus Weber and Stefan Wermter Hybrd Intellgent Systems, School of Computng and Technology Unversty of Sunderland, UK. Web:
More informationCURL: Co-trained Unsupervised Representation Learning for Image Classification
CURL: Co-traned Unsupervsed Representaton Learnng for Image Classfcaton Smone Banco, Ganlug Cocca, and Claudo Cusano arxv:0.08098v [cs.lg] Sep 0 Abstract In ths paper we propose a strategy for semsupervsed
More informationTraffic balancing over licensed and unlicensed bands in heterogeneous networks
Correspondence letter Traffc balancng over lcensed and unlcensed bands n heterogeneous networks LI Zhen, CUI Qme, CUI Zhyan, ZHENG We Natonal Engneerng Laboratory for Moble Network Securty, Bejng Unversty
More informationSiamese Multi-layer Perceptrons for Dimensionality Reduction and Face Identification
Samese Mult-layer Perceptrons for Dmensonalty Reducton and Face Identfcaton Lle Zheng, Stefan Duffner, Khald Idrss, Chrstophe Garca, Atlla Baskurt To cte ths verson: Lle Zheng, Stefan Duffner, Khald Idrss,
More informationArterial Travel Time Estimation Based On Vehicle Re-Identification Using Magnetic Sensors: Performance Analysis
Arteral Travel Tme Estmaton Based On Vehcle Re-Identfcaton Usng Magnetc Sensors: Performance Analyss Rene O. Sanchez, Chrstopher Flores, Roberto Horowtz, Ram Raagopal and Pravn Varaya Department of Mechancal
More informationMedium Term Load Forecasting for Jordan Electric Power System Using Particle Swarm Optimization Algorithm Based on Least Square Regression Methods
Journal of Power and Energy Engneerng, 2017, 5, 75-96 http://www.scrp.org/journal/jpee ISSN Onlne: 2327-5901 ISSN Prnt: 2327-588X Medum Term Load Forecastng for Jordan Electrc Power System Usng Partcle
More informationNovel Artificial Neural Networks For Remote-Sensing Data Classification
ovel Artfcal eural etwors For Remote-Sensng Data Classfcaton Xaol Tao * and Howard E. chel ξ Unversty of assachusetts Dartmouth, Dartmouth A 0747 ABSTRACT Ths paper dscusses two novel artfcal neural networ
More informationAnalysis of Time Delays in Synchronous and. Asynchronous Control Loops. Bj rn Wittenmark, Ben Bastian, and Johan Nilsson
37th CDC, Tampa, December 1998 Analyss of Delays n Synchronous and Asynchronous Control Loops Bj rn Wttenmark, Ben Bastan, and Johan Nlsson emal: bjorn@control.lth.se, ben@control.lth.se, and johan@control.lth.se
More informationNetwork Theory. EC / EE / IN. for
Network Theory for / / IN By www.thegateacademy.com Syllabus Syllabus for Networks Network Graphs: Matrces Assocated Wth Graphs: Incdence, Fundamental ut Set and Fundamental rcut Matrces. Soluton Methods:
More informationEffect of Time-Interleaved Analog-to-Digital Converter Mismatches on OFDM Performance
Effect of Tme-Interleaved Analog-to-Dgtal Converter Msmatches on OFDM Performance Vo-Trung-Dung Huynh, Nele Noels, Peter Rombouts 2, Jean Armstrong 3, Hed Steendam Department of Telecommuncatons and Informaton
More informationDay ahead hourly Price Forecast in ISO New England Market using Neuro-Fuzzy Systems Alshejari, A. and Kodogiannis, V.
WestmnsterResearch http://www.westmnster.ac.uk/westmnsterresearch Day ahead hourly Prce Forecast n ISO New England Market usng Neuro-Fuzzy Systems Alshejar, A. and Kodoganns, V. Ths s a copy of the author
More informationArtificial Neural Networks for Cognitive Radio Network: A Survey
Internatonal Journal of Electroncs and Communcaton Engneerng Artfcal Neural Networks for Cogntve Rado Network: A Survey Vshnu Pratap Sngh Krar Abstract The man am of a communcaton system s to acheve maxmum
More informationIEE Electronics Letters, vol 34, no 17, August 1998, pp ESTIMATING STARTING POINT OF CONDUCTION OF CMOS GATES
IEE Electroncs Letters, vol 34, no 17, August 1998, pp. 1622-1624. ESTIMATING STARTING POINT OF CONDUCTION OF CMOS GATES A. Chatzgeorgou, S. Nkolads 1 and I. Tsoukalas Computer Scence Department, 1 Department
More informationAlgorithms Airline Scheduling. Airline Scheduling. Design and Analysis of Algorithms Andrei Bulatov
Algorthms Arlne Schedulng Arlne Schedulng Desgn and Analyss of Algorthms Andre Bulatov Algorthms Arlne Schedulng 11-2 The Problem An arlne carrer wants to serve certan set of flghts Example: Boston (6
More informationGrain Moisture Sensor Data Fusion Based on Improved Radial Basis Function Neural Network
Gran Mosture Sensor Data Fuson Based on Improved Radal Bass Functon Neural Network Lu Yang, Gang Wu, Yuyao Song, and Lanlan Dong 1 College of Engneerng, Chna Agrcultural Unversty, Bejng,100083, Chna zhjunr@gmal.com,{yanglu,maozhhua}@cau.edu.cn
More informationChaotic Filter Bank for Computer Cryptography
Chaotc Flter Bank for Computer Cryptography Bngo Wng-uen Lng Telephone: 44 () 784894 Fax: 44 () 784893 Emal: HTwng-kuen.lng@kcl.ac.ukTH Department of Electronc Engneerng, Dvson of Engneerng, ng s College
More informationNew Parallel Radial Basis Function Neural Network for Voltage Security Analysis
New Parallel Radal Bass Functon Neural Network for Voltage Securty Analyss T. Jan, L. Srvastava, S.N. Sngh and I. Erlch Abstract: On-lne montorng of power system voltage securty has become a very demandng
More informationChapter 13. Filters Introduction Ideal Filter
Chapter 3 Flters 3.0 Introducton Flter s the crcut that capable o passng sgnal rom nput to output that has requency wthn a speced band and attenuatng all others outsde the band. Ths s the property o selectvty.
More informationDiscussion on How to Express a Regional GPS Solution in the ITRF
162 Dscusson on How to Express a Regonal GPS Soluton n the ITRF Z. ALTAMIMI 1 Abstract The usefulness of the densfcaton of the Internatonal Terrestral Reference Frame (ITRF) s to facltate ts access as
More informationMASTER TIMING AND TOF MODULE-
MASTER TMNG AND TOF MODULE- G. Mazaher Stanford Lnear Accelerator Center, Stanford Unversty, Stanford, CA 9409 USA SLAC-PUB-66 November 99 (/E) Abstract n conjuncton wth the development of a Beam Sze Montor
More informationSpeech bandwidth expansion based on Deep Neural Networks
INTERSPEECH 2015 Speech bandwdth expanson based on Deep Neural Networks Yngxue Wang 1,2, Shenghu hao 1, Wenbo Lu 3,4, Mng L 3,5,Jngmng Kuang 1 1 School of Informaton and Electroncs, Beng Insttute of Technology,
More informationAppendix E: The Effect of Phase 2 Grants
Appendx E: The Effect of Phase 2 Grants Roughly a year after recevng a $150,000 Phase 1 award, a frm may apply for a $1 mllon Phase 2 grant. Successful applcants typcally receve ther Phase 2 money nearly
More informationPOWER constraints are a well-known challenge in advanced
A Smple Yet Effcent Accuracy Confgurable Adder Desgn Wenbn Xu, Student Member, IEEE, Sachn S. Sapatnekar, Fellow, IEEE, and Jang Hu, Fellow, IEEE Abstract Approxmate computng s a promsng approach for low
More informationA MODIFIED DIRECTIONAL FREQUENCY REUSE PLAN BASED ON CHANNEL ALTERNATION AND ROTATION
A MODIFIED DIRECTIONAL FREQUENCY REUSE PLAN BASED ON CHANNEL ALTERNATION AND ROTATION Vncent A. Nguyen Peng-Jun Wan Ophr Freder Computer Scence Department Illnos Insttute of Technology Chcago, Illnos vnguyen@t.edu,
More informationOn Channel Estimation of OFDM-BPSK and -QPSK over Generalized Alpha-Mu Fading Distribution
Int. J. Communcatons, Network and System Scences, 010, 3, 380-384 do:10.436/jcns.010.34048 Publshed Onlne Aprl 010 (http://www.scrp.org/journal/jcns/) On Channel Estmaton of OFDM-BPSK and -QPSK over Generalzed
More information