Pattern Recognition System with Top-Down Process of Mental Rotation

Transcription

1 Pattern Recognition System wit Top-Down Process of Mental Rotation Sunji Sato, irotomo Aso, Sogo Miyake, and Jousuke Kuroiwa Department of Electrical Communications, Tooku University Aoba-yama05, Sendai , JAPAN ttp:// sun/index-e.tml Department of Applied Pysics, Tooku University Aoba-yama04, Sendai , Japan Te Division of Matematical and Information Sciences, irosima University, igasi-irosima , Japan Abstract. A new model wic can recognize rotated, distorted, scaled, sifted and noised patterns is proposed. Te model is constructed based on psycological experiments in a mental rotation. Te model as two types of processes: (i) one is a bottom-up process in wic pattern recognition is realized by means of a rotation-invariant neocognitron and a standard neocognitron and (ii) te oter is a top-down process in wic a mental rotation is executed by means of a model of associative recall in visual pattern recognition. In computer simulations, it is sown tat te model can recognize rotated patterns witout training tose patterns. Keywords. rotation-invariant neocognitron, rotated pattern, mental rotation, topdown process, pattern recognition 1 Introduction In usual we can recognize patterns even if tose are affected by distortions in sapes, sifts in positions, and/or rotations. owever it is difficult to realize a recognition system wic is robust for suc many kinds of deformation. Actually, some of systems [1] can recognize rotated patterns but can not recognize distorted patterns, a system proposed in [2] cause a combinational explosion of units, and a system proposed in [3] is robust only for translation and rotations at every. owever, tey are not necessarily satisfactory in te sense tat tey can not recognize rotated, distorted, sifted, scaled and noised patterns. For example, some models can recognize distorted patterns but can not recognize rotated ones. It is very important to realize a reliable recognition system wic is insensitive to most deformations described above. We ave proposed a rotation-invariant neocognitron [4, 5], wic is an extended model of a (standard) neocognitron [6], to satisfy most all demands of pattern recognition. It as been sown tat te rotation-invariant neocognitron is robust for rotation, distortion, sift, scaling, and noise by computer simulations. Te rotation-invariant neocognitron is constructed as a feedforward network based on te idea tat simple patterns could be recognized by means of a bottom-up process even if te patterns are rotated, sifted, scaled

2 and so on. Te rotation-invariant neocognitron, owever, learns implicitly rotated patterns in any angles wen a standard (non-rotated) pattern is presented at a training pase. Tis learning process of te rotation-invariant neocognitron may be effective for simple patterns but not be effective for complicated patterns, e.g. Cinese caracters, because uge number of cells/connections and very long learning time are needed to make possible to recognize suc rotated and complicated patterns. Altoug umans can correctly recognize rotated patterns, it does not necessarily mean tat umans learn or store all rotated patterns in any angles. We tink tat suc complicated and rotated patterns could not be recognized by a bottom-up process only, and top-down processes could play one of important roles in order to realize a complicated and rotated pattern recognition. Indeed, psycological experiments ave been sown tat information of mental rotation play one of important roles in recognizing rotated patterns. Separd and Metzler sowed by psycological experiments tat larger rotation angles of a pattern causes longer recognition time of te pattern []. In teir study, tey proposed te existence of mental rotation in recognition processes in suc a rotated pattern. We can also see from te result tat umans execute a mental rotation after a tentative result of recognition of te rotated pattern is obtained (see Fig. 1(b)), because umans can not calculate te difference of directions between te rotated pattern and a standard one (Fig. 1(d)) witout te tentative result for te rotated one. ere, te tentative result means tat te result is not necessarily coincide wit a correct category of a presented pattern. To know te direction of te rotated pattern, umans ave to make a ypotesis (tentative result) for te category of te rotated pattern. We consider tat mental rotation as an information in verifying te ypotesis, and te recognition processes of rotated patterns are done troug te verifications of iterative ypoteses by means of te information of a mental rotation and knowledges of stored patterns [8]. Terefore, we assume tat a mental rotation is a top-down process. n te contrary, it is apparent tat a recognition sequence by te neocognitron-type network is a bottom-up process. In tis paper, we examine te process of recognition of rotated patterns from a psycological point of view and construct a new model in wic te mental rotation is executed by means of feedback processes (top-down processes). Te aim of tis paper, owever, is not to construct a model wic realize a mental rotation but to construct a more realistic and reliable model to recognize rotated, scaled, distorted sifted and noised patterns. 2 Algoritm of te New Model From te study for recognition of complicated and rotated patterns discussed in te previous section, we give an algoritm to recognize complicated and rotated patterns in Fig. 1. Next, we will give more details about processes or structures sown in Fig ottom-up Process: ybrid-neocognitron asic Idea to Construct a ottom-up Process. We give a basic idea to construct a bottomup recognition system, wic is able to recognize distorted, scaled, sifted and noised patterns. If we recognize a rotated pattern, wat kinds of patterns do we use an information of mental rotation? We classify patterns into two classes, mental-rotation-required and mentalrotation-free patterns, by use of te criterion weter te mental rotation is needed to correctly recognize a rotated pattern or not. umans can correctly recognize rotated patterns

3 Rotated Pattern ottom-up : ybrid-neocognitron ottom-up (a): Pattern Recognition (b): ypotesis for Rotated Pattern Top-Down : Associative Recall (c): Associative Recall of Standard Pattern 3 (+, 30º) (e): Pattern Rotation (d): (rotated "3") Recalled Pattern Estimation of Angular Sift Fig. 1. verview of te system for recognition of complicated and rotated patterns. Te process is divided into two parts: (i) a bottom-up process of a recognition system and (ii) a top-down process of a system wit associative recall. Mental rotation, (d) and (e), requires a tentative recognition result (b) and te recalled pattern (c). and tose orientations in an instant witout elp of mental rotation if tose rotated patterns are comparatively simple like a T-saped pattern, cross-saped pattern or -saped pattern. Suc simple patterns are composed of two or more line segments wit different orientations (we call suc segments oriented segments [10]). We classify tose patterns into mental-rotation-free patterns. n te oter and, rotated numerals, alpabets or Cinese caracters are not correctly recognized in an instant and tose orientations are not determined in an instant. Tose complicated patterns are composed of many T-saped patterns, cross-saped patterns and so on. We classify suc patterns into mental-rotation-required patterns. Examples of eac classes in numerals are depicted in Table. 1. Te difference be- Table 1. Te classification of complexities of patterns from a psycological point of view and complexities to be detected in eac module of a rotation-invariant neocognitron or a standard neocognitron. Te lower layer detects comparatively simple patterns (mentalrotation-free patterns) and te iger layer detects complicated patterns (mental-rotationrequired patterns). module 23 5 Features Class mental-rotation-free mental-rotation-required

4 tween mental-rotation-free and mental-rotation-required patterns is due to weter umans ave already learned and stored all rotated patterns in any angles or not. In tis paper, we construct te bottom-up system for recognition of complicated and rotated patterns based on te system of a rotation-invariant neocognitron and a standard neocognitron [6]. In te rotation-invariant neocognitron and te standard neocognitron complexities of patterns can be classified into four classes. Te rotation-invariant neocognitron and te standard neocognitron are multi-layered networks and eac of tose as te retina,, and basically four modules,, and (eac model as nine layers since eac module!, consists of two layers, and $ ). Lower modules, and, detect comparatively simple patterns and iger modules, and, complicated patterns. Examples of patterns to be detected in eac module is depicted in Table 1. We can regard patterns detected in and as mental-rotation-free patterns because tose patterns are oriented segments or composed of two or more oriented segments and patterns detected in and as mental-rotation-required patterns because tose require a mental rotation for correct recognition. We can see from Table 1 tat te psycological classification of complexities of patterns corresponds to te classification by te serial number of modules in te rotationinvariant neocognitron or te standard neocognitron. Terefore, te model of a bottom-up type recognition system ave two modules: (i) te lower modules are composed of ones of a rotation-invariant neocognitron, in wic all te simple rotated patterns in any angles (mental-rotation-free patterns) are detected, and (ii) te iger modules are composed of a standard neocognitron, in wic only standard complicated patterns are detected. Te scematic structure is depicted in te upper part of Fig. 2. Te number of cells is reduced by replacing iger modules of a rotation-invariant neocognitron by iger modules of a standard neocognitron. We name te bottom-up system a ybrid-neocognitron. We note tat te ybrid-neocognitron is able to recognize distorted, scaled, sifted and noised patterns since te ybrid-neocognitron inerits te all functions of a standard neocognitron. Tentative Recognition of Rotated Patterns Using ybrid-neocognitron If tose values of tresold of feature-detecting cells and oter parameters remain fixed in te ybridneocognitron, te model can not recognize rotated patterns (all cells in #, gnostic cells, make no response) because a standard neocognitron can not recognize rotated patterns. owever, in order to produce an information of a mental rotation in te rotated pattern (see Fig. 1(b)), te ybrid-neocognitron, owever, as to give a tentative recognition result in some way. We note again tat te tentative result of recognition needs not to be correct one, and te tentative result is a ypotesis produced by a ybrid-neocognitron. Moreover, a mental rotation gives an information in verifying te ypotesis. Te problem is tat ow to make a tentative recognition result for te rotated pattern using a bottom-up model only wic as not learned or stored rotated patterns. ere, we give results of a psycological experiment wit respect to recognition of rotated patterns by twelve subjects. Te aim of te experiment is to analyze weter subjects can make a correct recognition in rotated patterns witout a mental rotation or not. Te conditions is an experimenter notes a subject tat a pattern is rotated, a pattern sown in Fig. 3 is presented in te front of te subject, and te subject is forced to report te category of te pattern as soon as possible.

5 ) ( '. - U) 0 U* S 1 U C U 1 U 2 n + y 1 U* S 2 U C n θ + 2 * 3 U+, C 3 U* S 4 U C U S U, 3 U Retina n& x (e) Rotation (d) Estimation of Angular Sift Cell-Plane Stack Cell-Plane (b) Associative Recalled Pattern (c) 3 W 0 W* S 1 W+ C 1 W* S 2 W+ C 2 W* S 3 W+, W* C 3 S4 W+ - C 4 Fig. 2. Upper part: te structure of a bottom-up system, a ybrid-neocognitron. Lower part: te structure of a model of associative recall. Te ybrid-neocognitron is composed of a rotation-invariant neocognitron in te lower module and a standard neocognitron in te iger module. Te structure of associative recall is same as one of te ybrid-neocognitron. (b) (e) are correspond to processes (b) (e) sown in Fig. 1. Te last item of te condition intends to force a subject to recognize a rotated pattern witout a mental rotation. Te result is very interesting. All of twelve subjects report tat te pattern is 2 beyond doubt wen te leftmost pattern in Fig. 3 is presented altoug te pattern is a rotated one of a mirror image of 2. We conclude tat te absolute positions of local features wic compose te wole of a pattern is ignored if one as to recognize te pattern using only a bottom-up-type process. In our bottom-up system, tese results are given by extending te region of blurring operation by C-cells in #, # and # and by decreasing tresold values of S-cells in " and "/. Tus, our bottom-up system wit te extending blurring operation and decreasing tresold values can give a tentative recognition in a rotated pattern. Details are explained in te Section 3. Fig. 3. Examples of patterns to test a recognition ability witout mental rotation. 2.2 Associative Recall of Standard Pattern After making ypotesis for a rotated pattern, te standard pattern is recalled using te knowledge about te pattern. Te associative recall is given by te model wic is con-

6 C b 9 FFE G f s r structed based on te selective attention and associative recall model given by Fukusima [11]. In te present paper te function of selective attention is omitted for simplicity. Te structure and connections of our associative recall model are similar to te one of te ybrid-neocognitron but te flow of information is opposite directions, from te igest module to te lowest module (see te lower part of Fig. 2). As te result, a recalled pattern is a blurred pattern (an average pattern) of te tentative recognition result for a rotated pattern. 3 Matematical description of te Model 3.1 ottom-up Process: ybrid-neocognitron Matematical description of te ybrid-neocognitron is same as te one of te rotationinvariant neocognitron because te rotation-invariant neocognitron includes a standard neocognitron. Te structure of te ybrid-neocognitron is sown in te upper part of Fig. 2. is a model of retina. denotes a layer consisting of S-cells in te t module, and $ a layer consisting of C-cells. An S-cell is a feature-detecting cell and a C-cell as a blurring function so tat te model is robust for rotation or distortion. Lower layers, ", #, " and #, is composed of a number of cell-plane stacks [4, 5], and different cell-plane stacks detect different features of inputs (a cell-plane stack is referred to as CPS ). A CPS is composed of a number of cell-planes (referred to as CP(s)) [6], and eac CP detects a different rotation angle of te local pattern (a local pattern wic composes an input pattern is called as features ). Eac cell in a CPS is located in a tree-dimensional space. A rotational information of a local pattern is represented by te serial number 021 assigned to a CP in a CPS, and te positional information by te position of a firing cell in te specific CP. An output response of an S-cell located on :6 0;1 of te < t CPS in te t module is denoted by 349 <>, and an output response of a C-cell by 349 <>. Te output response 39 <> is given by were "/ 349 J]_^a`Rbdc egf irj M;_v 3w< 3WV2 I YX[ZD\ IKJ?A NMP IL?A 3V o kmlan 3 k pq < 39 k p (3) `tbdcue yx 349zR{ (4) RQTS S U (1) (2) A binomial operator r wit } is defined by ~ 025Dr ƒ 5I W 026Ar ƒ 6P W 0;1r 1 W ; R X ˆŠ }Y{ (5)

7 b C C 9 f 9 ere?a denotes a tresold value of an S-cell, o 3 k pq < represents an excitatory connection from a C-cell to an S-cell and v 3w<> an inibitory connection from a V-cell to an S-cell a V-cell sends inibitory inputs to S-cells and are not depicted in Fig. 2 for simplicity. Eac connection is linked to a restricted number of C-cells in te preceding module, ŒŽ, denotes te number of CPSs in te # layer, and te number of CPs in te $ layer. An output response of a V-cell is given by >x 349z ^ `tbdcue f irj kl8n 3 k > 349 r s `Rbdc e were 3 k is an excitatory connection from a C-cell to a V-cell, wic takes a fixed value during a learning. An output response of a C-cell is given by 349 were te function is defined by E G kml8š br 3 k 349œr sa b k k p <> QU 3žV2 { (8) 3WV2 3 k Œ is an excitatory connection from an S-cell to a C-cell, Ÿ represents a te number of CPs in te layer. C-cells in # 39 <>, are gnostic cells, and so a maximumly firing C-cell represents a ere restricted region of te connection and te last layer, recognition result wit igest likeliood among te all of categories. Te parameters about te number of CPSs, CPs and cells are given in Table 2. Te, are given in Table 3. initial sizes of te areas, and Ÿ Table 2. Te number of cell-plane stack, and te number of cells in one cell-plane stack. Te numbers in columns marked by asterisks are not defined, and te numbers in parenteses in columns are ones after completion of learning. ª ; až a ay a@ 1 (6) (22) (10) «a q ay a@ a@ 1 1 (6) (22) (10) (6) () During a learning pase, excitatory connections o 3 k pq < and inibitory connections v 3w<> are modified according to an unsupervised learning using seed-cell [9]. An algoritm proposed by autors, auto-generating algoritm [10], by wic learning time is drastically reduced in te unsupervised learning is also adopted. During a recognition pase, all gnostic cell would make no response if a largely rotated pattern is presented in. At tat time, te ybrid-neocognitron execute te following two processes; (i): te model decreases te value of tresold of S-cells in " and "/ and (ii):

8 Ÿ " 3 ^ " 9 p < " 3 Ÿ Table 3. Te size of area connected wit one cell, and columns is one after completion of extending te size, discussed in section 2.1. a q a@ a@ ²± ³ µ³ ² ³ ²³ ³ µ³ ²± ³ ²³ 3w ³ ³. Te size in parenteses in spreads te blurring region of C-cells in #, # and as discussed in te section 2.1. Te process (i) is denoted by?8¹ º¼»²%?8 ½¾ 3, were te notation ḑ is used in te sense of computer language PASCAL and» is an attenuation constant. Extended regions of blurring operation is given in parenteses of Table Top-Down Process: Associative Recall Te structure of te model of associative recall (te lower part in Fig. 2) is same as te one of ybrid-neocognitron except te direction of te flow of information. Eac layer is [ denoted by, and, 3. Te output of te recognition layer in a ybrid-neocognitron, #, is sent back to lower modules troug backward pats of te associative recall model and te flow reaces te recall layer. Te output of a cell in and te cell Áx in in te backward pats are given by were  349 <> K Áx E Gàkilan dæ 349z ~ ^ båä e båä%e irj kl8n dæ 3 k?8dæ IË?8dÆ båä%e Áx irj o dæ 3 k dæ 349 is a positive constant. Te output of a 39 kilnš b båä e vdæ 3 3 k Ç s8 båä%e p k È É Ê cell in 39ÎÇ s QU ÌÆ 349 `tb "/dæ 39 p Ç sa båä e is given by were Íy is a positive constant. Te learning of te model of associative recall is executed by following te learning of te ybrid-neocognitron. For example, if a new CPS or CP is generated in te ybridneocognitron, te same CPS or CP is also generated in te associative recall model. Te network is designed so tat te variable backward connections is automatically reinforced in te following manner: after finising te reinforcement of te forward connections, te backward connections descending from a cell are automatically reinforced to ave a strengt proportional to te forward connections ascending to te cell wic makes a pair wit te cell. k <> k p (9) (10) (11)

9 Ï Ó f Ó Fig. 4. Examples of training patterns. 3.3 Angle Estimation Te difference in orientation between a rotated pattern and te recalled standard pattern is estimated in order to determine te angle of a mental rotation. In estimating te difference, as sown by te psycological experiment in Section 2.1 te absolute positional information is ignored at comparing tose two patterns. Te angle estimator compares projected patterns on 0;1 -axes in and because te absolute positional information of a feature is represented by a position 340:5 0:68. Te difference of orientations, Ï, is given by " 30;1 4 Simulation < " Ò 1 j Ô Ó Ô " " Z Ð/Ñ X[ZD\!Ó ÓÖÕÖÕÖÕ s8 w Ø 30;1 < Þ8ß ÞAà l8á g 349 <> " r s w ÚÙ Û <> ÞAß Þ8à l8á w <>ÝÜ " 39 (12) <>2{ (13) We examine te ability of te new model for te recognition of realistic and-written numerical patterns provided by ETL-1 database 1. We use five undreds training patterns (see Fig. 4 for examples) and one tousands test patterns. Te set of test patterns includes patterns wic are generated by rotating standard patterns in ETL1 database by use of a computer. Examples of correctly recognized patterns is sown in Table 4. Te value in column Ïãâåäçæ in Table 4 is an actual rotational difference between a pattern in te database and te rotated ones. Te difference of angles between columns is due to te resolution in orientation of te ybrid-neocognitron. Te resolution is since comparison of a standard pattern and a rotated one is executed in " " and and orientation is quantized into " è± direction in tose layers. Te error is not significant and does not influence te ability of recognition because and-written patterns cange te sape by writers and one can not estimate precise rotational angles of tose patterns. Recognition rate for non-rotated test patterns is 91.0% and rotated test patterns is 90.5% in spite of te fact tat te model does not learn or not store rotated patterns. Tis result sows te effectiveness of te model. Wen only te ybrid-neocognitron is used to recognize test patterns, te recognition rate for rotated patterns is 16.6%. It turns out tat te model of mental rotation is effective for recognition of rotated pattern in te case tat a bottom-up system does not learn or not store rotated patterns. 5 Conclusion We construct a new model for pattern recognition wic is robust for rotation as well as distortion, scaling, sift in position and noise. Te new model is designed so tat te process 1 A caracter database publised from te Electrotecnical Laboratory, Japan.

10 Ï ± ³ ³ Table 4. Samples of correctly recognized patterns. Ï âåäçæ is rotational angle from te standard pattern. Ï is an angle estimated by te angle estimator. test pattern Ï âåäéæ includes te result of psycological experiments. y computer simulations te effectiveness of te new model is sown by using a numerical and-written database. We intend to improve te recognition rate by adding an additional module used in a standard neocognitron, and expect tat te model will sow a ig recognition rate for realistic patterns. References 1. M. Fukumi, S. matu, and Y. Nisikawa, Rotation-Invariant Neural Pattern Recognition System Estimating a Rotation Angle, IEEE Trans., Neural Network, Vol. 8, pp , M.. Reid, L. Spirkovska, and E. coa, Rapid training of iger order neural networks for invariant pattern recognition, Proc. Int. Joint Conf. Neural Networks, Vol. 1, pp , Widrow, R. G. Winter and R. A. axter, Layered neural nets for pattern recognition, IEEE Trans. Acoust., Speec, Signal Processing, Vol. 36, pp , S. Sato, J. Kuroiwa,. Aso and S. Miyake, Recognition of rotated patterns using neocognitron, Proc. Int. Conf. Neural Information Processing, Vol. 1, pp , 199. ttp:// sun/postscript/iconip9.ps.gz 5. S. Sato, J. Kuroiwa,. Aso and S. Miyake, A rotation-invariant Neocognitron (in Japanese), IEICE Trans., Vol.J81-DII, K. Fukusima, Neocognitron : A ierarcical neural network capable of visual pattern recognition, Neural Networks, Vol. 1, No. 2, pp , R.N.Separd and J. Metzler, Mental rotation of tree-dimensional object, Science, Vol. 11, pp , C. Stanfill and D. Waltz, Toward memory-based reasoning, Communication ACM, Vol. 29, pp , K. Fukusima and N. Wake, An improved learning algoritm for te neocognitron, Proc. of te Int. Conf. on Artificial Neural Networks, pp. 4 (1992). 10. S. Sato, J. Kuroiwa,. Aso and S. Miyake, Recognition of and-written patterns by rotation-invariant neocognitron, Proc. Int. Conf. Neural Information Processing, Vol. 1, pp , ttp:// sun/postscript/iconip98.ps.gz 11. K. Fukusima, Neural network model for selective attention in visual pattern recognition and associative recall, Applied ptics, Vol.26, pp , 198.