10mW CMOS Retina and Classifier for Handheld, 1000Images/s Optical Character Recognition System

Transcription

1 TP mW CMOS Retina and Classifier for Handheld, 1000Images/s Optical Character Recognition System Peter Masa, Pascal Heim, Edo Franzi, Xavier Arreguit, Friedrich Heitger, Pierre Francois Ruedi, Pascal Nussbaum, Pascal Pilloud, Eric Vittoz Centre Suisse d Electronique et de Microtechnique SA, Neuchatel, Switzerland The optical character-recognition (OCR) system of Figure consists of a CMOS retina, an analog classifier IC and a microcontroller. The retina converts the parallel optical input into an oriented edge representation which is processed further and recognized by the classifier in real-time. The microcontroller postprocesses the time sequence of classifier outputs and provides timing and control. The system can be used as a handheld OCR scanner which processes the field of view 1000 times per second. The system output is the character string being scanned as shown in Figure The power consumption of the classifier IC is 1mW, more than two orders of magnitude lower than that of the AT&T s ANNA chip [1]. In contrast to the OCR system described in Reference 2, this present system is invariant to shift, and largely insensitive to distortions, scale, rotation, contrast, font type, and printing quality. The system also tolerates illumination variations as long as the spatial frequency spectrum of illumination inhomogeneity is below that of the character. The recognition rate is constant within the following operating ranges: font size variation of 100% (ratio of 2:1), ±10 o rotation, and % optical contrast. Performance degrades gradually outside these operating ranges and when processing exotic font types or poor print quality. To obtain robustness, it is important to achieve certain invariance at the image-acquisition stage. The retina performs illumination normalization by adapting integration time of photodiodes to impinging light intensity. Edges are extracted by binarizing the image gradient with a contrast-dependent threshold determined in the neighborhood of each pixel. Four orthogonal edge orientations are detected, North, East, South and West. Image gradient is coded on four bits with some redundancy. Zero, one or two orientations can be active at a time, providing 45 degrees or 3b resolution in the gradient orientation and 1 bit information about gradient intensity. Note, that the effective gradient resolution is more than one bit due to the local contrast dependent edge detection threshold. The advantage of the two-dimensional pixel array compared to linear one-dimensional scanners is that no image distortion is introduced due to scanning speed and scanning direction inhomogeneities. This advantage is particularly important for handheld devices. Furthermore, the two dimensional imaging introduces large redundancy compared to the onedimensional, which further increases robustness. The classifier architecture is similar to the convolutional neural network described in Reference 3 and is shown in Figure It is a feedforward multilayer perceptron architecture with meaningful constraints in the topology. The layers are divided into sublayers. All neurons in a sublayer have the same weight set ensuring shift invariance. All sublayers in a layer have the same input, but different weight set, forming feature extractor planes. The neurons have local receptive fields. This highly regular structure synthesizes the recognition process from consecutive multidimensional convolutions and thresholdings. A sample input and the corresponding output of each layer is shown in Figure Note, that layer 3 output encodes not only character class, but relative position within the field of view as well. The neuron threshold function is simplified from sigmoid to ternary, allowing only three different neuron states: -1, 0 or +1. This is achieved by modifying the learning process and is a key to simplifying communication. To achieve 1kHz sampling rate, the classifier executes 100M multiply and add operations per second. To combine high-speed computation with low power consumption, the most efficient solution in terms of area and power is the capacitive inner product circuit shown in Figure The classification task is encoded into weighting coefficients and hardwired into the circuit. The weights are obtained by the backpropagation training. The capacitive inner-product circuit is fully differential. Positive and negative weights are implemented as the difference of a length-weighted capacitor and a minimum-length dummy capacitor to cancel out the non-proportional fringing capacitance term. The weights are the top plates laid on a common bottom plate polysilicon-polysilicon capacitor. The filling factor (ratio of top plates area to bottom plate area) is optimized with a dedicated algorithm to get maximal signal-to-noise and signal-to-comparator offset ratios. Careful layout ensures absolute symmetry and eliminates crosstalk between neurons. The standard deviation of weight error is <1% of the largest weight of the corresponding sublayer. Except the outputs of layer 3, all communication is digital, allowing fast simple transfer between layers and easy intermediate storage. Inputs to the capacitors are CMOS rail-torail logic level transitions, ensuring perfect homogeneity and large dynamic range. Measured system performance is shown in Figure The system scans the character string slowly forward, stops at the digit zero then scans quickly backwards. The classifier outputs are plotted on the bitmap. Time (or snapshot index) is shown on the horizontal axis, character code on the vertical axis and confidence level is coded as pixel intensity. Bright pixel encodes high confidence. The white clusters clearly indicate a correct and robust recognition. Clusters follow each other, first with larger spacing in ascending order, then with smaller spacing in descending order corresponding to the actual scanning speed. Cluster-finding postprocessing is performed by the microcontroller. A special category, interdigit, indicates where the retina field of view is centered between two characters. This feature considerably increases the reliability of the cluster finding process. Both the retina and the classifier chips combine local analog computing with digital communication. The recognition performance of the analog classifier chip and the OCR system is practically equal to what can be achieved by 64b floating-point accuracy. Micrographs of the retna chip and the classifier chip appear in Figure and Figure , respectively. References: [1] E. Sackinger, B.E. Boser, J. Bromley, Y. LeCun, L. D. Jackel, Application of the ANNA Neural Network Chip to High-Speed Character Recognition, Journal of Solid-State Circuits,vol. 3, No. 3, May 1992, pp [2] John Platt and Tim Allen, A Neural Network Classifier for the I1000 OCR chip, Advances in Neural Information Processing Systems 8, pp , MIT Press, (1996). [3] J.D. Keeler, D.E. Rumelhart and W-K Leow, Integrated Segmentation and Recognition of Hand-Printed numerals, Advances in Neural Information processing Systems 3 pp

2 Figure : OCR system block diagram. Figure : (top) Scanned image. (midle) Measured classifier responses. (bottom) System output. Figure : Sample input and corresponding layer outputs of the classifier. Figure : Convolutional neural network classifier. System speed 100 images per second (retna illuminated with 5W/m 2 intensity) Check-reading recognition rate 99.95% (OCR B and similar font types) Classifier dissipation <1mW Retna dissipation 10mW Weight accuracy relative to largest <1% Neuron state resolution 3 levels Number of physical weights 54k Number of different weights 4k Multiply-and-add operation rate 100M/s Pixel array size 32x20 Pixel / photodiode size 125x125µm 2 / 50x50µm 2 CMOS process 0.5µm 2-poly 3-metal Retna chip size 5x6mm 2 Classifier chip size 6x7mm 2 Figure : Capacitive inner product computation circuit. Table : OCR system characteristics.

3 Figure : Micrograph of the retina chip. Figure : Micrograph of the classifier chip.

4 Figure : OCR system block diagram.

5 Figure : (top) Scanned image. (middle) Measured classifier responses. (bottom) System output.

6 Figure : Convolutional neural network classifier.

7 Figure : Sample input and corresponding layer outputs of the classifier.

8 Figure : Capacitive inner product computation circuit.

9 Figure : Micrograph of the retina chip.

10 Figure : Micrograph of the classifier chip.

11 System speed 100 images per second (retina illuminated with 5W/m 2 intensity) Check-reading recognition rate 99.95% (OCR B and similar font types) Classifier dissipation <1mW Retna dissipation 10mW Weight accuracy relative to largest <1% Neuron state resolution 3 levels Number of physical weights 54k Number of different weights 4k Multiply-and-add operation rate 100M/s Pixel array size 32x20 Pixel / photodiode size 125x125µm 2 / 50x50µm 2 CMOS process 0.5µm 2-poly 3-metal Retna chip size 5x6mm 2 Classifier chip size 6x7mm 2 Table : OCR system characteristics.