New foveated wide angle lens with high resolving power and without brightness loss in the periphery

New foveated wide angle lens with high resolving power and without brightness loss in the periphery K. Wakamiya *a, T. Senga a, K. Isagi a, N. Yamamura a, Y. Ushio a and N. Kita b a Nikon Corp., 6-3,Nishi-ohi 1-chome, Shinagawa-ku, Tokyo 14-861, JAPAN b National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba Central 2, Tsukuba 35-8568, JAPAN ABSTRACT A new foveated wide angle lens with high resolving power and without brightness loss in the periphery is developed. A foveated lens is an optical system which has both a large field of view and a high resolution in the center of the view field. While it is ideal to keep uniform brightness in the whole view field, widening the field of view without loosing brightness in the periphery had not been achieved by several previous designs and fabrications. In the new design, telecentric light is kept on the image side and a more negative distortion is added to the sine law. As a result, more than 14 degrees view field is achieved with very little loss of brightness in the periphery. The evaluation methods and results for the fundamental optical performance of the fabricated lenses are described. The performance of the proposed lens for several basic visual tasks, such as the readability of characters, the visibility in the view circumference, and the possibility of stereo vision, are compared with that of other optical systems which have a wide field of view. The prospective applications of the proposed lens is also discussed. Keywords: lens design, wide angle lens, foveated lens, illuminance, visual tasks 1. INTRODUCTION 1)2) 3) A foveated lens not only covers a large angle of view but also enables precise observation in the center of view. A new foveated optics of small size has been developed utilizing an electric image sensor of high sensitivity and high resolution. Instead of combining optics of narrow and wide angle view or using zooming optics, a simple and compact system that could take a survey of the whole view field instantaneously has been realized. The new foveated lens has a wide angle of view and maximum resolving power at the center of view field. High resolving power all over the view field and a projection curve of emphasized in center part of a field has been realized. By utilizing a telecentric optics, good adaptability to an image sensor(ccd) and precise measurement of an image size have been achieved. Maintaining illuminance at a high level in the periphery has resulted in good legibility performance In this paper, the performance and estimation method of the foveated optics that we developed is discussed in detail. Application where foveated optics are promising are examined. 2. TARGET SPECIFICATION The objective of this work was to develop a wide angle lens with more than 14 horizontal view with a large image in the center. For high resolving power all over the view, an image sensor device with a broad image size ( 2/3 inch 145million pixel CCD) was utilized. As a cell size of the CCD is 6.45μm, Nyquist frequency becomes 77 line pairs/mm. A resolving power of more than 8 line pairs /mm all over the view was the target level. An illuminance of more than 7% was also targeted in order to achieve legibility in the periphery. To prevent fluctuation of image size by focusing and to provide adaptability to a camera, telecentric optics at the image side was adopted. * Wakamiya.Koichi@nikonoa.net

3. LENS DESIGN 3.1 Camera The qualification of the camera is important in designing optics. HITACHI 2/3 inch CCD color camera (typekp- F12C/F12CL) has about 145million of effective pixels (1392 14 ) and a pixel size is 6.45μm square. Thus, the image size is 8.978 6.78mm and Nyquist frequency (1/2/pitch) reaches 77 line pairs/mm. Sensing linearity and dependence of sensitivity on the incident angle were measured to obtain the exact position of the exit pupil. The output of the CCD without lens was measured by varying the irradiation of a green-led. (1W, peak wavelength 53nm, half bandwidth 4nm). The LED was set in front of the CCD, at a distance of 54 cm and the average output of green on 5 pixels in a 3 3 area was used as plotting points. Figure.1 shows the 8bit-output of the CCD. The linearity of the CCD output in all area was confirmed. Camera Sensitivity by LED Intensity Sensor output 255 24 153 12 51 1 2 3 4 Illuminance(lx) Figure 1. Linearity of the CCD sensitivity The dependence on incident angle was measured using the same green-led and the camera on a goniometer. The incident angle was changed in steps of 5 and the output was measured. Figure 2 is a result of the dependence on the horizontal angle at the center of the CCD and Figure 3 shows the vertical angle dependence. 2 2 Camera Sensitivity 16 12 8 4 Camera Sensitivity 16 12 8 4-4 -3-2 -1 1 2 3 4-4 -3-2 -1 1 2 3 4 Figure 2. Output dependence on horizontal angle Figure 3. Output dependence on vertical angle Sensitivity is stable in the range of±1 for horizontal incidence and ±2 for vertical incidence. At larger incident angle, the sensitivity shows a rapid reduction. Very close values were obtained for both sides (right and left) of the CCD. Considering the symmetry of angle dependence, an infinite distance is appropriate as exit pupil position and telecentric optics has proved to be most suitable for this camera system.

3.2 Designed performance To achieve the target of view angle of 14 in the horizontal direction and of 18 in the diagonal, a particular arrangement of lenses is necessary. On the objective side from diaphragm position -front lens group-, negative power lenses were arranged. On the opposite image side -rear lens group-, positive power lenses were arranged. (The intersection point of the optical axis and the principal ray is defined as the diaphragm position.) Both sides have negative distortion aberration. As a result, a large distortion aberration occurs in the total optics that has nearly y=f*θ characteristics. For the target specification where negative distortion is smaller at small incident angle and larger at large incident angle, the distortion in the periphery should be emphasized. Using an aspherical lens is one solution. A concave high power lens would be necessary in the front group, and on the contrary, a positive power lens in the periphery would be needed in the rear group. An aspherical lens for wide angle that has a low power in the periphery has been utilized on the objective side. Using aspherical lenses causes some undesirable effects. One effect is the break of telecentric system. In particular, a high power lens in the rear group results in a dispersion at the image center and convergence in the periphery. This causes a complicated change of distortion by focusing. Thus, the insertion of an aspherial lens in the rear group image side was not pursued. Another undesirable effect of the aspherical lens is the loss of illuminance in the periphery by the reduction of aperture efficiency. An optical ray at large incident angle reaches the diaphragm position by reducing the incident angle gradually. The distortion level of optical ray at large incident angle is limited by lens shape and the loss of illuminance. As a balanced solution, one aspherical lens was used in the front group. After various simulated estimations of distortion and illuminance, an optics with negative distortion aberration in the periphery of about 17% to y=f*sin θ projection characteristics was designed. Figure 4 shows the designed projection curve compared to y=f*sin θ curve and y=f*θ curve. Image Height (mm) 5. 4.5 4. 3.5 3. 2.5 2. 1.5 1..5. Designed optics y=f*sinθ y=f*θ 1 2 3 4 5 6 7 8 9 Incident Angle (degree) Figure 4. Designed projection curve Figure 5 shows the MTF vs. focus position for a sine wave spatial frequency of 4 lp/mm. Each field is plotted, with Sagittal (solid line)and Meridional(dashed line) MTFs plotted separately. The MTF characteristics corresponds to an infinite plane. Figure 5 shows that at least more than 4lp/mm are resolvable throughout most image area. However, according to another observation, a value of 8lp/mm or more was almost achieved in the corner. To keep the high resolution in the manufacturing process, the fabricating process used for microscopic lenses was used.

Figure 5. Simulated MTF 3.3 Lens construction Optics is constructed with 6 groups with 9 pieces of lens. The focal length is 5.6mm and the F-number is 2.8. The diagonal view angle is 18, the horizontal view is 146 and the vertical view is 8. The diameter of the image circle reaches 9.25 mm. The total length from front surface of the first lens to image plane is 35mm and the outer diameter is 15mm. One aspherical lens is used at the front surface of the first lens. A color filter and a cover glass for the CCD are included in the optics. No low pass filter is included. A schematic diagram of the lens is shown in Figure 6. In this figure, ray tracings at, 3, 89 incident angle are indicated. Figure 6. Lens construction and ray tracing

4. ESTIMATION OF PERFORMANCE 4.1 Verification of specification In this section, the measurement process of resolving power, projection curve and illuminances is explained and a comparison between design targets and measurement data is presented. Projection curve and illuminance were estimated in a system including a camera. (HITACHI KP-F12C/F12CL) 4.1.1 Resolving power Usually a measurement apparatus of MTF is used for estimating resolving power. However, for a lens of wide angle of view, up to 18, the measurement is difficult because the object plane must extend to infinity. In this study, the reverse projection method was adopted. Reverse projected patterns obtained by lighting the resolution chart at the focus position are observed. The resolution chart that was prepared has 7 kinds of L/S pattern (line pairs/mm; 25/mm, 2/mm, 16/mm, 1/mm, 8/mm, 63/mm and 5/mm. ) arranged in the sagittal and meridional direction in the radial line(45 interval). The image height of the L/S pattern is mm,.93mm, 1.81mm, 2.8mm, 3.99mm for,1,2,32,54 respectively. Figure 7shows the arrangement of pattern images and the results of resolving power. The resolution of the readable chart is listed. A resolving power of more than Nyquist frequency(77 line pairs/mm) was obtained in almost all the view area. (<18 ) radial Incident angle direction 1 2 32 54 1 25/25 25/25 2/16 16/8 2 25/25 25/25 2/16 125/63 3 25/25 25/25 2/16 1/63 4 25/25 25/25 2/16 1/63 25 5 25/25 25/25 2/16 1/8 6 25/25 25/25 2/16 1/5 7 25/25 25/25 2/2 1/5 8 25/25 25/25 2/2 1/5 Figure 7. Resolving power measurements (Sagittal / Meridional) (line pairs /mm)

4.1.2 Image distortion (Projection curve) 4.1.2.1 Experimental setup Figure 8 shows the experimental setup for measuring the projection curve. The position of the image of the target (cross-shaped mark) in a CCD camera was measured. A cross-shaped mark was displayed on the LCD screen of 48cm 36cm placed at a distance of 1.6m from the CCD camera. The CCD camera was set on a goniometer and its rotation center was set at the entrance pupil of lens. The angular precision of the goniometer is 1 minutes. The coordinates of the cross-shaped mark and the angle of the goniometer could provide the image distortion. The position of the mark(cross point) could be detected sub-pixel precision. Figure 8. Measurement of image distortion 4.1.2.2 Entrance pupil The entrance pupil is on the backside of the objective first lens. Different incident angles cause the entrance pupil to shift along the axis. Figure 9 shows the relation between the incidence angle and the calculated position of the entrance pupil. Position of entrance pupil (mm) 1. 8. 6. 4. 2.. 1 2 3 4 5 6 7 8 9 Incident angle (degree) Figure 9. Relationship between incident angle and entrance pupil In the measurement of the projection curve, the point of the entrance pupil for maximum incidence angle was set as the rotation center of the goniometer. The shift of entrance pupil with incident angle reaches 6mm and causes an error in the detected angle of about.1 (maximum value). This affects the measurement position of the image by 3μ(~5pixels). 4.1.2.3 Focus point Though the fluctuation of image height for various focus points is small in the telecentric optics, the focus point was set to infinite distance to minimize the distortion. 4.1.2.4 Determination of the optical center

In the lens having a barrel type distortion, the track of the image accompanying the rotation of the camera becomes a straight line only when the image passes the optical axis of the lens. Thus, the optical center (intersection point of optical axis and image surface) could be defined by scanning the camera along X and Y directions and searching for a straight track of the image. The optical center could be adjusted with a precision better than 1 pixel. Figure1 shows image height at every 5 points. The solid line in Figure1 is the calculated projection curve. A good agreement with the designed target has been observed. 5. Image height(mm) 4. 3. 2. 1.. 1 2 3 4 5 6 7 8 9 Figure. 1. Measured image height (projection curve) 4.1.3 Illuminance in the periphery 4.1.3.1 Principle of illuminance Figure 11 shows the principle scheme of the measurement of illuminance. Figure 11. Principle of the measurement of illuminance The spherical diffuser that centers on the intersection of the entrance pupil and the optical axis is shown in Figure 11.

This spherical diffuser is a sphere where luminance has an uniform value in all aspects and always keeps a definite value. Diffusivity is necessary, but it may not be necessarily an ideally perfect diffusing surface. In this condition, the illuminance would be calculated from the model where the luminous flux from a small area da on the spherical diffuser reach da on the image surface. 4) The equation of illuminance normalized with respect to center illuminance E/ E is the following where the projection function is y=f*g(θ). In the equation, t is the optical loss by transmittance and reflection of the lens and S is the area of the projected luminous flux at the entrance pupil. E/E={(t/t)*(S/S) }* sinθcosθ*{ G ()} 2 /{ G(θ) * dg(θ)/dθ} 1 (E, t and S are the values at θ=) This expression is the same as the illuminance for an ideal Lambertian surface. In the case of G(θ) =tanθ (usual lens), the illuminance follows a cosine quadratic law as E/E={(t/t) *(S/S) }*cos 4 θ 2 The illuminance of a f θ lens ( G(θ)=θ) is given by the next equation. E/E={(t/t)*(S/S)*(sin2θ/2θ)} 3 If the value of (t/t)*(s/s) were 1(no loss by lens components), an illuminance of 64% at θ=45 would be expected for an fθ lens. When image height y is distorted to ŷ, the projection function is replaced by ŷ =f*h(θ) G(θ) ( ŷ / y = H(θ)) For the calculation of the present lens, sinθ was used as G(θ) and the polynomial approximation ΣCnθ n of the ray tracing curve was used as H(θ). The value oft (lost by glass transmittance and surface reflectance) is about 95% and decreases rapidly at incidence angles larger than 7. S/S become smaller when the incident angle increases, especially beyond 6. 4.1.3.2 Measurement of illuminance in the periphery Illuminance was measured in the experimental arrangement of Figure 8 that was used for estimating the projection curve. LCD displayed a grey screen instead of a cross- shaped mark for functioning as a diffused plane and outputs of the CCD were measured. As only the central area of the image was acquired, the diffuser could be regarded as a spherical surface in Figure 11. No effect by polarization characteristics of the screen could be observed by measuring at an incident angle of 6 while rotating the LCD plane. An output of 13 green pixels in the center of the image zone (5pixel 5pixel. 32μm area.) were detected.the measurements was repeated 3 times to reduce fluctuations. Figure 12 shows the results of illuminance varying the view angle by steps of 5. The ideal curve((t/t)*(s/s )=1) also shown by the dotted line. Rerative Illuminance(%) 16. 14. 12. 1. 8. 6. 4. 2.. -9-7 -5-3 -1 1 3 5 7 9 Figure 12. Illuminance of a foveated lens

4.2 Comparison to other type of wide angle optics Similar estimation were also carried out for other type of wide angle optics- a fθ lens for a 2/3 inch image( A optics ) and another foveated lens for 1/3 inch ( B optics ). Projection curve, brightness and resolving power were compared. A optics has a precise fθ projection characteristics all over the 18 view and shows a large reduction of illuminance in the periphery. Figure13 shows less than 56% illuminance at incident angle >7. The weakest point is that the center image size is not so large. Therefore the resolving power on the object side is rather low. B optics that has an emphasized foveated characteristics that results in an irregular and large reduction of illuminance in the periphery.figure14 shows less than 49% illuminance at incident angles >45. In this case the center image size is certainly large. However, the field angle of this optical system is somewhat small. Rerative Illuminance(%) 16 14 12 1 8 6 4 2-1 -5 5 1 Figure 13 Measured illuminance of A optics (fθ lens) Rerative Illuminance(%) 16 14 12 1 8 6 4 2-9 -7-5 -3-1 1 3 5 7 9 Image Height(mm) 2.5 2. 1.5 1..5. 1 2 3 4 5 6 7 8 9 Figure 14 Measured illuminance of B optics Figure 15 Measured image height of B optics Figure 16 and figure 17 are a comparison of an image obtained with our new foveated lens and with the B optics. These images were both taken from the distance of 45cm. Figure 16 Example obtained with the new foveated lens Figure 17 Exampleobtained with the B optics

5. APPLICATIONS The lens developed in the present study has the legibility in wide view and a large image at the center of view, and is expected to be useful in various cases. In addition to the bright and clear image all over the wide angle, the high resolving power at the center area of view is useful for monitoring. More detailed observation of an object in the periphery of view is possible by tilting the camera. Examples of application are monitoring of gate-in or out, monitoring a rather wide area as a children room or meeting room and monitoring some indicators simultaneosly etc. This would be also applicable to robot vision for a rather complicated work and mobile vision as for a camera in a vehicle. Figure 18. Image of the monitoring of the indicators with the new foveated lens Figure 19. Image of a meeting room with the new foveated lens 6. CONCLUSIONS A foveated lens for a 2/3 inch image size of telecentric optics has been developed. A resolving power over the Nyquist frequency (77line pairs/mm) was achieved in 86% of the area of the image circle. (at an incidence angle <18 ). The projection curve was verified to coincide to the designed target. Illuminance was also estimated and a small decrease in the periphery was measured. REFERENCES 1. Yasuo Kuniyoshi,Nobuyuki Kita,Sebastion Rougeaux,Takashi Suehiro: Active Stereo Vision System with Foveated Wide Angle Lenses, ACCV 95, PP.359-363 (1995). 2. Yasuo Kuniyoshi,Nobuyuki Kita,Kazuhide Sugimoto,Shin Nakamura, Takashi Suehiro : A Foveated Wide Angle Lens for Active Vision, IEEE international Conference on Robotics and Automation, PP.2982-2988 (1995). 3. Yoshikazu Suematu,Hironaro Yamada: A Wide Angle Vision Sensor with Fovea -Design of Distortion Lens and the Simulated Images,IEEE,PP.177-1773 (1993). 4. Max Born & Wolf : PRINCIPLES OF OPTICS fifth edition,pp.181-19.