A NEW SYSTEM FOR MEASURING EYE POSITION ON A PERSONAL COMPUTER.

Transcription

1 A NEW SYSTEM FOR MEASURING EYE POSITION ON A PERSONAL COMPUTER. K.Matsuda*, T.Nagami, K.Kawano and S.Yamane Information Science Division, Electrotechnical Laboratory, Tsukuba, Japan 744.2

2 Introduction We have developed a new system based on on-line image processing to measure eye position noninvasively. We use an infrared LED to illuminate the eye of a subject, whose head movement is lightly restricted, and record images of the subject s pupil using a CCD camera with a sampling frequency of 60Hz. The image of the pupil in each video-frame is digitized by a video capture board and transferred to a personal computer. The outline of the pupil and the reflection center of the illumination on the cornea are then detected using image-processing software, and the pupil is approimated by an ellipse. Using the shape, size of the ellipse and reflection center, we estimate the subject s view ais in the cameracoordinate system (the origin is the center of the eye rotation and the -y plane parallels the plane of the CCD camera) and the diameter of the pupil. To calculate the transformation matri to transfer a vector from the camera-coordinate system to the target-coordinate system (the origin is the same as above and the -y plane parallels the plane of the targets), we ask the subject to fiate each of the 9 targets displayed on a screen. We measure the subject s eye positions in the camera-coordinate system and compare them to the respective required eye positions in the target-coordinate system. After this calibration process, we are able to measure the subject s eye position in the targets plane. The accuracy of the measurement is better than 1 degree. We use Red Hat Linu 6.1 as the operating system, Video for Linu as the device driver for the video capture board, and the X Window system for display. This system can be used for both monkeys and humans. The program is available at

3 Method 1.Calibration. To obtain values listed below which are necessary for the measurement, we ask the subject to fiate each of the 5 (more than 3) targets displayed on a screen and capture images of its eye. The subject's head movement must be restricted. i) The eye rotation center (Xo, Yo). ii) The pupil center rotation radius (Rp) iii) The cornea curvature center rotation radius (Rc). iv)the length between the reflection on the cornea and the cornea curvature center in each ais ( α,β. v) The coordinate system transformation matri. 2.Measurement. i) Find the pupil center and the illumination's reflection center using image-processing. ii) Calculate the subject's view ais vector in the camera-coordinate system. iii) Transfer the subject's view ais vector from the camera-coordinate system to the target-coordinate system, using the transformation matri. iv) Calculate view ais angles in the target-coordinate system. v) Output angles as voltage by using D/A converter (option).

4 1 System outline Computer with video capture board Stimulus display CCD camera IrLED Eye Head(lightly restricted) CRT 1. All equipment can be bought on the market. 2. The CCD camera can be arranged at any position within the range where a picture of the pupil can be taken. 3. The IrLED can be arranged at any position within the range where a picture of the illumination's reflection on the cornea can be taken.

5 System equipment model memo price(about) CCDCamera XC-EI50 (Sony) Power supply DC-700 Lens 45mm Macro Lens <$3000 IrLED SLR931A(Sanyo) Use 56 LEDs. Power supply. <$15 Video Capture Card GV-VCP2/PCI(IO data) Any board adopt Video for Linu. <$120 DA converter DA12-4L(PCI)(Contec) D/A Converter PCI bus <$600 IBM PC Compatible Pentium III 700MHz RedHat Linu 6.1 <$2000 total <$6000

6 2 Image-processing The outline of the pupil and the reflection center on the cornea are detected using image-processing software. The Pupil is approimated by an ellipse epression. The center of the pupil, the pupil radius, minor-ais major-ais ratio and the minor ais slope are calculated from the epression. The processing time is less than 1ms.

7 There is an another method to obtain the pupil center, by using the pupil barycenter. But our method surpasses the barycenter method in accuracy when the pupil is hidden by the eyelid. Furthermore our method can obtain the eact diameter of a pupil even when the pupil is hidden by the eyelid. To compare the accuracy of these methods, we measured the center of a circle in y-ais, which was partly hidden, shown the following figure, by two methods. We also measured the diameter of a circle, which was partly hidden, to reveal our method's performance. 320 piel y hidden part Diameter 200piel hidden ratio[%]= hidden part diameter 100

8 1. In our method, the gap ratio is less than 0.1% when the hidden ratio is 30 %. 2. In our method, the diameter ratio is less than 0.1% when the hidden ratio is 30%. gap ratio[%]= 50 gap of center diameter 100 diameter ratio[%]= 10 diameter-measured diameter diameter berycenter method out method hidden ratio[%] Measured center in y-ais of partly hidden circle hidden ratio[%] Measured diameter of partly hidden circle.

9 3 Cornea curvature center is obtained from the illumination's reflection on the cornea. reflection α α β β CCD camera IrLED rotation center cornea curvature center eyeball reflection IrLED cornea cornea curvature center CCD camera reflection Suppose that the cornea is a part of a sphere, we refer to the center of that sphere as cornea curvature center. If the illumination and CCD camera is located farther enough than the cornea curvature center movements, the location relation of the cornea curvature center and the center of illumination's reflection on the cornea is always same. In this figure, α and β are fied.

10 4 Measurement Find the view ais vector and the rotation center in the camera-coordinate system from the pupil center and the cornea curvature center. pupil center (X p,y p ) y rotation center ( X o,y o ) R pc R p R c pupil center y cornea curvature center (X c,y c ) view ais vector (C,C y,c z ) The view ais in the camera-coordinate system. z eyeball cornea curvature center CCD camera (C,C y,c z )= X p-x c R pc The rotation center. (X o,y o )= R cx p -R p X c R c -R p, Y p-y c R pc,, R cy p -R p Y c R c -R p R 2 pc- X p -X c 2 - Y p -Y c 2 R pc

11 The view ais vector can be measured by using the pupil center and the cornea curvature center without fiing the head. The head movements can be measured at the same time as well. [deg] X-direction eye movements [piel] X-direction head movements [sec] 1[mm] [sec] Measured eye movements are not influenced by head movements. Movements of the eye rotation center is considered as head movements.

12 Find the view ais vector in the target-coordinate system by using the transformation matri. (C,Cy,Cz) The view ais vector in the camera-coordinate system (T,Ty,Tz) The view ais vector in the target-coordinate system T T y T z = a d g b e h c f i y C C y C z z The camera-coordinate system Z CRT Y CRT X X The target-coordinate system

13 Measurement eample X angle =arctan T T z Y angle =arctan T y T z stimulus movements eye movements X angle 5 [deg] Error average = [deg] 5 angle [deg] Error average = [deg] [sec]

14 5 Step 1) Input targets information Eample Input targets information in display order to the program. In this eample, we use 5 targets. Z CRT 200cm X

15 Step 2) Calibration We ask the subject to fiate each of the 5 targets displayed on a screen, and store values obtained from image-processing. [deg] [deg] movements 10[sec] y-movements 10[sec] Stimulus movements Captured computer image.

16 Step 3) Calculate parameters. i) Click mouse suitable position and find average pupil center Pn(Xpn, Ypn) and illumination's reflect center on cornea Rn(Xrn, Yrn) correspond to each target. P 1 X p1,y p1 P 2 X p2,y p2 P 3 X p3,y p3 P 4 X p4,y p4 P 5 X p5,y p5 R 1 X r1,y r1 R 2 X r2,y r2 R 3 X r3,y r3 R 4 X r4,y r4 R 5 X r5,y r5 ii) Calculate values listed below. The eye rotation center (Xo,Yo). The pupil center rotation radius (Rp). The cornea curvature center rotation radius (Rc). The length between reflection on cornea and the cornea curvature center in each ais (α, β). iii) Calculate the transformation matri from i) and ii).

17 Step 4) Measurement. The red line indicates view ais angle in the target-coordinate system. The green line indicates y view ais angle in the targetcoordinate system. The blue line indicates pupil radius.

18 Conclusions This system is very inepensive compared with other products. By using an ellipse approimation, this system can measure the eact eye position and pupil diameter even if the pupil is hidden by the eyelid. This system can measure eye position without fiing subject's head. This system can measure subject's head movement. The accuracy of the measurement is better than 1 degree. Sampling frequency is 60Hz(save to file) or 30Hz (D/A converter). This system can be used for human, monkey and mouse.

19 Appendi Ellipse epression : Center of pupil: 2 +ay+by 2 +c+dy+e=0 X p,y p = ay p+c, ac-2d 2 4b-a 2 r1>r2 a0 Pupil radius:r1 miner ais slope a2 r1<r2 a0 Pupil radius:r2 miner ais slope a1 r1>r2 a=0 Pupil radius:r1 miner ais slope r1<r2 a=0 Pupil radius:r2 miner ais slope 0 C 1 =X 2 p+ax p Y p +by 2 p +cx p+dy p +e a0 a 1 = b-1 a + b-1 a r 1 = -C 1 (a2 1 +1) ba 2 1 +aa a 2 = b-1 a r 2 = -C 1 (a2 2 +1) ba 2 2 +aa b-1 a 2 +1 a=0 r 1 = -C 1 r 2 = -C 1 b The rotation center (Xo,Yo). y y The rotation center projected on y plane eists on the etended line of ellipses' minor aes. The subject's head movement must be restricted. The rotation center is calculated from many images by using least squares. y (X o,y o )

20 The pupil rotation radius Rp. The angle between view ais and y plane is arcsin of (minor ais / major ais). The pupil rotation center is calculated from this angle and length between the rotation center projected on y plane (Xo, Yo) and the pupil center projected on y plane (Xp, Yp). The subject's head movement must be restricted. y eyeball (X o,y o ) z R p pupil θ ( X p,y p ) z y 90 Rotate the pupil. pupil z y θ=sin -1 ( minor ais major ais minor ais major ais ) CCD camera R p = (X p-x o ) 2 +(Y p -Y o ) 2 cosθ = (X p-x o ) 2 +(Y p -Y o ) 2 2 minor ais 1- major ais

21 The length between the reflection on cornea and the cornea curvature center in each ais (α,β). α β So α and β are calculated from captured images by using least squares. The subject's head movement must be restricted. α reflection (X r1,y r1 ) α reflection (X r2,y r2 ) β β (X c1,y c1 ) a 1 +b 1 y+c 1 =0 X c1 =X r1 -α Y c1 =Y r1 -β a 1 X c1 +b 1 Y c1 +c 1 =0 (X c2,y c2 ) X c2 =X r2 -α Y c2 =Y r2 -β a 2 X c2 +b 2 Y c2 +c 2 =0 a 1 (X r1 -α)+b 1 (Y r1 -β)+c 1 =0 a 2 (X r2 -α)+b 2 (Y r2 -β)+c 2 =0

22 The cornea curvature center rotation radius Rc. The cornea curvature center rotation radius Rc is obtained from length between the cornea curvature center and the rotation center projected on y plane, and length between the pupil center and the rotation center projected on y plane and the pupil rotation radius Rp. R c = (X c-x o ) 2 +(Y c -Y o ) 2 (X p -X o ) 2 +(Y p -Y o ) R 2 p pupil center (X p,y p ) R pc =R p -R c rotation center (X o,y o ) CCD camera pupil center cornea cornea curvature center cornea curvature center (X c,y c ) R pc R p R c eyeball rotation center

23 The transformation matri. To obtain the transformation matri, which transfers a vector from the camera-coordinate system to the target-coordinate system, we ask the subject to fiate each of the 9 (more than 3 points) targets displayed on a screen. We measure the subject's view ais vector (Cv1-9) in the camera-coordinate system, and calculate the transformation matri from their vectors and their respective required view ais vector (Tv1-9) in the target-coordinate system by using least squares. C C v1 =(C 1,C y1,c z1 ) C v2 =(C 2,C y2,c z2 )... C v9 =(C 9,C y9,c z9 ) T v1 = T v2 =... T v9 = -a a 2 +0+L 2, 0 a 2 +0+L 2, L a 2 +0+L 2 0, a, L 0+a 2 +L 2 0+a 2 +L 2 0+a 2 +L 2 a, -a, L a 2 +a 2 +L 2 a 2 +a 2 +L 2 a 2 +a 2 +L 2