(12) United States Patent

Transcription

1 USOO894757OB2 (12) United States Patent Silverstein (54) METHOD, APPARATUS, AND SYSTEM PROVIDING ARECTLINEAR PXEL GRID WITH RADALLY SCALED PXELS (71) Applicant: Micron Technology, Inc., Boise, ID (US) (72) Inventor: Amnon Silverstein, Palo Alto, CA (US) (73) Assignee: Micron Technology, Inc., Boise, ID (US) (*) Notice: Subject to any disclaimer, the term of this patent is extended or adjusted under 35 U.S.C. 154(b) by 0 days. This patent is Subject to a terminal dis claimer. (21) Appl. No.: 13/940,479 (22) Filed: Jul. 12, 2013 (65) Prior Publication Data US 2013/ A1 Dec. 12, 2013 Related U.S. Application Data (63) Continuation of application No. 11/739,010, filed on Apr. 23, 2007, now Pat. No. 8,502,898. (51) Int. Cl. H04N3/14 ( ) H04N 5/335 ( ) HO)4N 5/357 ( ) HOIL 27/46 ( ) H04N 5/369 ( ) H04N 5/3745 ( ) HOIL 3L/8 ( ) (52) U.S. Cl. CPC... H04N 5/357 ( ); HOIL 27/14609 (58) (56) JP JP (10) Patent No.: US 8,947,570 B2 (45) Date of Patent: *Feb. 3, 2015 ( ); H04N5/3572 ( ); H04N 5/3696 ( ); H04N 5/3745 ( ); HOIL 3 1/18 ( ) USPC /302:348/315 Field of Classification Search CPC... H04N 5/3696; H04N 5/3572; H04N 5/217; H04N 5/361; H04N 5/3575: H04N 5/335 USPC /251, 241, 311, 302,315 See application file for complete search history. References Cited U.S. PATENT DOCUMENTS 4,683,498 A * 7/1987 Topper ,302 5,736,732 A * 4/1998 Possin et al , ,796,095 A * 8/1998 Matsuyama et al , ,140,630 A 10/2000 Rhodes (Continued) FOREIGN PATENT DOCUMENTS A 2, A 4/2005 (Continued) OTHER PUBLICATIONS International Search Report and Written Opinion dated Jul. 24, 2008 issued in PCT2008/ Primary Examiner Nicholas Giles (74) Attorney, Agent, or Firm Dickstein Shapiro LLP (57) ABSTRACT Pixels in an imaging device pixel array are sized according to their geographic location in the pixel array to compensate for various optical characteristics/issues. In one example, pixel size is increased according to the distance of the pixel from the x-axis and/or the y-axis of the pixel array to correct for lens shading. 20 Claims, 9 Drawing Sheets y-axis X-axis

2 US 8,947,570 B2 Page 2 (56) References Cited 7,227,573 B2 * 6/2007 Stavely , ,408,572 B2* 8/2008 Baxter et al.. 348, U.S. PATENT DOCUMENTS 8,502,898 B2 * 8/2013 Silverstein , / A1* 9, 2003 Karellas et al ,140,980 A * 10/2000 Spitzer et al / A1* 1/2004 Stavely , ,204,524 B1 3/2001 Rhodes 2004/ A1* 5, 2004 Baxter et al ,302 6,310,366 B1 10/2001 Rhodes et al. 2004/ A1 12/2004 Cartlidge et al. 6,326,652 B1 12/2001 Rhodes 2006/ A1 2/2006 Jerdev et al. 6,333,205 B1 12/2001 Rhodes 2006, O A1 7, 2006 Hwan 6,376,868 B1 4/2002 Rhodes wang 6,455,831 B1* 9/2002 Bandera et al , / A1* 10, 2008 Silverstein ,308 6,563,101 B1* 5/2003 Tullis , ,636,185 B1 * 10/2003 Spitzer et al FOREIGN PATENT DOCUMENTS 6.738,057 B1 5/2004 Campbell 6,980,244 B1* 12/2005 Yonemoto et al /302 JP A 4, ,075,501 B1* 7/2006 Spitzer et al JP A 6, ,084,904 B2 * 8/2006 Liu et al ,106,374 B1* 9/2006 Bandera et al ,308 * cited by examiner

3 U.S. Patent Feb. 3, 2015 Sheet 1 of 9 US 8,947,570 B2 NN 32 FIG. 1 (PRIOR ART)

4 U.S. Patent Feb. 3, 2015 Sheet 2 of 9 US 8,947,570 B2 104 FIG. 3 (PRIOR ART)

5 U.S. Patent (6% s %

6 U.S. Patent Feb. 3, 2015 Sheet 4 of 9 US 8,947,570 B2 FIG. 5 FIG. 6

7 U.S. Patent Feb. 3, 2015 Sheet 5 of 9 US 8,947,570 B2 X-axis y-axis H I EH 304 FIG

8 U.S. Patent Feb. 3, 2015 Sheet 6 of 9 US 8,947,570 B2

9 U.S. Patent Feb. 3, 2015 Sheet 7 of 9 US 8,947,570 B2

10 U.S. Patent Feb. 3, 2015 Sheet 8 of 9 US 8,947,570 B ROW ROW DECODER

11 U.S. Patent Feb. 3, 2015 Sheet 9 Of 9 US 8,947,570 B2 --- S2 g f > O O s r ama 1.

12 1. METHOD, APPARATUS, AND SYSTEM PROVIDING ARECTILINEAR PXEL GRID WITH RADALLY SCALED PXELS CROSS-REFERENCE TO RELATED APPLICATIONS This is a continuation of application Ser. No. 11/739,010, filed Apr. 23, 2007, which is incorporated by reference in its entirety. FIELD OF THE INVENTION Generally, embodiments disclosed herein relate to adjust ing the outputted signals of imaging pixels according to their geographic location inapixel array to compensate for various optical problems. BACKGROUND A CMOS imager circuit includes a focal plane array of pixels, each one of the pixels including a photosensor, for example, a photogate, photoconductor or a photodiode over lying a substrate for accumulating photo-generated charge in the underlying portion of the Substrate. Each pixel has a readout circuit that includes at least an output field effect transistor formed in the Substrate and a charge storage region formed on the Substrate connected to the gate of an output transistor. The charge storage region may be constructed as a floating diffusion region. Each pixel may include at least one electronic device Such as a transistor for transferring charge from the photosensor to the storage region and one device, also typically a transistor, for resetting the storage region to a predetermined charge level prior to charge transference. In a CMOS imager, the active elements of a pixel perform the necessary functions of: (1) photon to charge conversion; (2) accumulation of image charge; (3) resetting the storage region to a known state before the transfer of charge to it; (4) transfer of charge to the storage region accompanied by charge amplification; (5) selection of a pixel for readout, and (6) output and amplification of a signal representing pixel charge. Photo charge may be amplified when it moves from the initial charge accumulation region to the storage region. The charge at the storage region is typically converted to a pixel output Voltage by a source follower output transistor. FIG. 1 shows a top down view of an individual four-tran sistor (4T) pixel 10 of a CMOS imaging device. A pixel 10 generally comprises a transfer gate 50 for transferring photo electric charges generated in a photosensor 21 (shown as a pinned photodiode) to a floating diffusion region FD acting as a sensing node, which is in turn, electrically connected to the gate 60 of an output source follower transistor. A reset gate 40 is provided for resetting the floating diffusion region FD to a predetermined Voltage in order to sense a next signal, and a row select gate 80 is provided for outputting a signal from the Source follower transistor to an output terminal in response to a pixel row select signal. The various transistors are coupled to each other via their source/drain regions 22 and coupled to other elements of the imaging device via the contacts 32. CMOS imagers of the type discussed above are generally known as discussed, for example, in U.S. Pat. No. 6,140,630, U.S. Pat. No. 6,376,868, U.S. Pat. No. 6,310,366, U.S. Pat. No. 6,326,652, U.S. Pat. No. 6,204,524 and U.S. Pat. No. 6,333,205, assigned to Micron Technology, Inc., which are hereby incorporated by reference in their entirety. FIG. 2 shows a grid layout for a conventional pixel array 100 in which all of the pixels and their respective photosen US 8,947,570 B sors are identically sized. Because the pixels are identically sized and spaced, the lines connecting the pixels to circuitry (not shown) located on the periphery of the pixel array 100 may also be identically spaced. It is known that pixels in various spatial locations in the pixel array 100 have different levels of output signals for the same level of light illumina tion. Optical image formation is not spatially shift invariant; that is, more light is transmitted to pixels located in-line with the optical axis, such as pixels 102 located at or near the center of the pixel array 100, than is transmitted to pixels 104 not in-line with the optical axis. In general, pixels may experience a cosine roll off in incident light intensity the greater their distance from the optical axis. Therefore, pixels located out of line with the optical axis, such as corner pixels 104, will receive less light than pixels located directly in-line with the optical axis, such as center pixels 102. This phenomenon is known in the art as lens shading and may cause an image generated by the pixel array 100 to be noticeably darker in the corners than in the center. FIG.3 shows, by shading, the amount of light captured and displayed as brightness by each of the pixels in the pixel array shown in FIG. 2. The darker shading indicates less light being captured by the pixels farther away from the center of the pixel array than pixels closer to the center of the pixel array. Conventional CMOS imaging devices have attempted to correct lens shading during post-processing of already-ac quired image data or during image acquisition (i.e., as the image is read out from the imaging device). There is a need and desire in the art for additional image correction methods that do not require post-readout image processing or special image acquisition techniques to compensate for lens shading. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 shows a top down view of a pixel of a CMOS imaging device. FIG. 2 shows a grid for a conventional pixel array. FIG.3 depicts the amount of light captured and displayed by the pixels of the pixel array of FIG. 2. FIG. 4 shows a top down view of pixels of various sizes according to an embodiment described herein. FIG. 5 shows a grid for a pixel array according to an embodiment described herein. FIG. 6 shows a grid for a pixel array according to an embodiment described herein. FIG. 7 shows a grid for a pixel array according to an embodiment described herein. FIG. 8 shows a grid for a pixel array according to an embodiment described herein. FIG.9 depicts the amount of light captured and displayed by the pixels of the pixel array of FIG. 8. FIG. 10 shows a grid for a pixel array according to an embodiment described herein. FIG. 11 illustrates a block diagram of a CMOS imaging device constructed in accordance with an embodiment described herein. FIG. 12 depicts a processor System, for example, a camera system constructed in accordance with an embodiment described herein. DETAILED DESCRIPTION In the following detailed description, reference is made to the accompanying drawings which form a part hereof, and in which are shown by way of illustration of specific embodi ments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those

13 3 of ordinary skill in the art to make and use them, and it is to be understood that structural, logical or procedural changes may be made to the specific embodiments disclosed herein. Various embodiments described herein modify the size of pixel photosensors according to their spatial location in a pixel array to compensate for different spatial output signal levels. The embodiments described can thus correct for spa tial optical effects such as lens shading, as well as other spatial optical effects. As discussed above, pixels located farther away from the optical axis of a lens in an imaging device may experience decreased light intensity, for example, caused by lens shad ing. One embodiment herein mitigates against this by increas ing the size of the pixel, and thus the size of the photosensor, according to the pixel s geographical distance form the opti cal axis in the pixel array. Although this disclosure refers to modifying pixel size, it is to be understood that what is most important is the size of the photosensor in accordance with spatial location of a pixel in the array. Accordingly, when the discussion herein refers to changes in the size of the pixel, it is understood that the size of the photosensor changes proportionately to the change in the pixel size. FIG.4, for example, shows an embodiment that may be used to correct lens shading in which a smaller pixel 210 with a proportionately smaller photosensor 221 may be located near the optical axis of a pixel array, and a larger pixel 310 with a proportionately larger photosensor 321 may be located farther away from the optical axis. Alternatively, the size of the photosensor could be changed in accordance with spatial location while keeping the overall size of the pixels COnStant. In one embodiment, the pixels may be arranged in the pixel array in a rectilinear grid. The pixels may be generally rect angular in shape or may comprise various other shapes. The size of the pixels and thus the size of the photosensor may be increased according to the distance of the pixels from the optical axis in the x-direction and/or the y-direction. This allows for differently sized pixels according to their geo graphic location, while maintaining a rectilinear pixel array. Furthermore, lines connecting the pixels to appropriate cir cuitry may be formed in straight lines, which may be parallel to the X-axis or the y-axis, and in which simplifies the con struction of the pixel array. FIG. 5 shows an embodiment in which the length of the sides 302 of the pixels parallel to the x-axis increases accord ing to the distance of the pixels from the y-axis of the pixel array. FIG. 6 shows an embodiment in which the length of the sides 304 of the pixels parallel to the y-axis increases accord ing to the distance of the pixels from the x-axis of the pixel array. FIG. 7 shows an embodiment in which the length of the sides 302 of the pixels parallel to the x-axis increases accord ing to the distance of the pixels from the y-axis of the pixel array and the length of the sides 304 of the pixels parallel to the y-axis increase according to the distance of the pixels from the x-axis of the pixel array. In FIG. 7, it can be seen that the pixels increase in size according to their distance from the center of the pixel array. In another embodiment, the pixels may be curved and may be arranged in a curved pixel array, as illustrated in FIG. 10. The relationship between the length of the sides of the pixels and the distance from the X-axis and/or y-axis of the pixel array may be determined according to the spatial optical problem to be corrected. As shown in the FIG. 8 embodiment, and as discussed below, to correct lens shading, the length of the sides of the pixels parallel to the X-axis may increase according to the distance of the pixels from the y-axis of the pixel array in accordance with a Gaussian normal distribution US 8,947,570 B curve and/or the length of the sides of the pixels parallel to the y-axis increases according to the distance of the pixels from the X-axis of the pixel array according to a Gaussian normal distribution curve. As shown in FIG. 8, The Gaussian normal distribution curve may be the same for the X-axis and the y-axis and thus forming a radially symmetric two dimensional curve. Alter natively, the Gaussian normal distribution curves for the length of the pixel sides parallel to the x-axis and the length of the pixel sides parallel to the y-axis may be different Gaussian normal distribution curves. FIG. 8 shows an example sampling grid with Gaussian step sizes for the sides of the pixels parallel to the x-axis and the y-axis. In this embodiment, the sides of the pixels parallel to the x-direction are 1.7 microns for the center pixels 602 and are 2.2 microns for the corner pixels 604. FIG. 9 shows the brightness output as a result of the light captured by each pixel in the pixel array. It can be seen from the grid that the bright ness does not vary greatly with distance from the center of the pixel array because of the pixel sizing illustrated in FIG. 8. For rectilinear pixel layout grids such as those shown in FIGS. 5, 6, 7, and 8, lines 306 (FIGS. 5, 6, 7, and 8) for connecting rows of pixels in the pixel array to circuitry Such as to the row driver 510 shown in FIG. 11, may be arranged Substantially parallel to each other and may be spaced apart from each other according to the various pixel sizing schemes. Similarly, lines 308 (FIGS. 5, 6, 7, and 8) for con necting columns of pixels in the pixel array to circuitry. Such as the column driver 520 shown in FIG. 11, may be arranged Substantially parallel to each other and may be spaced apart from each other according to the various pixel sizing schemes. The lines 306 for connecting rows of pixels in the pixel array to circuitry may be substantially perpendicular to the lines 308 for connecting columns of pixels in the pixel array to circuitry. The fabrication of the pixel array may be simplified by forming the various connecting lines in a recti linear grid. Sampling with a non-uniform pixel location grid may cre ate spatial distortion when the signals from the pixels are assembled to form a digital image. For example, the image may appear stretched near the center of the image because the pixels near the center of the image are Smaller than the pixels near the edges of the image. One method that may be used to correct the spatial distortion is digital resampling. The spatial distortion may also be corrected using an optical solution, Such as using a lens to correct the distortion. Alternatively, if the spatial distortion is Small enough, it may be ignored. As noted, in addition to changing the size of the pixels in order to change the size of the associated photosensors, an embodiment also keeps the pixel size the same throughout the array, but changes the size of the photosensors within the pixels in accordance with a spatial location of the pixels in an array. FIG. 11 illustrates a block diagram of a CMOS imaging device 500 having a pixel array505 arranged according to an embodiment described above. The pixels of each row in array 505 are all turned on at the same time by a row select line, and the pixels of each column are selectively output onto the output lines by respective column select lines. A plurality of row and column select lines are provided for the entire array 505. The row lines are selectively activated in sequence by a row driver 510 in response to row address decoder 515. The column select lines are selectively activated in sequence for each row activation by a column driver 520 in response to column address decoder 525. Thus, a row and column address is provided for each pixel.

14 5 The CMOS imaging device 500 is operated by a control circuit 530, which controls address decoders 515, 525 for selecting the appropriate row and column select lines for pixel readout. Control circuit 530 also controls the row and column driver circuitry 510,520 so that they apply driving voltages to the drive transistors of the selected row and column select lines. The pixel output signals typically include a pixel reset signal Virst read out of the pixel storage region after it is reset by the reset transistor and a pixel image signal Visig, which is read out of the pixel storage region after photo-generated charges are transferred to the region. The Virst and Visig sig nals are sampled by a sample and hold circuit 535 and are subtracted by a differential amplifier 540, to produce a dif ferential signal Vrst-Visig for each readout pixel. Vrst-Vsig represents the amount of light impinging on the pixels. This difference signal is digitized by an analog-to-digital converter 545. The digitized pixel signals are fed to an image processor 550 to form a digital image output. The digitizing and image processing can be located on or off the chip containing the pixel array 505. In some arrangements the differential signal Vrst-Visig can be amplified as a differential signal and directly digitized by a differential analog to digital converter and then fed to image processor 550, which assembles an image for output. Resampling of the image to correct image distortion caused by pixel arrays in accordance with embodi ments described herein may also be performed by the image processor 550. FIG. 12 shows a processor system 600, for example, a digital camera system, which includes an imaging device 500 constructed to include a pixel array arranged and operated in accordance with an embodiment described herein. The pro cessor System 600 is an example of a system having digital circuits that could include image sensor devices. Without being limiting, in addition to a digital camera system, such a system could include a computer system, Scanner, machine vision system, vehicle navigation system, video phone, Sur veillance system, auto focus system, star tracker system, motion detection system, image stabilization system, and other processing systems employing an imaging device 500. System 600, for example a camera system, generally com prises a central processing unit (CPU) 610. Such as a micro processor, that communicates with an input/output (I/O) device 640 over a bus 660. Image sensor 630 also communi cates with the CPU 610 over the bus 660. The system 600 also includes random access memory (RAM) 620, and can include removable memory 650, such as flash memory, which also communicate with the CPU 610 over the bus 660. Image sensor 630 may be combined with a processor, such as a CPU 610, digital signal processor, or microprocessor, in a single integrated circuit. In a camera, a shutter release button 670 is used to operate a mechanical or electronic shutter to allow light to pass through a lens 675 to the pixel array 505 of imaging device 200. While embodiments of the invention have been described in detail in connection with embodiments known at the time, it should be readily understood that they are not limited to the disclosed embodiments. Rather, they can be modified to incorporate any number of variations, alterations, Substitu tions or equivalent arrangements not heretofore described. For example, while embodiments are described in connection with a CMOS pixel imaging device, they can be practiced with any other type of pixel imaging device (e.g., CCD, etc.) or image source. Furthermore, the pixel sizing of the various embodiments could be used in other applications that require modified pixel sizes, such as in a display projector in which US 8,947,570 B bigger pixels located away from the center of the pixel array could be used to correct spatial distortion caused by the pro jector lens. In addition, although various embodiments have been described as being useful to correct lens shading, any spa tially varying image problem that can be corrected by chang ing pixel size in relation to pixel distance from a predefined area can be corrected. Such as barrel distortion or fish-eye lens. In one embodiment, each row and each column may have an arbitrary scale factor. Furthermore, if certain rows and/or columns of pixels have different characteristics, for example, due to the layout of the lines, transistors, or color filters, the row height or column width may be adjusted accordingly. By modifying the width of the columns and the height of the rows, various sizes of the pixels in the pixel array can be achieved, and the pixel array can include larger or Smaller pixels in various places to form a symmetric or asym metric pattern. The invention claimed is: 1. An imaging device, comprising: a pixel array comprising imaging pixels in which pixels located farther from an optical axis of the pixel array have a larger photosensitive area than a photosensitive area of pixels located closer to the optical axis of the pixel array, wherein the photosensitive area of the pixels increases with distance from the optical axis according to a normal distribution curve configured to correct pixel signal loss imposed on the pixel array by an associated lens. 2. The imaging device of claim 1, wherein the pixel array comprises a rectilinear grid. 3. The imaging device of claim 1, wherein the overall area of each pixel is Substantially the same. 4. The imaging device of claim 1, wherein the overall area of the pixels increases with distance from the optical axis and wherein the photosensitive area increases in proportion to the increase in overall area of the pixels. 5. The imaging device of claim 1, wherein the pixel array has an X-axis and a y-axis and wherein a length of a side of the pixels parallel to the X-axis increases according to the dis tance of the pixels from the y-axis. 6. The imaging device of claim 5, wherein the length of the side of the pixels parallel to the X-axis increases according to the distance of the pixels from the y-axis according to a Gaussian normal distribution curve. 7. The imaging device of claim 1, wherein the pixel array has an X-axis and a y-axis and wherein a length of a side of the pixels parallel to the y-axis increases according to the dis tance of the pixels from the X-axis. 8. The imaging device of claim 7, wherein the length of the side of the pixels parallel to the y-axis increases according to the distance of the pixels from the X-axis according to a Gaussian normal distribution curve. 9. The imaging device of claim 1, wherein said array has an X-axis and a y-axis and wherein a length of a side of the pixels parallel to the X-axis increases according to the distance of the pixels from the y-axis and a length of a side of the pixels parallel to the y-axis increases according to the distance of the pixels from the X-axis. 10. The imaging device of claim 9, wherein the length of the side of the pixels parallel to the X-axis increases according to the distance of the pixels from the y-axis according to a Gaussian normal distribution curve and the length of the side of the pixels parallel to the y-axis increases according to the distance of the pixels from the X-axis according to a Gaussian normal distribution curve.

15 7 11. An imaging processing System comprising: a processor; a pixel array; and an optical system for delivering an image to said pixel array, said optical system providing less light to photo sensors of pixels near a periphery of said array than to photosensors of pixels near a center of said array; wherein said photosensors of said pixels near a periphery of said array are larger than photosensors of said pixels near a center of said array and the photosensitive area of the pixels increases with distance from the optical axis according to a normal distribution curve configured to correct pixel signal loss imposed on the pixel array by an associated lens. 12. The image processing system of claim 11, further com prising a row driver and a column driver, wherein lines con necting rows of pixels to the row driver are parallel to each other, and wherein lines connecting columns of pixels to the column driver are parallel to each other. 13. A method of manufacturing an imaging device com prising: forming a pixel array comprising imaging pixels in which pixels located farther from an optical axis of the pixel array have a larger photosensitive area than a photosen sitive area of pixels located closer to the optical axis of the pixel array, wherein the photosensitive areas of the pixels are formed so that they increase with distance from the optical axis according to a normal distribution US 8,947,570 B curve configured to correct pixel signal loss imposed on the pixel array by an associated lens. 14. The method of claim 13, wherein said array has an X-axis and a y-axis and wherein a length of a side of the pixels parallel to the X-axis is formed to increase according to the distance of the pixels from the y-axis and the length of a side of the pixels parallel to the y-axis is formed to increase according to the distance of the pixels from the X-axis. 15. The method of claim 14, wherein the length of the side of the pixels parallel to the x-axis is formed to increase according to the distance of the pixels from the y-axis accord ing to a Gaussian normal distribution curve and the length of the side of the pixels parallel to the y-axis is formed to increase according to the distance of the pixels from the X-axis according to a Gaussian normal distribution curve. 16. The method of claim 13, wherein the photosensitive areas of the pixels are formed to increase in proportion to an increase in total area of the pixels. 17. The imaging device of claim 1, wherein the pixel signal loss represents a spatial distortion. 18. The imaging device of claim 1, wherein the pixel signal loss represents a barrel distortion or a fish-eye distortion. 19. The image processing system of claim 11, wherein the pixel signal loss represents a spatial distortion, barrel distor tion, or fish-eye distortion. 20. The method of claim 13, wherein the pixel signal loss represents a spatial distortion, barrel distortion, or fish-eye distortion.