3-D Quantitative Imaging of the Microvasculature with the Texas Instruments Digital Micromirror Device

Transcription

1 Invited Paper 3-D Quantitative Imaging of the Microvasculature with the Texas Instruments Digital Micromirror Device Y. Fainman, E. Botvinick*, J. Price*, D. Gough* Department of Electrical and Computer Engineering, 0407 *Department of Bioengineering University of California at San Diego ABSTRACT There is a growing need for developing 3-D quantitative imaging tools that can operate at high speed enabling realtime visualization for the field of biology, material science, and the semiconductor industry. We will present our 3-D quantitative imaging system based on a confocal microscope built with a Texas Instruments Digital Micromirror Device (DMD). By using the DMD as a spatial light modulator, confocal transverse surface (x, y) scanning can be performed in parallel at speeds faster than video rate without physical movement of the sample. The DMD allows us to programmably configure the source and the detection pinhole array in the lateral direction to achieve the best signal and to reduce the crosstalk noise. Investigations of the microcirculation were performed on 40 g to 45 g golden Syrian hamsters fit with dorsal skin fold window chambers. FITC-Dextran or Red blood cells from donor hamsters, stained with Celltracker CM-DiI, were injected into the circulation and imaged with the confocal microscope. We will present the measured results for the axial resolution, in vivo, as well as experimental results from imaging the window chamber. Keywords: Digital micromirror device, DMD, rapid parallel spot scanning, 3D microvasculature perfusion, in vivo 1. INTRODUCTION Confocal microscopes have become popular tools for the observation of biological specimens due to their depth sectioning ability. They provide images with superior resolution in the transverse axial (depth) direction and can be used to optically section a specimen for 3-D reconstruction. [1-3] Imaging of live specimens often requires laser light to penetrate deep within the specimen. To minimize thermal damage to the live specimen, it is important that laser power and exposure time are minimized. Meanwhile, to acquire high resolution images, light efficiency, scan rate, and collection/integration time must be maximized. We have built a prototype dynamically configurable array confocal microscope for in vivo imaging by using the Texas Instruments Digital Micromirror Device (DMD) to achieve high scan rate confocal imaging with low excitation power and high sensitivity. The confocal condition is achieved by geometrically matching three imaged conjugate focal points. The first point corresponds to a point illumination source defined by a pinhole aperture or coherent laser beam. The second point corresponds to a point source in the sample (whether reflection, scatter, or fluorescence) and the third point corresponds to a point detector defined by a pinhole aperture. Three-dimensional confocal scanning is performed by moving the relative position of these conjugate points with respect to the specimen in the transverse (x,y) and axial (z) directions. Transverse scanning is achieved by either moving the specimen with respect to the focal plane, or by exhaustively steering the excitation light across the focal plane. The specimen can be moved via stage scanning which, while providing the optical advantages of axial illumination, is generally slow. Optical raster scanning is a faster alternative to mechanical stage scanning. While providing higher scan rates, raster scanning exhibits very limited dwell times and low sensitivity (SNR). Without decreasing the scan area, speed can be increased only by sacrificing dwell time and SNR. Alternatively, array methods such as the rotating Nipkow disk, have high scan rates as compared with the methods above, but suffer from low illumination efficiency, and do not have a configurable spot pattern. Slit scanning sacrifices confocality and spatial resolution parallel to the direction of the slit in order to gain SNR and speed. Axial scanning is performed by moving the stage or the objective. Piezoelectric microscope objective micro-positioners are available and allow high-speed movement with high accuracy and repeatability. The rate limiting step in 3-D confocal imaging is the transverse scan. Spatial Light Modulators: Technology and Applications, Uzi Efron, Editor, Proceedings of SPIE Vol (2001) 2001 SPIE X/01/$

2 Previous work has shown that a configurable parallel array confocal microscope can be constructed using the DMD. [4-7] In this article we report the construction of a DMD based confocal microscope for imaging the microvasculature, in vivo. Included are both optical performance data recorded in the tissue and a demonstration of imaging a live specimen. The scan rate, depth of field, transverse resolution and optical efficiency of the system were measured. Then high rate in vivo imaging of blood flow through the microcirculation was demonstrated. 2. MATERIALS AND METHODS Light Source - ON - OFF MicroMirror Dichroic Mirror Objective Lens CCD Sample Fig. 1. Confocal reflection optics with a single micromirror. The micromirror acts as both the illumination and emission pinhole. Fig 2. Reflection spot confocal neighborhood on a micromirror array. White squares represent mirrors in the ON position and gray squares represent mirrors in the OFF position. All OFF mirrors are alternately positioned ON until the image is completely filled in. For a 6 x 6 neighborhood (indicated by dashed lines), the entire field can be scanned by alternating the ON mirror position 36 times Reflection confocal optical principle The original architecture of a confocal microscope uses an illumination point source (created by placing a small pinhole in the path of the illumination source) which is de-magnified by a microscope objective onto the specimen and re-magnified by an identical microscope objective onto a detection pinhole which is geometrically confocal to the point in the specimen and the point source.[8] Figure 1 demonstrates how a single micromirror can also be used to create confocal filtering. An excitation light source is reflected by a dichroic mirror towards a single micromirror. Only light striking the micromirror is reflected towards the objective lens to be focused into a diffraction limited spot in the specimen plane. In turn, only light originating from the focused spot in the specimen, whether fluorescence or reflection, will be focused by the objective lens onto the micromirror and reflected back through the dichroic mirror onto the CCD camera. Light originating from points other than the focal point in the specimen will pass by the micromirror and be rejected by the system Achieving a parallel array of confocal spots on the DMD The Texas Instruments DMD is a 1024 x 768 planar array of 16 x 16 µm 2 mirrors on a 17 µm pitch that are each bistable at ± 10 normal to the chip. [9] With proper alignment in an optical microscope system, the ± 10 positioning is used to direct light onto the sample (ON) or out of the optical system (OFF).[7] In this manner, an array of confocal areas can be configured on 138 Proc. SPIE Vol. 4457

3 the DMD to create a programmable multiple-aperture confocal microscope. [5, 6, 10] To create spot confocal areas, one ON mirror is surrounded by a neighborhood of OFF mirrors as shown in Figure 2. By successively alternating each OFF mirror to the ON position, the entire image is filled in rapidly. For the 6 x 6 neighborhood shown in Figure 2, an 1024 x 768 array of micromirrors has 21,845 neighborhoods and the entire image is filled in after alternating the position of the ON mirror 36 times. This results in a tremendously simplified scanning system with no macroscopically visible moving parts Pinhole configurability One tremendous advantage of the DMD over Nipkow disks, slit scanning, and other fixed-pattern parallel confocal systems is that the reflection pattern can be easily reconfigured. By changing the ON/OFF pattern of mirrors, pinhole size, pinhole spacing, and scan mode can be configured on the fly. Confocal imaging filters out light creating a tradeoff in image quality between sensitivity (SNR) and the degree of confocality. [7] This tradeoff makes it advantageous to increase the size of the pinhole and collect more light for dim specimens. With the DMD, the size of the pinhole can be adjusted by changing the number of adjacent ON micromirrors. Pinhole spacing is also a factor in image quality. Strongly scattering objects may require increasing the OFF mirror spacing between ON mirrors, whereas weakly scattering objects may allow closer ON mirror spacing. [7] With the DMD it is trivial to change pinhole size and spacing by changing the ON/OFF configuration. The DMD can also be used to generate a linear array of ON mirrors with configurable width and spacing to mimic a slitscanning confocal system. Any combination of these patters as well as random patterns can be configured. Bright field inspection of the sample can be achieved by turning on all mirrors or by setting up a second illumination system which utilizes the OFF mirrors for bright field imaging concurrent with confocal imaging. These examples demonstrate a degree of pinhole pattern configurability not possible with any other currently available confocal system (note that this would also be possible with light valve systems) Animal Preparation To image the microvasculature, hamster window chambers were placed on male golden hamsters according to the protocol of Endrich et al.[11] This chamber preparation allows for the direct observation of the microcirculation in conscious animals over prolonged periods of time permitting multi-day longitudinal studies of the microvasculature. [12, 13] An arterial catheter was inserted into the left carotid artery of the hamsters to allow injection of indicator dyes. Prior to imaging, FITC-Dextran (70,000 MW) or CM-DiI stained red blood cells were injected into the circulation via the catheter Optical layout 3. SYSTEM SETUP The optical layout of the DMD-based configurable confocal microscope is shown in Figure 3. Laser light (Melles-Griot Diode Pumped Solid State Green Lasers, 2.5 W) at 532 nm passes through a clean-up filter (532/10 nm Chroma Technology) and is expanded by lens 5 (from Melles-Griot beam-expander set; 5.1mm dia.,mgf 2 coated, f = 9.5 mm) and lens 6 (2 dia.,bk 7 optical glass,bi-convex f = 200 mm, Newport Corporation) to fill the active surface of the DMD. Mirrors 1 and 2 (2 dia., broadband metallic, Newport Corporation) and the dichroic mirror (2 dia. High Quality 565 nm long-pass, Chroma Technology) reflect the expanded laser beam onto the DMD surface. The angle of incidence of the beam onto the DMD must ensure that light reflecting from individual ON micromirrors will be parallel to the optical axis of the microscope. Lens 1 (2 dia, precision achromat f=150 mm, Newport Corporation) and lens 2 (1.5 dia., precision achromat f=100mm, Newport Corporation) telescopically image the DMD onto the image plane of the microscope side port (Nikon Eclipse TE300 inverted microscope). The microscope objective (Nikon 20x/0.40 Ph1 DL working distance 2.1 mm) then focuses the image at the side port into the specimen. Fluorescent emission from the specimen is focused onto the DMD surface by the same optics where it is spatially filtered and reflected towards mirror 2 and the dichroic mirror. The emission passes through the dichroic mirror and is focused by lens 3 (2 dia, precision achromat f=300 mm, Newport Corporation) and lens 4 (2 dia, precision achromat f=150 mm, Newport Corporation) onto the CCD surface. A bandpass filter (1 dia., High Quality 610/75 nm bandpass, Chroma Technology) and longpass filter (1 dia. Orange Glass, 3mm thick, Chroma Technology) are placed in front of the CCD camera ( bit digital, Cohu, 9.9µm pixels ) to remove any excitation light. A spatial filter consisting of a circular aperture and 3 parallel wires was constructed to filter out laser light from the emission path.. Proc. SPIE Vol

4 Fig. 3. Optical System 3.2. Configuring imaging lenses It is critical that lenses 1 and 2 are set such that their magnification ensures an adequate sampling interval in the specimen plane. The distance between confocal spots in the specimen plane defines the sampling interval, or spatial resolution. Shannon s sampling theorem states that to retain the spatial resolution present in the specimen plane, the sampling interval must be no greater that one-half the size of the smallest resolvable feature. [14] The Rayleigh limit of resolution for point objects states that the smallest distinguishable feature is equal to the radius of the Airy disk which is given by R = 0.61λ 0 /NA obj (1) 140 Proc. SPIE Vol. 4457

5 where λ 0 is the wavelength of the light in air, and NA obj is the numerical aperture of the microscope objective. [14] Thus lenses 1 and 2 must ensure that adjacent micromirrors are imaged down into the specimen plane with a spacing of R (see equation 1). Lenses 3 and 4 must be set to ensure that the DMD surface is imaged by the CCD with an adequate sampling interval. Assuming that lenses 1 and 2 have been set correctly, the microscope forms a discrete image of the specimen onto the DMD with 16 x 16 µm 2 pixels (mirrors). If the CCD has a pixel size of L x L µm 2, then lenses 3 and 4 should be set such that the image of each mirror is 2L x 2L µm 2 on the CCD. Or simply, each mirror of the DMD should be imaged to the size of four pixels on the CCD. The DMD was slightly undersampled to increase light intensity on the camera at the expense of some loss to transverse resolution. However for the application of imaging the microvasculature, the limit of transverse resolution was not necessary to achieve. 4. SYSTEM CHARACTERIZATION 4.1. Depth resolution Axial resolution was evaluated by graphical measurement of the full-width half max (FWHM) of the point spread function (PSF). Molecular Probes PS-Speck beads (0.175 µm diameter) were injected into the circulation of a hamster window chamber preparation. Beads were located that were at least 50µm deep within the tissue. Axial resolution was measured by a depth scan of single beads registered with single micromirrors Figure 4 shows the measured axial in vivo PSF. Axial resolution was 3.00 ± 0.16µm (n = 5) Light efficiency Light efficiency is very important in confocal microscopy. The highly periodic structure of the DMD surface creates two distinct diffraction patterns from the excitation light one in the direction of the OFF mirrors and one in the direction of the ON mirrors. Most micromirrors are in the OFF state during a scan and consequently the diffraction pattern in that direction contains most of the excitation light. Additional excitation light is lost to the small back aperture of the objective lens and in the aperture of the Nikon microscope s relay optics. To evaluate the light efficiency in the excitation pathway, illumination power was measured at three locations that can be viewed in Figure 3: between the laser and the DMD, at the side port of the microscope, and in the specimen plane. To measure the efficiency in the fluorescence emission pathway, the microscope halogen lamp was used and power was measured at the microscope side port and at the CCD image plane. Table 1 lists the measured efficiencies. The DMD Efficiency is the ratio of laser power at the side port to laser power before the DMD. Illumination Efficiency is the ratio of laser power at the specimen plant to laser power before the DMD. Emission Efficiency is the ratio of lamp power at the CCD to lamp power at the side port. Normalized Intensity Axial Point Spread Function All Mirrors ON 2x2 pinhole Confocal Depth (um) Fig. 4. PSF of confocal system. The PSF was measured by injecting microspheres into the circulation of a hamster window chamber preparation. The hamster was conscious during scanning. Shown are the PSFs for three imaging modes: 1) all mirrors ON 2) a 2x2 mirror reflective pinhole in a 4x4 mirror neighborhood and 3) a single mirror pinhole in a 2 x 2 mirror neighborhood. Only every 5 th data point is shown. Proc. SPIE Vol

6 Scan Mode DMD Efficiency Illumination Efficiency Emission Efficiency % % % All Mirrors Slit Confocal Table 1. Light efficiency Scan type 1 Hz refresh rate 15 Hz refresh rate 30 Hz refresh rate Raster Scan 3.3 µs 0.2 µsec 0.1 µs Neighborhood size = msec 16.7 msec 8.3 msec Neighborhood size = msec 2 msec 1 msec Table 2. Dwell Time per pixel for raster scanning and for DMD scanning with neighborhoods of 4 and 36 mirror positions. Calculations were for a 640 x 480 pixel 2 region 4.3. Scan rate and dwell time The parallel spot scanning provides both high scan rates and long dwell times. Consider a raster scanning system that scans a 640 x 480 pixel region. To achieve a 1 Hz frame rate, the beam must visit positions each second. This corresponds to, at best, a dwell time of 3.3 µs per pixel for a 1 frame/sec refresh rate. For the DMD confocal system, with a four-position neighborhood, the dwell time per pixel of a 640 x 480 pixel 2 scan is 250 msec. Table2 lists dwell times for different frame rates for the DMD confocal system at different neighborhood sizes and for a raster scan system all scanning a 640 x 480 pixel 2 region. The dwell time of the DMD confocal system is about 3 orders of magnitude longer than the raster scan dwell time. The specimen can be scanned with lower light intensity since the detector is integrating the signal over the increased dwell time. This feature of parallel spot scanning systems facilitate confocal imaging of live specimens. 5. IMAGING THE MICROVASCULATURE No visible damage was caused to the tissue during scanning, even with continuous laser exposure for over 30 s. The depth sectioning ability in vivo of the DMD confocal microscope is demonstrated in figure 5. In figure 5, the hamster was injected with FITC-Dextran (150K MW) to label the blood plasma. Image stacks of the same vessel s lumen were acquired in confocal mode, with a single mirror pinhole in a 2x2 neighborhood, and in nonconfocal mode will all mirrors ON. Y-Z projections through the image stacks at the dashed lines show that the confocal stack preserved the circular nature of the capillary while the nonconfocal stack blurred the capillary cross-section with no clear vessel boundaries. To measure blood flow rate, erythrocytes labeled with Molecular Probes cell tracker CM-DiI were injected into the circulation. 20 frame confocal videos were acquired at 30 frames/sec at various focal depths. Figure 5 shows a maximum projection image of a 20 frame confocal video of erythrocytes flowing in the microcirculation. Image processing, image segmentation and cell tracking algorithms were implemented to quantify cell paths and to measure cell flow rates. In figure 5, the black dots represent calculated cell centroids and the lines connected them are determined by the cell tracking algorithm. 6. CONCLUSIONS 142 Proc. SPIE Vol. 4457

7 Fig. 5. Object tracking. Shown is a maximum projection of a 20 frame confocal movie, enhanced with gamma correction to improve visibility. Black circles represent calculated cell centroids. The celltracking algorithm defined the lines connecting the centroids. Figure 5. Depth sectioning ability iv vivo. Top: Confocal image plane 18 from a 26 image stack with 5 µm steps. To the right is a Y-Z projection from the dotted line through the stack. Bottom: Brightfield (all mirrors ON) image plane 18 from a similar image stack of the same vessel at the same focal planes, and its Y-Z projection. The vessel extent in depth is more distinct (and round) in the confocal stack than in brightfield stack. The high degree of parallelism possible with the DMD makes it extremely compelling for high-resolution confocal biological microscopy. Use of the OFF mirrors can allow for concurrent transmitted and fluorescence microscopy. The high scan rate of the system allows extremely sensitive images of in vivo motion in confocal mode. The OFF mirrors create poor excitation light efficiency and add the requirement for a much higher power laser (typically 1-4 W) source compared to single spot raster scan techniques. This prototype could be further improved by increasing the optical efficiency and by building a controller that allows random access to each individual micromirror. The Texas Instruments controller only allows control of the micromirrors through standard VGA graphics and the mirrors themselves are capable of switching at much higher rates. The long dwell time associated with parallel spot scanning techniques can provided high resolution confocal images deep within living tissue while keeping excitation power below the damage threshold. Even with the current limitations, this new method for very high confocal parallelism creates a compelling combination of speed, sensitivity, and configurability that will allow confocal imaging of intact living biological specimens at substantially faster rates. ACKNOWLEDGMENTS This research was made possible by collaboration between the UCSD Departments of Bioengineering and Electrical and Computer Engineering through the Whitaker Institute of Biomedical Engineering. The work was supported in part by the Defense Advanced Research Projects Agency (DARPA) grant, the National Science Foundation grant BES , the Kangwon National University, and the WPC Research and Education Fund. REFERENCES 1. Bertero, M., et al., Super-resolution in confocal scanning microscopy. III. The case of circular pupils. Inverse Problems, (5): p Hamilton, D.K., T. Wilson, and C.J.R. Sheppard, Experimental observations of the depth-discrimination properties of scanning microscopes. Optics Letters, (12): p Proc. SPIE Vol

8 3. Sandison, D.R., et al., eds. Quantitative fluorescence confocal laser scanning microscopy. 2 ed. Handbook of Biological Microscopy, ed. J. Pawley. 1995, Plenum Press: New York. 4. Verveer, P.J., et al., Theory of confocal fluorescence imaging in the programmable array microscope (PAM). Journal of Microscopy, (pt.3): p Liang, M., R.L. Stehr, and A.W. Krause, Confocal pattern period in multiple-aperture confocal imaging systems with coherent illumination. Optics Letters, (11): p Hanley, Q.S., et al., An optical sectioning programmable array microscope implemented with a digital micromirror device. Journal of Microscopy, (Pt 3)(8): p Cha, S., et al. 3D profilometry using a dynamically configurable confocal microscope. in SPIE Photonics West. 1999: SPIE. 8. Inoue, S., ed. Foundations of Confocal Scanned Imaging in Light Microscopy. 2 ed. Handbook of Biological Confocal Microscopy, ed. J. Pawley. 1995, Plenum Press: New York. 9. Sampsell, J.B., An overview of Texas Instruments Digital MIcromirror Device (DMD) and its application to projection displays. Society for information display internatl. symposium digest of tech. paper, : p Eisner, M., N. Lindlein, and J. Schwider, Confocal microscopy with a refractive microlens-pinhole array. Optics Letters, (10): p Endrich, B., et al., Technical report--a new chamber technique for microvascular studies in unanesthetized hamsters. Research in Experimental Medicine, (2): p Freisenecker, B., A. Tsai, and M. Intaglietta, Capillary perfusion during iscemia-reperfusion in subcutaneous connective tissue and skin muscle. american journal of physiology, : p. H2204-H Sakai, H., et al., Changes in resistance vessels during hemorrhagic shock and resuscitation in conscious hamster model. American Journal of Physiology-Heart and Circulatory Physiology, (2): p. H563-H Inoue, S. and K.R. Spring, Video Microscopy. 1997, New York: Plenum Press. 144 Proc. SPIE Vol. 4457