Programmable array microscopy with a ferroelectric liquid-crystal spatial light modulator

Programmable array microscopy with a ferroelectric liquid-crystal spatial light modulator Patrick J. Smith, Cian M. Taylor, Alan J. Shaw, and Eithne M. McCabe We present a programmable array microscope that uses a ferroelectric liquid-crystal spatial light modulator SLM for dynamic generation of scanning apertures. A single SLM serves as both the source and the detector aperture array in a double-pass confocal system. Successive aperture frames scan the array across the viewing area for complete imaging of a sample while preserving depth discrimination. Integration of the microscope output across all aperture frames produces a confocal image. 2000 Optical Society of America OCIS codes: 180.1790, 230.6120. 1. Introduction Multiple-aperture confocal microscopes occupy the middle ground between conventional and confocal microscopy. A conventional microscope has effectively an infinite source aperture, an infinite detector aperture, and images the entire field of view simultaneously. An idealized confocal microscope has a point source, a point detector, and images a diffraction-limited point on the sample. 1 The confocal microscope must have some means of scanning the imaged point to build up a picture of the sample, and this carries a time penalty; however, the confocal arrangement confers depth resolution, which the conventional microscope does not. Multiple-scanning apertures offer a trade-off between these extremes. Some depth resolution is sacrificed, but a full image can be gained in a shorter time. 2 The best-known technique for multiply scanning confocal apertures is the spinning Nipkow Disk, a mechanical method of scanning an entire plane which dates from 1890 and was suggested for use in confocal microscopy in 1968. 3 In recent years it has become possible to perform a high-resolution, high-speed, multiaperture scan without mechanical scanning devices by use of a spatial light modulator SLM an electrically controlled two-dimensional modulation device; this idea was, for example, disclosed by IBM in 1993. 4 Such a device The authors are with the Department of Physics, Trinity College Dublin, Dublin 2 Ireland. Received 10 February 2000. 0003-6935 00 162664-06$15.00 0 2000 Optical Society of America allows an aperture function to be programmed into the system, hence the term programmable array microscopy PAM. 5 Not only does this promise a scanning microscope with no moving parts, it allows for arbitrary, novel aperture functions and scanning schemes under software control. Various SLM technologies exist, which make use of different techniques of encoding an electrical signal into a light beam; two that have been used to date for PAM are the Texas Instruments Digital Micromirror Device 6 and a twisted-nematic liquid-crystal SLM. 7 We use instead the Displaytech ferroelectric liquidcrystal FLC SLM 8 the first such application of this device to our knowledge. All these SLM s are fast, high-resolution modulators; the Digital Micromirror Device has the advantage in contrast ratio, but has not been generally available as an original equipment manufacture product. The FLC SLM has a more interesting modulation behavior; in particular, although here we consider it as an amplitude modulator, it can be used in a phase-only modulation mode. It also has a speed advantage over a twisted-nematic device. 2. Spatial Light Modulator Details The FLC SLM uses a birefringent crystal FLC layer with two stable optic axis orientations. An array of pixel mirrors behind the FLC make the device reflective; and the controlling circuitry allows the mirrors, and hence the FLC, to be electrically addressed. Each pixel acts as a reflective half-wave retardation plate with two electrically selectable optic axis directions 45 apart. Incident light that has been plane polarized in one of these optic axis directions will be reflected either unchanged or plane polarized at 90, 2664 APPLIED OPTICS Vol. 39, No. 16 1 June 2000

Fig. 1. PAM system, showing the reflective FLC SLM in its off-axis orientation. depending on the FLC state at a given pixel. Therefore the SLM is a binary modulator when operated between crossed or parallel polarizers. We use crossed polarizers so that the FLC state whose optic axis is aligned with the incident light results in the reflected light being blocked by the second polarizer. This is referred to as the off state of a SLM pixel. The other FLC state rotates the light polarization so that the reflected light is passed by the second polarizer, and it is referred to as the on pixel state. Unlike the Digital Micromirror Device, the modulated light is not angularly separated from the incoming beam when the SLM is normally illuminated. We set the SLM at an angle to the incoming beam so as to reflect the modulated light off axis toward a microscope objective, as shown in the system diagram in Fig. 1. This raises the question of how the FLC s modulation behavior suffers for off-axis illumination angles. A calculation based on a model of the FLC as a tilted array of optically active molecules 9 yields the transmission versus incidence-angle graph shown in Fig. 2 for the on-pixel state. The off state is not affected by the tilt. It can be seen that there is little modulation loss for angles less than 30 ; this allows us to angle the SLM at 22.5 and hence angle the second arm of the system at 45 to the first. To prevent degradation of the FLC, the system employs a so-called charge balancing: After the display of a useful frame, the SLM must display a reverse-video inverse version of the same frame for an equal time to keep the net charge across the FLC zero. It is necessary therefore to optically sample the display cycle of the SLM to exclude this inverse frame, as well as the frame update and crystal switching period. For this sampling we use continuous laser illumination but shutter the microscope output with a separate FLC shutter synchronized with the SLM display cycle. The Displaytech SLM has a 256 256 pixel active area, of side 3.84 mm. Pixel pitch is 15 m, and pixel mirrors cover 87% of the active area. The contrast of the device is quoted as 100:1 between crossed polarizers. The rate at which frames can be displayed on the SLM ranges from 2 Hz to 2.5 khz, and up to 128 frames can be held in the interface memory. Higher-resolution up to extended graphics array 1024 768 FLC miniature displays are available. However, the Displaytech 256 2 SLM is not addressed with a video signal and allows low-level pixel and frame-rate control. This makes it a versatile development tool. Its pixel and hence PAM aperture size is equal to that of larger SLM s. 3. Programmable Array Microscope System A confocal microscope system is usually folded into a reflective arrangement, in which a single objective lens serves to demagnify the illumination onto, and collect the reflection from, the sample of interest. 1 June 2000 Vol. 39, No. 16 APPLIED OPTICS 2665

Fig. 2. Transmission of a FLC modulator against incidence angle. This also means that a single aperture plane can serve as the source and detector, if placed at the front focal plane of the objective. In the PAM, this means that only one SLM is required for confocal behavior. Our objective lens has an infinity-corrected tube length, so an additional collimating lens is required between it and the SLM. This is shown in Fig. 1, which also shows the locations of the three polarizers that allow the SLM to function as a binary amplitude modulator. The first polarizer aligns the incoming light polarization with one of the FLC axis directions; the second is crossed to block the light reflected from SLM off pixels; the third is parallel to the first to block light that has not been rerotated into the original direction. The result is that the only light that passes all three has been through the SLM on pixels in both directions that is, the light has passed through the source and detector aperture just as in any confocal microscope. There are two SLM states, and the SLM is passed twice. Thus there are four cases of light path to deal with. The fate of each is summarized in Fig. 3. We assume that polarization is preserved, or affected in a spatially uniform way, through the reflection from the sample. Of course for some samples this will not be the case, and it is possible that this effect could be used to image sample birefringence. The FLC shutter can be placed in a variety of positions in the system to achieve the same result. It is actually placed immediately before the integrating camera for convenience; in this position the system behavior can be investigated before shuttering is done. 4. Scanning Aperture Arrays The choice of parameters for the aperture array patterns that are displayed on the SLM is determined by several factors. The contrast ratio of the modulator sets a lower bound on the fraction of the device area turned on in each frame, assuming that total coverage is required when integrating across all frames. This is because the integrating step sums the background contributions from off pixels; this sum must be less than a single on-pixel intensity for a confocal final image. Therefore, for a 100:1 contrast modulator, the fraction of on pixels must be 0.01 of the device area. For a square array of apertures, this means that the aperture separation must be less than 10 times the aperture diameter. This is a realistic level of contrast to expect. Each aperture is a block of on pixels and so is rectangular with sides that are multiples of 15 m. The number of frames is ultimately limited by the SLM interface memory, which can hold 128 frames. The SLM frame rate can vary from 2 Hz up to 2.5 Fig. 3. Light paths through a double-pass SLM system and their fate with polarizers set as in Fig. 1. 2666 APPLIED OPTICS Vol. 39, No. 16 1 June 2000

Fig. 4. Four frames of a 4:1 spaced set of 2 2 pixel apertures in a8 8 array. There are 16 frames in all for full coverage. khz, with every second frame contributing to the final image because of charge balancing. However, as the frame rate is increased, the fixed time intervals required for update and switching become more significant, and the useful frame is displayed for a smaller fraction of the cycle. For this reason it is desirable to keep the frame rate as low as is compatible with a uniform illumination in the integrated image. We tried aperture spacings of 4:1, 6:1, and 8:1 using n n arrays of square apertures of side 1, 2, or 4 pixels. Some example frames from a set of arrays of scanned apertures are shown in Fig. 4. In practice these 2 2 pixel apertures were later replaced by single-pixel apertures to improve the microscope resolution, once single-pixel alignment was achieved. Later arrays also used the full area of the SLM. 5. Experiment We have two different methods of judging the performance of the PAM. First, we image the PAM output as a reflective test sample is scanned axially through the focus of the objective; the confocal behavior is evident in a depth selectivity in the captured image. A series of pictures taken at different axial sample positions show a bright image forming, then disappearing as different parts of the object pass through focus. Height variation in the sample causes surface features to be optically sectioned. The test sample used was a piece of ordinary silicon VLSI. Captured confocal images of the latter are shown in Fig. 5 in comparison with a conventional microscope picture taken with the entire SLM area turned on. The pictures show different areas of the sample at maximum brightness for different axial positions because of the confocal operation. However, the background image remains visible, though defocused, throughout. It would be possible therefore for us to extract depth information by performing a range-by- Fig. 5. Captured images of a silicon VLSI sample scanned axially through the PAM focus. The sample area is approximately 200 m 200 m. a d show confocal images at progressive axial positions, spaced 2 m apart. Sample features become highlighted as they move through focus. maximum algorithm and also to build a single autofocus image. Second, we measure the total light throughput of the PAM as a plane mirror is axially scanned in a similar fashion the well-known axial resolution measurement of a confocal microscope. This yields a sharply peaked curve whose full width at halfmaximum FWHM quantifies the depth sectioning capability. Axial scanning is achieved with use of a piezoelectric actuator that moves the sample through a 20- m range. This is the only moving part during the operation of our microscope. To measure the FWHM, the background light level is first subtracted from the curve. This is the light level measured when the sample is defocused, and in our case it arises partly from the leakage of light through the SLM off state and the polarizers. This gives a measure of the depth sectioning capability, but neglects the detrimental effect of a highbackground on-microscope operation. Accordingly, we also give a measure for dynamic range, which we take to be the difference between the maximum light throughput and the background, as a fraction of that maximum level. Because the maximum is conventionally normalized to 100%, this is easily deduced. Figure 6 illustrates the relationship between these quantities. Actual FWHM and dynamic range measurements for single-pixel aperture arrays are shown in Table 1. Graphs of the light throughput versus axial position of a plane mirror, after background subtraction, are shown in Fig. 7. It can be seen that the axial resolution improves the peak becomes narrower for a higher aperture separation, as we would expect. However, the dynamic range as defined above drops because of a higher proportion of background light from off pixels reaching the integrated output. That 1 June 2000 Vol. 39, No. 16 APPLIED OPTICS 2667

Fig. 6. Relationship of FWHM, background, and dynamic range in an axial resolution measurement. Fig. 7. Measured axial resolution of the PAM with various singlepixel aperture spacings. is, the smaller proportion of SLM pixels that are being used to form the confocal image results in this confocal signal being progressively swamped by nonconfocal leaked light. This means that in captured images, there is a loss of distinction between the confocally sectioned features and the conventional image that leaks through the dark SLM. The measured FWHM values are comparable with, but larger than, previous experimental multiaperture results. 2 Our relatively low aperture spacing and extensive up to 64 64 array would both tend to reduce the dynamic range. However, Fewer et al. reported a measured FWHM of only 1.4 mfora10 10 array of 20- m pinholes at a 10:1 separation, which actually decreases at the cost of dynamic range for lower aperture spacings, or larger arrays, after background subtraction. 2 The inferior axial resolution of the SLM microscope probably stems from a defocus across the SLM array caused by the off-axis system configuration. It may also partly be a consequence of the SLM contrast, which is low relative to that of the static metal-etch mask used by Fewer et al. A more valid comparison than this can be made against other dynamically scanned array microscopes, such as those alluded to above that are based around a micromirror array. Hanley et al. 10 report a FWHM from their micromirror-based PAM of 0.5 m using an oil-immersion objective of N.A. 1.4 and an array of 34- m-square apertures. They note that the FWHM should be expected to scale roughly with the square of N.A., from which it follows that this Table 1. Measured Axial Resolution and Useful Response Range of the PAM for Single-Pixel Aperture Arrays of Varying Spacing Mark Space Ratio FWHM m Dynamic Range % 1:4 4.2 27 1:6 3.0 12 1:8 2.6 11 corresponds to a FWHM of 2.7 m with our 0.6 N.A. objective. The apertures used were of approximately twice the width of those we considered, so it is difficult to compare this directly with our findings, yet even so broad an agreement with the results in Table 1 can be inferred. The pupil function used in the theory to describe an aperture is idealized, with zero transmission outside the aperture. This corresponds more closely to an actual pinhole than to the pupil created by a SLM on pixel, which has a nonzero background level. Further theoretical research whereby a more realistic pupil function is fed into the theory will, we hope, explore the reason for this loss of resolution and also predict the associated rise in background. 6. Summary We have built and demonstrated a confocal microscope with a programmable aperture and detector plane consisting of a FLC SLM a programmable array microscope variant. Our system uses a double pass through the SLM, which is tilted off axis. The confocal nature of the system has been shown by its depth-sectioning ability. The main limiting factor of the FLC SLM for PAM is the relatively low achievable contrast of the device. We have demonstrated that more widely spaced aperture arrays result in improved depth resolution, but a loss of dynamic range because of the limited contrast. Depth information is recoverable, even though the out-of-focus parts of a sample cannot be completely visually suppressed. The authors acknowledge correspondence with Quentin Hanley and others at the Max Planck Institute for Biophysical Chemistry with regard to their paper published in the Journal of Microscopy. References 1. T. Wilson, Confocal Microscopy Academic, London, 1990. 2. D. T. Fewer, S. J. Hewlett, E. M. McCabe, and J. Hegarty, Direct-view microscopy: experimental investigation of the 2668 APPLIED OPTICS Vol. 39, No. 16 1 June 2000

dependence of the optical sectioning characteristics on pinholearray configuration, J. Microsc. 187, 54 61 1997. 3. M. Petran, M. Hadravsky, M. D. Egger, and R. Galambos, Tandem-scanning reflected-light microscope, J. Opt. Soc. Am. 58, 90 93 1968. 4. IBM Corporation, Electronically scanned confocal imaging system, IBM Tech. Discl. Bull. 36, 261 262 1993. 5. P. J. Verveer, Q. S. Hanley, P. W. Verbeek, L. J. van Vliet, and T. M. Jovin, Theory of confocal fluorescence imaging in the programmable array microscope PAM, J. Microsc. 189, 192 198 1998. 6. M. Liang, R. L. Stehr, and A. W. Krause, Confocal pattern period in multiple-aperture confocal imaging systems with coherent illumination, Opt. Lett. 22, 751 753 1997. 7. Q. S. Hanley, P. J. Verveer, and T. M. Jovin, Spectral imaging in a programmable array microscope by Hadamard transform fluorescence spectroscopy, Appl. Spectros. 53, 1 10 1999. 8. Displaytech, Inc., SLM Developer Kit User s Manual Displaytech Inc., 2602 Clover Basin Drive, Longmont, Colo., 1997. 9. G. Bader, R. Buerkle, E. Lueder, N. Fruehauf, and C. Zeile, Fast and accurate techniques for measuring the complex transmittance of liquid crystal light valves, in Liquid Crystal Materials, Devices, and Applications V, R. Shashidhar, ed., Proc. SPIE 3015, 93 104 1997. 10. Q. S. Hanley, P. J. Verveer, M. L. Gemkow, D. Arndt-Jovin, and T. M. Jovin, An optical sectioning programmable array microscope implemented with a digital micromirror device, J. Microsc. 196, 317 331 2000. 1 June 2000 Vol. 39, No. 16 APPLIED OPTICS 2669