Adaptive Scanning Optical Microscope (ASOM) for Large Workspace Micro-robotic Applications

Transcription

1 Adaptive Scanning Optical Microscope (ASOM) for Large Workspace Micro-robotic Applications Benjamin Potsaid and John T. Wen Center for Automation Technologies and Systems Rensselaer Polytechnic Institute Troy, New York Yves Bellouard Micro-/Nano- Scale Engineering, Mechanical Engineering Eindhoven University of Technology 5600 MB Eindhoven, The Netherlands Abstract Manipulation and assembly tasks associated with micro-systems, biotechnology, and product miniaturization demand that robots increasingly operate at microscopic dimensions. This paper discusses a new microscope design, called the Adaptive Scanning Optical Microscope (ASOM), that is particularly suitable for observing robotic activities at the micron scale. The ASOM combines a custom designed scanner lens, high speed steering mirror, and MEMS deformable mirror to offer the advantages of a greatly expanded field of view, rapid image acquisition, and no agitation to the workspace or specimen. After briefly discussing the challenges of micro assembly and micro manipulation, we present the ASOM theory of operation and include simulated performance results. A low cost proof-ofconcept experimental prototype of the ASOM is then described and used to demonstrate shape optimization of the MEMS deformable mirror for different field positions. Realtime tracking of multiple micromanipulators in a workspace and full area coverage are experimentally demonstrated. These results validate the ASOM concept and serve as a crucial step towards realizing a fully operational and high performance ASOM to enable the observation of micro-robotic activities over a large workspace. I. INTRODUCTION With the recent growth of biotechnology, micro-devices, and a trend towards product miniaturization, robots are increasingly being required to operate at the microscopic scale. Progress in these emerging fields will not only require robotic technologies that can sense, manipulate, navigate, and interact at the micron to sub-micron level, but also operate within a comparatively large workspace [1]. This dual requirement of high resolution over a large workspace poses a particular practical challenge. Currently, observation of these micro-robotic activities is commonly performed using a traditional optical microscope, but is significantly hampered by the microscope s small field of view at high resolution. The Adaptive Scanning Optical Microscope (ASOM) that we discuss in this paper is particularly suitable for observing micro-robotic activities and effectively addresses the shortcomings of the traditional optical microscope by offering: Two order of magnitude increase in observable field area when compared to a typical optical microscope. High scanning speed operation to allow for increased throughput or the observation of challenging spatialtemporal events, including the tracking of multiple moving objects. No agitation to the workspace or specimen during scanning. The challenges associated with micro-assembly and micromanipulation are significantly different than at the macroscopic scale. As the size of the objects passes below approximately 100μm, electrostatic, surface tension, and Van der Waals forces dominate over gravitational and inertial forces [2]. Consequently, micro-parts sometimes jump towards approaching manipulators or sensors before contact is actually made or fail to release from manipulators as expected. Additionally, part dimensions change due to thermal effects and manufacturing tolerances at this scale are generally quite large relative to part sizes, requiring that uncertainty with respect to part dimensions and locations be explicitly managed in the micro domain. For robust micro-assembly and manipulation, vision can help manage the uncertainty by providing non-contact measurements of part position and orientation, not only within the focal plane, but also normal to the focal plane by depthfrom-defocus techniques [3]. For high throughput applications, integrating vision information with force feedback allows for very high micro-part approach velocities, yet controlled and low impact forces upon part contact [4]. Especially important during today s early stages of micro-assembly research, the visual information provides researchers with a better understanding of the micro-process behavior to facilitate analysis and refinement. The three leading architectures for performing microassembly, manipulation, or inspection are: (1) macro scale tools and manipulators operating with micro to nano precision, (2) complete miniature manufacturing plants, called microfactories, consisting of small manipulators, lathes, millers, etc. [5], and more recently, (3) micro-robots cooperatively performing the tasks as a swarm of workers [6]. Notice that all three methods require a workspace that is typically much larger than a traditional microscope s field of view. Effective mechanical designs and workspace layouts are constrained by this limitation [1]. While the scanning electron microscope (SEM) has been proposed as a potential solution by offering a large field of view and a large depth of field, it is not suitable for many applications because the vacuum is incompatible with certain activities (e.g. manipulation of living biological cells) and

2 Science camera Final imaging optics System aperture Field lens / pupil imaging optics inverted eye-piece Steering mirror Deformable mirror (a) Scanner lens assembly Field lens / pupil imaging optics forward eye-piece Scale: 50mm rare event detection (b) ASOM sub-field scanning tracking moving object in time Object imaging only region of interest full area coverage Fig. 1. (a) Preliminary ASOM design (b) ASOM modes of operation the time required to pump down the chamber prevents high throughput. Additionally, the common solution of a moving stage with the microscope will continue to be limited to relatively low bandwidth and subject the workspace to vibration and agitation at high accelerations. The ASOM that we present in this paper effectively addresses these issues by combining a custom designed scanner lens, a high speed steering mirror, and a deformable mirror to effectively expand the field of view while preserving resolution. Previous work related to this research has demonstrated automatic tracking of moving objects in a robotic workspace and monitoring a large population of living biological cells with an ASOM-like post-objective scanner configuration [7]. The ASOM concept, theory of operation, and design methodology are discussed in greater detail in [8]. This paper discusses robotic applications of the ASOM and presents for the first time experimental results and demonstrations from a complete ASOM system that includes a deformable mirror and fast steering mirror. Section II describes the ASOM theory of operation and presents simulated results of an example ASOM design. Section III describes the experimental apparatus and Section IV describes the experimental shape optimization of the deformable mirror with the associated improvement in image quality. Section V demonstrates the ASOM capabilities by tracking two moving microgrippers in a large workspace. Conclusions and future work are presented in Section VI. II. THEORY OF OPERATION AND SIMULATED RESULTS The underlying concept of the ASOM is to use a low mass steering mirror located between the scanning lens and the imaging optics to form a post-objective scanning configuration as shown in Figure 1(a). This allows a small sub-field-ofview to be quickly scanned throughout the workspace so that multiple disjoint or overlapping regions of the workspace can be visited and imaged in rapid succession as illustrated in Figure 1(b). Through image warping and mosaic construction, a large and continuous virtual image of the object can then be constructed from the individual image tiles. The advantages of such an arrangement are: a large effective field of view at high resolution, no disturbance to the sample, and high scan rate operation. However, such a system configuration also poses significant design and implementation challenges. Compared to the moving stage or moving microscope designs, there is extensive off-axis imaging (i.e., images are obtained by looking diagonally through the scan lens), which introduces image distortion in addition to contrast degrading and resolution reducing aberrations (e.g. coma, astigmatism, field curvature, etc.) [9] that have typically reduced the effectiveness of such an approach. In the ASOM, we address the off-axis aberrations by: 1) Explicitly incorporating field curvature into the design to greatly reduce the complexity of the scanning lens. 2) Introducing an actuated deformable mirror (DM) into the optical path to correct for the residual aberrations. 3) Image processing to remove image distortion. Note that by combining dynamic components and algorithmic techniques with traditional static optical elements, the lens count in the scanner lens is greatly reduced compared to a comparable static optics only design. This potentially saves millions of dollars in manufacturing and assembly costs (lithography lenses with similar specifications can cost in the millions of dollars [10]), making the ASOM accessible for research and production activities. Figure 2 shows how the angle of the steering mirror selects the location of the field of view within the workspace. A specific deformable mirror shape corrects for the aberrations unique to each field position to obtain diffraction limited performance. Without the deformable mirror, images obtained at these field positions would be blurred with such a simple scanner lens design. Table I lists performance specifications for the ASOM shown in Figure 1(a) and the greatly enlarged field of view for the ASOM is compared to existing microscope technologies in Figure 3. A more thorough discussion of the ASOM theory of operation can be found in [7].

3 VIEWING FIELD 45.0 o 39.4 o 35.5 o 40.00mm Small sub-field can be steered within observable area shown in gray. 3.03mm 4096x4096 pixels ( FULL FIELD CAMERA) 0.76mm 1024x1024 pixels (COMMON TODAY) (a) 0.38mm 512x512 pixels (CURRENT ASOM DESIGN) * All field sizes at 0.21 NA Fig. 3. Field size comparison of ASOM to standard microscopes (0.00,0.00mm) (0.00,12.21mm) (0.00,20.34mm) (b) 1 0 Fig. 2. OPTIMAL DEFORMABLE MIRROR SHAPE 1 0 (a) Scanning (b) Optimal deformable mirror shapes TABLE I PRELIMINARY ASOM PERFORMANCE SPECIFICATIONS Specification Effective field of view diameter 40mm Total observable field area 1257 mm 2 Numerical aperture 0.21 Working distance 7 mm Operating wavelength 510 nm Resolution 1.5 μm Magnification 15.2 Camera pixel count Camera pixel size 10μm III. OVERVIEW OF EXPERIMENTS AND APPARATUS In simulation, perfect knowledge of the wavefront shape is readily available, and it is straightforward to numerically determine the required mirror shape to correct for the specific aberrations associated with each field position. However, in practice, the wavefront cannot be measured directly, but must be inferred from a related measurement (often related to light intensity) which can then be used to optimize the deformable mirror shape. Combined with the fact that the MEMS deformable mirrors exhibit significant internal residual stresses and manufacturing variations that prohibit accurate simulation based prediction of the resulting shape for a given set of actuator voltages, experimental use of the deformable mirror requires a method for the in-situ shape optimization of the mirror surface. Additionally, the simulations ignore the necessity of an alignment procedure, manufacturing and assembly tolerances, light reflections off the glass surfaces, and other stray light effects. For these reasons, the experimental results presented here provide further validation of the ASOM concept 1 0 and the experiences gained during the design, construction, calibration, and operation of this prototype significantly advance the research towards the ultimate goal of realizing a useful and deployable ASOM system. The purpose of this experimental ASOM apparatus was to demonstrate all essential optical aspects of the ASOM design, but at low cost and with a short development time. As such, off the shelf optics were used exclusively to avoid the considerable cost of custom fabricated optics and to take advantage of the existing stock of catalog available items that ship within days. However, most stock lenses are designed to be used in a particular manner (e.g. with infinite conjugates) for generic applications and are offered in a coarse range of focal distances, lens diameters, and glass selections. Considering the atypical imaging characteristics of the scanner lens, the experimental ASOM design using off-the-self optics only is far from optimal, and as such, exhibits a noticeably high lens count to achieve 0.1 NA over a nominal 20mm field size. However, even with the use of off-the-shelf optics only, this experimental apparatus has been carefully designed to demonstrate all of the critical optical characteristics that define the ASOM, including the curved field optical scanning approach and wavefront correcting optics using a deformable mirror. Future experimental work will utilize custom manufactured optics to fully realize the potential of the ASOM concept to achieve higher numerical aperture and a larger workspace. Figure 4 shows the optical layout of the experimental setup and Figure 5 shows a picture of the prototype ASOM. This initial prototype utilizes a transmitted lighting scheme and because the current design is very sensitive to chromatic aberration, a 510nm wavelength bandpass filter is included in the illumination stage to eliminate much of the light spectra below 500nm and above 520nm. Light transmits through the object contrast pattern and is then collected by the telecentric 12 element scanner lens assembly. An electromagnetically actuated fast steering mirror (FSM) with a flexure suspension (Optics in Motion OIM101) has two degrees of freedom to steer the sub-field of view within the workspace. Optics project the light onto the MEMS deformable mirror (Boston Micromachines Mini-DM). By precisely controlling the shape of the reflective surface of the mirror to be opposite the shape of the wavefront error (but at half the amplitude), the

4 Digital Camera Pulnix TM200 System Aperture Viewing off axis field positions Steering Mirror Optics in Motion OIM101 Object Plane Scanner Lens 12 element design using off-the-shelf Thorlabs 2 inch optics Deformable Mirror Boston Micromachines 32 actuator Mini-DM Fig. 4. ASOM Experimental Setup deformable mirror can correct for the wavefront aberrations to within the diffraction limit. This mirror has 32 electrostatic actuators with 400μm actuator spacing, a 2.5μm actuator stroke, and a 2.0mm diameter actively controlled area. The 2.5μm stroke is capable of correcting for up to several waves of aberration, which allows for high image quality even for the off-axis field positions and enables the greatly expanded field of view in the ASOM. Additional optics project the final image onto the CCD camera (Pulnix TM200). A host computer acquires (MATROX Meteor II frame grabber) and processes the images. The commands to the steering mirror are generated with a D/A board (Measurement Computing PCIM- DAS1602/16). Scanner Lens Object Light Source Fig. 5. Steering Mirror Science Camera Deformable Mirror Deformable Mirror Drive Electronics ASOM prototype built with off-the-shelf optics IV. DEFORMABLE MIRROR SHAPE OPTIMIZATION The deformable mirror shape is controlled by the actuator voltages and has a direct effect on the wavefront aberrations and hence the image quality. Experimental optimization of the deformable mirror shape for each field position requires: a metric to represent the image quality, Q(u) an optimization algorithm to minimize Q(u) This metric, Q(u), is in general a nonlinear function of the actuator voltages, u, and Q(u) is defined to decrease with improving image quality. The resulting optimization problem is also subject to upper and lower bounds on the actuator voltages and there are many possible image quality metrics and optimization methods that can potentially be combined. The image quality metric used for the optimization in this research was based on maximizing the energy of the high frequency image content and the specific formulation of the metric was obtained from [11]. First, a highpass filter was convolved with the acquired image. Then the summation of the absolute value of the filtered result was calculated. Given an i j matrix of intensity values to represent the image, I, the specific image quality metric, Q was: Q = 1 [I K] E i,j, (1) i j where E = i j I i,j, K =, and indicates convolution. Optimization of the image quality metric was performed with the parallel stochastic-gradient-descent (PSGD) algorithm, which is based on a parallel perturbation of the input signals [12]. It is applicable to systems where there are many input signals and a performance metric that can be quantified, but no model information is available to relate the performance metric to the inputs. A derivation and justification of the approach can be found in [12] and the algorithm found in [13] is as follows. Given an initial set of actuator signals, u n at time step n, and a corresponding objective function value, Q n (u n ), generate a set of equal magnitude, but random sign perturbations, δu, to apply to the actuator control signals in parallel.

5 Apply the perturbations to the system and measure the objective function: Q + n = Q(u n + δu). Reverse the polarity of the perturbations and apply the negative perturbations to the system and measure the objective function: Q n = Q(u n δu). Update the actuator signals with a gain on the step size of α as: Actuator Voltage (V) u n+1 = u n + α(q + n Q n )δu. (2) 100V all actuators Fig Iteration Example Deformable Mirror Convergence An example convergence of the 32 individual actuator voltages for an off-axis field located at 2.5mm off-axis is shown in Figure 6. The associated image of a USAF 1951 calibration target before and after optimization of the deformable mirror is shown in Figure 7. It is clear from these images that the image quality improves with optimization of the deformable mirror shape and similar results were obtained for other off-axis field positions. The smallest bar pattern (group 7 element 6) has 228 line pairs per mm and is resolved with the deformable mirror optimized. This bar pattern has a bar width of about 2 microns so the ASOM as tested has a resolving power of approximately 4 microns. The presence of faint vertically displaced ghost images, are likely caused by a protective glass cover plate mounted on the deformable mirror and we are currently working with Boston Micromachines Corp. on a new deformable mirror packaging design to eliminate this artifact. V. VISUAL TRACKING DEMONSTRATION Two microgrippers in a large workspace are shown in Figure 8, which is a 6 6 tile image mosaic. A silicon device ( gripper ) is mounted on a 2 axis manual precision linear stage and a shape memory alloy microgripper is mounted on a single axis precision linear stage. Each gripper is free to move independently in the workspace and the ASOM has been programmed to automatically track the gripper tips using real-time image processing techniques. Figure 9(a-h) shows selected frames from a gripper tracking demonstration. A first tile (each tile corresponds to a single camera exposure) automatically tracks the silicon device and a second tile monitors Optimized Voltages Fig. 7. Image of USAF calibration target with (a) 100V on all deformable mirror actuators (b) optimized voltages on deformable mirror actuators. The line width of the smallest bar pattern is about 2 microns. the region immediately to the right of the device. Third and fourth tiles also track the shape memory alloy microgripper. Notice that in frames (c) and (d), portions of the shape memory alloy gripper can be seen in the second tile, showing the ability of the system to track multiple moving objects with the possibility of overlapping regions. After completing the motion tracking, the ASOM performs an open loop (no visual feedback) full area scan of the workspace to construct the 6 6 image mosaic shown in Figure 9(i). This mode of operation can be used to identify and locate objects in the workspace, and indeed, the edge of a second shape memory alloy gripper can be seen in the upper right corner. Throughout the tracking exercise, the coordinates of the gripper tips as identified by the image processing algorithm are logged. The complete motion trajectory for each gripper from this data is shown in Figure 9(j). Current scan rates are quite slow (approximately 5 tiles per second) due to the settling time of the mirror, the software architecture, and time required to save the large image files to disk. Future work will soon address these issues and we intend to incorporate a high speed camera to operate at speeds greater than 30 frames per second. VI. CONCLUSION We have proposed that our new large field of view optical microscope design, called the Adaptive Scanning Optical Microscope (ASOM), is particularly suitable for providing vision information for micro-robotic activities. A proof of concept experimental apparatus demonstrates all of the critical

6 tile 1 tile 2 tile 3 a tile 4 b c d e frame 21 frame 97 frame 121 frame 147 frame 217 f g h i j frame 284 frame 339 frame 409 6x6 mosaic of workspace gripper trajectories Fig. 9. Gripper tracking demonstration 0.32mm 475 pixels 0.45mm 768 pixels Fig. 8. A silicon device ( gripper ) is manually positioned with a 2 axis precision stage A shape memory alloy microgripper is manually positioned with a single axis precision stage 6 6 tile robot workspace mosaic optical aspects of the ASOM using only off-the-shelf optics. The shape optimization of the deformable mirror, the resulting images of a calibration target, and initial experiments tracking microgrippers in the workspace demonstrate the validity of the ASOM concept. Future work will be to refine this prototype and to address the slow operational speed by improved control, software architecture, and management of the large amounts of data. After demonstrating this improved system on practical micro-robotic applications, we will design and construct a next generation ASOM prototype using custom manufactured optics and a 100 actuator MEMS deformable mirror for increased resolution and field size. ACKNOWLEDGEMENT This material is based in part upon work supported by the National Science Foundation under Grant No. CMS and by the Center for Automation Technologies and Systems (CATS) under a block grant from the New York State Office of Science, Technology, and Academic Research (NYSTAR). The second author is also supported in part by the NSFC two-bases project (No ), and the Outstanding Overseas Chinese Scholars Fund of Chinese Academy of Sciences (No ), China. REFERENCES [1] G. Yang, J. A. Gaines, and B. J. Nelson, Optomechatronic design of microassembly systems for manufacturing hybrid microsystems, IEEE Transactions on Industrial Electronics, vol. 52, no. 4, pp , [2] R. S. Fearing, Survey of sticking effects for micro parts handling, in IEEE Conference Proceedings Intelligent Robots and Systems 95. Human Robot Interaction and Cooperative Robots, vol. 2, Pittsburgh, PA, 1995, pp [3] B. Vikramaditya and B. J. Nelson, Visually guided microassembly using optical microscopes and active vision techniques, in Proceedings of the 1997 IEEE International Conference on Robotics and Automation, 1997, pp [4] B. J. Nelson, Y. Zhou, and B. Vikramaditya, Integrating force and vision feedback for microassembly, in Microrobotics and Microsystem Fabrication, Proceedings of SPIE, 1997, pp [5] Y. Okazaki, N. Mishima, and K. Ashida, Microfactory - concept, history, and developments, Journal of Manufacturing Science and Engineering, vol. 126, no. 4, pp , [6] H. Woern, J. Seyfried, S. Fahlbusch, A. Buerkle, and F. Schmoeckel, Flexible microrobots for micro assembly tasks, in Proc. of 2000 International Symposium on Micromechatronics and Human Science, Nagoya, Japan, 2000, pp [7] B. Potsaid, Y. Bellouard, and J. T. Wen, Adaptive scanning optical microscope (asom): A multidisciplinary optical microscope design for large field of view and high resolution imaging, Opt. Express, no. 17, pp , [Online]. Available: [8] B. Potsaid, Expanding the field of view in optical microscopy: A multidisciplinary approach, Ph.D. dissertation, Rensselaer Polytechnic Institute, Troy, NY, [9] R. E. Fischer and B. Tadix-Galeb, Optical system design, S. P.. McGraw-Hill, Ed. McGraw-Hill, [10] W. J. Smith, Modern Lens Design, Second Edition, SPIE, Ed. McGraw- Hill, [11] M. Cohen, G. Cauwenberghs, and M. A. Vorontsov, Image sharpness and beam focus vlsi sensors for adaptive optics, IEEE Sensors Journal, vol. 2, no. 6, pp , [12] M. A. Vorontsov and V. P. Sivokon, Stochastic parallel-gradient-descent techniques for high-resolution wave-front phase-distortion correction, Optical society of america A, vol. 15, no. 10, pp , Oct [13] G. Reimann, J. Perreault, P. Bierden, and T. Bifano, Compact adaptive optical compensation systems using continuous silicon deformable mirrors, in High-resolution wavefront control: methods, devices, and applications III, no. Proc. SPIE 4493, 2001, pp