Rapid Non linear Image Scanning Microscopy, Supplementary Notes Calculation of theoretical PSFs We calculated the electrical field distribution using the wave optical theory developed by Wolf 1, and Richards and Wolf 2, for imaging through optical systems with high numerical aperture. From the obtained field we calculated the intensity,, and the squared intensity,,. To describe the PSF we use the FWHM diameters of,, along and perpendicular to the z axis along which the light propagates. We approximated the incident laser beam at the back focal aperture of the objective by a Gaussian beam and calculated the extensions of the PSFs for back focal beam radii ranging from 2 mm to 4 mm. The values of the other parameters used were: numerical aperture NA 1.3; refractive index of immersion medium equal to 1.41; excitation wavelength 900 nm. Figure S1: (a) 3D representation of the 2PE PSF as an isosurface corresponding to ½ of its maximum value. (b) axial (+) and lateral (o) diameters (FWHM) of the 2PE PSF and their dependence on the beam diameter in the back focal aperture of the objective. 1. Wolf, E. Electromagnetic diffration in optical systems 1. An integral representation of the image field. Proceedings of the Royal Society, 349 357 (1959). 2. Richards, B. & Wolf, E. Electromagnetic diffraction in optical systems 2: Structure of the image field in an aplanatic system. Proceedings of the Royal Society, 358 379 (1959).
Rendered images of 3D SHG signals Figure S2 shows different views of a 3D stack of the SHG signal of a collagen I hydrogel as described in the online methods. Figure S2: 3D representations of conventional microscopy (a,c) and de convolved SHG ISM (b,d). (a) and (b) show pseudo colored z stacks; (c) and (d) show maximum intensity volumetric renderings. Image size: 63.5 µm x 29.5 µm.
Beam path in the ISM microscope The use of a galvo scanner with a resonant and a conventional scan axis has some implications to the beam path that are not obvious at first sight. The two mirrors create a lateral shift between the optical axis of the objective and the optical axis of the entrance port of the scanner. When feeding the emitted light back into the scanner, this shift needs to be compensated. The second implication is that the directions of the movement of the beam has to be matched when feeding the beam back to the scanner. This is illustrated in figure S3. There, the movement, generated by the first scan mirror (SM1) is upwards, as indicated by the green arrow on top of the beam. For the emitted beam (shown in blue) this movement is identical and if it was simply reflected upwards, the movement does not match the motion of SM1. Hence, in order to obtain a doubling of the scan angle, one needs to rotate and invert the beam in combination with the necessary lateral shift. In our setup this is achieved by a combination of three mirrors (M1 to M3). Figure S3: Schematic view of the beam path in the ISM microscope. For simplicity, the scan and tube lenses are not shown. The excitation light is shown in red, light
emitted by the sample is shown in blue. Sections where excitation and fluorescence overlap are shown in green. The arrows indicate the beam deflection due to movement of the resonant scan mirror (green) and the second scan mirror (red). The excitation beam enters the setup and is reflected by the dichroic mirror (DCM3) onto the optical axis of the scanner. The beam is scanned via the resonant scan mirror (SM1) and the second scan mirror (SM2). It propagates through the dichroic mirror (DCM1) and is reflected by the dichroic mirror (DCM2) into the microscope objective. The emitted light is collected by the objective, passes DCM2 and is reflected by a mirror to the arrangement of three mirrors (M1 M3). These mirrors align the beam with the axes of the scanner. Then, the beam passes DCM3 and is re scanned via SM1 and SM2. Finally, the light is coupled out by DCM1 and imaged onto a camera (not shown).
Image data processing Processing of the recorded imaging data is done according to the following steps: 1. Background subtraction. A background dataset consisting of 100 frames is recorded using the same settings as for the image recording, but without excitation. The frames are averaged and smoothed with a Gaussian filter. The resultant representation is then subtracted from the image data. Negative results are clipped to zero. It is also possible to simply subtract an average value taken from unexposed areas of the camera sensor if the background data is not available. 2. Removal of pixel artifacts. To remove artifacts from hot or cold pixels we proceed as follows. For each pixel we calculate the expected value E by linear interpolation for its surrounding pixels. We also compute the local variance based on the interpolation for the surrounding pixels. If the pixel value exceeds the range 3 we consider this as an artifact and set this pixel to the value of E. If one uses ImageJ one can use the Remove outliers function using a threshold based on the variance of the histogram of the image data. 3. Final processing. After an optional Fourier reweighting, the data is filtered using a 2D Gaussian filter with a radius of 1 pixel (about 46 nm) and, usually, 2x2 pixels are binned. Finally, the 16 bit data is downscaled to 8 bit dynamic range.
Fourier reweighting 1. The first aim is to compute the Fourier transform F(k) of the image data. To minimize problems by boundary effects, we pad the original data using accordingly flipped patches from each respective border. The aim is to obtain an image of the size N x N, where log ceil max, 1, and a, b denote the size of the image. 2. Additionally, we let the intensity of the synthesized image decay toward the borders using a linear ramp that falls from 1 to 0 in the range R = N/2 to R = N. 3. Next, we do the Fourier transform of the conditioned data. We separate the modulus and the phase angle of the FT data, since we only want to re weigh the modulus. The modulus can take values different from 0 only for max where max is the maximal length of a k vector that is transmitted by the objective: max 4 NA/ em. This, for all max F(k) has to be 0. 4. The ideal MTF curve of a microscope would be fi 2/π acos 1 with ma. Since the imaging process in ISM delivers a MTF that is basically raw fi one could use a weight function 1 fi to recover fi from raw. Since the fi has low amplitudes for 1 this would mean high values of and thus amplifying noise. Therefore one has to use a regularization avoiding this. A simple approach is 1 fi where takes reasonably small values. Typically 0.02 works well, but this depends on the SNR. 5. After applying the corrections on F(k) one can back transform the data into real space. 6. A MatLab script showing this procedure on sample data is shown below. 7. Since the ISM raw images already contain the expanded frequency support, one can also achieve the resolution enhancement using so called deconvolution algorithms, e.g. Richardson Lucy deconvolution. Classical deconvolution is not able to gain resolution improvements but sharpens blurring by trying to de convolve image data with the PSF. This essentially is the same operation that one implicitly does when reweighting the amplitudes in the MTF. Therefore, these algorithms can be applied in ISM, too.
An exemplary MatLab script showing the steps of this procedure: % read data from file name = 'actin.tif'; data = double(imread(name,'tiff')); data = max(data,0); % clip negative values [nx,ny] = size(data); N = 2^ceil(log2(max(nx,ny))+1); % size of image for Fourier transform % stich patches of the original image to form a periodic 'super-image' im = [data fliplr(data) data fliplr(data) data]; im = [flipud(im); im; flipud(im); im; flipud(im); im; flipud(im)]; % cut out the center part of the 'super-image' of size NxX [sx,sy] = size(im); im = im(floor((sx-n)/2)+(1:n),floor((sy-n)/2)+(1:n)); % let the signal decay to zero towards the edges [X,Y] = meshgrid(1:n,1:n); X = (X-N/2-1); Y = (Y-N/2-1); R = sqrt(x.^2+y.^2); W = 2-8*R/N; W = min(w,1); W = max(w,0); im = W.*im; % Do the Fourier transform fim = fftshift(fft2(im)); fa = abs(fim); % modulus fp = angle(fim); % Phase angle % calculate k-vectors of the Fourier-transformed image fs = 1/0.091; % per µm k = fs/2/(n/2).*r; km = 5.5; ind = double(k<km); % Step function to clip everything outside km to zero % 'Ideal' MFT OTF = ind.*2/pi.*(acos(k./km)-k./km.*sqrt(1-(k./km).^2)); % Re-weighting function ep = 0.015; w = ind./(otf + ep.*k./km); % Back-transform the re-weighted FT tim = abs(ifft2(ifftshift(w.*fa.*exp(1i.*fp)))); % cut out the original field of the data tim = tim((n-nx)/2+(1:nx),(n-ny)/2+(1:ny)); % display original data vs. re-weighted data imagesc([data; tim]) colormap(gray(256)) caxis([0 350]) axis image axis off
Description of Additional Files and Videos SupplementaryVideo 1.mov : This is a rendering of a collagen I gel as described in the paper. The data was recorded done using a conventional SHG confocal microscope. SupplementaryVideo 2.mov : This is a rendering of the same sample as in SHG.mp4. In this case the data was recorded using the ISM confocal microscope. SupplementaryVideo 3.mov : This video shows fluorescent beads (d = 100 nm) diffusing freely in water. The recording frame rate is 20 fps. The acquisition speed is sufficient to track individual beads for several frames until they diffuse out of focus. SupplementaryVideo 4.mov, SupplementaryVideo 5.mov, SupplementaryVideo 6.mov, and SupplementaryVideo 7.mov are rendered video sequences time series of 3D stacks of living Drosophila embryos as described in the Online Methods. The frame interval is 60 s or 120 s in the case of SupplementaryVideo 7.mov. Spacing of the z planes is 3 µm. The sections are taken around the center height of the embryo.