* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Theory of confocal fluorescence imaging in the
Gaseous detection device wikipedia , lookup
Phase-contrast X-ray imaging wikipedia , lookup
Photon scanning microscopy wikipedia , lookup
Diffraction topography wikipedia , lookup
Surface plasmon resonance microscopy wikipedia , lookup
Hyperspectral imaging wikipedia , lookup
Night vision device wikipedia , lookup
Preclinical imaging wikipedia , lookup
Image intensifier wikipedia , lookup
Interferometry wikipedia , lookup
Image stabilization wikipedia , lookup
Fluorescence correlation spectroscopy wikipedia , lookup
Chemical imaging wikipedia , lookup
Optical coherence tomography wikipedia , lookup
Optical aberration wikipedia , lookup
Harold Hopkins (physicist) wikipedia , lookup
Journal of Microscopy, Vol. 189, Pt 3, March 1988, pp. 192–198. Received 12 August 1997; accepted 1 December 1997 SHORT COMMUNICATION Theory of confocal fluorescence imaging in the programmable array microscope (PAM) P. J. VERVEER,* Q. S. HANLEY,* P. W. VERBEEK,† L. J. VAN VLIET† & T. M. JOVIN* * Department of Molecular Biology, Max Planck Institute for Biophysical Chemistry, Am Fassberg 11, D-37077 Göttingen, Germany † Faculty of Applied Physics, Pattern Recognition Group, Delft University of Technology, Lorentzweg 1, 2628 CJ Delft, The Netherlands Key words. Confocal microscope, spatial light modulator, digital micromirror device (DMD), tandem scanning microscopy, pseudorandom sequences, point-spread-function, optical sectioning. Summary The programmable array microscope (PAM) uses a spatial light modulator (SLM) to generate an arbitrary pattern of conjugate illumination and detection elements. The SLM dissects the fluorescent light imaged by the objective into a focal conjugate image, Ic, formed by the ‘in-focus’ light, and a nonconjugate image, Inc, formed by the ‘out-of-focus’ light. We discuss two different schemes for confocal imaging using the PAM. In the first, a grid of points is shifted to scan the complete image. The second, faster approach, uses a short tiled pseudorandom sequence of two-dimensional patterns. In the first case, Ic is analogous to a confocal image and Inc to a conventional image minus Ic. In the second case Ic and Inc are the sum and the difference, respectively, of a conventional and a confocal image. The pseudorandom sequence approach requires post-processing to retrieve the confocal part, but generates significantly higher signal levels for an equivalent integration time. 1. Introduction Confocal fluorescence microscopy has been widely used to obtain images of biological specimens (Pawley, 1995). In contrast to the conventional fluorescence microscope, confocal configurations allow optical sectioning of the object and thereby three-dimensional imaging (Wilson, 1989; Correspondence to: T. M. Jovin; tel: +49-551-201-1382; fax: +49-551-2011467; E-mail: [email protected] 192 Brakenhoff et al., 1989). This is achieved by illumination of the object with a point source and detection through a pinhole at a conjugate position in the emission path. To acquire a complete image, scanning of the light source or of the sample is required. An inherent limitation of the widely distributed confocal laser scanning microscope (CLSM) is its low duty cycle; the laser illuminates only one point at a time. In contrast, the conventional full-field microscope records the images much faster (no scanning is required) and is also more light efficient (no photons are rejected by a pinhole). A critical discussion of the relative merits of wide-field microscopy with deconvolution and confocal scanning microscopy for three-dimensional imaging involves complex issues (Shaw, 1995; Verveer et al., 1997). An alternative to the sequential operation of the CLSM is offered by the tandem scanning microscope (Egger & Petráň, 1967) and related real-time scanning instruments (Kino, 1995). These devices incorporate a rotating disc with multiple apertures for the simultaneous illumination and detection of a large number of points. However, since the spacing between the pinholes must be large to maintain the confocal effect, much of the illumination light is rejected by the disc ($ 97%; Fewer et al., 1997). This problem can be circumvented by using an array of closely packed apertures modulated in time such that they open and close in a completely uncorrelated fashion (Juškaitis et al., 1996; Wilson et al., 1996). The result is the sum of a conventional and a confocal image. A separate conventional image must be taken and subtracted in order to obtain the desired confocal image. q 1998 The Royal Microscopical Society P RO G R A M M A B L E A R R AY M IC RO SC O P E ( PA M ) Fig. 1. Schematic of a fluorescence PAM microscope based on a digital micromirror device (DMD). A DMD is illuminated uniformly. Only mirror elements orientated in the ‘on’ position (one is depicted) reflect light into the back-aperture of the objective, exciting the sample at conjugate positions. The fluorescent emission is imaged back to the DMD, where it is relayed to a 2-D detector (camera-1) by the same mirror elements, forming the focal conjugate image Ic. Fluorescent light arising from ‘out-of-focus’ positions in the sample is reflected from mirror elements in the ‘off ’ position, and may optionally be relayed to a second detector (camera-2) in order to record the nonconjugate image Inc simultaneously. In the figure, the fluorescent light from an ‘out-of-focus’ point is shown to be intercepted by both depicted mirror elements and is therefore detected by both camera-2 (predominantly) and camera-1. The fluorescence from ‘in-focus’ points is recorded exclusively on camera-1. 193 be tilted at an angle of 6 108 to the normal (Montamedi et al., 1997). The use of a DMD for laser-based confocal imaging in reflection mode was reported recently (Liang et al., 1997). Figure 1 presents a schematic for a fluorescence programmable array microscope system based on the DMD. The latter is uniformly illuminated by a lamp or laser source. Mirror elements switched to the ‘on’ position reflect the light to the object at conjugate positions in the focal plane of the objective. The fluorescent emission is imaged back to the corresponding elements of the DMD and reflected via relay optics to a 2-D detector (camera-1), yielding the focal conjugate image Ic. The fluorescence representing ‘out-of-focus’ contributions, i.e. originating from nonfocal planes, is reflected by the mirror elements in the ‘off ’ position and imaged onto a second detector (camera-2), thereby obtaining the corresponding nonconjugate image Inc. Forming the nonconjugate image on a second camera, a highly desirable option of the PAM, may be difficult or impossible with other kinds of SLM devices. During the frame integration time, the modulation pattern of the SLM is changed N times in order to generate a confocal image. We denote the ith modulation of the SLM by ¼ Si ðxd ; yd Þ 1 if ðxd ; yd Þ is on an SLM element that is ‘on’; 0 ifðxd ; yd Þ is on an SLM element that is ‘off ’; ð1Þ where xd and yd are continuous coordinates on the SLM. We include the shape and finite size of the SLM elements in Si(xd, yd). We assume an object O(xo, yo, zo) on a separate coordinate system (xo, yo, zo). It is scanned in the axial direction by displacing it over a distance zs. For each modulation pattern Si(xd, yd) the object is illuminated with an intensity given by integrating the contributions of all the illumination sources: þ∞ Si ðMu; MvÞHex ðxo ¹ u; yo ¹ v; zo Þ du dv; Ii ðxo ; yo ; zo Þ ¼ ¹∞ In this paper, we discuss two approaches for obtaining confocal fluorescence images using the newly developed programmable array microscope (PAM; Hanley, Verveer & Jovin, submitted manuscript). The PAM uses a programmable spatial light modulator (SLM) for generating an arbitrary pattern of conjugate illumination and detection apertures. We present the theoretical basis for image formation using the SLM-based system. 2. Theory 2.1. Confocal imaging with a PAM SLMs can be classified according to whether they operate in a transmission or reflection mode. A particular example of the latter class is the digital micromirror device (DMD), consisting of an array of square mirrors, each of which can q 1998 The Royal Microscopical Society, Journal of Microscopy, 189, 192–198 ð2Þ where M is the magnification of the lens, and Hex is the excitation PSF of the lens. We sum the contributions of all points in the object and over all different modulations of the SLM to find the conjugate image Ic: þ∞ ¹1 T NX S ðx ; y Þ Ic ðxd ; yd ; zs Þ ¼ N i¼0 i d d ¹∞ x y × Hem d ¹ u; d ¹ v; w Ii ðu; v; wÞOðu; v; w ¹ zs Þ du dv dw; M M ð3Þ where Hem is the emission PSF of the lens, and T is the total integration time. The expression for the nonconjugate image Inc is obtained by replacing the term Si(xd, yd) in Eq. (3) with 1¹ Si(xd, yd). The modulations Si(xd, yd) can be selected so as to implement two approaches for fast confocal imaging. 194 P. J. V E RV E E R E T A L . Switching SLM elements in a grid pattern and shifting them systematically provides a system analogous to a tandem scanning microscope. Alternatively, by modulating the SLM elements using pseudorandom sequences, an aperture correlation approach similar to that recently reported for a real-time microscope can be implemented (Juškaitis et al., 1996; Wilson et al., 1996). In the first approach, we select the modulation patterns according to Sa;b ðxd ; yd Þ ¼ Gðxd ¹ ah; yd ¹ bhÞ; ð4Þ where h is the size of an SLM element (assuming square elements with 100% fill factor) and the index i in Eq. (1) has been replaced by two-dimensional integer indices a, b, where 0 #a < nx, 0 #b < ny. The grid is defined by the properties Gðxd ; yd Þ ¼ Gðxd ¹ pdx ; yd ¹ qdy Þ p; q [ N; ð5Þ and nX y ¹1 x ¹1 n X Gðxd ¹ ah; yd ¹ bhÞ ¼ 1; ð6Þ a¼0 b¼0 where dx ¼ nxh and dy ¼ nyh are the lattice distances of the grid. The grid can have many different shapes, the simplest being square, but pseudohexagonal grids or line patterns are also possible and in some cases preferable. In the second approach, all SLM elements are ideally modulated in a completely uncorrelated fashion. This is difficult to achieve in practice, since the number of times that the SLM can change its pattern within a given frame integration time is limited. Thus, we repeatedly use the same sequence of relatively short length in order to modulate the complete plane. Define a sequence of N twodimensional patterns Ri(a, b) with the property N¹1 X Ri ða; bÞRi ðc; dÞ ¼ Ndða ¹ c; b ¹ dÞ a; b; c; d [ N; ð7Þ i¼0 where Ri(a, b) has value ¹1 or 1. A pseudorandom sequence of one-dimensional patterns having the desired property can be found using the rows of cyclic Hadamard matrices (Harwit & Sloane, 1979). These are mapped to twodimensional patterns so as to obtain the required property (Eq. 7). Since Si(xd, yd) can only assume the values 0 and 1, we use the sequence (1 þ Ri(a, b))/2 (see also Wilson et al., 1996). In this case 50% of all elements will be in the ‘on’ state in each modulation pattern. Other sequences may also be employed (Golay, 1961). We use a grid of the form in Eq. (5) to tile the ith pattern in order to form the complete modulation pattern Si(xd, yd) (see panel C of Fig. 3 for an example). That is, all SLM elements for which G(xd ¹ ah, yd ¹ bh) ¼ 1 are switched in an identical manner using the sequence (1 þ Ri(a, b))/2. Since we switch all SLM elements simultaneously, the ith modulation is given by a summation over all values of a and b: Si ðxd ; yd Þ ¼ nX y ¹1 x ¹1 n X a¼0 b¼0 1ÿ 1 þ Ri ða; bÞ Gðxd ¹ ah; yd ¹ bhÞ 2 nx ¹1 nX y ¹1 1 1X ¼ þ R ða; bÞGðxd ¹ ah; yd ¹ bhÞ: 2 2 a¼0 b¼0 i ð8Þ The resulting signal consists of two parts, the first one corresponding to a conventional image. We have derived the imaging model for these modulation schemes by substituting Eqs. (4) or (8) into Eq. (3). An outline of the derivations is given in Appendix A, and we present the results here. For the shifting grid approach the focal conjugate image is given by þ∞ x T y Hem d ¹ u; d ¹ v; w Ic;S ðxd ; yd ; zs Þ ¼ M M nx ny ¹∞ yd ¹ fy x ¹ fx ;v ¹ ; w Oðu; v; w ¹ zs Þ du dv dw; × IG u ¹ d M M ð9Þ where the total illumination is defined by: þ∞ IG ðxo ; yo ; zo Þ ¼ GðMu; MvÞHex ðxo ¹ u; yo ¹ v; zo Þ du dv: ¹∞ ð10Þ The total illumination IG enters into equation Eq. (9) with spatially varying shifts along the two orthogonal x and y directions, given by fx ¼ xd mod h; ð11aÞ fy ¼ yd mod h: ð11bÞ Equation (9) can be viewed as a three-dimensional convolution with a spatially varying PSF, due to the dependence of fx and fy on xd and yd, respectively. This is a consequence of the fact that the SLM cannot scan continuously. That is, G(xd, yd) can shift only over integer multiples of h, as opposed to a continuous motion in the tandem scanning microscope (Wilson & Hewlett, 1991). The distinction is negligible if h/M is small compared to the sampling density of the image. For large h/M the effect is a perceptible pixelation in the resulting image. The PSF could be made spatially invariant by scanning the SLM over h (dithering). For the pseudorandom sequence approach we obtain þ∞ x T y Hem d ¹ u; d ¹ v; w Ic;R ðxd ; yd ; zs Þ ¼ M M 4 ¹∞ þ∞ x T y × Oðu; v; w ¹ zs Þ du dv dw þ Hem d ¹ u; d ¹ v; w M M 4 ¹∞ yd ¹ fy xd ¹ fx × IG u ¹ ;v ¹ ; w Oðu; v; w ¹ zs Þ du dv dw; M M ð12Þ We can draw the following important conclusions about the various strategies for confocal image formation with the PAM. (i) In comparison with a CLSM, the PAM has the potential for increasing the acquisition speed by two orders of q 1998 The Royal Microscopical Society, Journal of Microscopy, 189, 192–198 P RO G R A M M A B L E A R R AY M IC RO SC O P E ( PA M ) magnitude with a concurrent increase of signal strength by an order of magnitude. Although the lamp source of the PAM yields a much lower irradiance (typically 100-fold) at the focal plane than the laser in a CLSM, this is compensated for by the large number of points that are scanned in parallel (allowing longer dwell times) and the higher quantum efficiency of the detector (a CCD sensor instead of a photomultiplier tube). (ii) The pseudorandom sequence approach is very light efficient, a result of the fact that 50% of all SLM elements are ‘on’ in each modulation pattern. This is also seen from the second term in Eq. (12), which is equal to Eq. (9) except for the coefficient T/4 instead of T/nxny. Thus, although both approaches yield the same confocal image for the same grid G(xd, yd), the pseudorandom sequence generates a much larger signal, the strength of which is independent of the length of the sequence. (iii) To obtain a confocal image by the pseudorandom sequence approach, a compensation term must be subtracted. This may be a conventional image acquired separately (Juškaitis et al., 1996; Wilson et al., 1996) or the nonconjugate image. The latter is preferable since it is acquired simultaneously and yields twice the intensity (see Section 2.2). However, we note that the subtraction operation inevitably increases noise, since the variance is the sum of the variances of the two images. 2.2. The nonconjugate image The nonconjugate image Inc can be imaged on a second camera, for instance using a DMD as the SLM. The model for Inc can be derived rigorously, but it is easier to note that the sum of the two images must be a conventional image, since the two cameras together collect all of the fluorescent light that falls on the DMD. Therefore, Inc is the difference of a conventional image and the image given by Eqs. (9) or (12). For the pseudorandom sequence, it follows that the difference between Ic and Inc equals the desired confocal image. In both approaches, the two images can be used with a deconvolution algorithm to generate an improved image. This includes simple enhancement algorithms such as the nearest-neighbour approach using three or more sections to generate a single enhanced image (Agard, 1984) or more powerful algorithms (see, for instance, Carrington et al., 1995) using 3-D stacks to generate a full 3-D reconstructed object. Both are used extensively in fluorescent imaging. We are currently working on a multichannel extension of a set of deconvolution algorithms described in Verveer & Jovin (1997) that will allow the use of both Ic and Inc for generating a single optimal restoration. 3. Results 3.1. Simulations Equation (9) was used to simulate numerically the response of an infinitely thin fluorescent plane as a function of its position q 1998 The Royal Microscopical Society, Journal of Microscopy, 189, 192–198 195 Fig. 2. The image of an infinitely thin plane as a function of its position relative to the focal plane obtained by confocal imaging using an SLM. A square grid was used with lattice distances dx ¼ dx ¼ {2, 3, 5, 10, 20} × h. The responses for a conventional microscope and an ideal confocal microscope are also plotted. Parameters: SLM element size h ¼ 17 mm, NA ¼ 1·4, M ¼ 100, excitation wavelength = 633 nm, emission wavelength = 665 nm, refractive index = 1·515. relative to the plane of focus. Simulation using Eq. (3) is also possible and yields identical results. We used a square grid with varying lattice distances. The conventional excitation and emission PSFs were computed using vector wave theory (van der Voort & Brakenhoff, 1990). Figure 2 presents the normalized results for different lattice distances in the PAM compared to a conventional microscope and an ideal confocal microscope. Even for relatively small lattice distances a significant sectioning effect is apparent, although at the cost of an increased background. With increased lattice spacing, the response becomes sharper and suppression of the background is more efficient. The conventional response deviates from the expected straight line as a result of simulating an infinitely large object in a finite sized image (truncation effect). Had we simulated the pseudorandom sequence approach, the result would have been identical except for a constant offset. These results are comparable to calculations for tandem scanning microscopy (Wilson & Hewlett, 1991). Figure 3 presents a simulation based on a more complex small 3-D object, consisting of a thin (0·1 mm) spherical shell with a diameter of 5 mm in which a solid sphere 1·6 mm in diameter was embedded. The peak grey value of the shell was twice that of the inner sphere. We have used a similar object for evaluating deconvolution algorithms (Verveer & Jovin, 1997). Equations (9) and (12) were used to simulate PAM microscope images using calculated excitation and emission PSFs. The panels in Fig. 3 show the following images: (A) single section through the middle of the simulated object; (B) and (C) single patterns generated by the SLM for the shifting grid and the pseudorandom 196 P. J. V E RV E E R E T A L . Fig. 3. Simulation of the imaging of a 3D object using the shifting grid approach and the pseudorandom sequence approach. The object consisted of a spherical shell with a thickness of 0·1 mm and a diameter of 5 mm and a small embedded 1·6-mm solid sphere. The peak grey value of the shell was twice that of the embedded sphere. Parameters: SLM element size h ¼ 17 mm, NA ¼ 1·3, M ¼ 100, excitation wavelength ¼ 488 nm, emission wavelength ¼ 520 nm, refractive index ¼ 1·515, sampling distances x and y ¼ 34 nm, z ¼ 100 nm. Shifting grid approach: square grid with lattice distances dx ¼ dy ¼ 10 × h. Pseudorandom sequence length: 128, tiled on a square grid with the same lattice distances. (A) A single slice through the middle of the simulated object. (B) Single SLM pattern for the shifting grid approach. (C) Single SLM pattern for the pseudorandom sequence approach. (D) Conjugate image Ic generated by the shifting grid approach. (E) Wide-field image. (F) Conjugate image generated by the pseudorandom sequence approach. (G) Nonconjugate image Inc generated by both approaches. D equals F ¹ G and E equals F þ G. See text for further explanation. sequence approaches, respectively; (D) conjugate image Ic for the shifting grid method. The pixelation effect due to the finite size of the SLM is clearly visible. For imaging of a real object with a less than ideal PSF, this effect will not be so apparent; (E) wide-field (conventional) image; (F) conjugate image Inc for the pseudorandom sequence approach; (G) nonconjugate image that apart from an intensity difference is equal for both approaches. The difference between the images of F and G is equal to the confocal image in D, and their sum is equal to the wide-field image in E. Note that images F and G are similar. That is, the difference that constitutes the reconstructed confocal image is small in magnitude compared to the underlying wide-field signal in both signals, reflecting the properties of this particular object. 3.2. Experimental verification of the sectioning properties The first implementation of the PAM is described in Hanley et al. (submitted manuscript). The instrument used a DMD to modulate the excitation light according to one of the two schemes described in Section 2. In contrast to the set-up of Fig. 1, the emission light was detected via a separate optical path that was not modulated by the DMD. By taking a separate exposure for each illumination pattern and applying image processing techniques to reconstruct the image, we were able to demonstrate both imaging modes of the PAM for biological specimens. The simulations based on the imaging of an infinitely thin fluorescent plane presented in the previous section have not been verified experimentally as a result of the technical difficulties in preparing a suitable sample. However, a similar experiment was performed in reflection mode by scanning a mirror through the plane of focus. Although the imaging model in reflection is somewhat different (Wilson & Hewlett, 1991), the effects of the lattice period on the sectioning and background suppression can be evaluated. Figure 4 presents the results of such an experiment for several lattice distances. The DMD was used to illuminate a mirror with a square pattern of points for a q 1998 The Royal Microscopical Society, Journal of Microscopy, 189, 192–198 P RO G R A M M A B L E A R R AY M IC RO SC O P E ( PA M ) Fig. 4. Experimental verification of the sectioning and background suppression in the PAM. A square grid was used with lattice distances dx ¼ dy ¼ {3, 5, 10, 20} × h. The response for conventional imaging is also plotted. SLM parameters: h ¼ 17 mm (16 mm þ 1 mm between-mirror gap) (DMD Texas Instruments, Dallas, TX). Objective NA ¼ 1·3, M ¼ 100 oil immersion, refractive index ¼ 1·515 (Plan-NeoFluar, Zeiss). Wavelength range 450–490 nm (bandpass filter, Zeiss). The apparatus for experimental verification is described in Hanley et al. (submitted manuscript). The sample was displaced along the z-axis with a computerized focus control (Ludl Electronic Products, Hawthorne, NY) and data acquisition was controlled with IPLAB Spectrum (Signal Analytics, Vienna, VA). Camera system: CH220 (Photometrics, Tucson, AZ) with KAF1400 sensor. All components were installed in an Axioplan microscope (Zeiss). given lattice distance. A single exposure was taken for each axial displacement. We used the image in which the mirror was at the focal plane to determine the centre positions of a number of illumination spots from their intensity distributions. The intensity as a function of the axial displacement was determined at these positions and averaged after normalization with the value at the focal plane. Although Fig. 2 models a fluorescent response and Fig. 4 is a measurement in reflection, both follow the same general trend. The background systematically increases as the lattice distance decreases. Thus, as the lattice period decreases the confocal signal becomes harder to distinguish from background noise. The peak widths in Fig. 4 are greater than expected for a confocal microscope operation in reflection. However, we expect this to improve in an instrument corresponding more closely to Fig. 1. 4. Conclusions Two modes of operation of the PAM for confocal imaging have been described. Both provide substantial advantages in speed and signal level compared to the widely used CLSM. The shifting grid approach has the advantage of being simple to q 1998 The Royal Microscopical Society, Journal of Microscopy, 189, 192–198 197 implement, as well as yielding a confocal image without postprocessing. The response is similar to that of the tandem scanning microscope in that the lattice distance of the grid plays the same role as the pinhole spacing in a Nipkow disc. However, this parameter is freely programmable in the PAM, a significant advantage. The use of a pseudorandom sequence of finite size tiled on a grid produces an image consisting of a conventional term plus a confocal term equal to that obtained with a shifting grid of equivalent spacing. The length of the sequence defines the lattice sizes of the grid upon which it is tiled and, thereby, the resolution and sectioning strength as in the shifting grid approach. Although some post-processing is needed, the frame integration time for a given signal level can be substantially shorter. The ‘out-of-focus’ light normally rejected in the CLSM can be collected with a second camera and used to enhance the primary ‘in-focus’ image. With the pseudorandom sequence approach, the difference of the two images constitutes a confocal image with twice the intensity obtained by correcting with a separately acquired conventional image. We are investigating the use of deconvolution algorithms for combining the two images obtained in the form of z-stacks using both approaches so as to achieve improved reconstructions. Acknowledgments We thank Dr Robert Clegg for constructive comments on the manuscript. P.J.V. is the recipient of a fellowship award (Starker–Werner Fonds) of the Max Planck Society. L.J.v.V. is supported as a Postdoctoral Fellow by the Dutch Royal Academy of Sciences (KNAW). References Agard, D.A. (1984) Optical sectioning microscopy: cellular architecture in three dimensions. Annu. Rev. Biophys. Bioeng. 13, 191–219. Brakenhoff, G.J., van Spronsen, E.A., van der Voort, H.T.M. & Nanninga, N. (1989) Three-dimensional confocal fluorescence microscopy. Meth. Cell Biol. 30, 379–398. Carrington, W.A., Lynch, R.M., Moore, E.D.W., Isenberg, G., Fogarty, K.E. & Fay, F.S. (1995) Superresolution three-dimensional images of fluorescence in cells with minimal light exposure. Science, 268, 1483–1487. Egger, M.D. & Petráň, M. (1967) New reflected-light microscope for viewing unstained brain and ganglion cells. Science, 157, 305– 307. Fewer, D.T., Hewlett, S.J., McCabe, E.M. & Hegarty, J. (1997) Directview microscopy: experimental investigation of the dependence of the optical sectioning characteristics on pinhole-array configuration. J. Microsc. 187, 54–61. Golay, M.J.E. (1961) Complementary series. IRE Trans. Inf. Theory, 7, 82–87. Harwit, M. & Sloane, N.J.A. (1979) Hadamard Transform Optics. Academic Press, New York. Juškaitis, R., Wilson, T., Neil, M.A.A. & Kozubek, M. (1996) 198 P. J. V E RV E E R E T A L . Efficient real-time confocal microscopy with white lightsources. Nature, 383, 804–806. Kino, G.S. (1995) Intermediate optics in Nipkow disk microscopes. Handbook of Biological Confocal Microscopy (ed. by J. B. Pawley), chap. 10, pp. 155–165. Plenum Press, New York, 2nd edn. Liang, M., Stehr, R.L. & Krause, A.W. (1997) Confocal pattern period in multiple-aperture confocal imaging systems with coherent illumination. Opt. Lett. 22, 751–753. Motamedi, M.E., Wu, M.C. & Pister, K.S.J. (1997) Micro-optoelectro-mechanical devices on-chip optical processing. Opt. Eng. 36, 1282–1297. Pawley, J.B. (1995) Handbook of Biological Confocal Microscopy. Plenum Press, New York, 2nd edn. Shaw, P. (1995) Comparison of wide-field/deconvolution and confocal microscopy for 3d imaging. Handbood of Biological Confocal Microscopy (ed. by J. B. Pawley), chap. 23, pp. 373–387. Plenum Press, New York, 2nd edn. Verveer, P.J. & Jovin, T.M. (1997) Efficient superresolution restoration algorithms using maximum a posteriori estimations with application to fluorescence microscopy. J. Opt. Soc. Am. A, 14, 1696–1706. Verveer, P.J., van Kempen, G.M.P. & Jovin, T.M. (1997) Superresolution MAP algorithms applied to fluorescence imaging. Three-Dimensional Microscopy: Image Acquisition and Processing IV (ed. by C. J. Cogswell, J.-A. Conchello & T. Wilson), Proc. SPIE, 2984, 125–135. van der Voort, H.T.M. & Brakenhoff, G.J. (1990) 3-D image formation in a high-aperture fluorescence confocal microscope: a numerical analysis. J. Microsc. 158, 43–54. Wilson, T. (1989) Optical sectioning in confocal fluorescent microscopes. J. Microsc. 154, 143–156. Wilson, T. & Hewlett, S.J. (1991) Optical sectioning strength of the direct-view microscope employing finite-sized pin-hole arrays. J. Microsc. 163, 131–150. Wilson, T., Juškaitis, R., Neil, M.A.A. & Kozubek, M. (1996) Confocal microscopy by aperture correlation. Opt. Lett. 21, 1879–1881. ⊗2 Hex ðu; v; wÞ Oðu; v; w ¹ zs Þdu dv dw; ð13Þ where ⊗2 denotes the two-dimensional convolution, defined by Eq. (2). We rewrite the coordinates xd and yd in terms of the SLM element size h and the lattice distances dx and dy by setting ð14aÞ xd ¼ ah þ pdx þ fx yd ¼ bh þ qdy þ fy where fx and fy are given by Eq. (11). Using Eqs. (5) and (6) we see that nX y ¹1 x ¹1 n X Derivation of the imaging equations ÿ ¼ G Mu ¹ xd þ pdx þ fx ; Mv ¹ yd þ qdy þ fy Þ × × ¹∞ Hem y ¹ u; d ¹ v; w M M Ic;R ðxd ; yd ; zs Þ ! nx ¹1 nX y ¹1 ¹1 T NX 1 1X ¼ R ða; bÞGðxd ¹ ah; yd ¹ bhÞ þ N i¼0 2 2 a¼0 b¼0 i " ! nx ¹1 nX y ¹1 þ∞ ÿ 1 1X R ðc; dÞG Mu ¹ ch; Mv ¹ dh þ × 2 c¼0 d¼0 i ¹∞ 2 # x y ⊗2 Hex ðu; v; wÞ Oðu; v; w ¹ zs ÞHem d ¹ u; d ¹ v; w du dv dw M M × nX y ¹1 x ¹1 n X T N x y × Oðu; v; w ¹ zs ÞHem d ¹ u; d ¹ v; w du dv dw M M þ∞ x T y ¼ Hem d ¹ u; d ¹ v; w M M 4 ¹∞ 1þ nX y ¹1 x ¹1 n X a¼0 b¼0 × ½GðMu ¹ ah; Mv ¹ bhÞ ⊗2 Hex ðu; v; wÞÿOðu; v; w ¹ zs Þ du dv dw þ∞ x T y ¼ Hem d ¹ u; d ¹ v; w M M nx ny ¹∞ þ∞ nx ¹1 nX y ¹1 n y ¹1 x ¹1 n X X N NX dða ¹ c; b ¹ dÞ þ 4 a¼0 b¼0 c¼0 d¼0 ¹∞ 4 ÿ × Gðxd ¹ ah; yd ¹ bhÞG Mu ¹ ch; Mv ¹ dh ⊗2 Hex ðu; v; wÞ × d ð15Þ Substituting this into Eq. (13) yields Eq. (9). To derive Eq. (12) we substitute Eq. (8) into Eq. (3): " x Gðxd ¹ ah; yd ¹ bhÞ ÿ ¼ G Mu ¹ xd þ fx ; Mv ¹ yd þ fy : nx ¹1 nX y ¹1 T X Gðxd ¹ ah; yd ¹ bhÞ nx ny a¼0 b¼0 þ∞ nX y ¹1 x ¹1 n X a¼0 b¼0 To derive Eq. (9) we substitute Eq. (4) into Eq. (3): Ic;S ðxd ; yd ; zs Þ ¼ ÿ Gðxd ¹ ah; yd ¹ bhÞG Mu ¹ ah; Mv ¹ bh a¼0 b¼0 ¼ Appendix ð14bÞ ÿ Gðxd ¹ ah; yd ¹ bhÞG Mu ¹ ah; Mv ¹ bh ! # ⊗2 Hex ðu; v; wÞ O ðu; v; w ¹ zs Þdu dv dw: ð16Þ Using Eq. (15) this reduces to Eq. (12). ! Gðxd ¹ ah; yd ¹ bhÞGðMu ¹ ah; Mv ¹ bhÞ a¼0 b¼0 q 1998 The Royal Microscopical Society, Journal of Microscopy, 189, 192–198