Download Active imaging lens with real-time variable resolution and constant

Active imaging lens with real-time variable resolution and constant
field of view
Jocelyn Parent and Simon Thibault1*
Centre for Optics, Photonics and Laser, Laval University, Québec, QC, Canada, G1V 0A6
ABSTRACT
We present a lens with a constant total field of view and real-time variable resolution in certain zones
of interest. This smart imaging lens uses an active optical element to modify as desired the local
distortion. This way, while keeping the total field of view constant, the resolution can be increased in a
zone of interest, at the expense of decreasing it somewhere in the remaining part of the field of view.
We first present the concept of this lens, using a deformable mirror as the active surface. Computer
simulations are done with Zemax in which a magnifying power of 2 in a zone of interest representing
10% of the full field of view is achieved, using a f=12.5 mm lens and a F/# of 18. Different
combinations of theses parameters would allow different performances and results. We then present
experimental results of this lens with a prototype built using a ferrofluidic deformable mirror as the
active element. Experimental results of a zone of increased resolution with a magnification of 1.32 and
a zone of decreased resolution with a magnification of 0.80 are obtained.
©2010 Optical Society of America
OCIS codes: (220.4830) Systems design; (220.1010) Aberrations; (220.1080) Active or adaptive optics
1. INTRODUCTION
The combination of modern technologies in optical systems, faster computers and better optical design programs has
allowed in the past few years to design new optical systems that were unthinkable before. These systems generally have
specific design goals to achieve: improving image quality, increasing resolution, enlarging the field of view, improving
the MTF, enhancing wavelength range, accelerating acquisition speed, decreasing bandwidth, lowering costs,
minimizing tolerancing errors and reducing design time. The optical system proposed in this paper achieves some of
these design goals, specifically a large field of view, increased resolution in pixels/degree where needed while
minimizing the total data bandwidth.
A greater image resolution of an object is clearly desirable. The more pixels on the detector the image is spread
on, the more information about the object can be perceived. For example, in security and surveillance applications, an
increase by a factor of 2 in resolution can be the difference between simply recognizing a human face and identifying the
person.1 These days, detectors with more and more pixels are possible, but in some systems, having a very high number
of pixels is not always a desired solution since it increases the data bandwidth and requires a trade-off between larger
* [email protected] and [email protected]
optics and/or smaller pixels. Often, only a fraction of the image’s entire field of view requires a high resolution and it is a
waste to capture a high resolution image of the full object space.
The idea behind the proposed system is to use an active optical element to adjust in real time the distortion of
the system. Distortion is not an aberration which affects image quality by changing the spot size, but an aberration that
changes the local lens magnification. Consequently, adding controlled distortion in an optical system can be used as a
design parameter. Of course, the produced image can be hard to directly interpret by a human if there is too much
distortion, but it is important to understand that if the object to image mapping is known, it is possible to unwrap the
image with a software code and produce unwrapped images.
Section 2 presents some background of similar optical systems to better understand the idea of the proposed
system. Subsection 3.1 describes in more depth the proposed system, followed by computer simulations in subsection 3.2
and then experimental results in subsection 3.3. A discussion is finally done in section 4.
2. BACKGROUND OF SIMILAR SYSTEMS
The proposed system has similarities with many existing ones. First, it shares some properties of a zoom system since it
can increase the resolution in a given region by changing the magnification. However, it is also very different since it
keeps the total field of view constant. Instead of increasing the resolution (in pixels/degree) by reducing the total field of
view, it is achieved here by increasing it in a zone, while decreasing it in a small zone around it. While doing so, it keeps
the average resolution constant because the total field is conserved. Another way to increase resolution is to use the
techniques of super-resolutions2. However, these are generally done at the software level and they could possibly be used
in combination with the increased resolution done at the optics level as in this paper.
Also, although the idea of using active optics to control distortion is novel, both the idea of using active
elements to enhance optical systems and the idea of controlling static distortion to adjust resolution have been used in the
past. Using deformable mirrors in adaptive optics to correct the phase of a wavefront and improve image quality is well
known because of their uses in modern astronomical telescopes. Another interesting system using an active optical
element comes from foveated imaging, where authors use a transmissive liquid crystal spatial light modulator (SLM)
near the stop to correct the phase of the wavefront for a given object field angle, giving a high quality image in a zone of
interest3. However, this system requires a detector with a large number of pixels because the image resolution is directly
related to detector resolution. The system is designed to allow a single zone of interest with a highly aberrated image
around it since the SLM is used to correct the phase of the wavefront in a defined image location while the other part of
the image is not corrected.
As for the use of controlled distortion as a design parameter in optical systems to increase the resolution in a
given zone, it is already used in Panomorph lenses4. In these lenses, as shown in fig. 1, the resolution is increased by a
factor 2.2 in the desired region of interest as compared to the 0 degree resolution. However, this increase is static, and if
a different zone of interest is required, a new design must be done. The idea of our system is to use the distortion in the
same way, but in real time and with a zone of interest moving across the field, of adjustable dimension and of variable
magnifying power (MP).
The use of distortion as a design parameter can appear strange at first since it is often considered as only an
undesired aberration. However, by looking at the equation of this aberration, the transverse distortion εy =-2fW311H3/rp
depends on the cube of the image height H, but not on the pupil coordinate5. This means that distortion affects each ray
in the system for a given object point in the same manner, that is only displacing the whole spot in the image plane
instead of enlarging it like other aberrations do. For this reason, a highly distorted image can have a very good image
quality and if required to be straightened for a human observer, software algorithm can convert it back to an undistorted
image if the object to image mapping is well known.
Fig. 1: The resolution curve in highly distorted systems as the Panomorph lens can vary greatly with the field
angle, offering an increased resolution in zones of interest. Here, the zone of interest is around 60 degrees, with a
resolution 2.2 times higher than at the center.
Finally, as an example of how distortion can produce different magnifications for different zones of interest, fig.
2 shows two simulated images6 of the same field of view of 140 degrees horizontally, but one with the center as the zone
of interest while the other has increased resolution near the edges.
a)
b)
Fig. 2: Simulation of images of the same scene of 140 degrees horizontally, but with different zones of interest.
a) The image on the left has a resolution of 11 pixels/deg at the center decreasing to 3 pixels/deg at the edge.
b) The image on the right has a center resolution of 3 pixels/deg increasing to 11 pixels/deg at the edges.
3. PROPOSED OPTICAL SYSTEM
3.1. Description of the lens
This novel optical system uses an active optical element to adjust in real time the distortion of the lens. For a given zone
of interest in the field of view, MP is achieved on the image by changing the direction of the rays only in a specific part
of the object. The angular deviation caused to the rays has to be done in a region of the lens where all the rays for a given
field point are very close around the chief-ray and where rays for each field are well separated. These two conditions are
usually satisfied for rays which are far from the aperture stop (close to the front element or close to the image plane). As
shown in previous work 7-8, in wide-angle lenses, where the entrance pupil is very small as compared to the diameter of
the first few lenses, a slight angular variation at the front surface will induce a local magnification change without
affecting the rest of the image. Of course, since the total field of view is constant, a region of increased resolution is
produced at the expense of a region of decreased resolution around it. Also, it must be understood that several possible
distributions of increased resolution exist, like a constant zone of increased resolution or Gaussian shaped increased
resolution, and the choice of which to use depends on the specific object in each case.
For a prototype of this system, the small footprint requirement on the front surface can also be achieved by
using a camera lens with a high F/# in front of the active surface. In this particular case, with the active surface as the
first optical element of the system, with an imager lens producing close to no distortion and having small angles, the
angular deviation produced by the deformable surface is, with a factor 2 because of the reflection, the angular variation
required. In more complex systems, with distortion in the imager or larger incident angles, the theory developed in
previous work must be used8.
The main advantage of using an active optical element is that the parameters (position, MP, dimension) of the
zone of interest can change in real-time. For this prototype, a deformable mirror is used as the active optical element.
Figure 3 shows a schematic of the prototype, showing half of the total field of view. The imager lens is pointed directly
at the deformable mirror at the bottom of the figure and the object at the top is seen by the reflection. Properties of the
deformable mirror and the imager lens, depending on the desired MP, allowed aberrations, dimension of the zone of
interest and total field of view are provided in the next subsection.
Fig. 3: Schematic of the system, with an imager pointing at the deformable mirror, and looking back at the object.
By changing the shape of the deformable mirror, chief-rays are deviated from the original angle (no distortion, in
red) to the desired angle (controlled distortion, in green).
3.2. Computer simulations
Using an optical design program like Zemax, it is possible to simulate some optical properties of a lens like the one
described in subsection 3.1. A reflective active surface is used in this simulation, and the active surface is the first one
after the object. For now, the considered zone of interest is at the center of the field of view, a radial symmetry is used
and a constant-resolution distribution is targeted. From figure 3, and for small angles, it can be understood that the
surface producing this constant resolution is the one where the slope varies linearly with the position. By direct
integration, it allows to conclude that the required surface for a constant-resolution distribution zone of interest is
approximately, for small angles, a parabola.
Now that the desired shape of the surface is found, it is possible to calculate the required amplitude on the
deformable mirror and also the maximum F/#. Since the amplitude required on the mirror depends on the dimension of
the zone of interest, a constant ratio of the angular zone of interest divided by the full field of view of 1/10 is chosen. For
this part of the paper, only the surface required for the augmented-resolution zone is considered, meaning that no shape
or amplitude is given for the reduced-resolution transition zone. A reason for this is that depending on the importance of
this transition zone, different angular lengths ΔΩ of this zone can be used. Figure 4 shows the calculated maximal
mirror amplitude required as a function of both the focal length of the imaging lens and the desired magnifying power
MP.
Fig. 4: With a zone of interest of 1/10 of the full field of view, the required amplitude on the deformable mirror
increases with the magnifying power, but decreases with the focal length of the imager.
For the maximal allowed F/# as a function of the focal length of the imaging lens and the desired MP, the
diameter of the entrance pupil for the imaging lens is calculated as the one which produces a peak-to-valley (PV) OPD of
λ/4 at 550 nm. From this choice, figure 5 is obtained.
From figures 4 and 5, the important parameters of the system can be chosen. Being limited by a
realistic F/# and a deformable mirror having maximum amplitudes of a few tens of µm, a MP of 2 and a focal
length of 12.5 mm seems a good choice, requiring a parabola of 55 µm PV in a diameter of 19.7 mm and a F/#
of 18 for the imager. To allow a better view of the shape of figure 4 and 5, 2D versions of these figures are
given respectively at figures 6 and 7 for the chosen values of MP=2 and f=12.5 mm.
Fig. 5: Given a limit of λ/4 PV OPD change on the wavefront, the minimum F/# of the imager lens tends to
increase with increasing magnifying power or increasing focal length of the imager.
a)
b)
Fig. 6: A better view of figure 4, required deformable mirror amplitude as a function of a single parameter: a) For
a fixed MP of 2. b) For a fixed focal length of 12.5 mm.
a)
b)
Fig. 7: A better view of figure 5, minimum F/# as a function of a single parameter: a) For a fixed MP of 2. b) For
a fixed focal length of 12.5 mm.
The full system is now simulated in Zemax. A perfect paraxial lens is used as the imager while a grid sag
surface is used to model the deformable mirror in the program. Using a 1/2.5’’ detector, the long-axis full angle is found
to be 12.85 deg, meaning a zone of interest of 1.285 deg. Here, the zone of settling back to normal resolution is a
parabola matching both the slope at the intersection with the zone of interest and the slope of 0 at a ΔΩ of 3 deg,
requiring a PV amplitude of 119 µm. Figure 8 shows the sag of the deformable mirror. Figure 9 shows the grid and linear
distortion of the resulting system. Figure 10 shows the RMS radius as a function of the field of view with the diffraction
limit radius of 12.08 µm given as reference.
Fig. 8: Amplitude of the deformable mirror as a function of the radial coordinate. The zone of interest is a positive
parabola, the transition a negative parabola and the rest is flat.
a)
b)
Fig. 9: Both figures show that a magnifying power of 2 is effectively achieved. a) With the grid distortion, it can
be seen in the different spacing between the crosses in the center and at the edge. b) With linear distortion, at high
angles, a -50% distortion relative to center field shows a MP of 2. It must also be noted that there is no distortion
in the zone representing 10% of the total field of view as desired by the zone of interest.
a)
b)
Fig. 10: RMS spot radius as a function of field angle. The dashed horizontal line is the diffraction limit of
12.08 µm with a F/# of 18. a) With the ideal focus of the paraxial lens b) With a best-focus shift of -0.347 mm.
3.3. Experimental results
To verify the simulations from subsection 3.2, the prototype is tested experimentally. As the deformable mirror, because
of the high amplitude required, a ferrofluidic liquid deformable mirror is used9. For the imager lens, a commercial lens
from Fujinon, model HF12.5 SA-1, having a focal length of 12.5 mm and F/# ranging from 1.4 to 22 is used. The camera
is an AVT Guppy F-503C having a 1/2.5’’ detector.
The ferrofluidic mirror can be placed in a closed-loop with a Shack-Hartmann sensor to measure the amplitude,
but has to be removed from the loop to be used in a setup with the imager lens. The mirror can generate amplitudes of
hundreds of µm, but because of limitations from the overheating electronics and the maximum slope measurable with the
Shack-Hartmann sensor, a maximum parabolic term of (35±5) µm PV in a diameter of 2.45 mm was used. The high
uncertainty on the amplitude of the mirror is due to not using the closed-loop with the Shack-Hartmann sensor to
measure it. Only the zone of interest with a parabola was generated, letting the liquid itself shape the transition zone.
Also, no reflective MELLF was deposited on the liquid mirror for this test, meaning that the mirror had a very low
reflectivity of about 4%. For this reason, the target object was a neon light source on the ceiling, with a regular pattern on
the plastic diffuser. Finally, since the imager lens was 310 mm vertically over the mirror, simulations allow calculating
that a MP of 1.405 and 0.776 should be expected with this amplitude for a positive and a negative parabola respectively.
For this series of measurements, a F/# of 16 was used. Figure 11 shows for a positive, null and negative parabola on the
mirror the resulting image of looking directly at the ceiling light. The regular pattern in the plastic diffuser allows
measuring a position, in pixels, for each of the small circular targets, obtaining figure 12 showing the position and the
derivative of the position for 20 targets going from the edge of the mirror to the center of the zone of interest.
4. DISCUSSION
4.1. Theoretical simulations
The proposed lens performances are greatly limited by the parameters of the system as shown by figures 4 and 5. For the
requirement regarding the active surface amplitude, as seen in figure 4, high amplitude is required for significant MP or
for a short imager focal length. The combination of both the F/# and the mirror was pretty restrictive in our case, but for
another version of the system, the active surface does not need to be the first one of the system, greatly reducing the
requirement on the amplitude. If the active surface is after an optical element which produces an angular magnification
as seen in figure 13, the required angular shift on the active surface is greatly reduced for the same total MP. Also visible
in the example of figure 13, the active surface is not restricted to continuous mirrors. For large angular variations, an
array of micro-mirrors or even a diffraction-based transmissive liquid crystal SLM could replace the presently used
continuous mirror.
For the limitation on the F/#, since a system with a MP close to 1 would be contrary to the idea of this paper of
increasing the resolution, figure 5 shows that a small focal length has to be used to obtain a realistic F/#. This includes
wide-angle lenses which generally have a very small focal length and a very small entrance pupil diameter. This is
another reason why the panoramic system of figure 13 could be a good example of a system using this technique. Also,
figure 5 is obtained by setting a maximum PV OPD produced by the mirror of λ/4. However, the dominating aberrations
produced by an off-axis parabola with a small beam diameter are defocus and astigmatism10, and as shown by figure 10,
using a best focus image position instead of the natural flat mirror image position could allow loosening this
requirement. In the experiments, a F/# of 16 was used, but images appeared to be as clearly focused with a F/# of 11
once the imager focus plane was adjusted.
The Zemax simulations presented in figures 8 to 10 show the performances of the simulated system. In figure 9,
the distortion is presented in two different ways, but the idea is the same. Since the distortion is calculated with a
reference in the center of the object, it is easy to understand why the region of 1/10 of the field of view in the center has
0 % distortion. For the region over [(2x1.285+3)/12.85]=0.43 of the field of view, since the mirror is flat, a constant
distortion is logical. The value of -50% is also in clear agreement with the choice of a MP of 2. As for the transition
zone, since a parabolic shape was set on the mirror, the distortion pattern is as expected.
4.2. Experimental prototype
The images presented in figures 11 a) and c) show controlled distortion to change the magnification in a zone, but still
keeping a constant field-of-view as shown by comparing the edges with figure 11 b). In these images, the zone in the
center with actuators is 24.5 mm in diameter, while the full mirror is 100 mm. The octagonal footprint from the 91
actuators is visible at the edge of this zone of interest. At the edge of the full mirror, surface tension in the liquid
produces a meniscus of about 1 mm, explaining the high distortion there. The asymmetry in the figure can be explained
by a wedge that was present on the mirror base, producing an unequal ferrofluid height. Also, the parabola produced was
not perfect, other Zernike terms were present. Even so, the results obtained clearly show the needed zone of interest, with
a tolerable focus in all three zones.
Figure 12 shows the position and resolution of each target. For a flat mirror, with no distortion in the imager
lens, there is no distortion in the complete system, explaining the linear relation between position and resolution. Small
differences in pixels are present because of the uncertainty of finding the real center of each target, explaining the noise
in the resolution graph. For negative and positive parabola on the mirror, there is now distortion in the system and the
position curve is no longer linear. The average resolution in the center zone can be calculated from the average of the last
5 values and resolutions of 28.7, 21.75 and 17.5 are respectively obtained for the positive, flat and negative shape on the
mirror. Relative to the flat surface, this equals MP of 1.32 and 0.80 in 1 dimension, values close enough to the 1.405 and
0.776 simulated earlier considering the high uncertainty that was present experimentally with the shape of the mirror. A
MP of 1.32 may seem low, but when considering a 2-dimensional object, this means 1.74 times more pixels, a nonnegligible improvement as seen visually in figure 11. Also visible in figure 12 a), for targets under # 5 and for targets
#19-20, the positions are about the same for all three cases. For targets under #5, the explanation lies in the flat mirror in
this zone for all three cases. For targets 19-20, it is a consequence of the cylindrical symmetry of mirror deformation.
a)
b)
c)
Fig. 11: Images obtained with different shapes on the deformable mirror in the zone of interest at the center of
each image: a) A positive parabola produces a zone of increased resolution in the center with a fast lowerresolution transition around it. b) The flat mirror makes a reference without distortion. c) A negative parabola
produces a zone of decreased resolution in the center with a fast higher-resolution transition around it.
a)
b)
Fig. 12: For 20 targets on a horizontal line from the 3 images of figure 11, a) the position in pixels and b) the
resolution in pixels/target. In the zone of interest (last 5 targets), the average resolution is 17.5 with the negative
parabola, 21.75 with the flat mirror and 28.7 with the positive parabola.
Fig. 13: Example of the position of a liquid crystal SLM in a wide-angle lens system. Here, the active surface is
not the first, and the required angular deformations are thus lowered by a factor depending on the shape of the
front lens. In the region where the SLM is placed, a small localized angular deviation causes an significant change
in the object angle, but only for this field angle, without affecting other object fields. This example lens is based
on a projection lens from Zebase.
5. CONCLUSION
The proposed lens in this paper offers capabilities never seen before in optical systems, a spatio-temporal variable
resolution with a constant total field-of-view. This lens has fundamental limits to the F/# to assure that acceptable
aberrations are present and on the active surface to achieve the required angular displacements. By respecting these
limits, a prototype was built using a commercial lens as the imager in front of a ferrofluidic deformable mirror. Since the
results obtained are comparable to what was expected by numerical simulations, it is possible to conclude that the idea
behind this system works. The lens is limited by current prices and performances of active surfaces, but this smart lens
could become more realistic with future technologies. Also, it must be noted that this system could be used in
combination with other imaging technologies like foveated imaging to improve the resolution and correct the aberrations
in a given area or like super-resolution to achieve even higher resolution gains. For all these reasons, it could be useful to
think about the idea presented here when designing the optical lenses of the 21st century.
ACKNOWLEDGMENTS
This research was supported by the Natural Sciences and Engineering Research Council of Canada and by the NSERC
Industrial Research Chair in Lens Design. Special thanks to the team of Ermanno F. Borra and Denis Brousseau for
letting us use their latest deformable mirror and to Immervision for letting us use the Panomorph Geometrical Model
Simulator.
5. References
[1] Smith, W. J., “Modern Optical Engineering,” 4th ed., McGraw-Hill (2007).
[2] Zalevsky, Z. and Mendlovic, D., “Optical Superresolution, ” Springer, (2003).
[3] Curatu, G., and Harvey, J., “Lens design and system optimization for foveated imaging,” Proc. SPIE, Vol. 7060, (2008).
[4] Thibault, S., Gauvin, J., Doucet, M. and Wang, M., “Enhanced optical design by distortion control,” Proc. SPIE, Vol. 5962, (2005).
[5] Greivenkamp, J. E., “Field guide to geometrical optics,” (2004).
[6] Panomorph Geometrical Model Simulator, Version 1.0.0.0, ImmerVision (Montreal, Canada).
[7] Parent, J. and Thibault. S., “Tolerancing panoramic lenses,” Proc. SPIE, Vol. 7433, (2009).
[8] Parent, J. and Thibault, S., “Spatial dependence of surface error slopes on tolerancing panoramic lenses,” Appl. Opt. 49, 2686-2693 (2010).
[9] Brousseau, D., Borra, E.F., Rochette, M. and Landry, D.B., “Linearization of the response of a 91-actuator magnetic liquid deformable mirror, ”
Opt. Express 18, 8239-8250 (2010).
[10] Malacara-Hernandez, D., “Some parameters and characteristics of an off-axis paraboloid,” Opt. Eng. 30, 1277-1280 (1991).