Download a PDF of the paper on this project published in the OCEANS 2005

Development of a Free-Drifting Submersible Digital Holographic
Imaging System
D.W. Pfitsch
Johns Hopkins University ([email protected])
E. Malkiel
Johns Hopkins University([email protected])
Y. Ronzhes
Johns Hopkins University
S.R. King
Johns Hopkins University
J. Sheng
Johns Hopkins University
J. Katz
Johns Hopkins University([email protected])
I. INTRODUCTION
Abstract – Submersible film-based holographic systems
have demonstrated the unique ability of holography to
provide high resolution, three-dimensional, in-situ images of
marine organisms. Inherently, use of film limits the frame
rate and total number of holograms. This paper describes a
submersible,
free-drifting,
digital
holographic
cinematography system that has a real-time fiber optic
communication link. This system collects dual-view digital
holograms at a rate of 15 frames per second, enabling the
user to observe the behavior of marine plankton and
distinguish motile organisms from abiotic particles. In order
to follow the particles in time, the sample should have as
little motion relative to the cameras as possible. To achieve
this goal, the submersible is neutrally buoyant, and has high
drag generating elements at the height of the sample volume.
In addition, the components surrounding the sample are
streamlined and designed to minimize the local flow
disturbance. The data from the two digital cameras and
other sensors are transmitted at 120 MB/s through a 1 km
long, 250 µm diameter, fiber optic cable to an acquisition
system located on a research vessel. The optical fiber is
spooled out from the submersible by a powered mechanism,
as the submersible drifts away from the vessel. Releasing
the fiber out at a rate greater than that of the drifting speed
minimizes the transmission of forces through the cable,
effectively decoupling the submersible from the cable and
vessel dynamics. A variable buoyancy system provides
vertical position control while still allowing the system to
drift vertically with the surrounding fluid, i.e. follow
internal waves. The dual-view holo-camera records two
in-line holograms from orthogonal directions, each with a
volume of 40.5 cm3. Without lenses, the resolution in the 3.4
cm3 volume where the beams cross each other, is about 7.4
µm in all three directions. Outside of the overlapping region,
the resolution in the beam axial direction is lower, but the
lateral resolution remains 7.4 µm. An optional 2X lens
doubles the resolution, but reduces the sample volume.
The first field deployment for this system took place in June
2005, in the Ria de Pontevedra, Spain. It was used for
examining thin layers of harmful algal blooms.
Harmful algal booms (HAB) are becoming an increasingly
significant concern for fisheries and for human health. The
specific motivation for the present study is the toxic
dinoflagellate, dynophysis, a 30 µm diameter motile organism
that causes shell fish poisoning [12]. While traditional water
sampling techniques combined with microscope examination
provide considerable insight into the behavioral characteristics
of micro-organisms [16], it does not provide information on their
behavior in their natural environment, such as interaction with
other organisms/particles and with the surrounding flow.
Observations on plankton behavior in-situ are essential for
understanding the reasons for the algae bloom formations and
life cycles. Determination of what combination of fluid
dynamics, e.g. horizontal shear, and migratory behavior would
result in formation of thin layers [11] of harmful algal blooms
has received little attention because of the difficulty of
performing such observations.
Two-dimensional digital and video imaging [4, 6, 15], while
informative, are limited in the extent of behavioral information
that can be gained due to the short time frame in which the
subject remains in focus. Submersible film holography has
been used to acquire 3-D still images of a sample volume
containing plankton [2, 8, 10, 17], and can even be used to
measure the instantaneous velocity by recording double
exposure holograms, as has been done in laboratory settings [1,
14, 18]. This approach has been successful in measuring 3D
velocity fields and determining particle distribution, but lacks a
suitable time frame that could be used to observe plankton
behavior. Furthermore, the data base is inherently limited, in part
due to the need to submerge film rolls or plates, and in part due
to the cumbersome process of film development followed by
optical reconstruction. However, the film resolution is much
higher than that of any digital recording medium, enabling use of
off-axis holography that can resolve an order of magnitude more
particles [18].
Three main criteria should be met in order to capture
organism behavior in-situ. First, the system must be able to
record a series of images at a framerate that enables observations
and analysis of behavior. Second, the imaging technique must
1
1230
GPS/RF Unit
Flashing Beacon
Sample Volume
15 fps Digital Cameras
Drag Generating
Elements
Streamlined
Hull
Laser
Beams
Pressure
Housing
Electronics, Battery and
Laser
Acoustic
Transponder
SeaBird CTD
Ballast Tank
Figure 1: The hull, drag generators, and sample volume
arrangement of the submersible holography system.
the (undisturbed part of the) illuminating beam with light
be able to measure 3D motions, and have extended depth to
scattered from objects in the sample volume, i.e. it contains all
track objects regardless of their direction. Third, the motion of
the information needed for reconstructing the location and shape
the system relative to the subjects of investigation should be
of particles over the entire sample volume. The holograms are
minimized in order to allow observations on the behavior of the
reconstructed
numerically
using
Fresnel
Huygens
same organisms for as long as possible.
transformation, as described in detail in [8] and [9], providing
The submersible digital holographic cinematography
detailed, in-focus images of any desired plane located within the
stystem described in this paper satisfies all of these criteria.
sample volume..
Holography maintains high lateral resolution over an extended
The 3D coordinates of particles measured by a single in-line
depth, enabling measurements of 3D motion of
hologram are accuarate to about pixel resolution in directions
micro-organisms, which would rapidly move out of the focal
that are perpendicular to optical axis of the illuminating beam,
plane of conventional, 2D imaging techniques. Digital
but are substantially less accurate in the beam direction due to
holographic cinematography can follow the same subject as it
the so-called depth of focus effect [8, 18]. In order to obtain 3D
passes through the sample volume. The number of frames
data at the same resolution, we use a dual-view system, which
containing the same object depends on magnification, relative
records two in-line holograms of the same sample volume from
motion, and frame rate. Thus, the submersible platform is
orthogonal directions.
designed to drift with the local current, and have minimal
As sketched in Figures 1 and 3, coherent illumination is
velocity relative to the fluid and particles in the sample volume.
achieved by spatially filtering, expanding and collimating a
The submersible holography system is described in Figures 1-3.
beam generated by a Q-switched, diode pumped ND-YLF laser,
Figure 1 shows the submersible platform highlighting elements
which is manufactured by Crystalaser. The red wavelength, 660
that enable it to drift with the local flow. Figure 2 provides
nm, is chosen specifically because few organisms in the ocean
details on the control and data transmission systems, and Figure
can detect red light [3], therefore minimizing changes in
3 is a schematic description of the optical setup. Details are
behavior due to phototrophic responses. The dual-view
provided below.
holograms are recorded using two 2K x 2K pixels, 15 frames per
second, 8 bit digital cameras manufactured by Pulnix. Electronic
shuttering synchronized with the laser pulses minimizes the
II. IN LINE DIGITAL HOLOGRAPHY
effect of ambient sunlight, which allows for daytime operation.
Without lenses, the total sample volme of each in-line
Capturing an in-line digital hologram consists of
hologram is 40.5 cm3. Over the 3.4 cm3 volume where the two
back-illuminating a sample volume with a collimated, coherent
light and recording the resulting diffraction pattern on a CCD
sample volumes intersect, the resolution is about 7.4 µm (pixels
array. This recorded image is a result of interference between
pitch) in all three directions. Installing a 2X lens in front of each
2
Vessel: Data Acquisition and Tracking Sytem
Fiber
Optic
MUX/
DEMUX
1510 nm
1530 nm
1570 nm
1550 nm
CTD
Camera
Camera
GPS/RF
Digital to
Optical
Converter
Digital to
Optical
Converter
Four
Terabyte
Data
Acquisition
System
Optical to
Digital
Converter
Digital
Compass
Tracking
Computer
Acoustic Transponder
1510 nm
1530 nm
Fiber
Optic
MUX/
DEMUX
1570 nm
1550 nm
250µm Single Mode
Fiber optic Cable
(1000m)
Ballast
System
Controller
Optical to
Digital
Converter
GPS
Acoustic USBL Transciever
Digital Holographic Submersible
Beacon
Control
Computer
Drop Weight
Figure 2: Components and communication lines of the system
area is selected to maintain a relative velocity of less than 10
mm/s based on a simplified analysis involving the system
inertia and drag forces on different components.
Recognizing that typical coastal flows are non-uniform, it is
essential to concentrate the drag force at the same elevation
as the sample volume. Thus, the main pressure housing of
the submersible is surrounded by a fiberglass, streamlined
hull (Figure 1).
Communication as well as data transfer to and from the
submersible are performed using a 1 km long (typically),
250 µm diameter, single mode fiber optic cable. The cable
spool is mounted on the bottom of the submersible, and is
spooled off using motorized drive at a rate that is slightly
faster than the drifting speed. This approach decouples the
submersible from the drag forces on the cable, and from the
dynamics of the researh vessel where the operator is sitting.
The submersible is neutrally buoyant, and is equipped
with a ballast system that enables control of its vertical
position.
The ballast is varied using a positive
displacement pump, which pumps water into and out of a
1.6 L rubber tube located within a pressure housing under
the main hull. This depth control method allows the
controller to bring the submersible to a desired depth,
without creating major disturbances. such as those caused
by thrusters or compressed air-ballast systems. Once
camera about doubles the resolution, but decreases the sample
volume. Use of microscope objectives [13] can increase the
resolution to sub-micron levels.
III. THE FREE DRIFTING PLATFORM
In accordance with the third criteria mentioned in the
introduction, the submersible platform is designed to drift with
minimal velocity relative to the fluid in sample volume, and to
minimize the disturbance to the local flow. Consequently, it can
view the behavior of the same organisms for as long as possible.
To minimize the flow disturbances, the sample volume is
located between two streamlined towers/fins, 367 mm away
from the main body. The fins are flat on inner (sample
volume) side, and convex on the outer side in order to
minimize the fluid acceleration as it passes between the
towers. As illustrated in Figures 1 and 3, the collimated
laser beams are directed into each tower, and then
transmitted through windows and sample volumes into the
opposite tower, where the corresponding digital cameras are
located.
Large drag-generating structures are attached to the
outer sides of each tower, at the same height as the sample
volume. While the system drifts, these drag generators are
aligned perpendicularly to the relative flow. Their surface
3
Overlapping sample region
Windows
Prism
30 mm
expanded beam
Digital Cameras
Laser
50-50 Beamsplitter
Figure 4: Deployment of the submersible holography
system.
lifting brackets attached to the towers. Once in the water
and floating, the submersible is realeased, leaving the fiber
optic cable as the only link to the research vessel. To track
the submersible during deployment, it is equipped with an
acoustic transponder. A TrackLink USBL transciever,
manufactured by LinkQuest is intalled off the side of the
vessel to query the transponder postion. Used in conjuction
with a digital compass and GPS unit, this acoustic tracking
system monitors the absolute position of the submersible as
it drifts away from the vessel.
In a typical deployment, after being released the ballast
is adjusted to make the submersible dive quickly to below
the target depth, and then rise slowly into the desired
sampling elevation. This procedure minimizes the effect of
vertical motion on the flow within the sample volume.
Limited by battery power, the submergence period is about
1 hour. When the desired data has been acquired, the
submersible rises to the surface. Once outside of the water, a
Garmin GPS unit with RF transmitter located at the top of
one of the towers can be polled to provide the location of the
Spatial Filter
Figure 3: Optical setup of the dual beam digital
holography system. Only the centerline of the beam is
shown for clarity. The blue lines depict the in-water
sample volume
adjusted, the neutrally buoyant platform can also drift
vertically with the surrounding fluid, aided by vertical drag
forces on the large surface area of the main hull. Prior to
deployment, the system is adjusted to be nuetrally bouyant
when the ballast tube is half full, by mounting syntactic
foam on the drag elements (Figure 4), which also helps in
maintaining the system stability. Additional elements are
installed within the streamlined hull.
IV. SYSTEM CONTROL & DEPLOYMENT
The system is internally powered by a Lithum-Ion
battery pack that enables deployment for about one hour,
and lasts for about two hours. As noted before and
sketched in Figure 2, we communicate and transmit all the
data through a single fiber optic cable. Two semi-custom
ARVOO Optilink units convert the digital images and
RS-232 signals of the CTD and communication lines into
four optical signals, each at a different laser frequency
(wavelengths of 1510, 1530, 1550 and 1570 nm). A
MUX/DEMUX unit from AFOP then combines all four
optical signals into a single optical fiber. An identical
MUX/DEMUX unit separates the four optical signals at the
other end of the cable on the research vessel, and each is
converted back to a digital signal using ARVOO Optilink
units. In the present configuration, the image acquisition
rate is about 120 MB/s.
A computer integrated by Boulder Imaging, containing
two 1 TB RAID drives and 2 TB of back-up hard-disk space,
acquires the digital images and CTD data. These data are
viewed in real time, enabling the operator to select whether
and which holograms to save. A separate laptop computer
runs the communication software that controls and monitors
the submersible “vital functions,” e.g. bouyancy, battery
voltage, fiber optic cable release mechanism and leak
detectors.
To deploy the submersible digital holography
system, it is lowered over the side of the vessel by two
time (minutes)
0
0
10
20
30
40
depth (m)
5
10
15
20
25
Figure 5: Sample depth record of the platform as
measured by the on-board CTD during a deployment
in the Ria de Pontevedra.
4
1mm
Figure 6: A Tiny medusa traversing downward at 45º at a rate of 6.8 mm/s during contraction and
virtually not moving during the resting phase of its swimming cycle.
platform. A flashing beacon located on the other tower
facilitates recovery of the submersible at night. The GPS
system and beacon have independent power sources, and
can operate for extended periods.
A drop-weight supported by a solenoid-controlled pin is
located at the very bottom of the submersible. It is designed
to be released, which quickly brings the system to the
surface in case of an emergency. The drop-weight is
released automatically if communication or power are lost,
but can also be released by a command over the fiber optic
cable.
When recoverd, the lithium-ion battery can be replaced
for a quick turn around or can be recharged, which can takes
up to 3 hours. The 1000 m optical fiber is clipped and a
new spool is installed after each deployment. Since we use
an unprotected fiber, its cost is minimal, and there is no
compelling reason or provision to re-spool it. If we only use
part of the fiber, the remaining spool can be reconnectorized.
Since the GPS system utilizes an independent power source,
it can still be used for finding the submersible platform even
if all other means of communication fail.
project called HABIT [5, 12], which studies thin layers of
the HAB, Dinophysis acuminata.
The European
collaborators
included
biological
and
physical
oceanographers from Spain, France, UK, and Ireland.
Various sampling methods were used to detect the
dinoflagellates, and provide approximate depths and
locations, where the submersible should be deployed to
observe their behavior.
As is common on the first deployment of complicated
instruments, we encountered many technical problems. We
resolved most, but due to time constraints could not solve
problems with the optical fiber release mechanism. Thus,
the 250 µm fiber was spooled out off the deck of the
research vessel. Furthermore, to ensure that the instrument
would not be lost during these preliminary runs, we attached
a bouyant safety tether to the submersible. The other end of
this tether was attached to a small zodiak, which followed
the platform, as it drifted with the current. By continuously
maintaining slack in the tether, we did our best to minimize
its effect on the trajectory and velocity of the submersible
platform.
Despite these setbacks, we completed 10 successful
deployments and recorded over 2 TB of digital holograms.
After reconstruction of multiple planes, these data will
amount to over 250 TB of images.
The longest
deployment took 50 minutes, during which the the
submersible performed two consecutive dives to a depth of
25 m, and the system drifted close to 150 m away from from
the research vessel. A sample depth record for one of these
V. RIA DE PONTEVEDRA
The first series of deployment tests of the free-drifting
digital holography system took place during June 2005, in
the Ria de Pontevedra, on the coast of northwest Spain.
The Johns Hopkins University group was working in
collaboration with a European research team working on
5
in studying ocean particle dynamics,” Opt. Eng. Vol
18, pp. 524-525, 1979.
[3] R. B. Forward, “Diel vertical migration: Zooplankton
photobiology and behaviour,” Oceanography and
Marine Biology—An annual Review, vol. 26, pp.
361-393, 1988.
[4] M. Gallager, C. S. Davis, A. W. Epstein, A. Solow,
and R. C. Beardsley, “High resolution observations of
plankton spatial distributions correlated with
hydrography in the Great South Channel, Georges
Bank,” Deep-Sea Res., vol. 2, no. 43, pp. 1627-1663,
1996.
[5] P. Gentien, M. Lunven, M. Lehaitre, and J. L. Duvent,
“In-situ depth profiling of particle sizes,” Deep-Sea Res.,
vol. 42, pp. 1297-1312, 1995.
[6] J. S. Jaffe, and P. J. S. Franks, “Simultaneous imaging
of phytoplankton and zooplankton distributions,”
Oceanography, vol 11, no. 1, pp. 24-29, 1998.
[7] E. Malkiel, O. Alquaddoomi, and J. Katz,
“Measurements of plankton distribution in the ocean
using submersible holography,” Meas. Sci. Technol.,,
vol. 10, pp. 529-551, April 1999.
[8] E. Malkiel, J. Sheng, J. Katz, and J. R. Strickler, “The
three-dimensional flow field generated by a feeding
calanoid copepod measuered using digital holography,”
J. of Exp. Bio., vol. 206, pp. 3657-3666, 2003.
[9] J. H. Milgram and W. C. Li, “Computational
reconstruction of images from holograms,” Appl. Optics,
vol 41, pp 853-864, 2002.
[10] T. J. O’Hern, L. D’Agostino, and A. J. Acosta,
“Comparison of Holographic and Coulter counter
measurements of cavitation nuclei in the ocean,” J.
Fluids Eng. Vol. 110, pp. 200-207, 1988.
[11] T. Osborn, “Finestructure, microstructure, and thin
layers,” Oceanography, vol. 11, no. 1, pp. 36-43, 1998.
[12] B. Reguera, I. Bravo, C. Marcaillou-Le Baut, P.
Masselin, M. L. Fernandez, A. Miguez, and A. Martinez,
“Monitoring of Dinophysis spp and vertical distribution
of okadaic acid on mussel rafts from Ria de Pontevedra
(NW Spain),” In: T. J. Smayda and Y. Shimizu (Eds.),
Toxic Phytoplankton Blooms in the Sea. Elsevier,
Amsterdam, pp. 553-558, 1993.
[13] J. Sheng, E. Malkiel, and J. Katz, “A digital
holographic microscope for measuring three
dimensional particle distributions and motions”
submitted to Applied Optics, 2005.
[14] B. Tao, J. Katz, and C. Meneveau, “Statistical
geometry of subgrid-scale stresses determined from
holographic
particle
image
velocimetry
measuremtents,” J. Fluid Mech., vol 457, pp. 35-78,
2002.
[15] P. Tiselius, “An in-situ video camera for plankton
studies: design and preliminary observations,” Mar.
Ecol. Prog. Series, vol. 64 pp. 293-299, 1998.
[16] C. R. Tomas (Ed.), Identifying Marine Phytoplankton,
San Diego, CA, Academic Press, 1997.
[17] J. Watson, S. Alexander, G. Craig, D. C. Hendry, P. R.
Hobson, R. S. Lampitt, J. M. Marteau, H. Nareid, M. A.
Player, K. Saw et al. “Simultaneous in-line and off-axis
subsea holographic recording of plankton and other
marine particles,” Meas. Sci. Technol., vol 12, pp.
L9-L15, 2001.
[18] J. Zhang, B. Tao, and J. Katz, “Turbulent flow
measurement in a square duct with hybrid holographic
PIV,” Exp. Fluids, vol. 23, pp. 373-381, 1997.
deployments, as measured by the CTD, is presented in
Figure 5. A very small fraction of the holograms has been
reconstructed to date. However, we have already
reconstructed selected sequences of holograms containing
images of organism behavior. In some cases, we are able to
observe the very same organism in up to 20 frames. Figure 6
is a sample time series of reconstructed holograms, showing
images of a 2 mm wide medusa jellyfish. Unlike organisms
of this size, whose signatures can be readily identified in the
original holograms, we have to reconstruct multiple planes
to detect the 20-30 µm Dinophysis acuminata. We are only
at the beginning of an extensive data analysis phase, which
will most likely require development of several new
processing tools. Some of these tools will be adopted from
procedures developed during earlier analysis of film-based
holograms [8].
VI. CONCLUSION
Several unique features of our free-drifting,
submersible digital holography system make it suitable for
in-situ observation on behavior of marine organisms.
Holography enables us to maintain the same lateral
resolution over an extended depth. Use of dual
perpendicular views provides the same resolution in all
directions. Cinematography and deployment as a
free-drifting platform enable us to follow and examine the
behavior of the same micro-organisms over many frames.
Preliminary analysis of data obtained during its first
deployment, demonstrates an ability to study the behavior
of organisms at unprecedented levels of detail. At this
point, we have just begun the data analysis phase. The full
potential of this instrument, and subsequent insights that the
data will provide will soon be realized.
As noted before, we still have to resolve some problems
encountered during the first deployment. As we gain more
experience in tracking and recovery operations, we will feel
more confident in removing the safety tether, which may
affect the motion of the system relative to the fluid in the
sample volume.
Acknowledgments
The development of the digital holographic submersible was
sponsered by the National Science Foundation under grant no.
0402792. Some of the equipment has been purchased using
funds provided by NSFMRI grant no. 0079674. The authors
would also like to thank Beatriz Reguera of IEO, our
collaborator and host in Vigo, Spain, Walt Krug & Mike
Franckowiak, superb machinists at JHU, Dan Ursu from JHU
for his help in Vigo, Tom Osborn of JHU for his input
regarding physical oceanography and plankton dynamics,
Louis Whitcomb of JHU for the useful suggestion of using
optical fiber, and finally the captain and crew of the Jose Maria
Navaz.
REFERENCES
[1] D. H. Barnhart, R. J. Adrian, and G. C. Papen,
“Phase-conjugate
holographic
system
for
high-resolution particle image velocimetry,” Appl.
Opt. vol. 30, pp. 7159-7170, 1994.
[2] K. L. Carder, “Holographic microvelocimeter for use
6