Download Design and characterization of a 256x64-pixel single

Design and characterization of a 256x64-pixel
single-photon imager in CMOS for a MEMSbased laser scanning time-of-flight sensor
Cristiano Niclass,* Kota Ito, Mineki Soga, Hiroyuki Matsubara, Isao Aoyagi, Satoru
Kato, and Manabu Kagami
Applied Optics Lab., Toyota Central R&D Labs, Inc., 480-1192, Nagakute, Aichi, Japan
*
[email protected]
Abstract: We introduce an optical time-of-flight image sensor taking
advantage of a MEMS-based laser scanning device. Unlike previous
approaches, our concept benefits from the high timing resolution and the
digital signal flexibility of single-photon pixels in CMOS to allow for a
nearly ideal cooperation between the image sensor and the scanning device.
This technique enables a high signal-to-background light ratio to be
obtained, while simultaneously relaxing the constraint on size of the MEMS
mirror. These conditions are critical for devising practical and low-cost
depth sensors intended to operate in uncontrolled environments, such as
outdoors. A proof-of-concept prototype capable of operating in real-time
was implemented. This paper focuses on the design and characterization of
a 256x64-pixel image sensor, which also comprises an event-driven readout
circuit, an array of 64 row-level high-throughput time-to-digital converters,
and a 16Gbit/s global readout circuit. Quantitative evaluation of the sensor
under 2klux of background light revealed a repeatability error of 13.5cm
throughout the distance range of 20 meters.
©2012 Optical Society of America
OCIS codes: (250.1345) Avalanche photodiodes (APDs); (270.5570) Quantum detectors;
(280.3640) Lidar; (280.3400) Laser range finder; (110.6880) Three-dimensional image
acquisition.
References and links
1.
H. K. Lim, “The 2nd wave of the digital consumer revolution: Challenges and opportunities!” in Proceedings of
IEEE International Solid-State Circuits Conference, Digest of Technical Papers (Institute of Electrical and
Electronics Engineers, New York, 2008), 18–23.
2. J. Dibbel, “Gestural Interfaces: Controlling computers with our bodies,” in 10 Emerging Technologies
Technology Review issue May/June 2011 (MIT, Boston, 2011).
3. R. Lange and P. Seitz, “Solid-state time-of-flight range camera,” IEEE J. Quantum Electron. 37(3), 390–397
(2001).
4. S. B. Gokturk, H. Yalcin, and C. Bamji, “A Time-Of-Flight Depth Sensor - System Description, Issues and
Solutions,” in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Workshops (2004).
5. T. Oggier, B. Büttgen, F. Lustenberger, R. Becker, B. Rüegg, and A. Hodac, “SwissRanger SR3000 and first
experiences based on miniaturized 3D-TOF cameras,” in Proceedings of 1st Range Imaging Research Day,
(ETH Zurich, Switzerland, 2005), 97–108.
6. C. Niclass, A. Rochas, P.-A. Besse, and E. Charbon, “Design and characterization of a CMOS 3-D image sensor
based on single photon avalanche diodes,” IEEE J. Solid-state Circuits 40(9), 1847–1854 (2005).
7. T. Ushinaga, I. A. Halin, T. Sawada, S. Kawahito, M. Homma, and Y. Maeda, “A QVGA-size CMOS time-offlight range image sensor with background light charge draining structure,” Proc. SPIE 6056, 605604, (2006).
8. G. Yahav, G. J. Iddan, and D. Mandelboum, “3D imaging camera for gaming application,” in Proceedings of
IEEE International Conference on Consumer Electronics (Institute of Electrical and Electronics Engineers, New
York, 2007), 1–2.
9. T. Ringbeck, “A 3D time of flight camera for object detection,” in Proceedings of the 8th Conf. on Optical 3D
Measurement Techniques (ETH, Zurich, Switzerland, 2007).
10. C. Niclass, C. Favi, T. Kluter, F. Monnier, and E. Charbon, “Single-photon synchronous detection,” IEEE J.
Solid-state Circuits 44(7), 1977–1989 (2009).
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11863
11. G. Zach, M. Davidovic, and H. Zimmermann, “A 16x16 pixel distance sensor with in-pixel circuitry that
tolerates 150klx of ambient light,” IEEE J. Solid-State Circuits 45(7), 1345–1353 (2010).
12. R. J. Walker, J. A. Richardson, and R. K. Henderson, “A 128x96 pixel event-driven phase-domain -based
fully digital 3D camera in 0.13m CMOS imaging technology,” in Proceedings of IEEE International SolidState Circuits Conference, Digest of Technical Papers (Institute of Electrical and Electronics Engineers, New
York, 2011), 410–412.
13. D. Stoppa, N. Massari, L. Pancheri, M. Malfatti, M. Perenzoni, and L. Gonzo, “A range image sensor based on
10µm lock-in pixels in 0.18µm CMOS imaging technology,” IEEE J. Solid-state Circuits 46(1), 248–258 (2011).
14. S.-J. Kim, J. D. K. Kim, S.-W. Han, B. Kang, K. Lee, and C.-Y. Kim, “A 640x480 image sensor with unified
pixel architecture for 2D/3D imaging in 0.11m CMOS,” in Proceedings of IEEE Symposium on VLSI Circuits
(Institute of Electrical and Electronics Engineers, New York, 2011), 92–93.
15. B. Schwarz, “LIDAR: Mapping the world in 3D,” Nat. Photonics 4(7), 429–430 (2010).
16. J. Levinson, J. Askeland, J. Becker, J. Dolson, D. Held, S. Kammel, J. Z. Kolter, D. Langer, O. Pink, V. Pratt, M.
Sokolsky, G. Stanek, D. Stavens, A. Teichman, M. Werling, and S. Thrun, “Towards Fully Autonomous
Driving: Systems and Algorithms”, in Proceedings of IEEE Intelligent Vehicles Symposium (IV), (Institute of
Electrical and Electronics Engineers, New York, 2011), 163–168.
17. K. Kidono, T. Miyasaka, A. Watanabe, T. Naito, and J. Miura, “Pedestrian recognition using high-definition
LIDAR” in Proceedings of IEEE Intelligent Vehicles Symposium (IV), (Institute of Electrical and Electronics
Engineers, New York, 2011), 405–410.
18. C. Niclass, M. Soga, H. Matsubara, and S. Kato, “A 100m-range 10-frame/s 340x96-pixel time-of-flight depth
sensor in 0.18m CMOS,” in Proceedings of IEEE European Solid-State Circuits Conference (Institute of
Electrical and Electronics Engineers, New York, 2011), 107–110.
19. S. Cova, A. Longoni, and A. Andreoni, “Towards picosecond resolution with single photon avalanche diodes,”
Rev. Sci. Instrum. 52(3), 408–412 (1981).
20. B. L. Stann, J. F. Dammann, M. M. Giza, W. B. Lawler, H. M. Nguyen, and L. C. Sadler, “MEMS-scanned ladar
sensor for small ground robots,” Proc. SPIE 7684, 76841E (2010).
21. T. Sandner, M. Wildenhain, T. Klose, H. Schenk, S. Schwarzer, V. Hinkov, H. Höfler, and H. Wölfelschneider,
“3D imaging using resonant large-aperture MEMS mirror arrays and laser distance measurement,” Proceedings
of IEEE/LEOS Optical MEMS and Nanophotonics (Institute of Electrical and Electronics Engineers, New York,
2008), 78–79.
22. T. Sandner, T. Grasshoff, M. Wildenhain, and H. Schenk, “Synchronized microscanner array for large aperture
receiver optics of LIDAR systems,” Proc. SPIE 7594, 75940C (2010).
23. K. Ito, C. Niclass, I. Aoyagi, H. Matsubara, M. Soga, S. Kato, M. Maeda, and M. Kagami are preparing a
manuscript to be called “System design and performance characterization of a MEMS-based laser scanning timeof-flight sensor based on a 256x64-pixel single-photon imager”.
24. I. Aoyagi, K. Shimaoka, S. Kato, W. Makishi, Y. Kawai, S. Tanaka, T. Ono, M. Esashi, and K. Hane, “2-axis
MEMS scanner for a laser range finder,” in Proceedings of IEEE International Conference on Optical MEMS
and Nanophotonics (Institute of Electrical and Electronics Engineers, New York, 2011), 39–40.
25. C. Niclass, M. Sergio, and E. Charbon, “A CMOS 64x48 single photon avalanche diode array with event-driven
readout,” in Proceedings of IEEE European Solid-State Circuits Conference (Institute of Electrical and
Electronics Engineers, New York, 2006), 556–559.
26. C. Niclass, C. Favi, T. Kluter, M. Gersbach, and E. Charbon, “A 128x128 single-photon imager with on-chip
column level 10b time-to-digital converter array,” IEEE J. Solid-state Circuits 43(12), 2977–2989 (2008).
27. C. Niclass and M. Soga, “A miniature actively recharged single-photon detector free of afterpulsing effects with
6ns dead time in a 0.18µm CMOS technology,” in Proceedings of IEEE International Electron Devices Meeting
(Institute of Electrical and Electronics Engineers, New York, 2010), 14.3.1–14.3.4.
28. L. Pancheri, N. Massari, F. Borguetti, and D. Stoppa, “A 32x32 SPAD pixel array with nanosecond gating and
analog readout,” Proceedings of the International Image Sensor Workshop (IISS, 2011), R40.
29. C. Veerappan, J. Richardson, R. Walker, D. U. Li, M. W. Fishburn, D. Stoppa, F. Borghetti, Y. Maruyama, M.
Gersbach, R. K. Henderson, C. Bruschini, and E. Charbon, “Characterization of Large-Scale Non-Uniformities
in a 20k TDC/SPAD Array Integrated in a 130nm CMOS Process,” in Proceedings of IEEE European SolidState Device Research Conference (Institute of Electrical and Electronics Engineers, New York, 2011), 331–334.
30. J. A. Richardson, L. A. Grant, and R. K. Henderson, “Low dark count single-photon avalanche diode structure
compatible with standard nanometer scale CMOS technology,” IEEE Photon. Technol. Lett. 21(14), 1020–1022
(2009).
1. Introduction
Consumer electronics applications have recently created a great need for low-cost threedimensional (3D) image sensors [1,2] and the sensor community is responding with a high
number of compelling technologies. Applications relying on 3D-enhanced computer vision
are consequently rapidly emerging. Interactive gaming applications have led the trend by
taking advantage of less demanding specifications, in particular with respect to the rejection
to external and uncontrolled background light – a major challenge to most active nearinfrared (NIR) depth sensors. In this context, a fast-growing class of light detection and
ranging (LIDAR) technologies are the so-called time-of-flight (TOF) 3D image sensors [3–
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11864
14]. In these systems, arrays of light emitting diodes (LEDs), or laser diodes (LDs), are
utilized to diffusely illuminate targets within a field-of-view (FOV) that preferably matches
the FOV of the typically employed standard imaging lens. Owing to their simple electrooptical design and the utilization of conventional integrated circuit technologies, such as
CMOS, these techniques commonly benefit from the obvious potential for low-cost and
compact implementations. In these systems, however, the pixels need to cope with extremely
low optical signal intensity due to the diffused light approach. While they operate reasonably
well over a range of several meters in controlled environments, such as indoors, their
performance tends to rapidly deteriorate in severe background light conditions. In these
conditions, higher optical signal to background ratio (SBR), and consequently better
performance, is typically achieved by laser scanner approaches, which take advantage of
collimated laser beams. Not until recently, scanner laser systems exhibited rather limited
vertical resolutions and their use has been mostly relegated to scientific, military and some
specific – typically low-volume – applications. With the introduction of commercially
available and relatively high vertical resolution LIDAR [15], 3D-enhanced computer vision
seems to have broken through the indoor barrier into fully uncontrolled environments. The
most actively researched applications for that technology have been in the automotive field
and, in particular, fully autonomous driving [16] and the detection of vulnerable road users
such as pedestrian and cyclists [17]. The high-resolution LIDAR of [15] thus serves a very
important purpose of providing the required level of performance to enable the research to
progress in these applications. However, its construction, based on an array of 64 discrete
905nm LDs and 64 carefully aligned photodetectors, makes its cost prohibitive for mass
production. Our research group has recently introduced a scanning LIDAR sensor achieving
comparable resolution and performance [18], yet utilizing a single LD coupled with a CMOS
photodetector. Accurate range imaging was obtained outdoors with a resolution of 340x96
pixels at 10 frames per second (fps) within a range of 100 meters. This performance was
partly enabled by taking advantage of arrays of avalanche photodiodes (APD) manufactured
in a readily available CMOS technology. Biased above their characteristic breakdown voltage
and therefore operating in the so-called Geiger mode, APDs become digital pixels [6], thus
enabling direct and virtually noise-free signal processing. These devices, commonly referred
to as single-photon avalanche diodes (SPADs), output a train of digital pulses upon the
detection of individual photonic events with a timing resolution on the order of one hundred
picoseconds [19]. The ability to digitally perform a combined spatial and temporal correlation
of photons within a pixel contributed significantly to the results achieved in [18]. While
continued optimization of that polygonal mirror-based approach is underway in our group, in
particular for applications requiring long range and higher immunity to background light, the
approach has limitations related to system size and cost, thus potentially narrowing its
applicability.
In this work, we propose a new concept of imaging LIDAR. In particular, in order to
allow for significant gains in system size and ultimately cost, we proposed a new microelectromechanical system (MEMS) scanning concept. The design flexibility and resilience
against electronic noise of SPAD signals played a key role towards the newly proposed sensor
architecture.
Another MEMS-based scanned LIDAR system capable of 256x128 pixels has been
recently introduced [20]. Based on a commercially available MEMS mirror, the sensor
consists of fully independent transmitting and receiving optical paths. On the receiving end, a
single InGaAs PIN photodiode allows for the detection of back-reflected laser pulses. Lateral
image resolution is therefore obtained sequentially as the laser beam scans the scene of
interest both in the horizontal and vertical directions. In that concept, the MEMS mirror is not
involved in the receiver path and, hence, a MEMS mirror small enough just to accommodate
the collimated transmitting beam may be effectively employed. A careful analysis of the
concept, however, reveals an important limitation with respect to SBR. Ideally, the receiver
optics should have a very narrow FOV so as to image only the point – or area – currently
being illuminated by the scanner. Only that point benefits from higher optical SBR. In [20],
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11865
since the receiver optics has a fixed and wide FOV, background light reflected from
unilluminated areas on the target are also received, thus practically revoking the advantages
of scanning over diffused light approaches.
Yet another type of MEMS scanned LIDAR concept was introduced in [21,22], whereby
the sensor benefits from a quasi-coaxial transmitting and receiving optics design. In coaxial
LIDAR sensors, transmitting and receiving paths share the same optical axis. Back-reflected
optical signals are thus imaged onto the optical receiver through the same scanning mirror.
The receiver optics may therefore have the desired FOV to match the laser spot size on the
target surface, thus leading to an advantageous SBR [18]. Unlike the previous situation,
however, large area MEMS mirrors are necessary to obtain sufficiently large aperture. Indeed,
TOF ranging performance is also limited by the amount of back-reflected light that is
collected in the receiver channel, thus calling for large aperture designs. In [21,22], a
monolithic array of synchronized MEMS mirrors is proposed to enable one-dimensional and
relatively large-aperture MEMS scanners. Provided that all the mirrors are accurately
synchronized, the ensemble fulfils the same function of a single larger mirror. The authors
also suggested that one synchronized MEMS mirror may be exclusively used for the
deflection of the laser beam on the transmitting path, thus leading to a quasi-coaxial design.
Although the complete LIDAR sensor has yet to be reported, this approach theoretically
addresses the main limitation of [20]. Matching properties between individual mirrors,
especially with respect to their resonance frequency, as well as the large scale
manufacturability of such large MEMS mirrors are still open issues too.
In this work, we propose a new concept of MEMS-scanned LIDAR that benefits from a
high optical SBR capability, yet employing a small area MEMS mirror. In our approach, the
design of the transmitting and receiving channels are fully independent. Similarly to [20], in
our system, the MEMS mirror may be small enough just to accommodate collimated laser
beams. Unlike [20], however, we take advantage of a 256x64 SPAD-based image sensor
whose pixels are actively gated in synchronization with the laser scanning pattern. Owing to
this synchronized gating approach, each pixel on the image sensor does not integrate
background light when its point imaged on the target surface is not illuminated by the laser
beam. Likewise, owing to the utilization of a standard imaging lens, background light
reflected on the remaining areas on the target surface do not interfere with the illuminated
pixel, to the extent that lens non-idealities may be neglected. Furthermore, since the imaging
lens may be chosen independently, large aperture, or equivalently low f-number, small size
lenses may be effectively employed. These benefits are enabled by the introduction of a novel
image sensor architecture based on an event-driven resource sharing principle, which, in turn,
is almost exclusively made possible by SPAD-based pixels. We believe that the concept
proposed here presents a good trade-off between high-performance scanning LIDARs and
their diffused light counterparts in terms of performance, compactness and cost.
While the overall system design issues and detailed ranging performance are reported in a
companion paper [23], this paper focuses on the design aspects and chip characterization of
the proposed CMOS SPAD-based image sensor. The paper is organized as follows. In the
next section, we introduce our depth sensor architecture; in section 3, we comprehensively
describe the architecture and design of the single-photon imager. Sensor characterization and
measurement results are reported and discussed in section 4. The paper is then concluded in
section 5.
2. Depth sensor architecture
Figure 1 illustrates the overall imaging rangefinder architecture. The transmitting optical
system consists of three NIR pulsed LDs, collimating lenses, and a two-axis MEMS scanner.
In order to minimize the oscillation amplitude of the 8x4mm2 MEMS mirror in the horizontal
direction, the FOV was divided in three adjacent scanning regions. The collimated beam from
each LD is then directed to its own region for raster scanning. 870nm LDs emitting pulses of
4ns at a repetition rate of 100 kHz, were utilized. Each LD outputs a maximum peak power of
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11866
40W, while its average power is approximately 16mW. The collimating optics consists of
three individual small lenses and a larger common lens.
Fig. 1. Proposed MEMS scanned TOF sensor architecture.
The MEMS device and its driving electronics, introduced in [24] and fully redesigned for
this work, performs uniform scanning within a FOV of 15 degrees horizontally and 11
degrees vertically, thus leading to a combined FOV of 45 by 11 degrees. Raster scanning is
performed along the Y-axis at the resonance frequency of the MEMS mirror, i.e. 1.3 kHz, and
then driven along the X-axis in a non-resonant mode at 10 Hz. Actuation of the MEMS mirror
is based on a moving magnet configuration. The MEMS driving electronics continuously
measures the actual mirror deflection angle in both directions, by means of Hall sensors
placed underneath the mirror, and provides a digital feedback signal for the main system
controller. On the receiving path, the design complexity is encapsulated into the SPAD-based
CMOS image sensor as much as possible so as to reduce system components. A single f/1.0
lens was therefore utilized to image the targets within the FOV onto the image sensor chip.
The latter also comprises an array of high-throughput time-to-digital converters (TDCs) in
order to measure the arrival time of photons impinging on the photo-sensitive area.
In addition to the MEMS and laser driver boards, the system electronics comprises a
sensor interface board, a digital board featuring a low-cost field-programmable gate array
(FPGA), and a USB transceiver to send data to and from a standard personal computer.
Although most of the digital interface circuits are designed in dedicated digital blocks on the
FPGA device, the latter also embeds a soft 32-bit microprocessor that acts as the main system
controller.
3. Image sensor design
The architecture of the image sensor was carefully devised so as to mitigate the number of
critical specifications in other sensor components. For instance, the maximum measurement
error on the actual MEMS mirror deflection angle is a critical specification as our system
relies on a virtually perfect pixel gating synchronization. This constraint stems from the fact
that an erroneous feedback signal would cause the image sensor to select pixels that are not
illuminated by any laser beam, thus resulting in wrong TOF evaluations. Alternatively, an
exceedingly strict error specification would call for more complexity at the system level, thus
rendering the approach less applicable.
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11867
Fig. 2. SPAD-based image sensor architecture. The image sensor is capable of achieving a
throughput of 16Gbit/s, or equivalently, 800 million TOF evaluations per second.
Another constraint that is unrelated, yet also specific to our concept, arises and should be
addressed. On the photo-sensitive area of our image sensor, silicon real-estate should as much
as possible be devoted to the design of the SPADs so as to maximize the fill factor. Yet, there
is a need to measure the time interval between the laser pulse emission and the instant the
same pulse is detected inside a pixel. An area-inexpensive and very precise circuit to achieve
this function in CMOS is a TDC; however, it still may take significant silicon area when
compared to the size of a pixel. It is therefore more effective in terms of fill factor to place
these circuits on the periphery of the image sensor. The introduction of an asynchronous
readout circuit capable of sharing resources is therefore required. In order to mitigate these
constraints, in particular the error constraint on the Hall sensor circuitry, we introduce a
hybrid pixel gating pattern simultaneously based on a raster scan and on an event-driven pixel
readout mechanism. The underlying idea is to select a slightly broader pixel area around the
location where the laser beam is expected to be imaged, and, at the same time, allow
peripheral TDC resources to be shared by the selected pixels on an effective event-driven
basis. This architecture is feasible provided that pixels featuring gating capability on the
nanosecond scale, and capable of outputting large and robust signals on extremely weak
incoming optical power, exist. To the best of our knowledge, only SPAD-based pixels
currently qualify for these functions in any solid-state technology. An event-driven readout
principle for single-photon pixels was introduced in [25], and was also utilized in other
sensors, e.g. in [26], where this resource sharing scheme is more effective than a simple raster
scan.
Figure 2 shows the architecture of our image sensor based on the proposed MEMSscanner and pixel synchronization approach. The sensor consists of an array of 256x64
SPAD-based pixels, a flexible column decoder circuit, an event-driven pixel readout circuit,
an array of 64 TDCs, a phase-locked loop (PLL), and a sensor readout circuit. Taking into
account our MEMS scanning pattern, we have opted for a row-based TDC layout. As the
number of TDCs match the number of rows in the pixel array, it is not necessary to disable
unused rows synchronously with the scanning pattern as enough resources are available. This
choice is also motivated by the much faster vertical scanning speed as well as by the fact that
the laser beams have a rather small aspect ratio of approximately 1:12. Synchronous column
activation is nonetheless necessary and it is implemented by means of a multi-phase column
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11868
decoder, explained in section 3.2. As photons are detected in the activated pixels, precise
timing trigger signals propagate horizontally to the TDC array, which, in turn, performs time
interval to digital conversions with a resolution of 208ps. A third-order PLL was implemented
on-chip to generate precise timing reference signals for the TDC array, thus keeping process,
voltage, and temperature (PVT) timing variations down to a minimum. The 12-bit TOF data
from the TDCs, together with an 8-bit pixel coordinate stamp indicating the pixel from which
the TOF evaluation originates, is synchronously read out off-chip by means of a pipelined
readout circuit. The latter is based on eight channels that operate as time-multiplexed readout
circuits, each one outputting the data of eight TDCs. In order to cope with the data rate, the
readout channels operate at a frequency eight times faster than the throughput of an individual
TDC.
3.1 Single-photon pixel
Individual photonic events are detected inside a pixel by means of a p+/p-well/deep n-well
SPAD whose cross-section is shown in Fig. 3. The device, a fill-factor-optimized version of
the SPAD presented in [27], measures 10.5µm in diameter, which leads to 13.9% of fill
factor. Its peak photon detection efficiency is 20% at 470nm. At the central wavelength of the
LD, i.e. 870nm, it is 3%. Premature edge breakdown is prevented by means of p-well guardrings. The overall pixel circuit, depicted in Fig. 4, performs passive quenching and recharge,
typical functions required in any SPAD front-end circuit, and implements an output buffer.
Fig. 3. Cross-sectional view of the p+/p-well/deep n-well SPAD. The left half-section shows
the device along a vertical imaginary cut, whereas the right half-section shows it along a
horizontal one. The avalanching p+/deep n-well junction has nonetheless a circular shape.
Fig. 4. In-pixel front-end and readout circuit. The 7-transistor circuit performs passive
quenching and recharge functions. M7 signals a photon detection by pulling the row output line
down for the duration of the SPAD recharge pulse.
Upon photon detection, the latter drives the output line down, thus generating an accurate
trigger signal for the TDC on the same row. All the transistors in the pixel are thick-oxide,
except for the output buffer M7, which requires higher driving strength and is fully isolated
from voltage swings larger than the 1.8V power supply voltage. The cathode terminal VAPD of
all the SPADs in the pixel array is biased at a constant voltage, typically at 25.6V, which is
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11869
3.3V of excess bias (VE) above the SPAD’s breakdown voltage at room temperature. Since
the cathode terminal remains at a constant voltage, the parasitic deep n-well/p-substrate
junction does not contribute to any increase in the parasitic capacitance of the SPAD [27].
The anode terminal VS, however, is rapidly raised by approximately VE upon photon detection
as the avalanche current is much larger than the maximum current set by transistors M1 and
M2. These transistors, biased as a cascode current source, achieve the quenching and recharge
functions. Column selection is performed by transistors M2 and M3 via the column-common
signal VSEL. When VSEL is set to zero logic state, the SPAD is unable to recharge, and,
simultaneously, the gate of M7 is driven to ground, thus fully disabling the pixels in the
corresponding column. When the pixel is selected, transistors M3 and M4 together with the
resistor R1 form a gated NMOS inverter, which is then followed by a CMOS inverter formed
by M5 and M6. These inverters reshape the triangular digital pulse generated on the node VS,
see [27], into a rectangular digital pulse that drives the output buffer. The utilization of a
NMOS inverter is justified by miniaturization constraints. PMOS transistors require a
relatively large clearance distance with respect to the SPAD’s deep n-well layer, so we have
minimized their utilization inside the pixel to just one device. While it would have been
possible to replace M6 by another resistor, resulting in a potential reduction of pixel size [28],
that modification would make the pixel draw quiescent current in our implementation, even
when the pixel is inactive. We believe our solution is a good trade-off between area and
power consumption since no static current is consumed, should the pixel be selected or not.
3.2 Programmable column decoder and event-driven pixel readout
In order to effectively share TDC resources on a row basis and yet allow for some tolerance
on the actual column position of the laser beams, the multi-phase column decoder and the
event-driven readout were designed to operate together. According to our estimates, a
maximum of eight adjacent columns is wide enough to accommodate the uncertainty on the
laser beam position as well as its worst case divergence. Furthermore, in our setup, shown in
Fig. 1, three laser beams are simultaneously directed to the scene. However, in order to
optimally share the same TDC on a given row, the lasers are pulsed at different times. Since
the repetition rate of the lasers is 100 kHz, i.e. with a period of 10µs, each LD is pulsed with a
delay of 3.33µs with respect to the other two. This delay gives enough time so the longest
TOF of interest corresponding to any given LD does not ambiguously enter the time slot of
the remaining ones. As shown in Fig. 2, the column decoder consists of four master-slave
shift registers and a 4:1 multiplexer. Each master-slave shift register is then programmed with
the desired column selection pattern associated with one LD. Once all the shift registers are
programmed, the 4:1 multiplexer allows for fast switching between them in synchronization
with the laser drivers. Figure 5 illustrates the timing diagram of the LDs and their relation
with the column activation pattern. Since our optical setup requires only three laser beams,
the fourth master-slave shift register is optionally used to actively disable all the columns.
Each master-slave shift register has 256 bits for column selection, and additional 8 bits that
can be programmed with a unique code stamp identifying the selection state of that particular
shift-register. This information is also synchronously readout off-chip at a later stage, as
explain in section 3.5. As the MEMS mirror deflects the laser beams horizontally, it is
necessary to continually reprogram the contents of the shift registers. The new state of each
slave shift register chain is serially programmed during operation without affecting the state
of the master one, thus keeping the selection state of the columns intact during this time. It
takes 264 clock cycles of 10ns, i.e. 2.64µs, to reprogram all the four slave shift registers, and
a single clock cycle to transfer their state into the master shift registers. The column decoder
ensures that, at any given time, there is a maximum of eight contiguous columns on a single
row that may utilize the TDC circuit. Based on this assumption, the row event-driven readout
of [25] was significantly simplified in this work. Figure 6 depicts the readout circuit of a
single pixel row, whereby only the first nine pixels are illustrated. In order to convey
asynchronous time interval, i.e. TOF, requests to the TDC, each row utilizes eight column
address lines. These lines are pulled up by properly biased PMOS transistors at the end of the
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11870
pixel row, shown as resistors on the left-hand side of Fig. 6. As can be seen in the figure, the
lines are connected to the output buffer of pixels following a regular pattern. The first line is
connected to the 1st, 9th, 17th, 25th pixel, and so on. The second line is then connected based
on the same pattern, i.e. connected to the 2nd, 10th, 18th, 26th pixel, and so on. Since there
are 256 pixels on a row, each address line is connected to 32 pixels. Despite the multiple
connections on a single line, since a maximum of eight contiguous pixels are activated at any
point in time, it is ensured that each line is connected to a single activated pixel.
Fig. 5. Timing diagram of the LDs and illustration of the column activation pattern. Since each
LD emits a laser pulse with a delay of 3.33µs with respect to the others, each TDC may be
effectively shared on a row basis. The maximum theoretical TOF is limited by the TDC range
of 853ns, or equivalently 128 meters.
Fig. 6. Simplified event-driven readout of a single pixel row. Column decoder lines are not
drawn in the picture.
Upon photon detection in one of the eight activated columns, the pixel pulls the
corresponding line down by means of its output buffer, which, in turn, generates an inverted
pulse along the row. On the right-hand end of the row, the eight lines are inverted, then fed
onto an 8-input OR gate. At the output of the OR gate, i.e. TRG, digital pulses whose leading
edge is accurately related to the photon arrival times are generated. This signal is utilized as a
trigger for the time-to-digital conversion, as well as a latching signal to store the state of the
address lines by means of eight flip-flops, as shown in Fig. 6. The delay through the OR gate
ensures that the setup time of the flip-flops is not violated. Once the state of the eight lines is
secured by the flip-flops, the row enters a dead time period of approximately 40ns, equivalent
to the dead time of the triggered SPAD. Subsequent photon detections on the remaining
pixels in the same row are simply missed during this dead time, not affecting therefore the
previous measurement. It is however possible that photons impinging on two or more pixels
trigger them simultaneously. In such situation, the state of the address lines will indicate that
two or more pixels were involved in that TOF event. Consequently, the TOF computed by the
TDC may be assigned to more than a single pixel. If, on the other hand, two pixel detections
are delayed by more than the margin delay of the flip-flop setup time, approximately 200ps in
our design, only the leading photon detection is taken into account. At the end of the pixel
dead time, the address lines are pulled up again, thus enabling subsequent TOF detections.
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11871
3.3 Time-to-digital converter array
Figure 7 shows the flash-conversion TDC array architecture. At the top of the TDC array,
fully-differential reference signals, namely 0, 1, 2, and 3, are generated by an 8-phase 600MHz PLL. These 600MHz signals, illustrated in Fig. 8, are uniformly delayed by 208ps and
are shared vertically by all the TDCs.
Fig. 7. Flash TDC array architecture.
Fig. 8. Example of waveforms of some TDC and PLL signals. In this example, two TOF
evaluations are illustrated. Prior to the first TOF evaluation, the START signal is asserted
once, thereby initiating the state of S[11:0]. Note that, when sampled and despite potential
setup time violations, the phase state {i}i3 always settles in the interval [0,7].
A coarse timing reference of 1.667ns for time-to-digital conversion is achieved by two 9bit counters placed near the PLL that are synchronously incremented by 0 and its inverted
replica, respectively. The output of these counters, i.e. C0[8:0] and C1[8:0], are also shared by
the remaining TDCs. A special TDC is utilized to register START signal assertions. Its
function is to sample the state of {i}i3, C0[8:0], and C1[8:0] and to encode this timing
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11872
information onto a 12-bit digital signal, i.e. S[11:0], which, in turn, is shared by the remaining
64 STOP TDCs. The encoding logic is implemented as follows: The state of {i}i3 is
encoded into the 3 least significant bits of S, i.e. S[2:0], according to the sampled phase delay
with respect to 0 within a single clock period. The most significant bits of S, i.e. S[11:3],
originate from either C0 or C1, depending on the 3-bit encoded sampled phase. As can be seen
in Fig. 8, it is possible that the sampled value of C0 or C1 be corrupted if the sampling occurs
during the state transition of either counter. However, when the state of one of the two
counters is in transition, the other counter state is safely stable. As a result, depending on the
decoded value of S[2:0], either C0 or C1 is safely utilized as S[11:3]. Upon assertion of its
trigger signal, each stop TDC synchronously registers the state of {i}i3, C0[8:0], C1[8:0],
and S[11:0]. According to Fig. 7, the time difference between the STOP and START is finally
computed by first generating the intermediate 12-bit value S’, based on {i}i3, C0, and C1, in
a similar way to which S is generated in the start TDC, followed by a subtraction operation
between S’ and the sampled S value. The signal VLD is asserted so as to schedule the readout
of the valid TOF result into a synchronization FIFO. Figure 8 illustrates the internal
waveforms for a TDC involved in two hypothetical time-to-digital conversions. As can be
seen in the Fig. 8, several STOP assertions may occur on a single TDC between two
consecutive START assertions. Furthermore, although the proposed architecture is capable of
performing flash conversion, the TDC undergoes a dead time cycle of approximately 40ns
due to the pixel event-driven readout mechanism.
3.4 Phase-locked loop
The PLL, shown in Fig. 9, is a type-II based on a commonly used zero dead-zone phasefrequency detector and charge pump. Based on an external reference signal of 75MHz and a
fully-differential voltage controlled oscillator (VCO), the PLL synthesizes the four uniformly
distributed 8-phase signals at 600MHz. The 75MHz reference signal is provided by a crystal
oscillator that is assumed to be uncorrelated to the laser repetition rate, thus preventing
systematic interference between the start TDC and the PLL’s VCO. A passive second-order
filter is placed off-chip near the PCB-bonded image sensor.
Fig. 9. Schematic of the third-order PLL. A passive second-order loop-filter LP() is placed
off-chip.
3.5 Sensor readout circuit
During the image acquisition time, the image sensor constantly outputs TDC data together
with binary words that identify the pixel that originated each TOF evaluation, independently
of the current column decoder selection state. The pipelined readout circuit is operated
synchronously with a clock frequency of 100MHz, thus achieving a maximum throughput of
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11873
16Gbit/s, or equivalently, 800Msamples/s. As can be seen in Fig. 2, the 64 TDCs are
combined in eight time-multiplexed readout groups of eight TDCs each. A 3-bit signal,
IDX[2:0], synchronously indicates the currently selected TDC in each group, thus allowing
the companion FPGA device to identify the TDC, or equivalently the pixel row, the data
should be assigned to. A readout word associated with each TDC comprises 21 bits of data,
i.e. the 12-bit TOF word, the 8-bit COL binary pattern registered by the event-driven readout
according to section 3.2, and a valid bit. The latter is also used by the readout circuit to
prevent chip output pads from toggling when no valid data are available for that particular
clock cycle. Since the COL pattern only indicates the pixel involved in the TOF evaluation
from among the eight selected columns, the coordinates of this contiguously selected column
block is read out on a separate bus, namely COLADD[7:0], see Fig. 2. Furthermore, since the
readout circuit takes some clock cycles from the moment a photon is detected until its TOF
data word is read out off-chip, it is important to register the column decoder coordinate data
word and to delay its readout by the same number of cycles. This is achieved by pipelining
the COLADD bus, thus ensuring that the companion FPGA is presented with a valid
COLADD word that matches the currently read out TOF data.
4. Experimental results
A photomicrograph of the single-photon image sensor is shown in Fig. 10(a). The chip was
implemented in a high-voltage 0.18µm CMOS technology. A full sensor characterization was
performed and the sensor chip was confirmed to be fully functional at its nominal clock
frequencies. As can be seen in Fig. 10(a), the effective active area of the image sensor
occupies approximately 1/3 of the total chip area, while the PLL and TDC array only
occupies a relatively small area. The sensor chip measures 8x5 mm2, mostly due to the 256pad ring limited design. While the chip could be readily mounted on a standard chip package,
we have opted for bonding the chip directly onto a printed circuit board.
Fig. 10. (a) Photomicrograph of the CMOS single-photon image sensor. The chip measures
8x5mm2. (b) Experimental setup of the overall LIDAR system.
The overall LIDAR system, which at the current form factor still benefits from a large
margin for miniaturization, is depicted in Fig. 10(b). In Fig. 10(b), the triple-LD driver and its
collimating optics can be seen with illustrations on how the laser beams are deflected by the
MEMS mirror. A standard imaging lens coupled with a NIR optical filter was utilized. The
system is capable of acquiring full 256x64-pixel images at 10 frames/s.
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11874
4.1Dark count rate
As is commonly the case with SPAD-based image sensors [6], the proposed imager benefits
from digital pixels whose signals are virtually free from readout noise. As a result, with the
exception of photon-shot noise, the imaging noise performance is typically dominated by
intrinsic SPAD noise, which, in turn, is mainly characterized in terms of the dark count rate
(DCR). Thermally or tunneling generated carriers within the p-n junction, which typically
generate dark current in linear mode photodiodes, cause unwanted trigger pulses in SPADs
that may not be directly discerned from regular photon triggered events. The rate of these
spurious Geiger events, i.e. DCR, was measured by acquiring an image in dark conditions for
a fixed acquisition time. Figure 11 shows the DCR image of a sample chip sensor at 30°C
with an excess bias (VE) of 3.3V above the SPAD breakdown voltage of 22.3V. As can be
seen in the figure, a low number of pixels present very high levels of DCR, which typically
appear in the intensity image as “salt and pepper” noise, without the “pepper” component.
DCR impact on the image may be conveniently understood by sorting the DCR values of all
the pixels and plotting them as a function of accumulated pixel count, or alternatively as
shown in Fig. 12, as the percentile of imaging pixels.
Fig. 11. DCR image acquired at 30°C, measured in counts per second.
Fig. 12. DCR distribution at 30°C. The measured DCR from all 16384 pixels were sorted and
plotted as a function of pixel percentile. On right-hand side, the same plot is shown with only
the 2% noisiest pixels.
As can be verified in Fig. 12, the median DCR at room temperature was 750 counts/s. On
the right-hand side of the figure, the noisiest two percentiles of the DCR are shown. A
relatively well defined curve “knee” can be seen approximately at 99%, at which the DCR is
33.2kcount/s. In our application, our sensor needs to cope with considerable high levels of
background light intensities, thus rendering DCR tolerable even for relatively noisy pixels. In
applications where the DCR performance is a more critical parameter, the top 1% of pixels
could be permanently disregarded at the image readout level. Since DCR is a strong function
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11875
of the SPAD area, it is convenient to report the DCR density, especially when a comparison
to other devices is of interest. In this work, the median DCR density was 8.66 counts/µm2,
whereas the average DCR density was 45 counts/µm2, mostly due to a low number of outlier
pixels. The median DCR density figure is slightly worse than the 6 counts/µm2 figure recently
reported at the same temperature for a SPAD array of comparable size and optimized for lowlight applications [29].
4.2 Time-to-digital converter array and overall timing evaluation performance
Timing performance evaluation was carried out after verification of the functionality of the
on-chip PLL. The TDC array was carefully characterized with respect to its typical
performance figures, namely differential non-linearity (DNL) and integral non-linearity
(INL). Since the array of 64 TDCs is utilized in an imaging configuration, rather than mean
performance, the worst-case TDC performance is essential. DNL was characterized at room
temperature for all the TDCs simultaneously. While a single column of pixels was selected by
the column decoder, the remaining chip functions were activated at their nominal clock
frequencies during the characterization. In particular, the sensor readout circuit was expected
to generate switching noise on-chip, which typically couples to critical reference signals
through substrate and/or power supply lines.
Fig. 13. (a) Measured DNL of individual TDCs in the array versus measurable time. (b)
Measured INL of individual TDCs as well as envelope curves of the maximum deviations.
Note that the worst INL among the 64 TDCs was 0.56LSB (116ps).
In Fig. 13(a), the DNL measured in picoseconds is plotted for all the 64 TDCs over the
full measurable time range. The worst-case DNL was considerably smaller than the TDC
resolution of 208ps, thus ensuring monotonic conversion performance. Figure 13(b) shows
INL curves measured in picoseconds as a function of measurable time range. The worst-case
INL among all the 64 TDCs and for all the TDC codes are also plotted as envelope curves in
the figure. As can be seen, the worst-case non-linearity error was 0.56LSB, or equivalently
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11876
116ps. When converting this timing deviation to distance, it corresponds to a non-linearity
error of 1.75cm, throughout the theoretical maximum distance range of 128 meters.
Fig. 14. Superimposed plot of 64 normalized histograms on photon arrival times. The
histograms were acquired while exposing a complete pixel column to a 40ps pulsed laser
source repeated at 40MHz. The laser trigger signal was divided by 34 inside a FPGA device
before being connected to the TDC start signal.
Fig. 15. Detailed view of the detected pulses in the set of 64 histograms. The average IRF at
FWHM was 519ps. Inset: semi-log plot of the same data showing the slightly asymmetric IRF.
In order to measure the timing resolution, also commonly referred to as the instrument
response function (IRF), of the overall detection circuit including the SPAD and TDC
contributions, an external 80ps full width at half maximum (FWHM) laser source was
utilized. Due to the unavailability of picosecond laser sources at 870nm in our laboratory, the
image sensor was directly exposed to a 635nm laser light at a repetition rate of 40MHz. The
laser beam intensity was attenuated by means of neutral-density filters and made spatially
uniform by means of a diffuser. Since the laser repetition rate was 40MHz, in order to
evaluate the timing performance over the whole TDC range, the digital trigger output signal
of the laser source was divided by 34 on the FPGA device, and then connected to the START
signal of the TDC array. Based on this setup, data from all the 64 TDCs were acquired
simultaneously for approximately one second. On the FPGA, 64 histograms were built as a
function of the measured TOF. These histograms are plotted in Fig. 14 after normalization.
As expected, the histograms comprise 34 peaks, corresponding to the 34 laser pulses
contained in the time interval between two consecutive START trigger events. Note that 64
histograms are superimposed in the figure.
In order to measure the actual timing IRF, a detailed view of the histogram peak plotted
on a linear scale is shown in Fig. 15, while the same data are plotted on a semi-log scale in the
inset. The mean timing IRF among these curves was 519ps FWHM. The obtained timing IRF
was considerably larger than the typical IRF of a SPAD, on the order of 100ps
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11877
[6,10,19,26,29]. Many factors are believed to have contributed to the overall IRF, major
contributors being the jitter sensitivity of the TDCs to power supply noise as well as crosstalk
between channels. Note that the IRF was measured under strong – nonetheless realistic –
photon detection activity. Although the IRF measurement was carried out with a 635nm laser,
as opposed to the 870nm LDs utilized in our LIDAR setup, we do not expect a significant
impact on the IRF figure. This assumption, which was valid for a similar SPAD in CMOS
[30], is also supported by the fact that our IRF figure is dominated by circuit jitter, which
does not depend on the signal wavelength. In addition, since the measured IRF is significantly
shorter than the laser pulse duration of 4ns utilized in our LIDAR setup, the overall IRF is
expected to have a negligible impact on LIDAR performance.
Furthermore, since the data were acquired for all the TDCs simultaneously, Fig. 15
already reveals the timing delays between the TDC channels, which are considerably shorter
than the IRF. In the inset of Fig. 15, the typical asymmetric IRF of SPADs is also revealed,
albeit considerably skewed by a large Gaussian jitter component.
4.3 Ranging performance and time-of-flight imaging
Quantitative evaluation of the proposed LIDAR sensor was carried out under 2klux of natural
background light within a distance range of 20 meters. A non-cooperative target was built by
wrapping a wide board in a white cotton fabric. It was placed in front of the sensor and
moved from 2 meters to 20 meters in steps of 1 meter. The actual target position was
controlled with a tape measure and the ambient light condition on the target surface was
measured with a digital luxmeter. At each distance, a set of 100 depth image samples were
acquired at 10fps.
Fig. 16. (a) Mean measured distance under 2klux of background light as a function of the
actual distance for a pixel on the center of the non-cooperative target. (b) Non-linearity error
(square marker, blue curve) and standard deviation (round marker, red curve) as a function of
the actual distance.
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11878
Table 1. Comparison of Performance to the Recently Published State-of-the-art in CMOS
Parameter
Unit
Niclass,
et al.,
JSSC,
2009
[10]
Zach,
et al.,
JSSC,
2010
[11]
Walker,
et al.,
ISSCC,
2011
[12]
Stoppa,
et al.,
JSSC,
2011
[13]
Kim, et al.,
VLSI Symp,
2011 [14]
This
Work
Depth image resolution
pix.
60x48
16x16
128x96
80x60
356x248
256x64
Technology
-
0.35µm
HV
CMOS
0.6µm
Bi
CMOS
0.13µm
imag.
CMOS
0.18µm
imag.
CMOS
0.11µm
imaging
CMOS
0.18µm
HV
CMOS
Optical fill factor
%
<1
66
3.2
24
34.5
13.9
Illumination wavelength
nm
850
850
850
850
850
870
Illumination repetition rate
or frequency
MHz
30
10
3.33
20
10
0.3
Illumination power (mean)
mW
800
900
50
80
Not stated
50
Field of view (HxV)
deg.
50x40
6x6a
40x40
8x6a
37x26a
45x11
Frame rate
fps
22
16
20
5
20
10
Target reflectivity
-
White
White
White
White
White
White
Distance range (measured)
m
2.4
3.2
2.4
6
3
20
Unambiguous
range
m
5
15
45
7.5
15
128
1 error
cm
3.8
<5
16
16
6.2
13.5
Max.
distance
m
2.4
3
2.4
6
3
20
Background
light
klux
0.15
Not
stated
0.11
Not
stated
Not
stated
2
Max. error
cm
11
5
(15)
0.5
4
Not
stated
10
Max.
distance
m
2.4
1.5
2.4
6
3
20
klux
0.15
100
(150)
0.11
Not
stated
Not
Stated
2
%
1.6
1.3
6.7
2.7
2.1
0.68
Precision
Nonlinearity
distance
Background
light
Relative precision
a
Calculated using published parameters as
FOV  2  tan
1
 
h
, where f is the lens focal length and h is the pixel array
2f
width (H) and height (V). The FOV was not explicitly reported in the respective references.
The depth images were computed in real-time on the FPGA by performing histogram
processing for each pixel. In Fig. 16(a), the mean value of 100 distance samples from one
pixel in the center of the target is plotted as a function of the actual distance. Based on the
same samples, the mean error with respect to the ground truth and the standard deviation were
computed. These figures, which correspond to the non-linearity error and the 1 repeatability
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11879
error of our depth image sensor, respectively, are plotted in Fig. 16(b). The worst-case
repeatability error was 13.5cm throughout the range of 20 meters, thus leading to a relative
precision of 0.68%. In the same range, the non-linearity error was better than 10cm.
Table 1 shows a performance comparison of this work to the recently published state-ofthe-art in CMOS technology. With the exception of this work, the repeatability errors of the
remaining sensors were characterized under low or undisclosed ambient light conditions. To
the best of our knowledge, our sensor compares favorably to the state-of-the-art in terms of
relative precision and range, in particular when taking into account the number of pixels,
signal power, FOV, and frame-rate.
The overall functionality of the proposed LIDAR system was also qualitatively evaluated
by acquiring depth images of common targets. An example of such images is shown in Fig.
17. Distance information is color-coded with near objects appearing in red while far objects
are shown in blue. In the figure, the imaged person was standing at approximately 3 meters in
front of the sensor, while the back of the room was situated at approximately 7 meters. Note
that pixels that exhibit high DCR were fully functional in the TOF evaluation. These noisier
pixels had a rather negligible impact on the resulting depth image. Column fixed-pattern noise
in the ranging image, however, may be noticeable in Fig. 17. This effect occurred due to the
simplified implementation of the interface circuit on the companion FPGA. Rather than
horizontally scanning the eight activated columns in sync with the MEMS mirror deflection
in steps of one column, see section 3.2, the selection window was moved in steps of eight
columns at a time. However, since the same effect cannot be perceived when the sensor
operates as a conventional image sensor, i.e. when it is not exposed to the LD signal, we
believe the column fixed-pattern noise may be mitigated when the column selection window
is scanned in smaller steps [23].
Fig. 17. Illustration of depth data acquired using the proposed sensor. The measured distance
in meters is color coded, red being close to the sensor and blue being farther from it.
5. Conclusion
We proposed a new concept for MEMS-scanned LIDAR that benefits from a high optical
signal-to-background ratio capability while employing a small area MEMS mirror. Our
system concept takes advantage of an image sensor whose pixels are actively gated in
synchronization with the laser scanning pattern. Owing to this synchronized gating approach,
each pixel of the image sensor does not integrate background light when its corresponding
image point on the target surface is not illuminated by the laser beam. Furthermore, since the
imaging lens may be chosen independently for the receiving optics, large aperture and small
size lenses may be effectively employed. These benefits are enabled by the introduction of a
novel image sensor architecture based on an event-driven resource sharing principle, which,
in turn, is nearly exclusively made possible by SPAD-based pixels.
A proof-of-concept prototype capable of operating in real-time was implemented. At the
core of the system, a single-photon image sensor comprising 256x64 pixels was designed and
characterized with respect to its performance metrics. The image sensor comprises fast
column synchronization decoders, an event-driven pixel readout circuit, an array of 64 rowlevel high-throughput time-to-digital converters, and a 16Gbit/s global readout circuit. The
image sensor was fabricated in a high-voltage 0.18µm CMOS technology and was confirmed
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11880
to be fully functional at its nominal clock frequencies. The median dark count rate, i.e. the
dominant intrinsic noise source in single-photon imagers, was 750 counts/s for circular
SPADs of 10.5µm in diameter. While higher to some extent than the state-of-the-art for lowlight imaging, this figure is assumed to be reasonably acceptable in LIDAR systems intended
to operate under strong background light conditions. Full characterization of the TDC array
was performed with respect to differential and integral non-linearities. The worst-case
differential non-linearity was approximately 35ps, much smaller than the TDC resolution of
208ps. The worst-case integral non-linearity at room temperature, taking into account all the
64 TDCs, was 116ps, equivalent to 0.56LSB. This worst-case systematic time deviation
corresponds to 1.75cm in distance error, throughout the theoretical maximum range of 128
meters. An 80ps laser source was utilized to measure the timing IRF of the overall detection
channels, including the jitter contributions from the SPADs and TDCs. The measured IRF
was 519ps under a realistic activity load, mostly due to the jitter sensitivity of the TDCs to
power supply noise. This value was considerably shorter than the duration of the laser pulses
utilized in the proposed LIDAR setup, i.e. 4ns, thus having a negligible impact on the ranging
performance.
Quantitative evaluation of the TOF sensor under 2klux of natural ambient light revealed a
repeatability error of 13.5cm throughout the distance range of 20 meters, thus leading to a
relative precision of 0.68%. In the same condition, the non-linearity error was better than
10cm. To the best of our knowledge, these results compare favorably to the state-of-the-art, in
particular when taking into account specifications that have a direct and strong impact on
precision. Examples of these specifications are the number of pixels, signal power, sensor
field-of-view, and frame-rate. The proposed setup was also qualitatively evaluated in this
work by acquiring depth images in ranges of several meters. The resulting images
demonstrated the full functionality of the sensor.
Acknowledgments
The authors would like to thank Mitsutoshi Maeda for helping in the design and assembly of
the optical system, Marc Lany and R. S. Popovic of EPFL as well as Toshiki Kindo of Toyota
Motor Corporation for their technical support on the design of the SPAD device.
#164400 - $15.00 USD
(C) 2012 OSA
Received 8 Mar 2012; revised 21 Apr 2012; accepted 21 Apr 2012; published 10 May 2012
21 May 2012 / Vol. 20, No. 11 / OPTICS EXPRESS 11881